Wednesday, 2020-08-05

*** Fl1nt has quit IRC00:17
oyrogergyoctozepto: Thank you for response re KA/OSA the other day. Also thank you to whoever set up the channel logging since my scrollback buffer wasn't big enough.00:34
oyrogergmgoddard: I would indeed like to hear any thoughts or links you might be willing to share. I will add to my earlier description: We were leaning toward KA last year before other priorities got in the way of the work, and now want to restart. I would like to inform a direction with some comparison that doesn't require full research into both, to record as our reasoning for colleagues and such to read.00:38
*** dastor has quit IRC01:40
*** dave-mccowan has joined #openstack-kolla01:45
*** dave-mccowan has quit IRC02:13
openstackgerritZihao Wang proposed openstack/kolla-ansible master: Change ssh-key to a variable  https://review.opendev.org/74459302:19
*** diurnalist has joined #openstack-kolla02:20
*** diurnalist has quit IRC02:42
*** diurnalist has joined #openstack-kolla02:44
*** brinzhang has joined #openstack-kolla02:54
*** JamesBen_ has quit IRC03:03
*** dcapone2004 has joined #openstack-kolla03:05
*** skramaja has joined #openstack-kolla03:21
*** skramaja has quit IRC03:36
*** skramaja has joined #openstack-kolla03:36
*** JamesBenson has joined #openstack-kolla03:38
*** diurnalist has quit IRC03:54
*** suryasingh has joined #openstack-kolla03:55
*** rohit02 has joined #openstack-kolla04:03
*** vishalmanchanda has joined #openstack-kolla04:15
*** brinzhang_ has joined #openstack-kolla04:24
*** brinzhang has quit IRC04:27
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-kolla04:33
*** brinzhang has joined #openstack-kolla04:46
*** brinzhang_ has quit IRC04:48
*** ladrua has quit IRC04:49
*** rohit02 has quit IRC04:58
*** rohit02 has joined #openstack-kolla04:58
*** GeraldS has joined #openstack-kolla04:59
*** JamesBenson has quit IRC05:01
*** diurnalist has joined #openstack-kolla05:01
*** diurnalist has quit IRC05:06
*** pbing19 has joined #openstack-kolla05:16
*** skramaja_ has joined #openstack-kolla05:39
*** skramaja has quit IRC05:39
*** rohit02 has quit IRC05:58
*** rohit02 has joined #openstack-kolla05:58
*** rohit02 has quit IRC06:07
oyrogergmgoddard: https://youtu.be/0liqSO0SZ60?t=5287 "I'll talk about reasons later on..." but I didn't catch anything after that. Can you fill me in?06:09
*** skramaja_ has quit IRC06:19
*** skramaja has joined #openstack-kolla06:19
*** e0ne has joined #openstack-kolla06:47
*** nikparasyr has joined #openstack-kolla07:06
*** bengates has joined #openstack-kolla07:07
*** skramaja has quit IRC07:13
mgoddardoyrogerg: https://youtu.be/0liqSO0SZ60?t=567007:31
mgoddardoyrogerg: I didn't do a full comparison between kolla and OSA, only described what I see as some of the main benefits of the kolla approach07:32
*** dougsz has joined #openstack-kolla07:32
mgoddardoyrogerg: honestly I expect OSA has changed enough since my initial evaluation that it would be unfair to comment on it07:33
mgoddardalthough I'm sure my decision would be the same today :)07:33
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/ussuri: Fix play hosts for ironic, monasca, neutron, nova  https://review.opendev.org/74483707:35
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Fix play hosts for ironic, monasca, neutron, nova  https://review.opendev.org/74483807:35
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix play hosts for ironic, monasca, neutron, nova  https://review.opendev.org/74484007:37
*** dastor has joined #openstack-kolla07:45
*** pbing19 has quit IRC07:46
*** pbing19 has joined #openstack-kolla07:47
*** e0ne_ has joined #openstack-kolla07:54
*** e0ne has quit IRC07:54
*** skramaja has joined #openstack-kolla07:56
mnasiadkamorning08:03
mgoddardmorning08:03
*** gfidente|afk is now known as gfidente08:03
hrwelo08:06
*** Fl1nt has joined #openstack-kolla08:10
Fl1ntHi everyone!08:10
openstackgerritMerged openstack/kolla-ansible master: Fix actions for Aodh and Swift  https://review.opendev.org/74469508:25
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible stable/ussuri: Fix actions for Aodh and Swift  https://review.opendev.org/74485208:28
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible stable/train: Fix actions for Aodh and Swift  https://review.opendev.org/74485308:28
openstackgerritMark Goddard proposed openstack/kayobe master: Add support for custom Masakari configuration  https://review.opendev.org/74422808:28
*** k_mouza has joined #openstack-kolla08:29
Fl1nt@hrw, are you there? I've run the SBSA test, 56 run, 24 passed and 9 failed, how can I extract them to something that can be shared ?08:32
hrwFl1nt: pastebin.com?08:39
hrwFl1nt: I only need output08:39
Fl1ntwell, that my question, how do I extract the output in a file ?08:41
Fl1ntthe sbsa.efi file is directly executed ^^08:41
*** dcapone2004 has quit IRC08:43
hrwFl1nt: priv08:47
*** devep has joined #openstack-kolla08:57
*** devep has quit IRC08:59
*** devep has joined #openstack-kolla08:59
openstackgerritBartosz Bezak proposed openstack/kolla master: Install Monasca plugin for Grafana by default  https://review.opendev.org/67618509:00
*** devep_ has joined #openstack-kolla09:04
*** devep__ has joined #openstack-kolla09:07
*** devep has quit IRC09:07
*** devep__ has quit IRC09:07
*** devep_ has quit IRC09:09
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Fix fernet bootstrap and key distribution - follow up  https://review.opendev.org/70708009:25
oyrogergmgoddard: Not much of a comparison with OSA rather with other ways or rather a description of why KA does things that way. All helpful stuff.09:33
oyrogergother ways e.g. container vs non-container, Docker vs K8s, ... perhaps in other words, avoiding us vs them talk, which can be unhelpful at times.09:34
oyrogergAnyway I'll keep prodding the topic, because at some point someone will ask me "why didn't you use OSA?" and I want to have a half decent answer given I don't work for a one-person IT shop any more.09:35
mgoddardoyrogerg: OSA either deploys services to bare metal or system containers (LXC). In either case, they're treated like hosts, and you lose the benefits of application containers listed09:35
mgoddardoyrogerg: last time I looked, the bridge networking setup for OSA was very complicated09:36
oyrogergAha, that is concrete useful comparison, thank you.09:40
*** e0ne_ has quit IRC09:42
*** e0ne has joined #openstack-kolla09:43
mnasiadkamgoddard: I think they also support systemd-nspawn, but that doesn't change anything compared to lxc09:45
mgoddardmnasiadka: thought they dropped that09:45
*** e0ne has quit IRC09:46
mnasiadkamgoddard: in centos8 yes, because machinectl needs btrfs :)09:46
*** e0ne has joined #openstack-kolla09:46
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: ubuntu: move to 20.04 Focal  https://review.opendev.org/74350209:51
hrwmnasiadka: it was on bionic due to ussuri not supporting focal09:52
hrwmnasiadka: check result of zuul run on 1st revision09:52
mnasiadkaah right09:53
mnasiadkawell, so we're in sort of dead spot :)09:53
*** e0ne has quit IRC09:53
hrwdeprecate ubuntu09:54
*** e0ne has joined #openstack-kolla09:54
mnasiadkahrw: or just use '3' as python version?09:55
*** born2bake has joined #openstack-kolla09:56
hrwmnasiadka: it should have 3.6/3.8 handled properly now09:56
hrwmnasiadka: but using 'python3' is more sane09:56
mnasiadkahrw: nope it doesn't, if you run focal containers on bionic - it will try to use bionic python3.6 as wsgi script interpreter on focal container images - and boom09:56
hrwok09:57
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: ubuntu: move to 20.04 Focal  https://review.opendev.org/74350209:57
mnasiadkaso let's check '3' as distro_python ;)09:57
mnasiadkaok, back to healthchecks, seems some are failing ;)09:58
Fl1ntuse 3 and have an alternative set, it use generic everywhere and get specific when needed that way ;-)09:59
mgoddardwe use the minor version in some places10:00
hrwmnasiadka: python_distro_version needs to be major.minor due to us playing with site-packages directory10:00
mnasiadkaoh boy10:00
oyrogergSo this leads me to the question I had already thought of... what are the first class citizen operating systems of Kolla Ansible?10:00
hrwmnasiadka: we may move to python3 from python{{ distro_python_version }}10:00
hrwoyrogerg: centos810:00
Fl1ntCentOS I would said @oyrogerg10:01
mnasiadkahrw: that's a bigger change, but makes sense10:01
mgoddardoyrogerg: CentOS, Ubuntu, Debian (in that order)10:01
hrwoyrogerg: we support centos 8, ubuntu 18.04 and debian 1010:01
mgoddardoyrogerg: https://docs.openstack.org/kolla/latest/support_matrix.html10:01
oyrogergAnd Ubuntu as second class, then... right. So Ubuntu 20.04 support will presumably come in time.10:01
mgoddardoyrogerg: https://docs.openstack.org/kolla-ansible/latest/user/support-matrix.html10:02
oyrogergThe best kind of RTFM comment: An actual link.10:02
mgoddardoyrogerg: we have quite a few ubuntu users. Majority CentOS I would say10:02
hrwoyrogerg: I handle Debian stuff because Debian/source/aarch64 is used by Linaro. Debian/x86-64 is kept in buildable state.10:02
hrwoyrogerg: most of devs deploy centos 8 from what I see10:03
oyrogergGood to know. We'll be building it from scratch so we might choose Centos8 to reduce the innovation token use.10:03
mgoddardoyrogerg: remember to change latest to the release name you are interested in in docs links10:03
hrwand for us it is easiest to talk to/with centos/rdo teams when we have issues10:03
oyrogergmgoddard: For sure. I spend a bit of time at docs.os.org10:03
mnasiadkahrw, mgoddard: or distinct host_python_version from container_python_version ;)10:04
hrwmnasiadka: go for it!10:04
*** e0ne has quit IRC10:04
mgoddardmnasiadka: it is distinct10:04
mnasiadkamgoddard: where10:04
*** e0ne has joined #openstack-kolla10:04
mnasiadka?10:04
mnasiadkawe use distro_python_version in wsgi scripts10:05
mgoddardmnasiadka: host version was in the baremetal role defaults in train. Looks like it was not needed after C7 removal10:05
mgoddarddistro_python_version is container versino10:05
mnasiadkaso then just setting it to 3.8 for focal should work10:06
*** e0ne_ has joined #openstack-kolla10:06
mgoddardmnasiadka: ignore me, was looking at stein. host_python_version still present10:06
oyrogergMini rant: I do want those "This release is under development. The current supported release is <link to version>" would link to current page at version, not to docs.os.org/version/ cf ah-may-zing Postgresql doco.10:07
mgoddardmnasiadka, hrw: we did this in train for CentOS 7/8. Just look at what we did10:07
oyrogergWrong channel I know. But, mini-rant over.10:07
mgoddardoyrogerg: indeed. wrong channel :)10:07
mnasiadkahrw: I think in PS4 everything went fine?10:08
mgoddardhrw, mnasiadka: https://opendev.org/openstack/kolla-ansible/src/branch/stable/train/ansible/group_vars/all.yml#L113210:08
hrwmnasiadka: yep10:09
*** e0ne has quit IRC10:09
mnasiadkamgoddard: it won't help, in upgrade job we are running focal containers on bionic host - taking distro_python_version from bionic host won't help, because it's 3.8 in focal and 3.6 in bionic :)10:09
mgoddardmnasiadka: ah10:09
mnasiadkahrw: so let's revert to PS4, fix the ceph upgrade job, because it was running on focal instead of bionic and we should be good :)10:10
mgoddardso override in config?10:10
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: ubuntu: move to 20.04 Focal  https://review.opendev.org/74350210:12
mnasiadkamgoddard: but why? distro_python_version 3.8 will work fine, at least it did in PS4 - the only jobs that failed were ceph ones - one because of mariadb, second (upgrade) because it was running on focal instead of bionic10:13
mgoddardit will fail on bionic10:14
mgoddardwe discussed this upgrade in last weeks meeting10:14
mgoddardwhether to upgrade on bionic or focal10:14
mnasiadkait will fail on focal, because we don't support focal in ussuri10:14
mgoddarddeja vu10:15
mnasiadkawhy will it fail on bionic?10:15
mgoddardhttp://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-07-29-15.00.log.txt10:15
*** e0ne has joined #openstack-kolla10:16
*** e0ne_ has quit IRC10:16
mgoddardoops, forgot my action item to email ML about this...10:18
mnasiadkamgoddard: well, I see that we want to support bionic in victoria, so the direction is correct to use bionic in the upgrade job10:20
mgoddardmnasiadka: I don't think it's quite decided yet, but that probably makes sense10:21
mgoddardand yes, I suppose given that this is the version in the container, 3.8 should be fine10:21
mnasiadkayeah, distro_python_version variable name is a bit funny :)10:21
jbalciunasmgoddard: any plans for 10.1.1 for Ussuri?11:29
*** kevko has joined #openstack-kolla11:30
*** JamesBenson has joined #openstack-kolla11:40
openstackgerritMichal Arbet proposed openstack/kolla master: Add missing horizon plugins for debian based OS  https://review.opendev.org/74445211:41
*** brinzhang_ has joined #openstack-kolla11:47
*** brinzhang has quit IRC11:49
mnasiadkahmm, wonder why this only happens on debian aarch64 upgrade job: 2020-08-05 10:14:11 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="out_file: `/var/log/kolla/glance-tls-proxy/glance-tls-proxy.20200805.log.gz` is not writable"11:53
*** k_mouza has quit IRC11:54
openstackgerritNikolaos Parasyris proposed openstack/kolla-ansible master: Enable glance role to copy extra configuration  https://review.opendev.org/74357812:04
*** brinzhang0 has joined #openstack-kolla12:21
*** brinzhang_ has quit IRC12:24
openstackgerritMerged openstack/kolla-ansible stable/ussuri: Fix play hosts for ironic, monasca, neutron, nova  https://review.opendev.org/74483712:42
openstackgerritMerged openstack/kolla-ansible stable/train: Fix play hosts for ironic, monasca, neutron, nova  https://review.opendev.org/74483812:44
openstackgerritMerged openstack/kolla-ansible stable/stein: Fix play hosts for ironic, monasca, neutron, nova  https://review.opendev.org/74484012:47
*** JamesBenson has quit IRC12:49
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Fix glance-tls-proxy logrotate  https://review.opendev.org/74490012:51
*** priteau has joined #openstack-kolla13:09
*** wuchunyang has joined #openstack-kolla13:10
*** wuchunyang has quit IRC13:16
*** wuchunyang has joined #openstack-kolla13:16
*** sluna has quit IRC13:18
*** sluna has joined #openstack-kolla13:18
*** wuchunyang has quit IRC13:19
*** wuchunyang has joined #openstack-kolla13:20
*** wuchunyang has quit IRC13:22
*** wuchunyang has joined #openstack-kolla13:22
*** wuchunyang has quit IRC13:23
*** wuchunyang has joined #openstack-kolla13:24
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638913:25
*** wuchunyang has quit IRC13:27
*** wuchunyang has joined #openstack-kolla13:27
openstackgerritBartosz Bezak proposed openstack/kolla master: Install Monasca plugin for Grafana by default  https://review.opendev.org/67618513:31
*** wuchunyang has quit IRC13:33
*** wuchunyang has joined #openstack-kolla13:34
*** wuchunyang has quit IRC13:36
*** wuchunyang has joined #openstack-kolla13:36
*** wuchunyang has quit IRC13:39
*** wuchunyang has joined #openstack-kolla13:40
*** wuchunyang has quit IRC13:42
*** wuchunyang has joined #openstack-kolla13:42
*** k_mouza has joined #openstack-kolla13:46
*** wuchunyang has quit IRC13:47
*** wuchunyang has joined #openstack-kolla13:48
openstackgerritBartosz Bezak proposed openstack/kolla master: Install Monasca plugin for Grafana by default  https://review.opendev.org/67618513:52
*** wuchunyang has quit IRC13:55
*** wuchunyang has joined #openstack-kolla13:55
*** TrevorV has joined #openstack-kolla13:56
*** wuchunyang has quit IRC13:58
*** wuchunyang has joined #openstack-kolla13:58
*** JamesBenson has joined #openstack-kolla13:59
*** k_mouza has quit IRC14:02
*** wuchunyang has quit IRC14:03
*** k_mouza has joined #openstack-kolla14:04
*** wuchunyang has joined #openstack-kolla14:04
mgoddardwuchunyang: you keep dropping your connection14:06
wuchunyangyeah, I don't have a good solution...14:08
*** wuchunyang has quit IRC14:10
*** wuchunyang has joined #openstack-kolla14:10
*** skramaja has quit IRC14:13
*** wuchunyang has quit IRC14:16
*** wuchunyang has joined #openstack-kolla14:17
openstackgerritMerged openstack/kolla-ansible stable/ussuri: Fix actions for Aodh and Swift  https://review.opendev.org/74485214:18
*** wuchunyang has quit IRC14:20
*** wuchunyang has joined #openstack-kolla14:20
*** wuchunyang has quit IRC14:22
*** wuchunyang has joined #openstack-kolla14:23
*** wuchunyang has quit IRC14:25
*** wuchunyang has joined #openstack-kolla14:25
*** JamesBen_ has joined #openstack-kolla14:27
*** JamesBenson has quit IRC14:30
*** wuchunyang has quit IRC14:32
*** wuchunyang has joined #openstack-kolla14:33
*** brinzhang_ has joined #openstack-kolla14:36
*** wuchunyang has quit IRC14:38
*** wuchunyang has joined #openstack-kolla14:38
*** brinzhang0 has quit IRC14:39
*** wuchunyang has quit IRC14:40
*** wuchunyang has joined #openstack-kolla14:41
yoctozeptomgoddard: I wonder if discord would help wuchunyang14:41
mgoddardyoctozepto: why don't you ask?14:41
yoctozeptomgoddard: kinda did that14:45
yoctozeptomgoddard: can rephrase14:45
yoctozeptowuchunyang: would discord work better than irc for you?14:45
*** JamesBenson has joined #openstack-kolla14:46
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/ussuri: [docker] Added a new flag to disable default iptables rules  https://review.opendev.org/74491914:46
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/ussuri: [docker] Added a new flag to disable default iptables rules  https://review.opendev.org/74491914:46
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: [docker] Added a new flag to disable default iptables rules  https://review.opendev.org/74492014:46
wuchunyangirc works for me ,, but always disconnect ..14:47
*** JamesBen_ has quit IRC14:48
wuchunyangi have never  used discord before...14:49
mgoddardmgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak14:51
mgoddardmeeting in 10 ^14:51
yoctozeptomgoddard: :-)14:51
yoctozeptowuchunyang: understood, thanks for feedback14:51
wuchunyangit‘s noting14:53
*** wuchunyang has quit IRC14:55
*** bengates has quit IRC14:55
*** wuchunyang has joined #openstack-kolla14:55
*** bengates has joined #openstack-kolla14:55
*** JamesBenson has quit IRC14:56
*** wuchunyang has quit IRC14:58
*** wuchunyang has joined #openstack-kolla14:59
*** bengates has quit IRC15:00
*** diurnalist has joined #openstack-kolla15:01
mgoddard#startmeeting kolla15:01
openstackMeeting started Wed Aug  5 15:01:52 2020 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
*** openstack changes topic to " (Meeting topic: kolla)"15:01
openstackThe meeting name has been set to 'kolla'15:01
mgoddard#topic rollcall15:01
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:01
mgoddard\o15:02
priteauo/15:02
yoctozeptoo/15:02
mnasiadkao/15:04
mgoddard#topic agenda15:04
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:04
mgoddard* Roll-call15:04
mgoddard* Announcements15:04
mgoddard** Kolla kall tomorrow15:04
mgoddard* Review action items from last meeting15:04
mgoddard* CI status15:04
mgoddard* Victoria release planning (kayobe)15:04
mgoddard* Victoria release planning (kolla & kolla ansible)15:04
mgoddard* Kolla klub and kall15:04
mgoddard#topic announcements15:04
*** openstack changes topic to "announcements (Meeting topic: kolla)"15:04
mgoddard#info Kolla kall tomorrow15:04
mgoddard#info mgoddard talked about Kayobe & Kolla: https://youtu.be/0liqSO0SZ60?t=484215:06
mgoddardvirtual open infra meetup london/manchester15:06
mgoddardAny others?15:06
mgoddard#topic Review action items from last meeting15:07
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)"15:07
mgoddardmgoddard to message openstack-discuss about focal & victoria upgrade15:07
mgoddardI did not15:07
mgoddardI was going to earlier, but thought I'd wait until after we discuss it today15:07
mgoddard#topic CI status15:07
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:07
bbezako/15:07
mgoddardwe have a few issue15:07
mgoddardKolla15:08
mgoddardTrain: gitdb issue https://review.opendev.org/#/c/74450115:08
patchbotpatch 744501 - kolla (stable/train) - requirements.txt: pin gitdb(2) versions - 2 patch sets15:08
mgoddardStein: ironic-conductor broken: https://27734da6eed83e608576-333f8b549856196606ae2476154ba69d.ssl.cf1.rackcdn.com/743923/1/gate/kolla-build-centos-source/10d084d/kolla/build/000_FAILED_ironic-conductor.log15:08
mgoddardhrw: around?15:08
mgoddardI guess not15:10
mgoddardmnasiadka: did you look into that ironic issue any further?15:11
mnasiadkamgoddard: nope, can do tomorrow15:11
mgoddardok15:11
*** bengates has joined #openstack-kolla15:12
mgoddard#action mnasiadka to look into kolla stein ironic-conductor failure15:12
mgoddardthanks15:12
mgoddardotherwise I think we are mostly ok15:12
mgoddard#topic Victoria release planning (kayobe)15:13
*** openstack changes topic to "Victoria release planning (kayobe) (Meeting topic: kolla)"15:13
mgoddardLooks like it's time to give mnasiadka access to kayobe +215:14
yoctozeptocongrats mnasiadka15:14
mnasiadkaYay15:14
yoctozepto:D15:14
mgoddardadded15:15
priteauJust in time for his holidays :)15:16
mgoddardpriteau, mnasiadka, dougsz: anything to say about kayobe this week?15:16
dougszNothing from me this week15:16
priteauNot really. I am slowly emerging from work on another project, hopefully more time for upstream in August15:17
mgoddardRelease still too far away to be busy :)15:17
mgoddard(on kayobe)15:17
*** bengates has quit IRC15:17
mgoddardI have some performance improvements in the works15:17
mgoddardfirst some easy wins15:17
mgoddardthen some less easy wins15:18
mgoddardbut hopefully all wins15:18
*** JamesBenson has joined #openstack-kolla15:18
mgoddardOne thing we do need to do this cycle is the configuration walkthrough in docs15:18
mgoddardis it something we should collaborate on, or just have one person do it?15:19
mgoddardtough crowd15:20
mgoddardLet's move on15:20
mgoddard#topic Victoria release planning (kolla & kolla ansible)15:20
*** openstack changes topic to "Victoria release planning (kolla & kolla ansible) (Meeting topic: kolla)"15:20
priteauSorry, I was in another conversation15:21
mgoddardpriteau: no problem15:21
priteauAbout previous topic, I guess if one person could do a draft to start with15:21
priteauThen we can review and improve collaboratively15:22
mgoddardmakes sense15:22
yoctozeptopriteau: am I seeing a volunteer? :D15:22
priteauIf I have the time15:23
*** dcapone2004 has joined #openstack-kolla15:23
mgoddardok, back to kolla15:24
mgoddardwas hoping hrw would be here to discuss focal15:24
yoctozeptoperf improvements going nicely thanks to mgoddard (and me a bit)15:24
mgoddardmnasiadka: I think you touched it recently. What's the status?15:24
yoctozeptohttps://review.opendev.org/#/q/project:openstack/kolla-ansible+branch:master+topic:bp/performance-improvements15:25
dcapone2004is it meeting time?  I lost power and just got it back, so I missed any replies to my questions from yesterday and any activity for that matter...15:25
yoctozeptodcapone2004: meeting running15:25
yoctozeptoalso a poc fix for the nasty restart-too-much bug: https://review.opendev.org/74420715:25
patchbotpatch 744207 - kolla-ansible - [WIP] Fix looped notifies and optimize further - 4 patch sets15:25
mgoddarddcapone2004: http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/15:26
yoctozeptoit has some impact on perf, makes some things better, some possibly worse15:26
mnasiadkamgoddard: k-a change looks green15:26
priteaudcapone2004: http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2020-08-04.log.html#t2020-08-04T22:58:1815:26
mnasiadkamgoddard: I need to analyse services logs and other stuff, but should be at least semi-okay ;)15:26
*** wuchunyang has quit IRC15:27
mgoddardmnasiadka: ok, guess we can push ahead with the kolla-ansible part?15:30
mnasiadkamgoddard: yeah, seems like it15:31
*** wuchunyang has joined #openstack-kolla15:32
mnasiadkamgoddard: well, actually it has depends-on to kolla change15:32
mgoddardmnasiadka: I think hrw wanted to reverse that15:33
mgoddardalthough it wouldn't work due to distro_python_version15:33
mgoddardI suppose we have a cyclic dependency15:33
mnasiadkamakes sense, that I just noticed that - needs some workaround - but I'll look into it15:33
mnasiadkamgoddard: we might need to move to python3 instead of python{{ distro_python_version }} in wsgi's at least15:34
mgoddardmnasiadka: I seem to remember the stuff in the wsgi config isn't actually used15:35
mgoddardso we might be able to remove it15:35
*** nikparasyr has left #openstack-kolla15:36
mnasiadkamgoddard: well, keystone fernet also fails on python{{ distro_python_version }}15:36
mnasiadkaThis one is also easy to remove - either way I'll follow up15:36
mgoddard#action mgoddard to message openstack-discuss about focal & victoria upgrade15:36
mgoddardstill need to do that I guess15:36
mgoddardok15:37
mgoddardyoctozepto: so this handlers change15:38
mgoddardwe have a few things to consider15:38
mgoddardbackporting15:38
mgoddardperformance15:38
mgoddardhaving spent a lot of time improving performance recently I don't really want to go backwards15:38
yoctozeptoI am inclined to merge the two tasks into one15:39
yoctozeptothe filtered loops seems to be improving perf a lot15:39
*** GeraldS has quit IRC15:39
yoctozeptothe only hit left would be the include_ indirection15:39
yoctozeptobut ansible kinda forces that on us15:39
mgoddardit makes a small difference at scale15:39
yoctozeptothe alternative is actually less performant15:39
yoctozeptodue to no loop filtering15:39
mgoddardtasks hurt performance far more than skips15:40
yoctozeptoso wdyt about merging tasks (to avoid skip and extra task) and pursuing this approach for all?15:40
mgoddardand the include method adds multiple tasks15:40
mgoddardplus the overhead of the include task15:41
mgoddardit's probably more overhead than I have saved in total so far :)15:42
yoctozeptomgoddard: for neutron the new approach is more performant at my scale15:42
mgoddardwhat is your scale?15:43
yoctozepto20-3015:43
yoctozeptothis is because the filtering helped A LOT15:43
yoctozeptoas neutron has many undeployed services15:43
yoctozeptoso the total amount of tasks went down15:43
mgoddardit will help with an include for sure15:43
yoctozeptobut it is a win for neutron15:43
yoctozeptonot every service out there15:43
yoctozeptobut most maybe hit less than you are afraid15:44
yoctozeptoI agree to merge the tasks15:44
yoctozeptoone is just perfect for that15:44
yoctozeptoalso easier to coordinate15:44
yoctozeptobut the include_ indirection must stay imho15:44
yoctozeptoas I see no other clean-enough approach15:44
yoctozeptoI wish ansible just fixed that handlers logic lol15:44
yoctozeptoI mean notifies* logic15:45
yoctozeptobut whatever15:45
mgoddard+1 for merging the tasks15:46
yoctozeptomgoddard: could you review https://review.opendev.org/743961 as it's prereq for that fix15:46
patchbotpatch 743961 - kolla-ansible - Performance: optimize genconfig - 4 patch sets15:46
yoctozeptowe would backport it like the other perf impv15:46
mgoddardinclude vs not needs further thought15:46
yoctozeptotrue that, but I can't imagine a solution without getting ansible fixed15:46
yoctozeptomight as well employ charms :-)15:47
mgoddardmakes sense15:47
yoctozepto:D15:47
mgoddardthe other solution is handlers that might do nothing. It's not clean, but if it's faster we should go with it15:48
mgoddardI'll propose it15:49
mgoddard#topic Kolla klub and kall15:49
*** openstack changes topic to "Kolla klub and kall (Meeting topic: kolla)"15:49
mgoddardWe have a kall tomorrow15:49
yoctozeptook, but you are getting the penalty of new horde of skipped tasks instead of filtered includes :-)15:49
mgoddardContinuing the docs discussion15:49
yoctozeptothis is why I'm not imagining other solutions15:49
yoctozeptook, kall15:49
yoctozeptoyes, we started exploring kayobe last time15:50
mgoddardI missed that part15:50
yoctozeptoI would like someone to chair that15:50
mgoddardwill be there tomorrow15:50
yoctozeptowe wrote it down for you15:50
mgoddardI can do it15:50
yoctozeptoor, well, I did15:50
mgoddardok15:50
yoctozeptothanks, happy to have kayobe head to chair kayobe discussion :-)15:50
yoctozeptowe'll try meetpad tomorrow again15:51
mgoddardre kolla klub, I was wondering if we should break for the summer?15:51
yoctozeptobut on problems switch to google meeting15:51
mgoddardattendance has been lower recently, maybe we've reached peak klub and need a break15:51
yoctozeptore kolla klub - agreed, the times are weird due to the pandemic but people want to go holidaying15:52
mgoddardwas also thinking we should try another time for people in APAC, if there is interest15:52
mgoddardcould potentially run the same onboarding & update sessions with a different group15:52
yoctozepto++15:54
*** dave-mccowan has joined #openstack-kolla15:54
mgoddard#topic Open discussion15:55
*** openstack changes topic to "Open discussion (Meeting topic: kolla)"15:55
mgoddardanyone have anything?15:55
yoctozeptonot me15:55
*** e0ne has quit IRC15:56
headphoneJamesDoes the Nova API wsgi txn need any additional changes?15:57
mgoddardheadphoneJames: good question. Haven't looked lately15:57
headphoneJameshttps://review.opendev.org/72479415:57
patchbotpatch 724794 - kolla-ansible - Add support for encrypting Nova API - 29 patch sets15:57
*** wuchunyang has quit IRC15:59
*** wuchunyang has joined #openstack-kolla15:59
mgoddardwill try to look this week.16:01
mgoddardthanks all16:01
mgoddard#endmeeting16:01
*** openstack changes topic to "Vote on Victoria priorities https://bit.ly/2ActPtx | IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"16:01
openstackMeeting ended Wed Aug  5 16:01:09 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-08-05-15.01.html16:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-08-05-15.01.txt16:01
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-08-05-15.01.log.html16:01
*** wuchunyang has quit IRC16:02
*** wuchunyang has joined #openstack-kolla16:03
*** wuchunyang has quit IRC16:05
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Get distro_python_version from kolla_toolbox  https://review.opendev.org/74493816:05
*** wuchunyang has joined #openstack-kolla16:05
*** wuchunyang has quit IRC16:07
*** pbing19 has quit IRC16:07
*** wuchunyang has joined #openstack-kolla16:08
*** wuchunyang has quit IRC16:09
*** wuchunyang has joined #openstack-kolla16:09
*** Reepicheep has quit IRC16:10
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Fix glance-tls-proxy logrotate  https://review.opendev.org/74490016:10
*** JamesBenson has quit IRC16:10
*** wuchunyang has quit IRC16:11
*** Reepicheep has joined #openstack-kolla16:11
*** diurnalist has quit IRC16:12
*** wuchunyang has joined #openstack-kolla16:12
hrwsorry about missing meeting16:13
hrwfocal changes? I sent them to gerrit and hope that someone will be able to get it sorted16:14
*** wuchunyang has quit IRC16:14
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638916:14
*** JamesBenson has joined #openstack-kolla16:15
*** pbing19 has joined #openstack-kolla16:17
*** diurnalist has joined #openstack-kolla16:17
*** dcapone2004 has quit IRC16:23
*** muhaha has joined #openstack-kolla16:24
muhahaguys ? is podman supported in kolla install  ?16:24
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Fix looped notifies mk2  https://review.opendev.org/74494416:26
*** dcapone2004 has joined #openstack-kolla16:26
*** dave-mccowan has quit IRC16:26
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Fix looped notifies mk2  https://review.opendev.org/74494416:27
hrwmuhaha: no. docker only16:27
hrwmuhaha: we would welcome patch adding podman or even moving us to podman16:28
hrwbut no one has time to write such16:28
*** dougsz has quit IRC16:30
muhaha@hrk ah, ok, thanks :)16:31
*** pbing19 has quit IRC16:31
*** gbatir_ has quit IRC16:32
openstackgerritMerged openstack/kolla-ansible stable/train: Fix actions for Aodh and Swift  https://review.opendev.org/74485316:32
openstackgerritMerged openstack/kolla-ansible master: Update TLS documentation  https://review.opendev.org/73914416:32
*** brinzhang0 has joined #openstack-kolla16:37
dcapone2004ok, so I switched to Ussuri...and my reliance/switch to using KA for deployment has me a bit behind on changes to openstack deployment and manual configuration files, but I have run into 2 issues.... 1. the ceph.client.nova.keyring file, didn't copy into the nova_compute container despite being in the /etc/kolla/config/nova directory (cinder one16:40
dcapone2004was in same directory and copied fine) and 2, more importantly because I haven't fixed yet, I do not have external network connectivity16:40
*** brinzhang_ has quit IRC16:40
dcapone2004in looking at the neutron.conf files and ml2_conf.ini files, I have noticed that there is no bridge mapping section in ml2_conf.ini [ovs] bridge_mappings = physnet1:br-ex .... in trying to find openstack documentation, it seems like this configuration directive might have been removed or moved....is their a new mechanism for associating physnet16:42
dcapone2004names to bridge names in Ussuri?16:42
*** gfidente is now known as gfidente|afk16:44
priteaudcapone2004: For your first issue, do you have nova_backend set to rbd16:58
priteaudcapone2004: for the bridge_mappings issue, take a look at the template and what variables it expects16:59
priteauansible/roles/neutron/templates/openvswitch_agent.ini.j216:59
dcapone2004priteau: think I just figured out my second issue....found an article where the mapping moved and it is set correctly, so think it is a physical networking issue....let me check on globals.yml file17:01
openstackgerritMerged openstack/kolla master: Add missing horizon plugins for debian based OS  https://review.opendev.org/74445217:01
dcapone2004priteau:  I dont see that option in globals.yml,  I have nova_backend_ceph: "yes"17:03
priteauThat does the same thing17:03
dcapone2004ok, so yes it is set to yes17:03
priteauAh. By default ceph_nova_keyring is the same as ceph_cinder_keyring17:04
dcapone2004also have, rbd_user=nova in nova-compute.conf in [libvirt] section in the same /etc/kolla/config/nova dirrectory17:04
priteauIf you want to use a different key, set ceph_nova_keyring to ceph.client.nova.keyring in globals17:04
dcapone2004ah, that must be a change from train....because I thought kolla docs said that it copies ALL keyring files in the respective /etc/kolla/config directories...17:05
dcapone2004but i see it now in the globals.yml17:05
dcapone2004I found it odd that the default "USER" is different but the keyrings are the same17:06
dcapone2004considering the keyring files designate a user don't they?17:07
priteauYeah it's odd, like a mix of two separate configuration17:08
dcapone2004yeah, I'm not an expert but would that default config even work?  Can keyring files maybe contain the keys of multiple users?17:09
*** e0ne has joined #openstack-kolla17:09
*** brinzhang_ has joined #openstack-kolla17:26
*** brinzhang0 has quit IRC17:28
*** brinzhang0 has joined #openstack-kolla17:35
*** brinzhang has joined #openstack-kolla17:37
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Fix glance-tls-proxy logrotate  https://review.opendev.org/74490017:37
*** priteau has quit IRC17:38
*** brinzhang_ has quit IRC17:39
*** brinzhang0 has quit IRC17:39
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Get distro_python_version from kolla_toolbox  https://review.opendev.org/74493817:41
*** k_mouza has quit IRC17:43
*** kevko has quit IRC17:44
*** dciabrin has quit IRC17:57
*** dciabrin has joined #openstack-kolla17:58
*** jonaspaulo has joined #openstack-kolla17:58
*** e0ne has quit IRC18:10
*** k_mouza has joined #openstack-kolla18:13
*** k_mouza has quit IRC18:16
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Get distro_python_version from kolla_toolbox  https://review.opendev.org/74493818:20
*** e0ne has joined #openstack-kolla18:26
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Get distro_python_version from kolla_toolbox  https://review.opendev.org/74493818:28
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Get distro_python_version from kolla_toolbox  https://review.opendev.org/74493818:30
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Fix looped notifies mk2  https://review.opendev.org/74494418:35
*** brinzhang_ has joined #openstack-kolla18:37
*** dave-mccowan has joined #openstack-kolla18:39
*** brinzhang has quit IRC18:40
*** dcapone2004 has quit IRC18:50
*** e0ne has quit IRC18:51
*** patchbot has quit IRC18:54
mgoddardyoctozepto: https://github.com/stackhpc/ansible-scaling/commit/d5b05413e0f1a1aae742d8023508ca4c2bcf5ec019:33
mgoddardyoctozepto: for 100 hosts, 1m without includes, 1m30 with includes19:35
mgoddardyoctozepto: for 10 hosts, 7s without, 9s with19:35
*** brinzhang0 has joined #openstack-kolla19:36
*** jonaspaulo has quit IRC19:39
*** brinzhang_ has quit IRC19:39
*** Fl1nt has quit IRC19:41
*** dave-mccowan has quit IRC19:45
openstackgerritPaul Leimer proposed openstack/kolla master: Add collectd-libpod-stats to image  https://review.opendev.org/74498120:20
*** vishalmanchanda has quit IRC20:25
*** muhaha has quit IRC20:55
*** ladrua has joined #openstack-kolla20:55
*** brinzhang_ has joined #openstack-kolla21:36
*** brinzhang0 has quit IRC21:39
*** markmcclain has quit IRC21:53
*** TrevorV has quit IRC21:57
openstackgerritJames Kirsch proposed openstack/kolla master: Refactor httpd install to base image  https://review.opendev.org/74403722:06
openstackgerritJames Kirsch proposed openstack/kolla master: Refactor httpd install to base image  https://review.opendev.org/74403722:29
*** jbadiapa has joined #openstack-kolla22:49
*** jbadiapa has quit IRC22:50
*** born2bake has quit IRC23:01
*** markmcclain has joined #openstack-kolla23:12
*** diurnalist has quit IRC23:24
*** irclogbot_1 has quit IRC23:34
*** irclogbot_3 has joined #openstack-kolla23:37
*** yoctozepto3 has joined #openstack-kolla23:44
*** yoctozepto has quit IRC23:45
*** yoctozepto3 is now known as yoctozepto23:45

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!