Wednesday, 2020-07-08

*** born2bake has quit IRC00:21
*** stokvis has quit IRC00:22
*** zhanglong has joined #openstack-kolla00:26
*** zhanglong has quit IRC01:12
*** zhanglong has joined #openstack-kolla01:13
*** ricolin has joined #openstack-kolla01:40
*** wuchunyang has joined #openstack-kolla02:04
*** antonym has quit IRC02:15
*** antonym has joined #openstack-kolla02:31
*** dave-mccowan has quit IRC02:41
*** dave-mccowan has joined #openstack-kolla02:50
*** Manheim has joined #openstack-kolla02:55
*** Manheim has quit IRC02:59
*** zhanglong has quit IRC03:16
*** zhanglong has joined #openstack-kolla03:16
*** zhanglong has quit IRC03:27
*** zhanglong has joined #openstack-kolla03:28
*** wuchunyang has quit IRC03:31
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Manage octavia health manager worker through openstack_service worker  https://review.opendev.org/73988203:36
*** hjensas has quit IRC03:42
*** hjensas has joined #openstack-kolla03:43
*** skramaja has joined #openstack-kolla04:01
*** zhanglong has quit IRC04:09
*** zhanglong has joined #openstack-kolla04:10
*** ykarel|away is now known as ykarel04:14
*** diurnalist has quit IRC04:19
*** dave-mccowan has quit IRC04:19
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-kolla04:33
*** wuchunyang has joined #openstack-kolla04:33
*** abdysn has joined #openstack-kolla05:05
*** iniazi_ has joined #openstack-kolla05:22
*** iniazi has quit IRC05:24
*** diurnalist has joined #openstack-kolla05:32
*** wuchunyang has quit IRC05:35
*** wuchunyang has joined #openstack-kolla05:36
*** diurnalist has quit IRC06:04
*** Manheim has joined #openstack-kolla06:15
*** cah_link has joined #openstack-kolla06:16
*** zijlboot has joined #openstack-kolla06:17
*** Manheim has quit IRC06:19
*** wuchunyang has quit IRC06:21
*** xinliang has joined #openstack-kolla06:36
*** Manheim has joined #openstack-kolla06:37
*** ianw is now known as ianw_pto06:38
*** wuchunyang has joined #openstack-kolla06:39
*** xinliang has quit IRC06:42
*** Manheim has quit IRC06:50
*** Manheim has joined #openstack-kolla06:50
*** dskontra has joined #openstack-kolla06:54
*** Manheim has quit IRC06:55
*** luksky has joined #openstack-kolla07:00
*** wuchunyang has quit IRC07:05
*** jbadiapa has joined #openstack-kolla07:16
*** nikparasyr has joined #openstack-kolla07:17
*** wuchunyang has joined #openstack-kolla07:17
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Update ceph-ansible to v5 - deploy Ceph Octopus  https://review.opendev.org/73790307:20
*** bengates has joined #openstack-kolla07:26
*** bengates has quit IRC07:26
*** bengates has joined #openstack-kolla07:26
*** mnasiadka has joined #openstack-kolla07:27
*** gfidente has joined #openstack-kolla07:29
*** bbezak has joined #openstack-kolla07:30
*** also_stingrayza is now known as stingrayza07:35
*** xinliang has joined #openstack-kolla07:36
*** Manheim has joined #openstack-kolla07:44
*** hjensas has quit IRC07:49
*** dougsz has joined #openstack-kolla07:51
mgoddardmorning07:54
mnasiadkamorning07:56
Manheimmorning07:59
*** zhanglong has quit IRC08:01
openstackgerritBartosz Bezak proposed openstack/kolla-ansible master: Logstash 6 support  https://review.opendev.org/73975708:01
openstackgerritPierre Riteau proposed openstack/kolla-ansible stable/ussuri: Fix incorrect value of [storage]/ceph_keyring in gnocchi.conf  https://review.opendev.org/73991708:11
*** jaicaa_ has quit IRC08:14
*** zhanglong has joined #openstack-kolla08:14
*** Manheim_ has joined #openstack-kolla08:15
*** zhanglong has quit IRC08:19
*** Manheim has quit IRC08:19
*** zhanglong has joined #openstack-kolla08:21
Manheim_Is anyone using Monasca with Ceilometer? Or only Monasca08:25
dougszManheim_ we do08:26
*** k_mouza has joined #openstack-kolla08:30
*** bsanjeewa has joined #openstack-kolla08:30
Manheim_what do you do with the metrics afterwards, push them fo influxDB? I am interested in exposing them to Grafana in the end08:31
Manheim_since Gnocchi is not working well with Grafana, I am looking into Monasca08:32
*** zhanglong has quit IRC08:33
openstackgerritliumengke proposed openstack/kolla-ansible master: fix typo for om_rpc_transport default value.  https://review.opendev.org/73992508:34
*** bsanjeewa has quit IRC08:43
hrwmorning08:53
*** zhanglong has joined #openstack-kolla08:55
*** wuchunyang has quit IRC08:56
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638908:56
*** diurnalist has joined #openstack-kolla09:00
*** jaicaa has joined #openstack-kolla09:00
hrwwho uses fluentd?09:00
dougszhrw: everyone09:01
*** hjensas has joined #openstack-kolla09:01
hrwok, next questin: who will review and help fixing update to td-agent4?09:01
hrwI can prepare patch09:01
dougsz Manheim_: We have a blog post on it here which might be useful: https://www.stackhpc.com/cloudkitty-and-monasca-1.html09:02
*** luksky has quit IRC09:03
dougszhrw: Remind me, why are we using td-agent4?09:03
dougszlatest stable Fluentd uses 3.7.1?09:04
Manheim_dougsz: thanks09:04
hrwdougsz: or rather: why are we using td-agent at all09:04
dougszProbably because that's what they link to here: https://www.fluentd.org/download09:04
hrwdougsz: treasuredata finally released 4.0.0 version09:04
*** diurnalist has quit IRC09:04
hrwubuntu/x86-64 images use td-agent3 and there is no td-agent3 for focal09:05
*** zhanglong has quit IRC09:05
*** wuchunyang has joined #openstack-kolla09:05
dougszI see09:05
*** zhanglong has joined #openstack-kolla09:06
hrwmove to td4 would require changed to fluentd image etc09:07
dougszI just hope the config is backwards compatible..09:07
mnasiadkahrw: you want to build td-agent by yourself? I haven't seen td4 packages anywhere apart "testing-uploading-artifacts-3" :D09:11
hrwmnasiadka: they are in repos now09:11
mnasiadkaah I see09:12
mnasiadkaso let's bump it everywhere09:12
mnasiadkaand wait for the world to collapse09:12
hrwish09:12
mnasiadkaI see centos 8 x86/aarch64, Ubuntu Focal and Debian Buster, at least all x86 packages are built.09:13
mnasiadka(all that we need)09:13
hrwaarch64 too09:13
hrwfluentd Dockerfile.j2 is a mess09:13
mnasiadkayeah, it needs some major cleanup09:13
*** born2bake has joined #openstack-kolla09:13
mnasiadkahrw: I can look into td4 and cleanup of the Dockerfile - unless you prefer to do it by yourself ;)09:14
hrwmnasiadka: I will send simple version and then you continue, ok?09:14
mnasiadkahrw: sure09:14
*** wuchunyang has quit IRC09:16
*** luksky has joined #openstack-kolla09:16
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638909:17
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: fluentd: migrate to td-agent 4 where possible  https://review.opendev.org/73993509:18
hrwhere it is09:18
hrwmnasiadka: left comments09:21
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: fluentd: migrate to td-agent 4 where possible  https://review.opendev.org/73993509:24
hrwfixed syntax errors09:24
openstackgerritMerged openstack/kolla stable/train: Drop systemd support from nsswitch.conf on RHEL-based distros  https://review.opendev.org/73950809:26
openstackgerritMerged openstack/kolla stable/ussuri: Drop systemd support from nsswitch.conf on RHEL-based distros  https://review.opendev.org/73950709:26
openstackgerritMerged openstack/kolla stable/stein: Drop systemd support from nsswitch.conf on RHEL-based distros  https://review.opendev.org/73950909:26
mnasiadkahrw: replied on some of those comments, so we have some background :)09:27
hrwthx09:29
hrwmnasiadka: I think that we can go with td4 on all distros on x86-64/aarch64 and make it simple.09:30
mnasiadkathat would be a great plan09:30
hrwmnasiadka: if someone join to get ppc64le/s390x/riscv/whatever then they can restore from git and work09:30
mnasiadkaespecially whatever :)09:31
hrwmnasiadka: openstack goes to m68k!09:32
mnasiadkahrw: OpenStack on Atari ST?09:33
hrwmnasiadka: and amiga 50009:33
hrwI remember using 'dmesg' command under amigaos to check why my Linux kernels failed to boot on my a1200.09:36
hrwno memory protection so command was going through whole memory to find kernel log buffers09:36
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: fluentd: migrate to td-agent 4 where possible  https://review.opendev.org/73993509:39
hrwenabled centos/aarch6409:39
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ubuntu: move to 20.04 Focal  https://review.opendev.org/73899409:46
hrwfocal has python 3.8 not 3.609:46
*** bsanjeewa has joined #openstack-kolla09:46
*** dskontra has quit IRC09:54
mnasiadkahrw: infra has focal images to use in Zuul jobs?10:03
hrwyes10:04
hrwI added focal to infra month ago10:04
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Evaluate PASSWORDS_FILE later  https://review.opendev.org/73994310:10
openstackgerritMark Goddard proposed openstack/kolla-ansible master: [docker] Added a new flag to disable default iptables rules  https://review.opendev.org/69205210:10
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Load br_netfilter module in nova-cell role  https://review.opendev.org/73994410:10
*** e0ne has joined #openstack-kolla10:12
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Load br_netfilter module in nova-cell role  https://review.opendev.org/73994410:14
openstackgerritMark Goddard proposed openstack/kolla-ansible master: [docker] Added a new flag to disable default iptables rules  https://review.opendev.org/69205210:14
*** zhanglong has quit IRC10:14
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ubuntu: move to 20.04 Focal  https://review.opendev.org/73899410:15
mnasiadkahrw: I see that now aarch64 CI fails on docker hub connection - so it wasn't long in a good state :)10:16
hrwmeh10:16
*** brinzhang_ has quit IRC10:16
*** Manheim__ has joined #openstack-kolla10:18
Manheim__oh, I see monasca is deploying it's own grafana server, so I don't have to enable grafana, no?10:19
dougszManheim__: Yeah, that is correct. Monasca deploys a fork (old) with Keystone integration10:19
Manheim__pre 6.7 grafana I guess10:20
dougszyeah, but you can still use that, you just need to manually insert a keystone token into the datasource and manage the Grafana accounts outside of Keystone10:20
dougszSo the integration is not as nice, but you can use a more recent Grafana..10:21
Manheim__so I can use the latest Grafana if I edit the datasource fine in grafana?10:21
Manheim__only with token10:22
*** Manheim_ has quit IRC10:22
dougszyeah, that's correct10:23
dougszyou can run both as well - they use different ports10:23
*** zhanglong has joined #openstack-kolla10:23
Manheim__yeah, 3001 (monasca grafana) and 300010:23
Manheim__but when I used grafana past 6.7 i would not work with keystone no matter how I tried to connect them10:23
Manheim__token or no token10:24
Manheim__it would not*10:24
dougszYou've tried?10:24
Manheim__yes10:24
Manheim__token, no token, browser mode, server mode10:24
Manheim__but I can retry it, maybe it did not cover all cases then10:25
dougszHmm, maybe the datasource is broken, any grafana logs?10:25
dougszI forget exactly which version of Vanilla Grafana I tried with the token method10:25
Manheim__the grafana gnocchi plugin is broken past 6.710:26
Manheim__the plugin ppl confirmed it10:26
Manheim__it hangs on keystone integration10:26
Manheim__so I guess that's why monasca grafana deployed a fork of grafana right?10:27
dougszNot sure about the gnocchi plugin, never used it (not maintained any longer)10:27
*** zhanglong has quit IRC10:27
Manheim__sorry, I got confused, you are right10:28
Manheim__ I will deploy both versions of grafana, thanks for the info so far10:28
Manheim__will let you know hot it works10:28
dougsznp - thanks! Monasca is working on upgrading Grafana support, but it will be a while10:28
Manheim__well, as long as it is maintained and ppl are working on the project this is good!10:29
*** zhanglong has joined #openstack-kolla10:34
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ubuntu: move to 20.04 Focal  https://review.opendev.org/73899410:37
hrwtestsuite10:37
*** bhagyashris is now known as bhagyashris|brb10:42
*** wuchunyang has joined #openstack-kolla10:42
*** zhanglong has quit IRC10:44
*** zhanglong has joined #openstack-kolla10:45
*** wuchunyang has quit IRC10:47
*** xinliang has quit IRC10:54
hrwhttps://review.opendev.org/#/c/739446/3 opendaylight abomination waits for second core10:59
patchbotpatch 739446 - kolla - Remove opendaylight image - 3 patch sets10:59
*** bhagyashris|brb is now known as bhagyashris10:59
*** bsanjeewa_ has joined #openstack-kolla11:11
*** bsanjeewa has quit IRC11:12
*** zhanglong has quit IRC11:29
Manheim__dougsz: another question, if I also enable ceilometer, it will automatically push metrics to monasca? or do I have to do anythig?11:39
mgoddardHi. I should be able to make the start of the meeting, but if not please continue without me and I will join when available11:39
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: novajoin: install 'freeipa-client' on both Debian and Ubuntu  https://review.opendev.org/73996011:51
*** zijlboot has quit IRC11:59
*** mark_ has joined #openstack-kolla11:59
yoctozeptomgoddard: ok12:03
yoctozeptomgoddard: since you wrote the octavia docs - a question for you - are the amphora images only debuntu? (hence debootstrap always required)12:03
yoctozeptoWu seems to have misunderstood my question there12:04
dougszdougsz: I don't think it is automatic currently12:08
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638912:09
*** zijlboot has joined #openstack-kolla12:11
yoctozeptodougsz: self-replies are the best :-)12:15
dougszlol :) Manheim__ ^12:15
Manheim__:)12:15
*** wuchunyang has joined #openstack-kolla12:16
wuchunyanghttps://blueprints.launchpad.net/kolla-ansible/+spec/implement-automatic-deploy-of-octavia12:17
wuchunyanghi, i have created this bp . could any help review ? i will implement it next.12:18
hrwmay I alter description a bit?12:19
wuchunyangsure12:20
wuchunyangthanks hrw12:20
hrwa bit of formatting and rephrase12:22
hrwwuchunyang: make a patch, add proper flag to it and submit to gerrit. this will join it with bp12:23
*** zhanglong has joined #openstack-kolla12:24
wuchunyangwhat is this patch need to do ?12:25
hrw"Implements: blueprint implement-automatic-deploy-of-octavia" in commit message12:26
wuchunyangok, thanks , i will work on it  next12:27
*** wuchunyang has quit IRC12:32
*** mnasiadka has quit IRC12:33
*** luksky has quit IRC12:36
openstackgerritMerged openstack/kolla-ansible master: Fix incorrect value of [storage]/ceph_keyring in gnocchi.conf  https://review.opendev.org/73984312:40
openstackgerritMerged openstack/kolla-ansible master: [doc] Add missing required packages  https://review.opendev.org/73980812:40
openstackgerritMerged openstack/kolla master: [doc] Update example in release management  https://review.opendev.org/73922612:45
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638912:52
*** mnasiadka has joined #openstack-kolla12:53
*** Manheim has joined #openstack-kolla13:01
*** Manheim__ has quit IRC13:05
*** dougsz has quit IRC13:17
*** skramaja has quit IRC13:26
*** TrevorV has joined #openstack-kolla13:28
*** dougsz has joined #openstack-kolla13:31
mark_Hi Folks.13:39
mark_I got the following error during the installation of Openstack USSURI in CentOS 8 through Kolla-Ansible: https://pastebin.com/yLcYuckk .13:39
mark_Hope to have your support13:39
yoctozeptomark_: looks like it passed an empty string to ceilometer-upgrade for some reason13:41
mark_I followed this guide for installation: https://docs.openstack.org/project-deploy-guide/kolla-ansible/ussuri/quickstart.html13:43
mark_should I just destroy and deploy again?13:43
yoctozeptomark_: nah, I suspect ceilometer-upgrade got stricter about its argument passing13:44
yoctozeptoneed to fix images to pass13:45
yoctozeptoeh, lack of ceilometer testing in CI bites again13:45
yoctozeptocould you write a bug report on kolla?13:47
mark_ok, appreciate your support ...13:47
yoctozeptohere https://bugs.launchpad.net/kolla13:48
yoctozeptoit looks like the regression was introduced here https://review.opendev.org/71819013:50
patchbotpatch 718190 - kolla - Allow operators to use "ceilometer-upgrade" parame... (MERGED) - 4 patch sets13:50
openstackgerritRadosław Piliszek proposed openstack/kolla master: [WIP] Fix ceilometer-upgrade params  https://review.opendev.org/73999213:56
yoctozeptomark_: here's the likely patch ^ https://review.opendev.org/73999213:57
patchbotpatch 739992 - kolla - [WIP] Fix ceilometer-upgrade params - 1 patch set13:57
yoctozeptoit needs to be applied on kolla and the image needs to be rebuilts13:57
*** zijlboot has quit IRC13:58
*** zijlboot has joined #openstack-kolla14:02
*** zijlboot has quit IRC14:03
*** zijlboot has joined #openstack-kolla14:03
*** mark_ has quit IRC14:07
*** bhagyashris is now known as bhagyashris|afk14:07
*** wuchunyang has joined #openstack-kolla14:07
*** mnasiadka has quit IRC14:09
admin0when my interface is a bridge, kolla precheck fails .. my bridge is called br-mgmt14:16
admin0i am trying to use the same test rig to test both openstack-ansible as well as kolla-ansible14:16
admin0ansible.vars.hostvars.HostVarsVars object' has no attribute u'ansible_br-mgmt'14:16
*** bhagyashris|afk is now known as bhagyashris14:17
hrwyoctozepto: do you remember does gerrit allows patchA to depend on patchB from same project?14:17
hrwyoctozepto: Focal needs TD4 and Novajoin - I can do rebase Focal on top of TD4 but prefer not to pile TD4 on top of Novajoin (or viceversa)14:18
*** dave-mccowan has joined #openstack-kolla14:23
*** cah_link has quit IRC14:29
*** dave-mccowan has quit IRC14:31
*** abdysn has quit IRC14:31
*** bbezak has quit IRC14:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ubuntu: move to 20.04 Focal  https://review.opendev.org/73899414:35
hrwrebased on top of TD414:35
hrwubuntu-binary should built14:35
hrwubuntu-binary should build14:35
yoctozeptohrw: it can depend-on but the behaviour is not much better from rebases14:36
admin0where can i find a list of tags that can be used in kolla-ansible deploy14:40
admin0like --tags nova neutron etc14:40
*** dmsimard3 has joined #openstack-kolla14:41
*** dmsimard has quit IRC14:41
*** dmsimard3 is now known as dmsimard14:41
*** priteau has joined #openstack-kolla14:48
*** zhanglong has quit IRC14:51
admin0if i make changes after deployment, do I run deploy again , or do i run reconfigure ?14:52
*** jovial[m] has joined #openstack-kolla14:52
admin0and how to setup SSL for the endpoints ?14:53
*** zhanglong has joined #openstack-kolla14:53
yoctozeptomgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak - meeting in 7 minutes14:53
admin0kolla_external_fqdn_cert: "{{ node_config_directory }}/certificates/mycert.pem" -- just this is enough ?14:54
admin0does the .pem contain both the ca,crt and key ?14:54
*** ysirndjuro has joined #openstack-kolla14:57
* hrw present14:58
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: remove test tasks(planned)  https://review.opendev.org/74001214:58
hrwwuchunyang: 'tests: remove cinder and glance as planned' + short description14:59
hrwhttps://chris.beams.io/posts/git-commit/ as usual15:00
yoctozepto#startmeeting kolla15:00
hrwyoctozepto: #startmeeting?15:00
openstackMeeting started Wed Jul  8 15:00:14 2020 UTC and is due to finish in 60 minutes.  The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: kolla)"15:00
openstackThe meeting name has been set to 'kolla'15:00
*** mnasiadka has joined #openstack-kolla15:00
yoctozeptohrw: you guessed it15:00
yoctozepto#topic rollcall15:00
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:00
*** k_mouza has quit IRC15:00
hrwo~15:00
*** bbezak has joined #openstack-kolla15:00
mnasiadkao/15:00
osmanlicilegio/15:00
yoctozeptomgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak - meeting ongoing15:00
*** zhanglong has quit IRC15:00
yoctozepto\o/15:00
yoctozepto\o\15:01
yoctozepto /o/15:01
admin0\o15:01
*** k_mouza has joined #openstack-kolla15:02
jovial[m]o/15:02
*** diurnalist has joined #openstack-kolla15:02
yoctozeptopriteau, dougsz: meeting, meeting?15:02
yoctozepto#topic agenda15:03
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:03
yoctozepto* Roll-call15:03
yoctozepto* Announcements15:03
yoctozepto* Review action items from last meeting15:03
yoctozepto* CI status15:03
yoctozepto* Keystone fernet key sync followup https://review.opendev.org/70708015:03
yoctozepto* Victoria release planning (kayobe)15:03
yoctozepto* Victoria release planning (kolla & kolla ansible)15:03
patchbotpatch 707080 - kolla-ansible - Fix fernet bootstrap and key distribution - follow up - 14 patch sets15:03
yoctozepto* Kolla klub15:03
yoctozepto#topic Announcements15:03
*** openstack changes topic to "Announcements (Meeting topic: kolla)"15:03
*** zhanglong has joined #openstack-kolla15:03
bbezako/15:03
yoctozepto#info The first Kolla Kall is happening tomorrow15:04
yoctozepto#link https://wiki.openstack.org/wiki/Meetings/Kolla/Kall15:04
priteauyoctozepto: sorry I won't be available for the meeting today15:04
*** ykarel is now known as ykarel|away15:04
yoctozeptoit is open to all interested in developing kolla and friends15:04
*** priteau has quit IRC15:04
jovial[m]I assume you will only be discussing kolla and not kayobe?15:04
yoctozeptopriteau: ack, no probies15:04
openstackgerritgugug proposed openstack/kolla-ansible master: Remove the planned merge_configs  https://review.opendev.org/73952515:05
yoctozeptojovial[m]: kayobe too15:05
yoctozeptoin the agenda we have the documentation review15:05
yoctozeptoit is very likely kolla docs will take up enough time to fail to fit kayobe in this session though15:05
yoctozeptobut we will try15:05
jovial[m]awesome, thanks :)15:05
mnasiadkadocs review? I just found out I can't join! ;)15:05
yoctozeptomnasiadka: ha-ha15:05
yoctozeptojovial[m]: :-)15:06
yoctozeptoanything else to share as announcement?15:06
mnasiadkayoctozepto: kayobe also needs docs review I guess :)15:06
yoctozeptoany new kolla babies? (in progress or no)15:06
yoctozeptomnasiadka: yes, it is scheduled in the agenda15:06
yoctozeptobut I am worried we will eat up enough time for the previous two15:06
yoctozeptothe Kall happens every other week so the next will be in two weeks time15:07
yoctozeptoit alternates with Klub15:07
yoctozeptook, I assume no announcements15:07
yoctozeptolet's go15:07
yoctozepto#topic Review action items from last meeting15:07
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)"15:07
yoctozeptohrw to switch kolla to use stable branches and update release documentation15:07
yoctozeptoI did not notice but I was a bit overwhelmed by other things15:08
openstackgerritDoug Szumski proposed openstack/kolla-ansible stable/train: Use kolla_logs volume for Elasticsearch  https://review.opendev.org/74001515:08
mnasiadkathere is a change backport to stable/ussuri - but haven't seen release docs update15:08
yoctozeptoit looks started https://review.opendev.org/73890715:08
patchbotpatch 738907 - kolla (stable/ussuri) - version-check: handle using of stable branch tarballs - 2 patch sets15:08
yoctozepto"cherry-picked from master + updated SOURCES in config.py"15:09
yoctozeptoit seems gerrit lost the connection between the two15:09
hrwyoctozepto: patch for stable in gerrit15:09
yoctozeptohttps://review.opendev.org/73890615:09
patchbotpatch 738906 - kolla - version-check: handle using of stable branch tarballs - 3 patch sets15:09
hrwfirst we need to merge one into master15:09
yoctozepto#link https://review.opendev.org/73890615:10
patchbotpatch 738906 - kolla - version-check: handle using of stable branch tarballs - 3 patch sets15:10
hrwussuri one was CI test more15:10
yoctozeptoack, that makes sense15:10
yoctozeptono other tasks15:10
yoctozeptoI spent some time with timesync and family but did not finish15:11
yoctozeptoduties, duties15:11
yoctozeptomoving on15:11
yoctozepto#topic CI status15:11
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:11
hrwrocky is RED still15:11
yoctozeptoyeah, but rocky is em15:11
yoctozeptoyet we want it fixed for other reasons15:11
hrwand we have fix in gerrit which fails anyway15:12
yoctozeptoanyhow, it generally looks green15:12
yoctozeptothe timeouts are annoying though15:12
hrwyep15:12
yoctozeptothe horizon slowness is ---15:12
hrwand aarch64 issues15:12
mgoddardo/15:12
mgoddardsorry I'm late, thanks for getting started15:12
yoctozepto#chair mgoddard15:12
openstackCurrent chairs: mgoddard yoctozepto15:13
yoctozeptowelcome mgoddard15:13
yoctozepto#topic Keystone fernet key sync followup https://review.opendev.org/70708015:13
patchbotpatch 707080 - kolla-ansible - Fix fernet bootstrap and key distribution - follow up - 14 patch sets15:13
*** openstack changes topic to "Keystone fernet key sync followup https://review.opendev.org/707080 (Meeting topic: kolla)"15:13
yoctozeptomnasiadka, the floor is yours15:13
mnasiadkaI'm afraid I didn't have time to even look in that change, just a quick peek15:13
yoctozeptomnasiadka: so nothing to add? sad :-(15:14
yoctozeptobut I obviously understand, in a similar position15:14
dougsz\o15:14
mgoddardI did another review of it15:14
mnasiadkayoctozepto: I'll spawn some VMs on Friday and try to fix it though - if there's nobody that can pick it up15:15
mnasiadkamgoddard: saw that, thanks - but I think that change needs a major rework, I just left it in some weird state - might be easier to have a fresh approach15:15
*** wuchunyang has quit IRC15:15
yoctozeptomnasiadka: I only thought you would be the best guy to continue this, already told you that15:16
mgoddardI was wondering if there is some part of it we can merge to make an improvement, without looking for perfection15:16
*** priteau has joined #openstack-kolla15:16
yoctozeptobut if you strongly disagree, then we can reorganize it15:16
yoctozeptomgoddard: ++15:16
yoctozeptowe seem to have made some parts of the process better but worsened others, could limit to reversing the worsening and obvious wrongs15:17
mnasiadkatomorrow I'm pretty busy, but I'll move this to top of my priorities list, so at least I can look into what we could split out of that change15:17
yoctozeptobut I guess it does not end up being too far away from perfection then15:17
yoctozeptomnasiadka: sounds great to me15:17
mgoddardanother thing - should keystone wait for keys on startup, or just fail and retry at the docker level?15:17
mnasiadkayoctozepto: fernet won't change approach in keystone, so we just need to make it semi-good :)15:17
yoctozeptomnasiadka: haha, that true15:17
yoctozeptoquasi-good would be probably even more appropriate15:18
mnasiadkamgoddard: retrying at Docker level might be good enough15:18
mgoddardit's more visible15:18
mgoddardand we should be able to restructure the ansible to wait for keys to be distributed15:18
*** zhanglong has quit IRC15:19
mgoddardor actively sync in handlers15:19
mnasiadkamgoddard: good idea, let me look into that on Friday - tomorrow I'm a bit busy15:19
mnasiadkaand we can discuss on the channel15:19
mgoddardmaybe a handler like: include_tasks: distribute_fernet.yml15:20
mgoddardor maybe not15:20
mgoddardit could work15:20
yoctozeptoanyhow, as far as chrony/ntp/timesync is concerned I did some analysis that I share with infra and you might be interested in when debugging https://etherpad.opendev.org/p/ci-timesync15:21
yoctozepto#link https://etherpad.opendev.org/p/ci-timesync15:21
yoctozepto(side note I forgot to share)15:21
openstackgerritjacky06 proposed openstack/kolla-ansible master: Remove the waiting for ironic-api to be accessible  https://review.opendev.org/74001815:21
mnasiadkayoctozepto: regarding timesync I'm really unhappy that we merged a change, that uninstalls chrony on the OS level without giving the option for not doing so :)15:22
yoctozeptomnasiadka: how so? it is controlled by enable_chrony15:22
mnasiadkayoctozepto: yeah, still hard to override it on host/group level :)15:23
mnasiadkayoctozepto: anyway, just laughing, it's a bit irritating to see that, if you don't track changes15:24
mgoddardlogically it makes sense, but in practice might cause issues15:24
mgoddardyou could use the chrony group to control where it lands15:24
*** JamesBenson has joined #openstack-kolla15:25
yoctozeptoanyhow, we agreed to deprecate the switch and switch it to False by default15:25
yoctozeptowe don't recommend containerizing chrony15:26
yoctozepto(unless we do)15:26
mgoddardso from your notes yoctozepto it looks like ntpd is slow to sync, but chrony and timesyncd are better15:27
yoctozeptoin their current infra config15:27
mgoddardoften with ntpd it's combined with a one time sync15:27
yoctozeptoit probably failed15:27
mgoddardmaybe newer daemons have some better startup logic15:27
yoctozeptoanyway, infra will likely switch to the distro defaults15:27
yoctozeptontpd is no longer a default anywhere15:28
mnasiadkaso bye bye ntpd15:28
yoctozeptomore or less, yes15:29
mgoddardshall we move on?15:30
mgoddard#topic Victoria release planning (kayobe)15:30
*** openstack changes topic to "Victoria release planning (kayobe) (Meeting topic: kolla)"15:30
mgoddard#link https://etherpad.opendev.org/p/KollaWhiteBoard15:30
mgoddardL29215:30
mgoddardjovial[m]15:31
jovial[m]hey15:31
mgoddarddougsz15:31
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Remove the planned merge_configs  https://review.opendev.org/73952515:31
dougszhi there15:31
mgoddardis anyone working on victoria features yet?15:31
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Remove the ml2_conf.ini merging for agents  https://review.opendev.org/73952515:31
dougsznot I15:32
jovial[m]priteau: was working on multiple environments, right?15:32
mgoddardyes, there was a patch15:32
priteauI’ve not been able to work on multiple environments recently.15:32
mgoddardhttps://review.opendev.org/73486715:32
patchbotpatch 734867 - kayobe - Support multiple environments from a single config... - 1 patch set15:32
priteauWill resume as soon as I’ve finished the project keeping me busy15:32
mgoddardI should give it a review15:33
mgoddardhttps://review.opendev.org/#/q/project:openstack/kayobe+status:open15:34
priteauIt’s not fully done yet but happy to have initial comments15:34
*** nikparasyr has left #openstack-kolla15:35
mgoddardwe should pick a few patches to prioritise to keep on top of reviews15:35
jovial[m]sounds good to me15:35
mgoddardbackports of course15:35
mgoddardeveryone pick a (non-backport) patch and add RP+1 :)15:35
*** k_mouza has quit IRC15:36
admin0how to ask kolla ansible to use nfs for glance15:36
admin0i see the docs saying glance_file_datadir_volume  .. is that done on the controllers ?15:37
mgoddardadmin0: we're in a meeting at the moment15:37
mgoddardcall back in 25 minutes15:37
admin0oh .. sorry15:37
mgoddardany other kayobe things?15:37
admin0carry on15:37
dougszhmm, nothing I can think of15:38
priteauI’ve pushed minor changes to the train image regex please review15:38
mgoddardthanks15:38
mgoddard#topic Victoria release planning (kolla & kolla ansible)15:39
*** openstack changes topic to "Victoria release planning (kolla & kolla ansible) (Meeting topic: kolla)"15:39
mgoddardwhiteboard L178+15:39
*** zijlboot has quit IRC15:39
*** priteau has quit IRC15:39
mgoddardgood to see hrw working hard on infra images, ubuntu focal and stable branch lifecycle15:40
hrwyo15:40
hrwfocal needs td-agent4 and novajoin fixes to merge15:40
mgoddardplease review his patches to keep him interested :)15:40
hrwthen it will work for both x86 and aarch6415:40
hrws/work/build15:40
mgoddardok. Let's keep an eye on fluentd logs for any issues15:41
hrwI can add RP+1 if someone needs more pinging15:41
yoctozeptothe good news in this regard is that I need designate in one of my projects so it is easier to fit it in my schedule15:41
mgoddardnice15:41
*** k_mouza has joined #openstack-kolla15:41
mgoddardhrw: how does td-agent map to fluentd version?15:42
hrwmgoddard: have to check15:42
*** bengates has quit IRC15:42
dougszhaven't seen a stable fluentd release using td4 yet15:42
hrwmgoddard: 'latest v1'15:42
hrwdougsz: they did it last week15:43
hrwdougsz: https://review.opendev.org/73993515:43
patchbotpatch 739935 - kolla - fluentd: migrate to td-agent 4 where possible - 3 patch sets15:43
dougszyeah, but that's not a stable fluentd release15:43
hrwdougsz: I wanted to respond with 'since when td-agent is sane' but this is logged15:44
hrw;D15:44
hrwdougsz: if we want focal then either td-agent4 or fluentd from gem15:44
hrwchoose your poison15:44
mgoddarddo we support fluentd on PPC?15:45
mgoddard'support'15:45
dougszperhaps we should ask them what the plans are for the next stable release15:45
hrwI would go with just fluentd everywhere but I am outvoted on 'just use distro/foss instead of random binaries'15:45
hrwmgoddard: do we support ppc? we just have basic support15:45
hrwdougsz: go and ask. I prefer not to. they barely made 4.0.0 release15:46
dougsz:) OK, I will ask15:46
* hrw ends bashing and wait for next topic15:46
mgoddardAny other features we should discuss?15:46
mnasiadkadougsz: first read hrw's history of asking15:47
mgoddardI've been pushing patches for performance improvements, reviews appreciated15:47
hrwdougsz: https://github.com/treasure-data/omnibus-td-agent/issues/209 is fun read15:47
mnasiadkadougsz: since when fluentd has stable releases? :)15:47
hrwmgoddard: your patches remind me to finally learn ansible..15:47
mgoddardhrw: you should! it's really not hard15:48
hrw;)15:49
mgoddard#topic Open discussion15:49
*** openstack changes topic to "Open discussion (Meeting topic: kolla)"15:49
hrwanyone wants to say something?15:49
mgoddardseems not15:50
mgoddardLet's finish early15:50
mnasiadkaI revived docker healtchecks on kolla-ansible - would be happy if somebody reviewed15:50
mgoddardThanks everyone, and thanks yoctozepto for chairing15:50
mgoddard#endmeeting15:50
*** openstack changes topic to "Vote on Victoria priorities https://bit.ly/2ActPtx | IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"15:50
openstackMeeting ended Wed Jul  8 15:50:53 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-07-08-15.00.html15:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-07-08-15.00.txt15:50
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-07-08-15.00.log.html15:50
mnasiadkahttps://review.opendev.org/#/c/676389/15:51
patchbotpatch 676389 - kolla-ansible - Use Docker healthchecks for core services - 53 patch sets15:51
mgoddardtoo slow mnasiadka15:51
hrwmgoddard: my problem with ansible is the same as with python - started with projects where it is heavy used and learnt that part. never basics15:51
mnasiadkamgoddard: you need to learn to wait! ;)15:51
mgoddardmnasiadka: 90 seconds is enough if you're paying attention :p15:52
mnasiadkamgoddard: I noticed some other projects are following the trend in being more inclusive and removing slavery themes and blacklist themes - are we going that path as well? :D15:52
*** Alan_Ortiz has joined #openstack-kolla15:54
yoctozeptomnasiadka: we probably shouldn't create new references for that but can't really decide for deployed projects15:55
mgoddardI found two references to blacklist in k-a. If we remove them it will break configuration :)15:56
mgoddardif we can make changes without introducing regressions its fine by me15:56
hrwand few slave mentions15:56
mnasiadkadougsz: td-agent 4 is fluentd 1.11.1 - isn't that stable?15:57
dougszNot sure, just going off: https://www.fluentd.org/download15:57
mnasiadkadougsz: and there's 1.11.1 on the bottom when installing from gem15:58
mnasiadkadougsz: I would say it's the most recent version of stable ;)15:58
dougszthat's true about the gem15:58
mnasiadkahttps://github.com/fluent/fluentd/blob/master/CHANGELOG.md - this one also lists 1.11.1 as the latest one15:58
mnasiadkaok, need to wrap up16:00
hrwcan someone remind me why we not just 'gem install fluentd'?16:01
*** bengates has joined #openstack-kolla16:02
hrwor is it usual devops thing 'I do not trust distros, I trust random binaries on the internet'?16:02
mgoddardhrw: gem install is rubygems, not distro16:03
hrwmgoddard: whatever16:03
hrwmgoddard: td-agent bundles ruby iirc16:03
hrwand fluentd needs 2.1+ so each distro we cover is fine16:07
hrwok, install docs say 2.4+ - still fine16:07
mnasiadkahrw: we got bitten by old ruby in distros, and got stuck on old fluentd versions - and td-agent ships embedded ruby16:07
mnasiadkahrw: and this was the only issue I remember16:07
hrwmnasiadka: rocky times...16:08
mnasiadkahrw: centos 7 times16:08
hrwok16:08
hrwnow we have 2.5+16:08
hrwgem time?16:08
mnasiadkastill if you go to fluentd site - rpm/deb installation links to td-agent16:08
*** bbezak has quit IRC16:08
mnasiadkaMaybe I'm old fashioned, but I like deb/rpm over gem :)16:09
*** bengates has quit IRC16:09
hrwI like sane solutions ;)16:09
hrwanyway, as long as it builds and someone takes care that it works I am fine16:09
mnasiadkahrw: yeah, and as long as it is similar on all distros/archs - if td-agent fails, we may go with gem16:10
admin0ok .. to my question .. glance_file_datadir_volume:  // is this shared storage only mounted on the controllers where the glance docker runs, or need to also mount in hypervisors ?16:10
*** mnasiadka has quit IRC16:11
* hrw out16:11
mgoddardadmin0: wherever glance-api runs16:22
mgoddardcontrollers16:22
admin0ok .. and does kolla work with bridges ?16:22
admin0for network interface16:22
admin0if it works, i think it fails when the bridge name is br-X format16:23
admin0with a -16:23
admin0i have 3 different config of serves with 3 different network names .. so i cannot make a common in globals.yml16:23
admin0which is why i am creating a bridge and putting that interface inside the bridge16:23
*** gouthamr has quit IRC16:26
mgoddardadmin0: you can use host_vars or group_vars16:31
mgoddardnext to your inventory16:31
*** dougsz has quit IRC16:32
*** Manheim_ has joined #openstack-kolla16:35
*** Manheim has quit IRC16:38
openstackgerritMerged openstack/kolla-ansible stable/ussuri: Fix incorrect value of [storage]/ceph_keyring in gnocchi.conf  https://review.opendev.org/73991716:40
Manheim_I can deploy without heat, right?16:41
Manheim_I see this in keystone https://paste.ubuntu.com/p/nMhmrsjkMT/16:42
Manheim_it's on stable train16:42
*** ykarel|away has quit IRC16:46
*** whoami-rajat has quit IRC16:46
openstackgerritMerged openstack/kolla-ansible master: Manage octavia health manager worker through openstack_service worker  https://review.opendev.org/73988216:50
*** k_mouza has quit IRC16:53
*** bsanjeewa_ has quit IRC16:55
*** jonaspaulo has joined #openstack-kolla17:12
Manheim_found the bug https://bugs.launchpad.net/kolla-ansible/+bug/184678917:17
openstackLaunchpad bug 1846789 in kolla-ansible train "Creating the Keystone service and endpoint Failed: Internal Server Error (HTTP 500)" [High,Triaged]17:17
openstackgerritPierre Riteau proposed openstack/kayobe master: Add support for custom options in static routes  https://review.opendev.org/73697617:26
*** jcmdln has quit IRC17:42
*** Manheim has joined #openstack-kolla17:46
*** Manheim has quit IRC17:48
*** Manheim has joined #openstack-kolla17:48
*** Manheim_ has quit IRC17:49
*** Manheim has quit IRC17:52
*** ricolin has quit IRC18:00
*** e0ne has quit IRC18:03
*** Manheim has joined #openstack-kolla18:11
*** bsanjeewa_ has joined #openstack-kolla18:14
*** Manheim has quit IRC18:16
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Add elasticsearch-curator to kolla-ansible inventory template  https://review.opendev.org/74007918:18
*** also_stingrayza has joined #openstack-kolla18:19
*** stingrayza has quit IRC18:21
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Performance: Split out prometheus node exporter and cadvisor  https://review.opendev.org/74008218:32
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: CI: add prometheus-efk scenario  https://review.opendev.org/74008318:32
admin0the volume group is called __DEFAULT__  in kolla for cinder .. how to change this name to something meaningful ?18:47
admin0Ubuntu release focal version 20.04 is not supported. Supported releases are: bionic"   -- how do I test in focal ?18:49
*** kemopq_ has quit IRC18:49
admin0or is focal not supported at all ?18:49
*** kemopq_ has joined #openstack-kolla18:49
*** bsanjeewa_ has quit IRC19:13
*** gmann_ has joined #openstack-kolla19:17
*** gmann_ is now known as gmann19:18
openstackgerritMerged openstack/kolla-ansible stable/train: Use kolla_logs volume for Elasticsearch  https://review.opendev.org/74001519:31
*** r3ap3r has joined #openstack-kolla19:33
*** cz3 has joined #openstack-kolla19:34
*** TheJulia has joined #openstack-kolla19:35
*** kklimonda has joined #openstack-kolla19:36
*** v1k0d3n has joined #openstack-kolla19:36
*** niedbalski has joined #openstack-kolla19:37
*** rpittau has joined #openstack-kolla19:40
*** jonmasters has joined #openstack-kolla19:41
openstackgerritMerged openstack/kolla-ansible master: Remove the ml2_conf.ini merging for agents  https://review.opendev.org/73952519:47
*** rm_work has joined #openstack-kolla19:53
*** vishalmanchanda has joined #openstack-kolla19:53
*** bbezak has joined #openstack-kolla19:57
*** headphoneJames has joined #openstack-kolla19:58
*** mnasiadka has joined #openstack-kolla20:04
*** johnsom_ has joined #openstack-kolla20:10
*** mwhahaha has joined #openstack-kolla20:10
*** emccormick has joined #openstack-kolla20:14
*** vkmc_ has joined #openstack-kolla20:17
*** sri_ has joined #openstack-kolla20:17
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [WIP] [CI] Ceilometer scenario  https://review.opendev.org/74010620:17
*** andrein has joined #openstack-kolla20:21
*** crindi has joined #openstack-kolla20:22
*** stackedsax has joined #openstack-kolla20:23
*** alanmeadows has joined #openstack-kolla20:26
*** gfidente is now known as gfidente|afk20:31
*** egonzalez has joined #openstack-kolla20:52
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [WIP] [CI] Ceilometer scenario  https://review.opendev.org/74010620:57
dkingDoes anybody know how to get a list of plays that  are run by "kolla-ansible bootstrap-servers"?20:58
*** gouthamr has joined #openstack-kolla21:11
*** gouthamr_ has joined #openstack-kolla21:15
*** johnsom_ is now known as johnsom21:19
*** johnsom has joined #openstack-kolla21:19
admin0how to test in focal .. i get message Ubuntu release focal version 20.04 is not supported. Supported releases are: bionic21:46
*** JamesBenson has quit IRC21:47
*** TrevorV has quit IRC21:53
*** Manheim has joined #openstack-kolla22:00
*** vishalmanchanda has quit IRC22:03
*** ysirndjuro has quit IRC22:08
*** jonaspaulo has quit IRC22:23
openstackgerritMerged openstack/kolla-ansible master: Remove the waiting for ironic-api to be accessible  https://review.opendev.org/74001822:56
*** Manheim has quit IRC22:56
*** gfidente|afk has quit IRC23:06
*** evrardjp has quit IRC23:25
*** evrardjp has joined #openstack-kolla23:26
*** born2bake has quit IRC23:42
*** Alan_Ortiz has quit IRC23:42
*** wuchunyang has joined #openstack-kolla23:45
*** wuchunyang has quit IRC23:49
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for encrypting Nova API  https://review.opendev.org/72479423:56

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!