Wednesday, 2021-04-14

*** kevko_ has quit IRC00:02
*** kevko_ has joined #openstack-kolla00:36
*** LinPeiWen has joined #openstack-kolla00:49
*** kevko_ has quit IRC00:52
*** wuchunyang has joined #openstack-kolla01:01
*** zzzeek has quit IRC01:07
*** zzzeek has joined #openstack-kolla01:08
*** zzzeek has quit IRC01:33
*** zzzeek has joined #openstack-kolla01:35
*** k_mouza has joined #openstack-kolla01:50
*** k_mouza has quit IRC01:54
*** zzzeek has quit IRC01:54
*** zzzeek has joined #openstack-kolla01:57
*** zzzeek has quit IRC02:55
*** zzzeek has joined #openstack-kolla02:57
*** zzzeek has quit IRC03:30
*** zzzeek has joined #openstack-kolla03:31
*** skramaja has joined #openstack-kolla03:52
*** dasTor has quit IRC03:57
*** wuchunyang has quit IRC04:01
*** wuchunyang has joined #openstack-kolla04:22
*** wuchunyang has quit IRC04:26
*** vishalmanchanda has joined #openstack-kolla04:44
*** ivan_lin has quit IRC04:46
*** ivan_lin has joined #openstack-kolla04:46
*** cah_link has joined #openstack-kolla05:25
*** wuchunyang has joined #openstack-kolla05:36
*** dasTor has joined #openstack-kolla05:37
openstackgerritLin PeiWen proposed openstack/kolla-ansible master: Use Docker healthchecks for rabbitmq services  https://review.opendev.org/c/openstack/kolla-ansible/+/78040705:47
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: mariadb: Fix wsrep notify script  https://review.opendev.org/c/openstack/kolla-ansible/+/78603606:31
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: mariadb: Update wsrep notify script for Debian Bullseye  https://review.opendev.org/c/openstack/kolla-ansible/+/78603606:32
*** emccormick has quit IRC06:34
*** macz_ has joined #openstack-kolla06:34
*** PrinzElvis has quit IRC06:34
*** kevko_ has joined #openstack-kolla06:35
*** lxkong has quit IRC06:35
*** emccormick has joined #openstack-kolla06:37
*** lxkong has joined #openstack-kolla06:37
*** PrinzElvis has joined #openstack-kolla06:37
*** macz_ has quit IRC06:38
*** luksky has joined #openstack-kolla07:11
oklhostmorning07:14
*** andrewbonney has joined #openstack-kolla07:14
*** bengates has joined #openstack-kolla07:17
*** bengates has quit IRC07:18
*** bengates has joined #openstack-kolla07:19
*** bengates has quit IRC07:26
*** bengates has joined #openstack-kolla07:27
*** k_mouza has joined #openstack-kolla07:30
*** k_mouza has quit IRC07:34
*** e0ne has joined #openstack-kolla07:54
*** rpittau|afk is now known as rpittau07:55
openstackgerritMichal Nasiadka proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247907:57
mnasiadkahrw, yoctozepto: we need  https://review.opendev.org/c/openstack/kolla-ansible/+/786036 and https://review.opendev.org/c/openstack/kolla/+/772479 to move to Bullseye - seems it's working :)07:58
hrwjust as I was looking for it at gerrit ;D07:58
openstackgerritPierre Riteau proposed openstack/kayobe stable/stein: Revert "Performance: skip LVM configuration if no groups configured"  https://review.opendev.org/c/openstack/kayobe/+/78333707:59
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Revert "Performance: skip LVM configuration if no groups configured"  https://review.opendev.org/c/openstack/kayobe/+/78331607:59
hrwmnasiadka: please add comment "as we have mariadb 10.5, this and that was changed so we have to do this'07:59
hrw+1 + comment08:01
*** gfidente|afk is now known as gfidente08:01
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247908:02
hrwdropped changelog from commit message08:02
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: mariadb: Update wsrep notify script for Debian Bullseye  https://review.opendev.org/c/openstack/kolla-ansible/+/78603608:03
*** fuhrmannb has joined #openstack-kolla08:03
mnasiadkaI updated the second change in the same time, let me run recheck on Kolla change ;)08:06
hrw+208:06
yoctozeptomnasiadka: does not the timeout affect other versions as well?08:07
mnasiadkayoctozepto: well, if by affect you mean if we saw it failing due to that - I think not, but really we should drop that wsrep-notify script08:08
mnasiadkaI just don't want to do it at this state of a cycle :)08:08
yoctozeptomnasiadka: what will we replace it with?08:09
yoctozeptodid I miss something?08:09
mnasiadkayoctozepto: we don't need it since we have clustercheck08:09
mnasiadkayoctozepto: that script only renames the haproxy user - we needed that when we did do haproxy level healthchecks on the DB08:09
mnasiadkayoctozepto: clustercheck makes more ,,advanced'' checks to check if the galera node is usable08:10
yoctozeptomnasiadka: ah, it's pre-xinetd then08:10
yoctozeptoI remember08:10
yoctozeptodid not we deprecate this monstrosity?08:11
yoctozeptoperhaps we can drop it indeed08:11
mnasiadkaI have no clue08:11
yoctozeptoat least we should not use it with clustercheck08:11
mnasiadkawell, right08:12
mnasiadkaWe can skip connect-timeout - but I think mariadb locally should respond in 3 seconds :)08:12
mnasiadkaAnd in Bullseye I've seen that this script when called by mariadb - it made the DB ,,hang'' - no connections could be made to MariaDB even over network08:13
mnasiadkabut maybe it was because mysql.user is a view now :)08:13
*** ivan_lin has quit IRC08:21
*** ivan_lin has joined #openstack-kolla08:21
kevko_i think connection timeout is ok, if it is less then time when haproxy will start report backend is down08:26
kevko_(and I don't know how much it is )08:26
kevko_mnasiadka and yes , wsrep notify is dirty thing08:26
yoctozeptomnasiadka: can we skip wsrep-notify if clustercheck is used?08:33
mnasiadkayoctozepto: sure we can08:33
yoctozeptolet's run all deprec/obsolete/drop we can08:33
yoctozeptothanks mnasiadka08:33
*** macz_ has joined #openstack-kolla08:35
hrwwill have to build images for aarch64 and check locally08:35
*** ricolin has quit IRC08:39
*** macz_ has quit IRC08:39
openstackgerritPierre Riteau proposed openstack/kayobe master: Remove ntp.yml configuration file  https://review.opendev.org/c/openstack/kayobe/+/78618109:04
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep-notify script if clustercheck enabled  https://review.opendev.org/c/openstack/kolla-ansible/+/78618209:06
openstackgerritPierre Riteau proposed openstack/kayobe master: Fix typo in CI playbook  https://review.opendev.org/c/openstack/kayobe/+/78619009:30
*** hjensas|afk is now known as hjensas09:36
*** jovial[m] has quit IRC09:43
*** kevko_ has quit IRC09:44
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep-notify script if clustercheck enabled  https://review.opendev.org/c/openstack/kolla-ansible/+/78618209:46
*** jovial[m] has joined #openstack-kolla09:48
yoctozeptomnasiadka: have you verified that deprec/obsoletion? happened/not?09:56
openstackgerritMerged openstack/kayobe stable/victoria: Fix overcloud provision with fast track enabled  https://review.opendev.org/c/openstack/kayobe/+/78533109:57
mnasiadkayoctozepto: nope it didn't09:57
*** kevko_ has joined #openstack-kolla10:00
yoctozeptomnasiadka: then prime time, started failing on bullseye - no longer supported :-)10:03
yoctozeptosince we already want to get rid of it10:03
yoctozeptoI somehow missed that we did not10:03
yoctozeptoI wonder if it has a good reason10:03
mnasiadkawell, now that we default to clustercheck - we don't have any good reason to keep it10:04
mnasiadkaand it keeps failing on renaming the haproxy user when it can't connect to the DB and so on10:04
mnasiadkaso, do we want to deprecate wsrep-notify, or deprecate haproxy/mariadb without clustercheck?10:05
*** wuchunyang has quit IRC10:05
yoctozeptomnasiadka: I guess wsrep-notify; we can let users break themselves by disabling clustercheck, why not ;-)10:06
yoctozeptodepends on what to guard against10:06
yoctozeptoon single node it does not make sense to use either10:06
yoctozeptoso10:06
yoctozeptoah this lovely time when we go berserk in deprecations...10:07
hrw;D10:07
hrwI checked deprecation of binary images. Debian and Ubuntu build fine without external openstack repos. CentOS needs openstack repos10:08
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247910:09
yoctozeptohrw: though I wonder how they work10:10
yoctozeptodid not UCA bump up things like libvirt?10:10
hrwprobably10:25
kevko_hi,10:30
kevko_how much time between joined and synced is normal  ?10:31
kevko_[mariadb]10:31
kevko_i have something about 5 minutes (test cloud deployed on vms on openstack)10:32
*** Luzi has joined #openstack-kolla10:46
parallaxkevko_: depends on the amount of data in the cluster10:47
kevko_parallax: well, empty dbs .. it's just fresh env ...10:48
kevko_parallax: but maybe really slow disks .. before they dies10:48
kevko_parallax: because when I m reconfiguring ..I almost always faild on wait for wsrep synced ..so trying to figure out how to change not only "hard coded" retries and deplay10:49
parallaxShould be instantaneous then10:50
mnasiadkakevko_: 5 minutes for syncing an empty log? that's awesome! :)10:56
kevko_mnasiadka: :)11:02
kevko_mnasiadka: well it's actually bigger :)11:02
kevko_mnasiadka: | wsrep_local_state_comment     | Joined                                        |11:02
kevko_| wsrep_local_state_uuid        | 7e0a8cca-9b90-11eb-a7ca-9b5bedd1b90f          |11:02
kevko_| wsrep_open_connections        | 0                                             |11:02
kevko_| wsrep_open_transactions       | 0                                             |11:02
kevko_| wsrep_protocol_version        | 9                                             |11:02
kevko_| wsrep_provider_name           | Galera                                        |11:02
kevko_| wsrep_provider_vendor         | Codership Oy <info@codership.com>             |11:02
kevko_| wsrep_provider_version        | 3.25(rddf9876)                                |11:02
kevko_| wsrep_ready                   | OFF                                           |11:02
kevko_| wsrep_received                | 1684                                          |11:02
kevko_| wsrep_received_bytes          | 142541211:02
mnasiadkakevko_: 1.5 megabyte? nice11:13
kevko_mnasiadka: well, i really don't know why it is so slow ..11:13
hrwmeh. grafana repo broke builds11:21
mnasiadkamaybe it's time to setup a mirror@infra ;)11:23
hrwno11:27
hrwas then you would follow with elasticsearch, rabbitmq, erlang...11:28
mnasiadkahrw: https://review.opendev.org/c/openstack/kolla-ansible/+/786182 - need another +2 and we have this mariadb crap solved11:33
hrwdone11:34
* hrw deploys on aarch6411:34
yoctozeptomnasiadka: could you kickstart the wsrep-notify deprecation in gerrit?11:37
mnasiadkayoctozepto: that's my plan11:37
yoctozeptogreat! :D11:37
mnasiadkado we have the meeting today?11:45
*** hrw has quit IRC11:50
*** hrw has joined #openstack-kolla12:08
yoctozeptoyes, I can chair12:12
yoctozeptowe have agenda ;-)12:12
*** luksky has quit IRC12:20
*** luksky has joined #openstack-kolla12:20
*** luksky has quit IRC12:21
hrwyepFAILED - RETRYING: Wait for MariaDB service port liveness (4 retries left).12:21
*** luksky has joined #openstack-kolla12:21
hrw(kolla) 14:21 (1s) hrw@jagular:kolla$ mysql -h 192.168.200.241 -u root -p12:21
hrwEnter password:12:21
hrwbut then hangs after providing whatever12:21
*** wuchunyang has joined #openstack-kolla12:35
*** macz_ has joined #openstack-kolla12:36
*** hrw has quit IRC12:36
openstackgerritMerged openstack/kolla-ansible master: mariadb: Disable wsrep-notify script if clustercheck enabled  https://review.opendev.org/c/openstack/kolla-ansible/+/78618212:37
*** macz_ has quit IRC12:41
*** wuchunyang has quit IRC12:42
*** hrw has joined #openstack-kolla12:44
*** Luzi has quit IRC12:44
hrwhad anyone said something after '14:21 < hrw> but then hangs after providing whatever'?12:46
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: mariadb: Deprecate wsrep-notify.sh  https://review.opendev.org/c/openstack/kolla-ansible/+/78624513:05
mnasiadkahrw: do you have the change for disabling wsrep-notify.sh?13:05
mnasiadkahrw: that merged some minutes ago?13:05
mnasiadkahrw: https://review.opendev.org/c/openstack/kolla-ansible/+/78618213:05
yoctozeptoit's green in the CI so I would assume hrw does not13:11
*** hrw has quit IRC13:11
yoctozeptohrw is b0rken13:13
*** hrw has joined #openstack-kolla13:16
mnasiadkamaybe now hrw is unbroken13:18
*** hrw has quit IRC13:21
*** k_mouza has joined #openstack-kolla13:24
mnasiadkaor not :)13:27
*** ricolin has joined #openstack-kolla13:34
*** skramaja has quit IRC13:34
*** k_mouza has quit IRC13:47
*** sorin-mihai has quit IRC13:50
*** k_mouza has joined #openstack-kolla13:54
*** samcat116 has joined #openstack-kolla13:59
openstackgerritJan Horstmann proposed openstack/kolla-ansible master: Allow override of rabbitmq config in kolla toolbox  https://review.opendev.org/c/openstack/kolla-ansible/+/78559514:00
*** baha has joined #openstack-kolla14:13
*** k_mouza has quit IRC14:20
*** samcat116 has quit IRC14:28
*** wuchunya_ has joined #openstack-kolla14:28
yoctozeptooh well14:30
yoctozeptomgoddard is vacationing14:30
yoctozeptohrw b0rken14:30
yoctozeptoit seems it's only me and you, mnasiadka14:30
yoctozeptowe have to proteccc the gatezzz14:30
*** wuchunya_ has quit IRC14:33
mnasiadkayoctozepto: seen this https://review.opendev.org/c/openstack/kolla/+/785350 ?14:35
*** samcat116 has joined #openstack-kolla14:37
yoctozeptomnasiadka: yeah, but hrw told me the gates are back on track14:37
mnasiadkaso should we ask Rico Lin to abandon?14:38
yoctozeptostill, I think hrw does not care about centos aarch6414:38
mnasiadkaso if hrw does not care, why should we waste Linaro CI cycles on that?14:38
yoctozeptoso it's droppable but due to different reasons14:38
yoctozeptoadd him to that before approving14:38
mnasiadkayoctozepto: there's a second change to add it back ;)14:39
mnasiadkaI think it's only to delete some stale jobs or something...14:39
yoctozeptoadded him14:39
yoctozeptomnasiadka: yeah, it was14:39
yoctozeptoit's written in the very message14:40
yoctozeptoaarch64 queue was stuck14:40
yoctozeptoI got a spam afterward14:40
yoctozeptostill left some around14:40
yoctozeptono time to delete14:40
*** samcat116 has quit IRC14:50
*** macz_ has joined #openstack-kolla14:50
*** k_mouza has joined #openstack-kolla14:50
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Additional small changes in role/mariadb  https://review.opendev.org/c/openstack/kolla-ansible/+/78565714:54
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Rename role haproxy to loadbalancer  https://review.opendev.org/c/openstack/kolla-ansible/+/77061814:54
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database  https://review.opendev.org/c/openstack/kolla-ansible/+/77021514:54
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021614:54
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario  https://review.opendev.org/c/openstack/kolla-ansible/+/77062114:55
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Move config.yml to config-node.yml in mariadb role  https://review.opendev.org/c/openstack/kolla-ansible/+/78356314:55
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add mariadb arbitrator to mariadb role  https://review.opendev.org/c/openstack/kolla-ansible/+/78081114:55
*** samcat116 has joined #openstack-kolla14:55
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: [CI] Test Mariadb-Arbitrator with shards in the nova cells scenario  https://review.opendev.org/c/openstack/kolla-ansible/+/78097014:55
yoctozeptomnasiadka: https://review.opendev.org/c/openstack/kolla-ansible/+/78565714:56
yoctozeptomake kevko happy14:56
yoctozeptowell, kevko_*14:56
yoctozeptoand mnasiadka: https://review.opendev.org/c/openstack/kolla-ansible/+/74423514:57
* yoctozepto starting the meeting14:59
yoctozeptomgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt15:00
yoctozeptomeeting now15:00
yoctozepto#startmeeting kolla15:00
openstackMeeting started Wed Apr 14 15:00:13 2021 UTC and is due to finish in 60 minutes.  The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: kolla)"15:00
openstackThe meeting name has been set to 'kolla'15:00
yoctozepto#topic Roll-call15:00
*** openstack changes topic to "Roll-call (Meeting topic: kolla)"15:00
yoctozepto\o/15:01
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add mariadb arbitrator to mariadb role  https://review.opendev.org/c/openstack/kolla-ansible/+/78081115:01
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: [CI] Test Mariadb-Arbitrator with shards in the nova cells scenario  https://review.opendev.org/c/openstack/kolla-ansible/+/78097015:01
headphoneJameso/15:01
mnasiadkao/15:01
*** wuchunya_ has joined #openstack-kolla15:02
yoctozeptoit's not quantity but quality that counts they say15:03
yoctozepto#topic Agenda15:03
*** openstack changes topic to "Agenda (Meeting topic: kolla)"15:03
yoctozepto* Roll-call15:03
yoctozepto* Announcements15:03
yoctozepto  ** mgoddard on holiday this week15:03
yoctozepto  ** Xena PTG next week! https://etherpad.opendev.org/p/kolla-xena-ptg15:03
yoctozepto  ** OpenStack User Survey 2020 is in! https://www.openstack.org/analytics15:03
yoctozepto* Review action items from the last meeting15:03
yoctozepto* CI status15:03
yoctozepto* Review requests15:03
yoctozepto* Wallaby release planning15:03
yoctozepto  ** Debian bullseye progress15:03
yoctozepto* Xena PTG planning https://etherpad.opendev.org/p/kolla-xena-ptg15:03
yoctozepto#topic Announcements15:04
*** openstack changes topic to "Announcements (Meeting topic: kolla)"15:04
yoctozeptomgoddard on holiday this week15:04
yoctozeptoall wish him a good rest; he deserves it ;-)15:04
yoctozeptoXena PTG next week! https://etherpad.opendev.org/p/kolla-xena-ptg15:04
yoctozeptoyeas! it's this time of the year again15:05
yoctozeptograb your entry cards and join us on the ptg15:05
yoctozeptoand add discussion topics to the etherpad15:05
mnasiadkait's this week where you end up exhausted like after running a marathon? ;)15:06
yoctozeptomnasiadka: yeah, I already feel it in my bones, ouch15:06
yoctozeptobut it's worth the effort! :D15:06
yoctozepto#info OpenStack User Survey 2020 is in! https://www.openstack.org/analytics15:06
yoctozepto15% of deployments are powered by kolla15:07
yoctozeptokolla stronk15:07
mnasiadka17% of deployments on Stein? :)15:07
mnasiadkaand Queens15:08
mnasiadkayay15:08
yoctozeptoyeas ;-)15:08
yoctozeptomany sad numbers there15:08
mnasiadka2% of people using Kolla-Kubernetes15:09
mnasiadkagood one15:09
yoctozeptomore masakari than blazar priteau15:09
* yoctozepto feels the pressure15:09
yoctozeptoyeah, kolla-kubernetes kills the system15:09
mnasiadkaoh, TripleO only used in 10% deployments and K-A in 15%15:09
mnasiadkathat's interesting15:09
yoctozeptoyeah, I wonder what those using RHOSP chose15:10
yoctozeptosome might be remembering the good ol' days of puppet15:10
mnasiadkaWhich OpenStack Network (Neutron) drivers are you using? - this is my readability winner15:10
yoctozeptowell, and pure "ansible" is interesting as the deployment choice15:11
yoctozeptoyeah, re neutron I have no idea who looked at this and thought: yeas, it's perfect to display ;D15:12
yoctozeptoonly table makes sense for that15:12
yoctozeptoand sorted by pop15:12
yoctozeptobut meh15:12
yoctozeptowell, anyhow15:13
wuchunya_ubuntu used 40%15:13
yoctozeptoin storage ceph is the king15:13
mnasiadkanothing new15:14
yoctozeptowuchunya_: yeah but it's for 2020 and centos+rhel = 4715:14
*** wuchunya_ is now known as wuchunyang15:14
yoctozeptobut hard to tell if they overlap or not15:14
yoctozeptoperhaps they do15:14
yoctozeptonowadays ubuntu would probably be getting a boost15:14
yoctozeptomany simply decided to migrate to ubuntu15:14
yoctozeptofor lolz: more windows than suse15:15
yoctozeptoall right, enough dwelling on stats15:15
yoctozepto#topic Review action items from the last meeting15:15
*** openstack changes topic to "Review action items from the last meeting (Meeting topic: kolla)"15:15
yoctozeptomgoddard email openstack-discuss about quay.io credentials15:16
yoctozeptohe did not15:16
yoctozepto#action mgoddard email openstack-discuss about quay.io credentials15:16
yoctozeptoyoctozepto deprecate enable_host_ntp15:16
yoctozeptohe did not either15:16
yoctozepto#action yoctozepto deprecate enable_host_ntp15:16
yoctozepto#topic CI status15:16
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:16
yoctozeptoI was not angry at CI so it must be pretty green15:17
yoctozeptoI like quay so far15:17
yoctozeptoit only failed once since15:17
yoctozeptomove to Bullseye needs mariadb work15:17
yoctozepto^ that has happened already15:17
yoctozeptoremoving15:17
mnasiadkayep, we're waiting for hrw to test on aarch15:18
yoctozeptonobody cares about monasca-grafana I guess?15:18
mnasiadkaseems so15:19
mnasiadkaare we deprecating in W or in X?15:19
yoctozeptokayobe looks green too15:19
yoctozeptomnasiadka: would ask dougsz15:19
yoctozeptobut I love to deprecate early15:20
yoctozeptoand drop early :-)15:20
yoctozeptoit's a monstrosity anyway15:20
mnasiadkaWill ask when I'll see him online15:20
* yoctozepto feels bad recommending it15:20
mnasiadkahttps://review.opendev.org/c/openstack/kolla/+/78490115:21
mnasiadkawe have this, seems there's consensus15:21
yoctozeptomnasiadka: does your cephadm fix helps with that random issue mentioned on the whiteboard?15:21
*** k_mouza has quit IRC15:21
*** k_mouza_ has joined #openstack-kolla15:21
mnasiadkayoctozepto: well, I haven't seen it in there, but I'll do a couple of rechecks to be sure.15:22
yoctozeptomnasiadka: thanks15:22
yoctozeptolet me check that monasca-grafana15:22
yoctozepto+215:23
yoctozeptomnasiadka: you +2 too?15:25
mnasiadkasure, I'll try to ask Doug if he needs help in K-A deprecation15:25
yoctozepto#topic Review requests15:26
*** openstack changes topic to "Review requests (Meeting topic: kolla)"15:26
yoctozeptomnasiadka: very well, thanks!15:26
yoctozeptowhat do you want reviewed?15:26
yoctozeptoyour favourite plant?15:26
yoctozeptoyour favourite meal?15:26
headphoneJameshttps://review.opendev.org/c/openstack/kolla-ansible/+/741340/15:26
yoctozeptoI'm here to grant you your wish15:26
yoctozeptoah yes15:27
yoctozepto+96615:27
yoctozepto-1815:27
headphoneJamesI also enjoy orchids15:27
yoctozeptoand now I know why it is still not merged :-)15:27
yoctozeptoorchids can be pretty15:27
mnasiadkaheadphoneJames: well, my main issue is with supervisord (so multiple processes) in haproxy image... can we do anything to not do it?15:28
headphoneJamesI'm open to other approaches.15:29
mnasiadkaI'll try to look into, but that might happen in two weeks time...15:30
headphoneJamesThe issue is that we need a process that monitors for new certificates, and causes an haproxy reload15:30
mnasiadkaor we need a newer haproxy, which is a crazy thing to do - because we would need additional repos...15:30
headphoneJamesI have a suspicion this is not going in the current  release, so two weeks should be fine15:30
*** hrw has joined #openstack-kolla15:31
hrwmorning15:31
yoctozeptoyou might be right, headphoneJames, I am sorry15:31
yoctozeptoI really wished to get this in this cycle :/15:31
mnasiadkawe could run a separate container with shared ipc namespace15:31
headphoneJamess'all good15:31
mnasiadkaheadphoneJames: thought about that?15:31
headphoneJamesI certainly have not. I would have to learn with that means ;)15:32
mnasiadkaheadphoneJames: https://docs.docker.com/engine/reference/run/#ipc-settings---ipc - look at shareable15:32
mnasiadkaprobably we could then restart haproxy from another container15:32
yoctozeptowe could send a signal15:33
priteauIs there a less automated version that could be merged in wallaby, with a manual command to update certificates and reload haproxy?15:33
mnasiadkabut what happens when you restart haproxy and if it joins the same ipc group, that is a mystery :)15:33
yoctozeptobut that would need share pid namespace, no?15:33
mnasiadkayoctozepto: but it doesn't need to be host namespace, it seems in Docker you can create a ,,shareable'' one15:33
yoctozeptopriteau: yeah, my thoughts exactly15:33
yoctozeptoI remember we discussed some layering to this15:33
yoctozeptomnasiadka: I wonder what the lifetime is ;-)15:34
yoctozeptoas you need that PID 115:34
yoctozeptoit would be fugly to have either break the other15:34
yoctozeptoperhaps it's better to push newer haproxy for this15:34
mnasiadkayoctozepto: that's why I mentioned a mystery, meaning it would need some degree of testing15:34
yoctozeptoheadphoneJames: do you have an answer for priteau?15:34
mnasiadkayoctozepto: both centos and ubuntu would need a repo15:34
yoctozeptoeven current ubuntu? doh15:35
mnasiadkayoctozepto: which in centos case there's a nfv repo - but for Ubuntu it's just somebody's PPA15:35
yoctozeptowe need15:35
yoctozepto2.2?15:35
*** hrw has quit IRC15:35
headphoneJamesWe can definitely push a version that doesn't do the automatic restart, and would require customers to restart ha proxy manually15:36
*** hrw has joined #openstack-kolla15:36
openstackgerritMerged openstack/kolla master: centos: enable repos where needed  https://review.opendev.org/c/openstack/kolla/+/78587115:36
mnasiadkayoctozepto: ideally 2.215:36
headphoneJamesHowever, the test case would need to then manually restart haproxy15:36
yoctozeptoheadphoneJames: let's do it in another change then15:37
yoctozeptoperhaps we can merge that15:37
yoctozeptomnasiadka: meh15:37
mnasiadkaheadphoneJames: well, how do we notify them to restart haproxy? :) and how often do they need to do that?15:37
headphoneJamesThey would have to check logs :)15:37
mnasiadkayoctozepto: yeah, so centos nfv repo and ubuntu PPA https://launchpad.net/~vbernat/+archive/ubuntu/haproxy-2.2 - no other option with Focal15:37
headphoneJamesKeep in mind supervisord is only deployed and used with the let's encrypt option enabled15:38
headphoneJamesAnd works really well over several months of testing15:38
mnasiadkaI don't doubt it does work, but we don't run more than one process in any container image :)15:38
headphoneJamesAnyhow, my expectations are properly set for next release15:39
yoctozeptoI made a few updates to https://ethercalc.openstack.org/kolla-infra-service-matrix15:40
yoctozeptoI assumed we move to bullseye15:40
yoctozeptoheadphoneJames: how long will it take you to simplify it as we discussed?15:41
*** hrw has quit IRC15:41
*** samcat116 has quit IRC15:41
headphoneJamesI can easily remove the supervisord part.15:42
*** hrw has joined #openstack-kolla15:42
headphoneJamesI will look into the shared ipc today15:42
priteaumnasiadka: don't look at the bifrost image then ;-)15:43
yoctozeptopriteau: bifrost is on the death path15:43
headphoneJamesHaving customers manually restart haproxy is probably not the right approach15:43
yoctozeptojust no executioners around15:43
yoctozeptoheadphoneJames: yes, it's not15:43
yoctozeptobut I have a feeling we are better off merging it in parts rather than postponing till the end of the world15:44
headphoneJamesLet me see how far I get today. I'm happy to discuss breaking it into smaller chunks15:45
yoctozeptoall right, ping us whenever15:45
*** wuchunyang has quit IRC15:46
yoctozeptoooh, I have a last minute announcement15:46
yoctozepto#topic Last minute announcements15:46
*** openstack changes topic to "Last minute announcements (Meeting topic: kolla)"15:46
yoctozepto#info OpenStack Wallaby released! yahoo!15:46
hrwyay!15:46
mnasiadkapriteau: bifrost is a monster15:47
hrwso when we branch?15:47
yoctozeptothat's the question for ptg15:47
yoctozepto#topic Wallaby release planning15:47
*** openstack changes topic to "Wallaby release planning (Meeting topic: kolla)"15:47
mnasiadkaI think we can branch Kolla once we merge all hrw's patches ;) (UCA/Debian/RDO)15:47
yoctozeptoDebian bullseye progress15:47
yoctozeptoI think it's only pending a review15:47
yoctozeptohttps://review.opendev.org/c/openstack/kolla/+/77247915:47
yoctozeptoI pasted it on the whiteboard already15:48
yoctozeptoanything to discuss?15:48
*** samcat116 has joined #openstack-kolla15:48
*** hrww has joined #openstack-kolla15:49
mnasiadkawell, since Debian is now on good path, I guess it's time to look into UCA/RDO changes to get them merged and images built15:49
hrwwI will read meeting log :(15:49
mnasiadkaI'll try to look into those15:49
hrwwUCA is upgrade V->W issue iirc15:50
mnasiadkaupgrade is always a release change issue :)15:50
*** e0ne has quit IRC15:50
*** e0ne has joined #openstack-kolla15:51
*** hrw has quit IRC15:51
mnasiadkaah, looking for somebody brave enough to merge this: https://review.opendev.org/c/openstack/kolla-ansible/+/78366915:51
*** hrw has joined #openstack-kolla15:52
* hrw drops - too many lags and reconnects today to be able to follow discussion15:53
hrwupc--15:53
mnasiadkayoctozepto: anything else left on agenda? because it seems I'm speaking with myself...15:53
yoctozeptocheck the recorded logs hrw15:53
*** e0ne has quit IRC15:53
yoctozeptomnasiadka: open discussion15:54
mnasiadkaok then15:54
yoctozepto#topic Xena PTG planning https://etherpad.opendev.org/p/kolla-xena-ptg15:54
*** openstack changes topic to "Xena PTG planning https://etherpad.opendev.org/p/kolla-xena-ptg (Meeting topic: kolla)"15:54
yoctozeptoit was mentioned already15:54
yoctozeptoplease add yourself and your discussion topics15:54
*** hrww has quit IRC15:55
yoctozepto#topic Open discussion15:55
*** openstack changes topic to "Open discussion (Meeting topic: kolla)"15:55
yoctozeptomnasiadka wanted to participate in open discussion15:56
mnasiadkanaah, I speak too much and mainly to myself, it doesn't sound like an open discussion ;)15:56
mnasiadkaand we will have too much time for discussions next week15:56
*** hrw has quit IRC15:57
yoctozeptomnasiadka: well, it's open-ended then!15:57
yoctozeptobut you are right15:57
yoctozeptowe are all tired simply by thinking how tired we will be after the ptg15:58
yoctozeptothat said15:58
yoctozeptothank you for attending15:58
yoctozepto#endmeeting15:58
*** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"15:58
openstackMeeting ended Wed Apr 14 15:58:28 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:58
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-04-14-15.00.html15:58
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-04-14-15.00.txt15:58
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-04-14-15.00.log.html15:58
headphoneJamesthx!15:58
*** hrw has joined #openstack-kolla15:58
*** samcat116 has quit IRC16:00
*** hrw has quit IRC16:14
*** hrw has joined #openstack-kolla16:15
*** samcat116 has joined #openstack-kolla16:17
openstackgerritMerged openstack/kolla master: deprecate monasca-grafana image  https://review.opendev.org/c/openstack/kolla/+/78490116:20
*** hrw has quit IRC16:21
*** rpittau is now known as rpittau|afk16:21
*** hrw has joined #openstack-kolla16:22
*** gfidente is now known as gfidente|afk16:25
*** hrww has joined #openstack-kolla16:26
hrwwok. now network should work16:26
*** hrw has quit IRC16:26
*** hrww is now known as hrw16:26
*** k_mouza_ has quit IRC16:32
hrwricolin: -1 on your job removal patch16:35
*** fuhrmannb has quit IRC16:38
*** bengates has quit IRC16:38
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: centos: clean up repo enable/disable/keys  https://review.opendev.org/c/openstack/kolla/+/76586416:40
*** bengates has joined #openstack-kolla16:40
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: switch to RDO final release of Wallaby  https://review.opendev.org/c/openstack/kolla/+/78562216:42
ricolinhrw, the image build issue solved already, I think we can abandon the job directly16:42
ricolinI mean abandon the patch :p16:43
hrwricolin: feel free16:43
ricolinhrw, done16:43
hrwricolin: next time ask why images are not building16:43
hrwricolin: it was lack of powertools repo in both failing images16:43
openstackgerritMerged openstack/kayobe stable/ussuri: Fix overcloud provision with fast track enabled  https://review.opendev.org/c/openstack/kayobe/+/78533216:44
hrwricolin: as I wrote in comment - I may not use most of aarch64 images but I atleast try to keep them buildable16:44
hrwricolin: that's why ubuntu/aarch64 CI job was dropped - it is not buildable/deployable due to too old erlang16:45
*** bengates has quit IRC16:45
*** k_mouza has joined #openstack-kolla16:46
*** ivan_lin has quit IRC16:49
*** ivan_lin has joined #openstack-kolla16:50
hrw    "msg": "timedatectl sees the system clock as unsynchronized. Please wait for synchronization."16:52
hrwhm.16:52
*** e0ne has joined #openstack-kolla16:55
ricolinhrw, thanks for these suggestions, actually learned some more infra stuff (like nb03.opendev.org ) through this chance.16:55
hrwricolin: base-jobs  diskimage-builder  openstack-zuul-jobs  project-config  system-config  zuul-jobs - check all of them ;D16:57
hrwmy ~/devel/linaro/openstack/Infra/ directory16:57
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247916:59
hrwadded back rabbitmq for debian/aarch6416:59
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: handle APT keys in proper way  https://review.opendev.org/c/openstack/kolla/+/78492317:02
hrwand here as well17:02
*** k_mouza has quit IRC17:06
ricolinhrw, will do! BTW, I upload the devstack arm64 job patch with your suggestion, all building wheels things are gone.17:06
ricolinStill try to fine out why volume backup/restore is so slow17:06
ricolinit takes like 5 mins to do one backup or restore action17:07
hrwglad to help17:07
*** cah_link has quit IRC17:08
*** k_mouza has joined #openstack-kolla17:10
*** samcat116 has quit IRC17:10
*** bengates has joined #openstack-kolla17:12
*** bengates_ has joined #openstack-kolla17:17
*** bengates has quit IRC17:17
hrwrabbitmq crashes on debian/aarch64 :(17:18
*** samcat116 has joined #openstack-kolla17:21
*** bengates_ has quit IRC17:21
*** k_mouza has quit IRC17:25
*** hrw has quit IRC17:30
*** hrw has joined #openstack-kolla17:32
*** hrw has quit IRC17:39
*** hrw has joined #openstack-kolla17:41
*** hrw has quit IRC17:49
*** bengates has joined #openstack-kolla17:52
*** e0ne has quit IRC17:56
*** andrewbonney has quit IRC17:59
*** ivan_lin has quit IRC18:04
*** ivan_lin has joined #openstack-kolla18:04
*** bengates has quit IRC18:05
openstackgerritMerged openstack/kolla-ansible master: Additional small changes in role/mariadb  https://review.opendev.org/c/openstack/kolla-ansible/+/78565718:07
*** k_mouza has joined #openstack-kolla18:24
*** k_mouza has quit IRC18:27
openstackgerritMerged openstack/kayobe stable/train: Revert "Performance: skip LVM configuration if no groups configured"  https://review.opendev.org/c/openstack/kayobe/+/78331618:36
*** vishalmanchanda has quit IRC18:44
*** jonaspaulo has joined #openstack-kolla18:51
*** hrw has joined #openstack-kolla19:02
*** k_mouza has joined #openstack-kolla19:28
*** k_mouza has quit IRC19:32
*** jonaspaulo has quit IRC19:33
openstackgerritMerged openstack/kolla-ansible master: masakari: support host monitor  https://review.opendev.org/c/openstack/kolla-ansible/+/74423520:02
*** samcat116 has quit IRC20:30
*** samcat116 has joined #openstack-kolla20:43
*** samcat116 has quit IRC21:13
*** jaicaa has quit IRC21:52
*** jaicaa has joined #openstack-kolla21:53
*** luksky has quit IRC22:34
*** luksky has joined #openstack-kolla22:34
*** luksky has quit IRC22:40
*** luksky has joined #openstack-kolla22:53
*** macz_ has quit IRC23:05
*** ivan_lin has quit IRC23:15
*** ivan_lin has joined #openstack-kolla23:15
*** luksky has quit IRC23:18
*** jovial[m] has quit IRC23:23
*** irclogbot_3 has quit IRC23:50
*** irclogbot_2 has quit IRC23:50
*** irclogbot_3 has joined #openstack-kolla23:55
*** irclogbot_2 has joined #openstack-kolla23:56

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!