Tuesday, 2019-12-17

« Monday, 2019-12-16 Index Wednesday, 2019-12-18 »
*** schwicht has quit IRC00:05
*** xinliang has joined #openstack-kolla01:02
*** goldyfruit has quit IRC01:14
*** schwicht has joined #openstack-kolla01:23
*** goldyfruit has joined #openstack-kolla01:29
*** goldyfruit has quit IRC01:43
*** k_mouza_ has quit IRC01:47
openstackgerritYongjun Bai proposed openstack/kolla-ansible master: Fix deploy mariadb failed to reach primary node  https://review.opendev.org/69917202:27
openstackgerritYongjun Bai proposed openstack/kolla-ansible master: Fix deploy mariadb failed to reach primary node  https://review.opendev.org/69917202:40
*** ricolin has joined #openstack-kolla03:03
iniaziHello guys, is there some doc/guide somewhere that talks about cleaning up previous openstack/kolla install/configs (on centos 7).  have some hosts that were running/configured with kolla + openstack.  it was all with rpms, etc., so maybe not something i need to worry about?  it was about 4 versions ago03:30
iniazioh i think there is a kolla command to cleanup.03:33
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Include a reference to the globally configured Certificate Authority to all services. Services use the CA to verify HTTPs connections.  https://review.opendev.org/69931203:37
openstackgerritDincer Celik proposed openstack/kolla-ansible master: WIP: Docker image pruning  https://review.opendev.org/69869403:42
*** k_mouza has joined #openstack-kolla03:47
*** k_mouza has quit IRC03:52
*** zainub_wahid has joined #openstack-kolla03:58
*** goldyfruit has joined #openstack-kolla04:01
*** xinliang has quit IRC04:02
openstackgerritDincer Celik proposed openstack/kolla-ansible master: WIP: Ansible max version check  https://review.opendev.org/69933304:24
*** goldyfruit has quit IRC05:00
*** factor has quit IRC05:05
*** shyamb has joined #openstack-kolla05:30
*** raukadah is now known as chkumar|rover05:33
*** xinliang has joined #openstack-kolla05:33
*** factor has joined #openstack-kolla05:34
*** shyamb has quit IRC05:36
*** shyamb has joined #openstack-kolla05:59
*** shyamb has quit IRC06:05
*** shyamb has joined #openstack-kolla06:05
*** ricolin_ has joined #openstack-kolla06:10
*** ricolin has quit IRC06:13
*** kozhukalov has joined #openstack-kolla06:59
*** ricolin_ is now known as ricolin07:00
*** jbadiapa has joined #openstack-kolla07:16
osmanlicilegimorning07:17
*** cah_link has joined #openstack-kolla07:23
hrwmorning07:26
hrwcloudnull: (enable_repos) you can also install repos in base image and ignore enable_repos functionality. This is also supported.07:28
hrwmjturek: it is a bit more complicated. we run create_dockerfiles() to create final dockerfiles, then build_image_list() uses those templates to build list of images to build. find_parents() switch flat list to tree and then filter_images() removes not needed entries from that tree.07:41
hrwmjturek: and build is using that tree to create build queue. each entry has path to dockerfile07:41
hrwmjturek: filter_images() also removes all not needed entries as you may want to build just subset of images (like 'nova ^grafana skydive').07:43
*** shyamb has quit IRC07:44
hrw08:46 (503s) hrw@puchatek:del$ find kolla-hrw-test -name "*j2"|wc -l07:46
hrw28007:46
hrw08:46 (8s) hrw@puchatek:del$ find kolla-hrw-test2 -name "*j2"|wc -l07:46
hrw23107:46
hrwall templates, buildable templates07:47
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: build: remove not buildable templates for --template-only  https://review.opendev.org/69936107:50
xinlianghrw: Thanks, the answer of https://review.opendev.org/#/c/699340/ makes thing clear. That's why it will take more time for pip install on aarch64 than x86_64 righ?07:50
patchbotpatch 699340 - magnum - Fix heat-container-agent image building error on a... - 2 patch sets07:51
hrwxinliang: yes07:51
hrwmjturek: https://review.opendev.org/699361 would be interesting for you07:51
patchbotpatch 699361 - kolla - build: remove not buildable templates for --templa... - 1 patch set07:51
xinlianghrw: that  makes me confusion for a while. Now i know why :-)07:52
*** pcaruana has joined #openstack-kolla07:52
hrwxinliang: https://github.com/pyca/cryptography/issues/508307:53
xinlianghrw: o/07:54
*** rpittau|afk is now known as rpittau08:05
openstackgerritDincer Celik proposed openstack/kolla-ansible master: WIP: Ansible max version check  https://review.opendev.org/69933308:12
openstackgerritDincer Celik proposed openstack/kolla-ansible master: WIP: Ansible max version check  https://review.opendev.org/69933308:14
yoctozeptomorning08:21
*** shyamb has joined #openstack-kolla08:24
mnasiadkamorning08:41
hrwyoctozepto: replied to you in review08:42
*** sri_ has joined #openstack-kolla08:47
*** shyamb has quit IRC08:53
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: add more tests for enable_repos template method  https://review.opendev.org/69735408:57
hrwrebased and applied mgoddard's comments08:57
*** priteau has joined #openstack-kolla08:57
hrw~kill monasca-grafana...08:59
*** priteau has quit IRC09:02
*** kevinz_ has joined #openstack-kolla09:02
*** dougsz has joined #openstack-kolla09:03
*** rphillips_ has joined #openstack-kolla09:03
*** priteau has joined #openstack-kolla09:03
*** skramaja_ has joined #openstack-kolla09:04
*** klindgren_ has joined #openstack-kolla09:04
*** jistr_ has joined #openstack-kolla09:05
*** dasp has joined #openstack-kolla09:05
*** benj_- has joined #openstack-kolla09:05
*** shyamb has joined #openstack-kolla09:07
*** sri_ has quit IRC09:10
*** kozhukalov has quit IRC09:10
*** klindgren has quit IRC09:10
*** osmanlicilegi has quit IRC09:10
*** baha has quit IRC09:10
*** tbarron has quit IRC09:10
*** skramaja has quit IRC09:10
*** jistr has quit IRC09:10
*** benj_ has quit IRC09:10
*** rphillips has quit IRC09:10
*** dasp_ has quit IRC09:10
*** andreykurilin has quit IRC09:10
*** _Cyclone_ has quit IRC09:10
*** kevinz has quit IRC09:10
*** markmcclain has quit IRC09:10
*** yankcrime has quit IRC09:10
*** kevinz_ is now known as kevinz09:12
*** benj_- is now known as benj_09:12
*** xinliang has quit IRC09:12
*** skramaja_ is now known as skramaja09:13
*** k_mouza has joined #openstack-kolla09:14
openstackgerritPierre Riteau proposed openstack/kayobe master: Fix Train release notes  https://review.opendev.org/69937609:22
openstackgerritPierre Riteau proposed openstack/kayobe master: Sync comment wth default value of pip_upper_constraints_file  https://review.opendev.org/69937709:28
openstackgerritPierre Riteau proposed openstack/kayobe master: Sync comment with default value of pip_upper_constraints_file  https://review.opendev.org/69937709:28
mgoddardmorning09:30
hrwhi boss09:32
yoctozeptohe ain't no boss, he a leader09:33
yoctozeptoProject Team Leader 8-)09:33
hrwmgoddard: https://review.opendev.org/697354 refreshed.09:33
patchbotpatch 697354 - kolla - add more tests for enable_repos template method - 9 patch sets09:33
mgoddardhrw: merci09:33
mgoddardyoctozepto: Project Team Boss09:34
mgoddardnow get back to work all of you :p09:34
yoctozeptorude09:34
dougsz:D09:35
hrwo dougsz!09:35
hrwdougsz: monasca-grafana - are there plans for finally retire it or sth?09:36
hrwdougsz: it now failed on ubuntu09:36
dougszhrw: it is proposed to do so this cycle in Monasca, but the work hasn't been done yet09:36
hrwok09:37
dougszI looked at the build logs for it on Centos yesterday and the list of deprecated and stagnant packages makes horrific reading09:37
*** jbadiapa has quit IRC09:37
hrwdougsz: that's the part I do not even wanted to start discussing ;D09:38
dougszI know, I know :)09:39
*** vmixor has joined #openstack-kolla09:39
hrwdougsz: I would vote for patch which just drops it from CI ;D09:46
dougszhrw: I think that is a realistic option given the state of it09:47
yoctozeptodeprecate everything09:47
yoctozeptodrop everything09:47
hrwyoctozepto: just base. easier to review09:47
yoctozeptohrw: right!09:47
yoctozeptohrw, dougsz: if that's so, I agree to remove building it on master09:48
yoctozeptowhy endure the pain09:48
*** jbadiapa has joined #openstack-kolla09:51
*** shyamb has quit IRC09:51
*** xinliang has joined #openstack-kolla09:57
dougszyoctozepto: I will look at the Ubuntu build later today and see how terminal it is, and report back09:57
openstackgerritPierre Riteau proposed openstack/kayobe-config master: Sync comment with default value of pip_upper_constraints_file  https://review.opendev.org/69938210:10
openstackgerritPierre Riteau proposed openstack/kayobe-config-dev master: Sync comment with default value of pip_upper_constraints_file  https://review.opendev.org/69938310:10
*** zainub_wahid has quit IRC10:11
hrwhttps://review.opendev.org/698875 https://review.opendev.org/697354 https://review.opendev.org/694797 anyone?10:14
patchbotpatch 698875 - kolla - docker: fix 'Empty continuation line found' warnings - 2 patch sets10:14
patchbotpatch 697354 - kolla - add more tests for enable_repos template method - 9 patch sets10:14
patchbotpatch 694797 - kolla-ansible - Some ansible-lint fixes - 12 patch sets10:14
*** shyamb has joined #openstack-kolla10:15
*** yankcrime has joined #openstack-kolla10:15
openstackgerritMerged openstack/kayobe-config-dev master: Synchronise with kayobe-config for Train  https://review.opendev.org/69926110:17
openstackgerritViktor Michalek proposed openstack/kolla-ansible master: Adds support of CADF notifying for other APIs next to keystone.  https://review.opendev.org/67457910:22
*** xinliang has quit IRC10:22
*** shyam89 has joined #openstack-kolla10:25
*** shyamb has quit IRC10:27
openstackgerritMerged openstack/kayobe master: Fix Train release notes  https://review.opendev.org/69937610:31
*** priteau has quit IRC10:38
openstackgerritMerged openstack/kayobe-config master: Sync comment with default value of pip_upper_constraints_file  https://review.opendev.org/69938210:42
yoctozeptomgoddard, mnasiadka: any good advice to offer Taisto? https://bugs.launchpad.net/kolla-ansible/+bug/185628110:52
openstackLaunchpad bug 1856281 in kolla-ansible "deploy fails at rabbitmq because epmd does not start" [Undecided,Incomplete]10:52
mgoddardyoctozepto: I wondered about firewalls at one point10:52
yoctozepto(I have no children so cannot say)10:52
yoctozeptohmm, yeah, that could be it but then running epmd in kolla-toolbox helped10:53
yoctozeptopuzzling failure10:53
yoctozeptostrace clearly show epmd starting up10:53
*** kozhukalov has joined #openstack-kolla10:56
openstackgerritViktor Michalek proposed openstack/kolla-ansible master: Adds support of CADF notifying for other APIs next to keystone.  https://review.opendev.org/67457911:07
yoctozeptomgoddard: https://bugs.launchpad.net/kolla-ansible/+bug/1856281/comments/26 I hope I am wrong11:07
openstackLaunchpad bug 1856281 in kolla-ansible "deploy fails at rabbitmq because epmd does not start" [Undecided,Incomplete]11:07
mgoddardyoctozepto: sounds like a line from a disaster movie11:07
yoctozeptomgoddard: indeed it is considering the train release with such a bug ;p11:08
yoctozeptowhich fails with IPv4-only :D11:08
mgoddardwhat has changed that could have caused that?11:08
yoctozeptomgoddard: no idea, all IPv6 stuff is conditional11:08
yoctozeptoIPv4-only but without IPv6 disabled passes11:09
yoctozeptoit's a wild guess he disabled IPv6 in such a way as to break epmd11:09
yoctozeptobut no simple idea why11:09
yoctozeptoerlang ecosystem is weird11:09
openstackgerritViktor Michalek proposed openstack/kolla-ansible master: Adds support of CADF notifying for other APIs next to keystone.  https://review.opendev.org/67457911:12
mgoddardhrw: when you said ~kill monasca-grafana earlier, were you trying to announce that the gate is broken, due to monasca-grafana builds on ubuntu?11:12
*** shyam89 has quit IRC11:12
mgoddardhttps://d879b1d41db92bf012a3-2bffcf55b823b0a6b0373b3bb2b87412.ssl.cf1.rackcdn.com/697354/9/check/kolla-build-ubuntu-binary/623faf9/kolla/build/000_FAILED_monasca-grafana.txt.gz11:12
yoctozeptoand we are running non-IPv6 friendly erlang there, it could have lots of bugs with ipv6 integration in the first place11:12
openstackgerritMerged openstack/kolla-cli stable/train: Update .gitreview for stable/train  https://review.opendev.org/69926211:12
openstackgerritMerged openstack/kolla-cli stable/train: Update TOX/UPPER_CONSTRAINTS_FILE for stable/train  https://review.opendev.org/69926311:18
openstackgerritMerged openstack/kolla-cli master: Update master for stable/train  https://review.opendev.org/69926411:18
Blinkizyoctozepto: Hi. Remember I tried a couple of days ago to get IPv6 prefix delegation to work? It never worked so I have nothing to give back at the moment. I gave up today and open a bug against neutron. https://bugs.launchpad.net/neutron/+bug/185667511:23
openstackLaunchpad bug 1856675 in neutron "IPv6 Prefix Delegation do not work" [Undecided,New]11:23
yoctozeptoBlinkiz: yeah, I remember11:23
yoctozeptook, will follow it11:23
yoctozeptoBlinkiz: analyzed and replied11:25
yoctozeptoI see you have the Bash Warrior Set applied11:26
yoctozeptoset -o errexit11:26
yoctozeptoset -o pipefail11:26
yoctozeptoset -o nounset11:26
yoctozeptoset -o xtrace11:26
yoctozepto;D11:26
BlinkizI just trying to follow common practices :-)11:27
BlinkizBash Warrior Set.. Never heard that expression before :-P11:28
yoctozeptoBash Warrior Set, Bash Knight Oath, ...11:29
yoctozeptoI promise to exit on errors11:29
yoctozeptoI promise to keep my pipes clean11:29
yoctozeptoI promise to refrain from acting upon unset variables11:30
BlinkizNice. Have you come up with this?11:30
yoctozeptoI will trace my all my actions, so help me Unix11:30
yoctozeptoyeah, I'm making this up in real time :-)11:30
mgoddardDeveloper rule #1: don't use bash :)11:31
yoctozepto^11:31
BlinkizThat is really good. Sweet.. You should go professional with Bash Warror Set. Spread your Ideology11:31
yoctozeptoDeveloper rule #2: shoot your friend suggesting PowerShell as an alternative11:32
yoctozepto:D11:32
mgoddards/Should I use bash?/Can I use python?/11:32
yoctozeptoBlinkiz: nah, I'm good at making other things up as well, wrote better11:33
yoctozeptoand in all seriousness11:33
mgoddardI regret not using python for almost every bash script I write that grows beyond 20 lines11:33
yoctozeptothese 4 lines really help11:33
BlinkizOkay.. Do not use bash.. I did wonder about that when I notice most scripts in /tests/ are in bash. Why not use Python?11:33
yoctozeptomgoddard: unless it has close to no logic11:33
mgoddardbut +2, if you must use bash then use the warrior set11:33
yoctozeptoBlinkiz: /we lazy11:34
yoctozeptowe should really have them in proper ansible and/or python depending on case11:34
Blinkizgot it11:35
*** osmanlicilegi has joined #openstack-kolla11:37
mnasiadkabash warrior set... wonder what is the python warrior set11:38
openstackgerritAndrey Kurilin proposed openstack/kolla master: Fix building sensu-client by pinning network-checks version  https://review.opendev.org/69939511:38
yoctozeptomnasiadka: well, python already incorporates first 3 by default, and you get 4th by debugging11:41
mnasiadkayoctozepto: I was just laughing11:42
yoctozeptomnasiadka: ha-ha-ha11:42
yoctozepto:D11:42
*** shyamb has joined #openstack-kolla11:45
hrwmgoddard: I want to kill monasca-grafana each time I look into it's log11:47
yoctozeptohrw: stop looking11:47
mgoddardbut seriously, the kolla gate is broken11:49
yoctozeptoI said disable building11:49
yoctozepto:-)11:49
dougszI'll look at it after lunch11:49
mgoddardthanks dougsz11:49
yoctozeptoThis is an error with npm itself. Please report this error at: http://github.com/npm/npm/issues11:51
yoctozeptothat's what the error logs say11:51
dougszI guess that is why there are 2167 open issues11:51
yoctozeptotypeerror at fetchPackageMetadata11:52
yoctozeptowell, the ^ does really make sense if npm itself is to blame11:52
yoctozeptowhere do we get npm from?11:53
yoctozeptoah, repos11:53
dougszyeah11:53
yoctozeptoINFO:kolla.common.utils.monasca-grafana:Get:170 http://mirror.gra1.ovh.openstack.org/ubuntu bionic/universe amd64 npm all 3.5.2-0ubuntu4 [1,586 kB]11:53
yoctozeptopff11:53
yoctozeptoit's an old sh*t11:54
yoctozeptono wonder it broke11:54
yoctozeptocannot parse current metadata11:54
dougszI think it can upgrade itself like pip11:54
yoctozeptodougsz: if it can read its own metadata that is xD11:56
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: add more tests for enable_repos template method  https://review.opendev.org/69735411:57
openstackgerritRadosław Piliszek proposed openstack/kolla master: Fix monasca-grafana building due to old npm  https://review.opendev.org/69939911:59
yoctozeptolet's have it checked in the CI then12:00
*** shyamb has quit IRC12:00
hrwyoctozepto: nice change12:00
yoctozeptoI think I forgot sudo though12:00
yoctozeptoor is the build as root?12:01
yoctozeptook, as root, uff12:01
yoctozeptomgoddard, mnasiadka, hrw, dougsz: I'm running bets12:07
yoctozepto1) it will succeed12:07
yoctozepto2) it will fail to parse metadata12:07
yoctozepto3) it will fail due to old node.js12:07
yoctozeptoplease bet before it builds12:07
yoctozepto:-)12:08
mnasiadka4) you will get a mental disease just from trying to fix it12:08
hrw5) will fail due to amount of obsolete/deprecated modules12:08
yoctozeptoI have systems on node js in production :D12:08
*** jbadiapa has quit IRC12:09
yoctozeptomy mental health is already impacted12:09
hrwyoctozepto: you do not have to say. we know that you are insane without it12:09
mnasiadkabut I agree with hrw, monasca-grafana is like a closet closed by 7 locks hosting the most ugly troll with a big sign "DON'T OPEN"12:09
mnasiadkaand it can't be unseen12:09
yoctozeptomnasiadka: +212:09
yoctozeptologs make me cringe, but the error message was quite clear now12:09
yoctozepto"no because no"12:09
yoctozeptotypical12:09
hrwhow many of you noticed arm64 jobs in recent checks?12:11
hrwI am tempting to fake some change just to see it failing ;D12:11
yoctozeptohrw: I did12:12
yoctozeptoglad it builds12:12
hrwand takes just 1h12:12
yoctozeptoand who is insane?12:12
*** kplant has joined #openstack-kolla12:12
yoctozeptoyeah, it takes less than the others' total12:12
hrwyoctozepto: it;s called 'testing'12:12
yoctozeptonope, it comes under masochism there12:13
yoctozepto6) it managed to install it but did it elsewhere and failed with the same error12:18
yoctozeptohere is the winner12:18
yoctozeptowe all lost12:19
*** jbadiapa has joined #openstack-kolla12:19
yoctozeptoI guess it needs local inspection then, meh12:20
yoctozeptohrw, mgoddard: I'm guessing all kolla branches are affected?12:26
mgoddardyoctozepto: probably12:29
yoctozepto:/12:29
hrwgit rm -r docker/monasca/monasca-grafana; git commit -m"monasca-grafana: get rid due to build failures";git review12:32
*** goldyfruit has joined #openstack-kolla12:41
yoctozeptoBash scripting12:43
*** dave-mccowan has joined #openstack-kolla12:44
openstackgerritViktor Michalek proposed openstack/kolla-ansible master: Adds support of CADF notifying for other APIs next to keystone.  https://review.opendev.org/67457912:48
openstackgerritMerged openstack/kayobe-config-dev master: Sync comment with default value of pip_upper_constraints_file  https://review.opendev.org/69938312:50
*** gfidente has joined #openstack-kolla12:54
hrwyoctozepto: scripting?12:55
hrwyoctozepto: https://review.opendev.org/697354 waits for you. or for mnasiadka12:56
patchbotpatch 697354 - kolla - add more tests for enable_repos template method - 10 patch sets12:56
openstackgerritRadosław Piliszek proposed openstack/kolla master: Fix monasca-grafana building due to old npm  https://review.opendev.org/69939912:58
yoctozeptohrw: well, it's not really going to merge12:59
yoctozeptobut you have my blessing12:59
hrwyoctozepto: it will merge. just later12:59
yoctozeptoyeah12:59
yoctozeptomuch later12:59
* hrw out13:00
dougszyoctozepto: thanks for trying that - got a meeting now, but i'll try a local build right after that13:00
*** mixor has joined #openstack-kolla13:02
*** vmixor has quit IRC13:02
yoctozeptonow I see ubuntu fixed but centos broke since it somehow cannot upgrade itself in-place argh13:09
*** cah_link has quit IRC13:09
yoctozeptowhat a mess13:10
mnasiadkawe need a third distro, that will be totally different from centos and ubuntu, so it can also break ocassionally ;)13:12
yoctozeptoanyone aware why we are not using grafana binaries? https://grafana.com/grafana/download13:13
kplantmnasiadka: how about windows??13:13
mnasiadkakplant: well, we have support for Cloudbase stuff I think ;)13:14
yoctozeptodougsz, mgoddard: why are we not using grafana binaries? https://grafana.com/grafana/download13:15
yoctozeptothen no need to build grafana13:15
dougszyou won't like this, but the monasca-grafana image is a fork of Grafana13:15
dougszto add keystone integration13:15
yoctozeptoah, missed that13:15
dougszIt is not developed really, so you could potentially just host the binaries somewhere13:16
mnasiadkafork of version 4 if I remember correctly13:16
dougsztry not to13:16
dougszbut yes13:16
mnasiadkadougsz: so if there is a security bug in Grafana that affects version 4 - is anybody fixing that in monasca-grafana? :)13:16
yoctozeptodougsz: first I would have to get the binaries13:16
dougszmnasiadka: no13:17
mnasiadkagood to know13:17
yoctozeptosomehow it's managed up to the point of breaking npm metadata :-)13:17
mnasiadkadougsz: so... do we really need that keystone integration? or is there some other reason to use this old Grafana fork?13:17
yoctozeptoif only I get an idea how to fix centos without breaking it further... I can always condition on being in ubuntu13:18
dougszThat is the main reason - you log in with you OpenStack crendentials13:18
yoctozeptobackport and remove on master13:18
*** cah_link has joined #openstack-kolla13:18
dougszYou can use vanilla grafana with Monasca, but you need to manage accounts separately from Keystone, unless you have something like LDAP driving the whole thing13:18
dougszAnd then you need to do an openstack token issue to get the datasource to work, which is one option13:19
dougszIf there is any time, the plan is to try app credentials this cycle with vanilla grafana13:19
dougszWe might need this: https://github.com/monasca/grafana/pull/1113:21
yoctozeptoor nodejs is much older, don't worry13:25
yoctozeptoour*13:25
openstackgerritMerged openstack/kayobe master: Sync comment with default value of pip_upper_constraints_file  https://review.opendev.org/69937713:27
mnasiadkadougsz: well, we deploy normal grafana without keystone authentication, so probably monasca-grafana can live with the same13:29
dougszmnasiadka: Not really because the whole point is that is that's multi-tenant13:30
yoctozeptothe problem is flaky dependence on multiple environments13:31
yoctozeptoflaky ruby, flaky node13:31
dougsz*is that it's mu..13:31
yoctozeptoone this, one that13:31
kplantso i did mention that my builds were exploding yesterday, seems they still are. here's one example: http://paste.openstack.org/show/787668/13:36
kplantthat's from stable/train13:36
*** mixor has quit IRC13:37
yoctozeptosorry, I forgot about flaky go13:37
*** wuchunyang has joined #openstack-kolla13:40
dougszFWIW, I think it builds a .deb and an .rpm regardless of the distro, so if it builds on centos, we could just host both those binaries. Not sure where though13:41
*** _Cyclone_ has joined #openstack-kolla13:41
*** wuchunyang has quit IRC13:42
openstackgerritRadosław Piliszek proposed openstack/kolla master: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69939913:42
yoctozeptodougsz, hrw, mnasiadka, mgoddard: ^13:42
yoctozeptotake that, you grafana13:43
yoctozeptobuilds here locally13:43
mnasiadkaI'm not even taking a peek, this is mental health unsafe :)13:43
yoctozeptomnasiadka: it's actually very nice13:44
yoctozeptobbl13:44
openstackgerritMark Goddard proposed openstack/kolla master: CentOS 8: work around mod_ssl certificate issue  https://review.opendev.org/69844513:44
openstackgerritMark Goddard proposed openstack/kolla master: WIP: move to CentOS 8 in containers  https://review.opendev.org/69236813:44
openstackgerritMark Goddard proposed openstack/kolla master: WIP: Remove support for CentOS 7  https://review.opendev.org/69245013:44
openstackgerritMark Goddard proposed openstack/kolla master: CentOS 8: base and openstack-base images  https://review.opendev.org/69941313:44
openstackgerritMark Goddard proposed openstack/kolla master: CentOS 8: Update packages in images  https://review.opendev.org/69941413:44
yoctozeptolol, what a spam13:44
yoctozeptomgoddard: are you a bot?13:44
mgoddardyoctozepto: yes13:45
*** mixor has joined #openstack-kolla13:47
kplantany ideas on what's causing my image builds to fail?13:49
*** vmixor has joined #openstack-kolla13:51
*** mixor has quit IRC13:51
hrwkplant: wind on Artcic today flows into wrong direction.13:55
hrwkplant: it also caused grafana failure13:55
kplantfoiled again13:55
openstackgerritMerged openstack/kolla-ansible stable/train: [neutron] Adjust neighbour table thresholds  https://review.opendev.org/69918913:56
openstackgerritMerged openstack/kolla-ansible stable/rocky: [neutron] Adjust neighbour table thresholds  https://review.opendev.org/69919113:56
openstackgerritMerged openstack/kolla-ansible stable/stein: [neutron] Adjust neighbour table thresholds  https://review.opendev.org/69919013:57
hrwmgoddard: nice work on centos8 patches. thanks!14:12
mgoddardhrw: thank you. Planning to write a short spec on migration too14:12
hrwmgoddard: left some comments. mostly missing commas14:13
hrwcontrary to yoctozepto's commas I care about middle list ones ;D14:13
mgoddardthose are important14:14
mgoddardnon-negotiable14:14
yoctozeptocomma fetish14:14
yoctozeptocome on, be tolerant14:14
hrwLGBTC14:16
hrwmgoddard: have to check how dnf reacts to those lacks14:17
hrwuf. errors out14:18
hrwproper behaviour by default. contrary to yum14:19
hrwcheck-arm64 gets nice use. 6/8 nodes are occupied by us now14:25
hrwmgoddard: shouldn't 'remove centos7' patch depend on 'move to centos8' one?14:27
hrwah, ignore - they are in set...14:27
mgoddardhrw: it's a patch chain14:27
mgoddarddepends-on doesn't work when they touch the same code14:28
hrwreviewing14:30
hrw+2 to simple ones14:33
*** factor has quit IRC14:39
mjturekhrw: I'm still a bit confused on https://review.opendev.org/#/c/699361/ don't UNBUILDABLE_IMAGES already get skipped? https://github.com/openstack/kolla/blob/master/kolla/image/build.py#L1381-L138714:44
patchbotpatch 699361 - kolla - build: remove not buildable templates for --templa... - 1 patch set14:44
mjturekis the intent to remove unbuildable images before we even reach here?14:45
*** TrevorV has joined #openstack-kolla14:47
openstackgerritMark Goddard proposed openstack/kolla master: WIP: Remove support for CentOS 7  https://review.opendev.org/69245014:48
mjturekhrw: more details in comments https://review.opendev.org/#/c/699361/14:52
patchbotpatch 699361 - kolla - build: remove not buildable templates for --templa... - 1 patch set14:52
*** kozhukalov has quit IRC15:07
openstackgerritViktor Michalek proposed openstack/kolla-ansible master: Support for custom api-paste files  https://review.opendev.org/69505415:11
openstackgerritViktor Michalek proposed openstack/kolla-ansible stable/train: Neutron: Added loading ip6_tables kernel module  https://review.opendev.org/69943615:14
mgoddardcloudnull: hi, we're seeing some rhel build failures: http://logs.rdoproject.org/45/698445/4/openstack-check/tripleo-build-containers-rhel-8-master/bfaf3a5/logs/buildah-builds/kolla-fa5lz_c6/docker/haproxy/haproxy-build.log.txt.gz15:16
*** dougsz has quit IRC15:19
* cloudnull looking 15:24
*** wuchunyang has joined #openstack-kolla15:26
cloudnullthat's probably ok for now. its likely that the repos are out of sync. that said, I'll ping the job maintainers and see what's what15:26
chkumar|rovercloudnull: mgoddard poked rdo people they are looking into it15:29
cloudnullchkumar|rover ++ thanks!15:29
*** wuchunyang has quit IRC15:32
mgoddardthanks chkumar|rover + cloudnull15:33
chkumar|rovermgoddard: we are discussing currently on #rdo channel15:34
*** dougsz has joined #openstack-kolla15:36
*** skramaja has quit IRC15:36
*** wuchunyang has joined #openstack-kolla15:39
*** wuchunyang has quit IRC15:43
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Check for supported OS distributions in prechecks  https://review.opendev.org/69944315:48
*** wuchunyang has joined #openstack-kolla15:50
*** chkumar|rover is now known as raukadah15:50
openstackgerritDoug Szumski proposed openstack/kayobe master: Support custom Kolla group_vars  https://review.opendev.org/69888215:53
openstackgerritDoug Szumski proposed openstack/kayobe master: Support customising nova-conductor group for cells  https://review.opendev.org/69897015:53
openstackgerritDoug Szumski proposed openstack/kayobe master: Cells documentation  https://review.opendev.org/69897115:53
*** wuchunyang has quit IRC15:54
yoctozeptohrw: https://review.opendev.org/#/c/699399/3 <- did aarch64 hang on it?15:55
patchbotpatch 699399 - kolla - Fix monasca-grafana being unbuildable due to old npm - 3 patch sets15:55
yoctozeptoah no, it's close15:56
yoctozepto2019-12-17 15:54:26.163816 | primary |  - Worker 0 (1 tests) => 2:03:21.34302815:56
yoctozeptowell, took some time huh15:56
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Add Ansible group check to prechecks  https://review.opendev.org/69944515:57
openstackgerritDoug Szumski proposed openstack/kayobe master: Support custom Kolla group_vars  https://review.opendev.org/69888215:57
openstackgerritDoug Szumski proposed openstack/kayobe master: Support customising nova-conductor group for cells  https://review.opendev.org/69897015:57
openstackgerritDoug Szumski proposed openstack/kayobe master: Cells documentation  https://review.opendev.org/69897115:57
yoctozepto2h 15min - I guess we praised too early15:58
raukadahmgoddard: it is fixed now, please check-rdo in sometime15:59
kplantso the first element of the image.plugins being passed here https://github.com/openstack/kolla/blob/stable/train/kolla/image/build.py#L572 is null16:00
kplantany idea why that would be? that seems to be the source of my failures16:00
openstackgerritRadosław Piliszek proposed openstack/kolla master: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69939916:01
*** vmixor has quit IRC16:03
openstackgerritRadosław Piliszek proposed openstack/kolla master: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69939916:04
yoctozeptomgoddard, mnasiadka, hrw, dougsz: all right, make it happen ^16:05
yoctozeptokplant: as the name suggests - plugins16:05
yoctozeptowhere does it fail?16:05
kplanthttp://paste.openstack.org/show/787668/16:05
kplantthere's a few things that fail this way, horizon is just the quickest failure16:06
*** gkadam has joined #openstack-kolla16:06
*** baha has joined #openstack-kolla16:06
yoctozeptoyeah, it uses plugins16:09
yoctozeptohow did you install new kolla?16:09
kplanti just switched branches from stein to train, pip install -r requirements.txt16:10
kplantand tried a build16:10
yoctozeptohmm, odd16:10
yoctozeptoare you using a custom config?16:11
kplantooo, i did copy kolla-build.conf into /etc/kolla because i was going to16:12
kplantbut i never changed it16:12
kplantthat's probably an issue16:12
cosmicsoundanyone has seen this or know some fix for it? FAILED - RETRYING: Waiting for nova-compute services to register themselves (3 retries left).16:12
yoctozeptocould be, still, the error could be better16:12
cosmicsoundi added a new server to multinode16:13
yoctozeptocosmicsound: check the nova logs16:13
kplantyeah there's no check for it, just trying to access a key in the dict that doesnt exist and it dies16:13
kplantyoctozepto: i think that was the problem, thank you!16:13
yoctozeptocosmicsound: most notably nova-compute on the new node16:13
yoctozeptokplant: yw16:13
openstackgerritDaniel Alvarez proposed openstack/kolla stable/stein: ovn: Create the ovn dbs only if the schema file exists  https://review.opendev.org/69944716:14
*** gkadam has quit IRC16:15
*** wuchunyang has joined #openstack-kolla16:36
*** wuchunyang has quit IRC16:41
mgoddardhrw: when will we see an ARM64 deploy job then?16:42
*** ricolin has quit IRC16:44
openstackgerritYongjun Bai proposed openstack/kolla-ansible master: Fix unable to connect to epmd when deploy rabbitmq by train with ipv6  https://review.opendev.org/69945816:44
hrwre16:44
hrwmgoddard: write one?16:44
hrwmjturek: UNBUILDABLE_IMAGES are skipped when you build. when all you want is templates then you get every template.16:44
mjturekahhhhhhh got it16:45
mgoddardhrw: you are the ARM expert :)16:45
hrwmgoddard: you know k-a better16:46
hrwmjturek: we generate all templates, then use them to create list of images to build, transform list to tree, filter tree, do build.16:47
*** wuchunyang has joined #openstack-kolla16:47
mgoddardhrw: don't you want to see ARM working though?16:47
hrwmjturek: my patch switches --template-only to: generate all templates, then use them to create list of images to build, transform list to tree, filter tree, remove dockerfiles16:48
hrwmgoddard: I may copy some job from k-a and run it on aarch64.16:48
mgoddardhrw: that would be neat16:48
hrwmgoddard: which would be best? aio probably16:48
mgoddardhrw: yes16:48
yoctozeptoaio definitely16:50
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: Add AArch64 CI job  https://review.opendev.org/69946416:52
hrwjust zuul stuff16:52
*** wuchunyang has quit IRC16:52
hrwmay need some playbook changes. will see16:52
yoctozeptohrw: yea, since we don't publish arm16:53
hrwyoctozepto: hm... we maybe should start then16:54
yoctozeptoNeed periodic aarch64 :D16:54
mgoddardwe can always start by building in the job like we did for debian16:55
hrwmgoddard: this is debian/source job D:16:55
hrwanother set of xmas gifts arrived16:57
hrwwill see how it goes17:01
openstackgerritDoug Szumski proposed openstack/kayobe master: Support custom Kolla group_vars  https://review.opendev.org/69888217:01
openstackgerritDoug Szumski proposed openstack/kayobe master: Support customising nova-conductor group for cells  https://review.opendev.org/69897017:01
openstackgerritDoug Szumski proposed openstack/kayobe master: Cells documentation  https://review.opendev.org/69897117:01
openstackgerritDoug Szumski proposed openstack/kayobe master: Cells documentation  https://review.opendev.org/69897117:09
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Add AArch64 CI job  https://review.opendev.org/69946417:11
hrwdropped all CI jobs17:12
*** reddhatt has joined #openstack-kolla17:17
reddhattHi all!17:17
reddhattWe are trying to deploy OS using kolla ansible. Very interesting approach.  We are getting and error "internal endpoint for compute service in RegionOne region not found" in task "nova : Waiting for nova-compute services to register themselves".17:20
reddhattWe had a look in the keystone db and figured out that the endpoints were not created.17:20
reddhattTask "nova/tasks/register.yml", apparently, was not properly executed.17:21
reddhattany tip?17:23
*** cah_link has quit IRC17:23
*** rpittau is now known as rpittau|afk17:25
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Add Ansible group check to prechecks  https://review.opendev.org/69944517:27
mgoddardhi reddhatt, have you changed the region name? Try looking at the nova compute logs in /var/log/kolla/nova/nova-compute.log17:28
reddhattRegion name not changed. The compute endpoints was not created. The table endpoints has only these records:17:31
*** wuchunyang has joined #openstack-kolla17:31
reddhatt id                               | legacy_endpoint_id | interface | service_id                       | url                    | extra | enabled | region_id |17:31
reddhatt+----------------------------------+--------------------+-----------+----------------------------------+------------------------+-------+---------+-----------+17:31
reddhatt| 48872c2a52c44841bf91d7252874c5bb | NULL               | admin     | a236d353043b4dd1a7aa81f91e630537 | http://10.2.1.10:35357 | {}    |       1 | RegionOne |17:31
reddhatt| c2e6de76dc844702b872902f675130c5 | NULL               | internal  | a236d353043b4dd1a7aa81f91e630537 | http://10.2.1.10:5000  | {}    |       1 | RegionOne |17:31
reddhatt| c8b18983300e4bccbdbb1908e2fb5706 | NULL               | public    | a236d353043b4dd1a7aa81f91e630537 | http://10.2.1.10:5000  | {}    |       1 | RegionOne |17:31
openstackgerritDoug Szumski proposed openstack/kayobe master: Support customising nova-conductor and proxy groups for cells  https://review.opendev.org/69897017:31
openstackgerritDoug Szumski proposed openstack/kayobe master: Cells documentation  https://review.opendev.org/69897117:32
openstackgerritMerged openstack/kayobe master: Bump stack.libvirt-vm role to 1.8.0  https://review.opendev.org/69890617:33
mgoddardreddhatt: can you see why the tasks in register.yml were not executed for nova?17:35
*** cah_link has joined #openstack-kolla17:35
mgoddardwhich release of kolla-ansible are you using?17:35
reddhattWe was using the version 8. But we updated to the latest version, cleaned the servers and are going to run again.17:36
reddhattIn this new run, we are going to check why task for register is not performed, if the error occurs again...17:37
mgoddardhrw, mnasiadka: https://review.opendev.org/#/c/69939917:37
patchbotpatch 699399 - kolla - Fix monasca-grafana being unbuildable due to old npm - 5 patch sets17:37
hrwdone17:38
mgoddardgreat: CI++17:38
*** wuchunyang has quit IRC17:39
*** dave-mccowan has quit IRC17:39
hrwnow have to rebase patches on top of it. but first let it merge17:42
hrwarm64 k-a failed. good. now have some logs17:44
hrw    "stderr": "Error response from daemon: Container a7de800d7203ab111cf339b084d0bcbd4bc75b37ca2f12328ef72b32ed428852 is not running",17:45
hrwtime to enable building images17:45
*** cah_link has quit IRC17:45
*** wuchunyang has joined #openstack-kolla17:46
*** k_mouza has quit IRC17:48
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Add AArch64 CI job  https://review.opendev.org/69946417:48
hrwif aarch64 then build images17:48
hrwsimple17:48
*** wuchunyang has quit IRC17:50
*** wuchunyang has joined #openstack-kolla17:54
openstackgerritDoug Szumski proposed openstack/kayobe master: Cells documentation  https://review.opendev.org/69897117:54
mgoddardI'm out until 2nd Jan now. Well done all for getting the Train out of the station :)17:58
*** wuchunyang has quit IRC17:59
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: CentOS 8 spec  https://review.opendev.org/69947217:59
yoctozeptomgoddard: have fun18:03
hrwCould not find or access '/var/lib/zuul/builds/9b3798b83b7540bf985ed4d5d1052af2/work/src/opendev.org/openstack/kolla/tests/vars/zuul.yml' on the Ansible Controller.18:05
hrwyay18:05
hrwmgoddard: have fun. you deserved18:05
*** wuchunyang has joined #openstack-kolla18:06
hrwyoctozepto: according to NOTE() you know what next18:07
*** cah_link has joined #openstack-kolla18:07
hrwyoctozepto: or I can add publisher to kolla, wait until it do it's work and recheck ;d18:07
yoctozeptohrw: add kolla to required projects for this job18:09
hrwthx18:09
yoctozeptonp18:09
*** wuchunyang has quit IRC18:10
dougszyoctozepto: where do we send beer?18:11
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: WIP: Add AArch64 CI job  https://review.opendev.org/69946418:12
yoctozeptodougsz: haha, too baad I'm a water-drinker huh :D18:13
*** wuchunyang has joined #openstack-kolla18:17
*** k_mouza has joined #openstack-kolla18:18
mnasiadkaback18:19
mnasiadkadougsz: you need to travel in person to Bialystok ;-)18:20
mnasiadkayoctozepto: there was some conference planned, right? ;)18:20
hrwand speakers are needed probably...18:21
yoctozeptowell, one position is still open18:22
*** wuchunyang has quit IRC18:22
*** k_mouza has quit IRC18:23
openstackgerritDincer Celik proposed openstack/kolla-ansible master: WIP: Ansible max version check  https://review.opendev.org/69933318:28
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Configure services to use Certificate Authority  https://review.opendev.org/69931218:31
*** wuchunyang has joined #openstack-kolla18:35
*** pcaruana has quit IRC18:38
*** wuchunyang has quit IRC18:40
*** dougsz has quit IRC18:55
*** wuchunyang has joined #openstack-kolla19:02
openstackgerritMerged openstack/kolla master: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69939919:06
*** dougsz has joined #openstack-kolla19:11
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: docker: fix 'Empty continuation line found' warnings  https://review.opendev.org/69887519:11
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: add more tests for enable_repos template method  https://review.opendev.org/69735419:11
hrwrebase19:12
* hrw out19:14
*** wuchunyang has quit IRC19:14
*** pcaruana has joined #openstack-kolla19:18
openstackgerritRadosław Piliszek proposed openstack/kolla stable/train: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69948219:19
openstackgerritRadosław Piliszek proposed openstack/kolla stable/stein: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69948319:19
openstackgerritRadosław Piliszek proposed openstack/kolla stable/rocky: Fix monasca-grafana being unbuildable due to old npm  https://review.opendev.org/69948419:20
*** TrevorV has quit IRC19:20
*** dougsz has quit IRC19:21
*** wuchunyang has joined #openstack-kolla19:21
yoctozeptomgoddard, mnasiadka, hrw: unbreakers for stable kolla branches ^19:24
*** wuchunyang has quit IRC19:26
*** wuchunyang has joined #openstack-kolla19:31
*** TrevorV has joined #openstack-kolla19:37
*** wuchunyang has quit IRC19:40
*** wuchunyang has joined #openstack-kolla19:46
*** ianw is now known as ianw_pto19:50
*** wuchunyang has quit IRC19:51
*** pcaruana has quit IRC19:59
openstackgerritDincer Celik proposed openstack/kolla-ansible master: WIP: Ansible max version check  https://review.opendev.org/69933320:12
*** reddhatt has quit IRC20:15
*** kplant has quit IRC20:31
*** Chaserjim has joined #openstack-kolla20:38
*** pcaruana has joined #openstack-kolla20:42
*** wuchunyang has joined #openstack-kolla20:44
*** gfidente has quit IRC20:48
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Add AArch64 publisher for Debian/source  https://review.opendev.org/69949720:48
*** wuchunyang has quit IRC20:49
*** kozhukalov has joined #openstack-kolla21:02
*** wuchunyang has joined #openstack-kolla21:03
*** wuchunyang has quit IRC21:08
*** wuchunyang has joined #openstack-kolla21:15
*** wuchunyang has quit IRC21:19
*** wuchunyang has joined #openstack-kolla21:26
*** born2bake has joined #openstack-kolla21:26
born2bakehi guys, did anyone have any errors related to chrony container on controller? http://prntscr.com/qc95sw21:29
*** kozhukalov has quit IRC21:29
born2bakecant even stop container...its stuck after stop command21:29
*** wuchunyang has quit IRC21:30
*** TrevorV has quit IRC21:40
*** pcaruana has quit IRC21:41
noxoidits possible containerd lost track of it, ive had issues with certain docker daemon upgrades causing orphaned containers21:48
*** cah_link has quit IRC21:58
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Configure services to use Certificate Authority  https://review.opendev.org/69931222:25
openstackgerritMerged openstack/kolla master: add more tests for enable_repos template method  https://review.opendev.org/69735422:32
*** born2bake has quit IRC22:41
*** reddhatt has joined #openstack-kolla23:17
openstackgerritMerged openstack/kolla master: docker: fix 'Empty continuation line found' warnings  https://review.opendev.org/69887523:26
*** reddhatt has quit IRC23:34
*** jbadiapa has quit IRC23:38
« Monday, 2019-12-16 Index Wednesday, 2019-12-18 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!