Monday, 2019-03-25

*** Sravan has quit IRC00:00
*** Sravan has joined #openstack-kolla00:02
*** luksky has quit IRC00:02
*** Sravan has quit IRC00:13
*** Sravan has joined #openstack-kolla00:15
*** Sravan has quit IRC00:18
*** Sravan has joined #openstack-kolla00:23
*** Sravan has quit IRC00:25
*** Sravan has joined #openstack-kolla00:35
*** wolverineav has joined #openstack-kolla00:38
*** Sravan has quit IRC00:39
*** iniazi has joined #openstack-kolla00:41
*** wolverineav has quit IRC00:42
*** iniazi_ has quit IRC00:43
*** wolverineav has joined #openstack-kolla00:51
*** Sravan has joined #openstack-kolla00:53
*** hongbin has joined #openstack-kolla01:05
*** andrein has quit IRC01:30
*** iniazi has quit IRC01:46
*** iniazi has joined #openstack-kolla01:57
*** Sravan has quit IRC02:22
*** Sravan has joined #openstack-kolla02:38
*** shyamb has joined #openstack-kolla02:43
*** Sravan has quit IRC02:54
*** shyamb has quit IRC02:57
*** shyamb has joined #openstack-kolla02:58
*** iniazi_ has joined #openstack-kolla02:58
*** iniazi has quit IRC03:01
*** Sravan has joined #openstack-kolla03:03
*** hongbin_ has joined #openstack-kolla03:21
*** Sravan has quit IRC03:21
*** hongbin has quit IRC03:23
*** Sravan has joined #openstack-kolla03:24
*** shyamb has quit IRC03:28
*** Sravan has quit IRC03:34
*** skramaja has joined #openstack-kolla03:41
*** wolverineav has quit IRC03:54
*** wolverineav has joined #openstack-kolla03:56
*** janki has joined #openstack-kolla03:56
*** hongbin_ has quit IRC03:59
*** Sravan has joined #openstack-kolla04:05
openstackgerritLucas.hua proposed openstack/kolla-ansible master: Add feature to merge_config to support extra_config  https://review.openstack.org/64728004:09
*** Sravan has quit IRC04:09
*** wolverineav has quit IRC04:29
*** Sravan has joined #openstack-kolla04:31
*** sshnaidm|rover has joined #openstack-kolla04:38
*** Sravan has quit IRC04:42
*** wolverineav has joined #openstack-kolla04:47
*** sshnaidm|rover has quit IRC04:49
*** shyamb has joined #openstack-kolla05:04
*** vblando has joined #openstack-kolla05:11
*** shyamb has quit IRC05:11
*** shyamb has joined #openstack-kolla05:21
*** wolverineav has quit IRC05:21
openstackgerritLucas.hua proposed openstack/kolla-ansible master: Add feature to merge_config to support extra_config  https://review.openstack.org/64728006:02
*** wolverineav has joined #openstack-kolla06:11
*** shyamb has quit IRC06:34
*** iniazi_ has quit IRC06:34
*** ivve has joined #openstack-kolla06:37
*** shyamb has joined #openstack-kolla06:41
*** jbadiapa has joined #openstack-kolla06:50
*** vblando has quit IRC06:52
*** Luzi has joined #openstack-kolla06:53
*** owalsh has quit IRC06:54
openstackgerritLucas.hua proposed openstack/kolla-ansible master: Add feature to merge_config to support extra_config  https://review.openstack.org/64728006:54
*** jbadiapa has quit IRC06:57
*** jbadiapa has joined #openstack-kolla06:58
*** wolverineav has quit IRC07:00
*** wolverineav has joined #openstack-kolla07:01
*** pcaruana has joined #openstack-kolla07:07
*** shyamb has quit IRC07:17
openstackgerritMerged openstack/kolla-ansible stable/queens: Fix swift reconfigure  https://review.openstack.org/64513207:34
openstackgerritMerged openstack/kolla-ansible stable/pike: Fix swift reconfigure  https://review.openstack.org/64513307:34
*** shyamb has joined #openstack-kolla07:34
*** wolverineav has quit IRC07:34
*** wolverineav has joined #openstack-kolla07:35
*** wolverineav has quit IRC07:36
*** wolverineav has joined #openstack-kolla07:37
*** oanson has joined #openstack-kolla07:37
*** wolverineav has quit IRC07:37
*** owalsh has joined #openstack-kolla07:41
openstackgerritMartin Schuppert proposed openstack/kolla master: Install trousers to get user tss for nova_libvirt  https://review.openstack.org/64730808:00
*** shyamb has quit IRC08:00
openstackgerritMerged openstack/kolla-ansible stable/rocky: Fix swift reconfigure  https://review.openstack.org/64513108:15
*** luksky has joined #openstack-kolla08:18
*** andrein has joined #openstack-kolla08:23
*** iniazi_ has joined #openstack-kolla08:24
*** tosky has joined #openstack-kolla08:28
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Test upgrades in CI  https://review.openstack.org/61664008:30
*** andrein has quit IRC08:30
*** sshnaidm|rover has joined #openstack-kolla08:38
openstackgerritMerged openstack/kolla master: Bump skydive version to 0.22.0  https://review.openstack.org/64552208:42
openstackgerritMerged openstack/kolla master: Add some irrelevant files to build jobs  https://review.openstack.org/64336808:42
*** wolverineav has joined #openstack-kolla08:43
openstackgerritMartin Schuppert proposed openstack/kolla master: Install trousers to get user tss for nova_libvirt  https://review.openstack.org/64730808:44
*** shyamb has joined #openstack-kolla08:44
*** zbr has quit IRC08:53
*** wolverineav has quit IRC09:02
*** ivve has quit IRC09:10
*** andrein has joined #openstack-kolla09:15
*** zbr has joined #openstack-kolla09:20
*** iniazi has joined #openstack-kolla09:25
*** iniazi_ has quit IRC09:28
mgoddardmorning all09:38
hrwmorning09:43
*** ivve has joined #openstack-kolla09:52
*** shardy has joined #openstack-kolla09:54
openstackgerritMerged openstack/kolla-ansible stable/pike: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644510:02
openstackgerritMerged openstack/kolla-ansible stable/queens: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644610:02
mgoddardUpgrade job now passing on centos and ubuntu, including building images when the patch depends on kolla \o/10:08
mgoddardWhich implicitly means we now have fixes for upgrade bugs10:08
*** wolverineav has joined #openstack-kolla10:17
*** shyamb has quit IRC10:19
*** ivve has quit IRC10:21
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Test upgrades in CI  https://review.openstack.org/61664010:21
openstackgerritMerged openstack/kolla master: cleanup in base for yum install entries  https://review.openstack.org/64518510:22
*** gfidente has joined #openstack-kolla10:25
*** tolisbar has joined #openstack-kolla10:26
*** iniazi has quit IRC10:30
*** wolverineav has quit IRC10:30
openstackgerritMark Goddard proposed openstack/kolla master: Remove unused scripts  https://review.openstack.org/64738310:33
*** sshnaidm|rover is now known as sshnaidm|pto10:34
*** iniazi has joined #openstack-kolla10:37
openstackgerritMerged openstack/kolla-ansible stable/rocky: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644710:39
openstackgerritMerged openstack/kolla-ansible stable/ocata: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644410:39
openstackgerritMerged openstack/kolla master: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64643810:39
openstackgerritMerged openstack/kolla stable/rocky: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644210:39
*** wolverineav has joined #openstack-kolla10:39
hrwmgoddard: good10:40
mgoddardhrw: what did we decide re python3? The conversation didn't continue after the meeting from what I saw10:42
hrwmgoddard: we merge in train10:42
mgoddardhrw: are we ok to leave the images as they are?10:42
hrwmgoddard: yes. ubuntu ones are mix of py2/py3 as their packages are py2/3 anyway10:42
mgoddardhrw: ok. So they'll just be a bit fatter in stein?10:43
hrwmgoddard: yep10:43
mgoddardhrw: I think we can live with that10:43
hrwmgoddard: https://review.openstack.org/#/c/616640 looks great. out of my knowledge so will not +2 but +1 as i like it10:44
openstackgerritMark Goddard proposed openstack/kolla stable/rocky: Bump OpenStack service versions  https://review.openstack.org/64739010:44
mgoddardhrw: thanks :) it's long overdue10:45
hrwmgoddard: "cleanup in base for yum install entries" patch adds '-' for whitespace stripping10:45
mgoddardhopefully will stop the 'test upgrades and fix bugs at the end of the release' issues10:45
hrwmgoddard: code it touch is very fragile to that10:46
mgoddardhrw: yeah, my question was about where to put the -10:46
hrwok10:46
mnasiadkamorning10:46
mgoddarde.g. all four {%- if foo -%}{%- endif -%}10:46
mgoddardor {% if foo -%}{%- endif %}10:47
hrwok10:47
mgoddardor other combinations10:47
hrwmnasiadka: hej10:47
mgoddardmorning mnasiadka10:47
mnasiadkahi guys10:47
mgoddardhrw: btw, that upgrade test patch has lots of dependencies to actually make upgrades work. Listed on https://etherpad.openstack.org/p/KollaWhiteBoard10:48
openstackgerritMark Goddard proposed openstack/kolla stable/queens: Bump OpenStack service versions  https://review.openstack.org/64739210:50
*** ivve has joined #openstack-kolla10:50
hrwmnasiadka: https://review.openstack.org/#/c/64425010:51
openstackgerritMark Goddard proposed openstack/kolla stable/pike: Bump OpenStack service versions  https://review.openstack.org/64739410:52
mnasiadkahrw: looking at it10:54
hrwmgoddard: went through list in whiteboard10:54
hrwmnasiadka: thx10:54
mgoddardhrw: nice one, thanks10:54
mnasiadkamgoddard: ceph jobs are timing out on the PR hrw mentioned, I don't think it's first time - should we bump up the 2 hour limit on those?10:55
*** sshnaidm|pto has quit IRC10:55
hrwmnasiadka: and welcome in kolla core!10:56
mgoddardmnasiadka: looking10:56
mnasiadkahrw: thanks, felt like a half-core previously :D10:56
hrwmnasiadka: you are 2nd case iirc10:56
mnasiadkamgoddard: not counting the ubuntu job, which failed because of some environmental issues I guess10:57
mgoddardmnasiadka: looks like centos got very close to the end before timing out10:58
mnasiadkamgoddard: yeah, I noticed that10:59
mgoddardmnasiadka: I can't even work out what part it didn't do10:59
mnasiadkamgoddard: well, status is status - probably it missed 2 more minutes and would be green :)10:59
mgoddardmnasiadka: yeah11:00
*** shyamb has joined #openstack-kolla11:01
mgoddardmnasiadka: could be because it had to build images that it was pushed over the 2 hour limit11:01
mgoddardmnasiadka: I only see that job timing out in the last 50: http://zuul.openstack.org/builds?job_name=kolla-ansible-centos-source-ceph&branch=master11:02
mgoddardmnasiadka: could try bumping to 2.5 hours?11:03
mgoddardthese two need a +2 to unblock stable branches: https://review.openstack.org/#/c/645197/ https://review.openstack.org/#/c/645165/11:05
mnasiadkamgoddard: that's what I'm thinking, but have no clue where to bump it up ;)11:08
mgoddardmnasiadka: zuul.d/jobs.yaml11:09
mnasiadkamgoddard: yeah, need to read zuul docs ;)11:09
mnasiadkait's funny to +2 a change, where you have already only +1'ed because of missing rights :D11:10
*** wolverineav has quit IRC11:14
openstackgerritMerged openstack/kolla master: Install trousers to get user tss for nova_libvirt  https://review.openstack.org/64730811:19
*** ivve has quit IRC11:19
hrwmnasiadka: ;D11:24
hrwmnasiadka: I have reviews where I give +1 despite having +2.11:25
mnasiadkahrw: I know, makes sense sometimes - but I was rather having the other way around ;)11:25
hrwmnasiadka: 'patch is good, lgtm' == +2, 'patch is good but have comments' == +1 'patch is okish but needs work' == comments11:25
hrwthat's more or less my way11:26
hrwand 'patch is WRONG' == -111:26
mgoddardor, 'patch is good but I don't feel comfortable approving == +1'11:27
*** shyamb has quit IRC11:30
mnasiadkamgoddard: yeah, that's what I meant ;)11:34
*** hrw has quit IRC11:35
*** hrw has joined #openstack-kolla11:36
*** henriqueof has quit IRC11:38
*** luksky has quit IRC11:39
*** iniazi has quit IRC11:40
*** iniazi has joined #openstack-kolla11:41
openstackgerritMark Goddard proposed openstack/kolla master: Remove unused scripts  https://review.openstack.org/64738311:44
*** shyamb has joined #openstack-kolla11:44
*** wolverineav has joined #openstack-kolla11:48
openstackgerritMerged openstack/kolla-ansible master: Wait for cinder volume to become available in CI  https://review.openstack.org/64527711:52
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Bump up timeout for ceph jobs  https://review.openstack.org/64740511:52
openstackgerritMerged openstack/kolla master: MariaDB upgrade: specify host and port  https://review.openstack.org/64599011:54
openstackgerritMerged openstack/kolla stable/pike: Fix multiple issues  https://review.openstack.org/64519711:54
*** wolverineav has quit IRC11:56
openstackgerritMerged openstack/kolla-ansible stable/rocky: Upgrade test: add support for being the source release  https://review.openstack.org/64508912:01
openstackgerritMerged openstack/kolla-ansible stable/rocky: Use new Docker packages in Ubuntu 18.04+  https://review.openstack.org/64581612:02
*** jroll has quit IRC12:14
*** luksky has joined #openstack-kolla12:15
*** jroll has joined #openstack-kolla12:15
openstackgerritMartin Mágr proposed openstack/kolla master: Allow mitral-executor to perform undercloud backups  https://review.openstack.org/64000812:24
*** logan- has quit IRC12:27
*** sshnaidm|pto has joined #openstack-kolla12:30
*** logan- has joined #openstack-kolla12:31
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Fix up config file permissions on the host  https://review.openstack.org/64586112:33
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Check configuration file permissions in CI  https://review.openstack.org/64741012:33
*** edleafe has joined #openstack-kolla12:39
*** henriqueof has joined #openstack-kolla12:42
*** iniazi has quit IRC12:46
*** skramaja has quit IRC12:49
openstackgerritMerged openstack/kolla-ansible stable/rocky: Fix Neutron precheck to not fail with newer Docker  https://review.openstack.org/64584012:53
openstackgerritMerged openstack/kolla-ansible master: Fix neutron rolling upgrade  https://review.openstack.org/64510212:53
openstackgerritMerged openstack/kolla-ansible master: Update openstack_previous_release_name to rocky  https://review.openstack.org/64561312:53
openstackgerritMerged openstack/kolla-ansible master: Fix booting instances after nova-compute upgrade  https://review.openstack.org/64561412:53
*** iniazi has joined #openstack-kolla12:54
*** shyamb has quit IRC13:13
*** ivve has joined #openstack-kolla13:15
*** wolverineav has joined #openstack-kolla13:16
*** goldyfruit has joined #openstack-kolla13:18
*** tolisbar has quit IRC13:24
*** irclogbot_2 has joined #openstack-kolla13:28
*** sshnaidm|pto has quit IRC13:28
*** tolisbar has joined #openstack-kolla13:29
hrwmgoddard: updated whiteboard - marked merged patches13:30
mgoddardhrw: nice, thanks13:30
*** altlogbot_0 has quit IRC13:31
*** altlogbot_3 has joined #openstack-kolla13:32
*** sshnaidm|pto has joined #openstack-kolla13:37
*** mchlumsky has joined #openstack-kolla13:37
*** tosky has quit IRC13:38
*** irclogbot_2 has quit IRC13:38
*** iclon has quit IRC13:39
*** irclogbot_3 has joined #openstack-kolla13:39
*** iclon has joined #openstack-kolla13:40
*** mchlumsky has quit IRC13:45
*** mchlumsky has joined #openstack-kolla13:48
*** mjturek has joined #openstack-kolla13:50
openstackgerritLucas.hua proposed openstack/kolla-ansible master: Add feature to merge_config to support extra_config  https://review.openstack.org/64728013:52
*** sshnaidm|pto has quit IRC13:53
*** sshnaidm has joined #openstack-kolla13:53
*** serhatd has joined #openstack-kolla13:58
openstackgerritMerged openstack/kolla stable/queens: Pin sensu mailer and slack plugins, fix rabbitmq  https://review.openstack.org/64516513:59
*** sshnaidm is now known as sshnaidm|pto14:01
*** baha has joined #openstack-kolla14:01
*** munimeha1 has joined #openstack-kolla14:04
*** ivve has quit IRC14:11
v1k0d3nhey everyone...i could be missing an update, since i'm not involved with openstack as much these days, but can anyone tell me if this is still an outstanding issue: https://bugs.launchpad.net/kolla/+bug/181102014:18
openstackLaunchpad bug 1811020 in kolla "Grafana image build failed" [Undecided,New]14:18
v1k0d3ni'm using kolla/kolla-ansible from the pip installation (latest, rocky, 7.0.1)14:20
*** hamzy_ is now known as hamzy14:22
*** ivve has joined #openstack-kolla14:23
mnasiadkamgoddard: I was thinking recently about doing a new release in rocky at least - a lot of people come here hitting the mariadb resolveip bug... and some other bugs already fixed in stable14:27
mnasiadkamgoddard: what do you think?14:27
mgoddardmnasiadka: good idea.14:27
mgoddardmnasiadka: we have the change to bump versions that seems like a good point to release14:27
mgoddardmnasiadka: might as well release all branches14:28
mnasiadkamgoddard: makes perfect sense14:28
mgoddardmnasiadka: I think there is some text saying we commit to releasing every 45 days, I should probably put something in my calendar14:28
mnasiadkamgoddard: yeah, this is something I always was missing in kolla/kolla-ansible - release often after some major bugs are fixed14:29
mgoddardmnasiadka: yeah. I normally just use stable branches, but it's good practice to release14:30
mnasiadkamgoddard: me too, but all the users don't :)14:30
serhatdHello Everyone14:30
serhatdany progress on ttps://bugs.launchpad.net/kolla-ansible/+bug/1792496?14:30
serhatdhttps://bugs.launchpad.net/kolla-ansible/+bug/179249614:31
openstackLaunchpad bug 1792496 in kolla-ansible "No column dpdk_version in table Open_vSwitch" [Undecided,New]14:31
serhatdWe hit this bug today while upgrading from queens to rocky14:31
v1k0d3nmnasiadka:  that's the bug i am ultimately hitting in Rocky...the mariadb resolveip. so that is a known issue in the "stable" rocky release right now?14:32
mnasiadkav1k0d3n: yeah, fixed it some time ago - or we think we fixed it, so if you can test and give feedback - would be nice :)14:32
v1k0d3nin Rocky...still hitting it and not really sure where to start with this one...14:32
v1k0d3ni was going to build images too, but then hit the gpg issue....so kinda of getting stuck14:33
v1k0d3nwhat am i doing wrong?14:34
v1k0d3nthis is an AIO too, so i would think that would be a bit easier.14:34
mnasiadkaserhatd: that bug is a bit old and related to some pre-release versions, so would be good if you can follow up on it, or create a new one - then we can try to troubleshoot14:34
v1k0d3nhttps://www.irccloud.com/pastebin/lxCrod71/14:35
mnasiadkav1k0d3n: I would start with using stable/rocky from git instead of latest release from pip14:35
v1k0d3ni'm using stable rocky14:35
mnasiadkav1k0d3n: connection reset by peer doesn't sound like kolla-ansible or ansible problem ;)14:35
mnasiadkabut it's wait_for module, so probably some service is not working as it should?14:36
v1k0d3nhttps://www.irccloud.com/pastebin/uuSrvSmT/14:36
v1k0d3nhttps://www.irccloud.com/pastebin/bVvrZChX/14:37
mnasiadkav1k0d3n: so this is not stable/rocky from git, it's the latest release from pypi.org14:37
v1k0d3nit's an AIO...so not sure why it would be getting a reset by peer, since there's nothing running on that port but what kolla is attempting to bring up.14:38
mnasiadkapip install git+https://github.com/openstack/kolla@stable/rocky14:38
v1k0d3nok...so sorry about that. just following the directions in docs.openstack.org14:38
v1k0d3nso what am i missing here?14:38
v1k0d3nok cool i can try force reinstalling that.14:39
*** wolverineav has quit IRC14:39
serhatdmnasiadka: I will update bug report14:39
mnasiadkav1k0d3n: and similar for kolla-ansible, and start from there ;)14:40
v1k0d3non it.14:40
v1k0d3nnow...14:40
v1k0d3nhttps://www.irccloud.com/pastebin/r9WZYFvq/14:41
v1k0d3nso i would imagine that folks are reaching out because on docs it's not exactly that clear (as it just says to pip install, and that will be "latest stable"14:41
mnasiadkav1k0d3n: I'll add updating the docs on my to-do list ;)14:44
v1k0d3ndidn't mean that.14:44
v1k0d3ni meant only that it could be the reason folks are pinging the group if they are.14:44
v1k0d3ni can update the docs if you want. doesn't matter to me. i was just trying to be helpful.14:45
mnasiadkano problem - whoever does that - it will be better for the users to understand :)14:46
mnasiadkaand it doesn't change the fact that kolla and kolla-ansible should be released more often :)14:46
v1k0d3ni was thinking that it would cause them to reach out less, and avoid get frustrated and dropping it for something else (as i was about to do).14:47
v1k0d3ntrue. but i wasn't trying to give an opinion on that. just noticed that, tangentially, i was having issues with mariadb14:49
serhatdmnasiadka: updated https://bugs.launchpad.net/kolla-ansible/+bug/179249614:50
openstackLaunchpad bug 1792496 in kolla-ansible "No column dpdk_version in table Open_vSwitch" [Undecided,New]14:50
*** ktibi has joined #openstack-kolla14:53
mnasiadkaok, then it seems like we need to take care of that14:53
mnasiadkamgoddard: you were running some rocky to stein upgrade tests lately, no similar errors?14:53
mgoddardmnasiadka: I don't see it on ubuntu14:58
openstackgerritMerged openstack/kolla-ansible master: Bump up timeout for ceph jobs  https://review.openstack.org/64740514:58
mnasiadkamgoddard: but I guess it makes sense to add this check and a handler to do upgrade of the ovsdb schema when required - let me add this to my list :)15:03
mgoddardmnasiadka: yeah, seems like a sensible post upgrade activity15:04
*** Luzi has quit IRC15:05
hrwv1k0d3n: grafana?15:07
v1k0d3nthink i linked the wrong issue, sorry about that hrw.15:09
hrwv1k0d3n: np15:09
v1k0d3ni meant to link an issue related to the gpg keys needing updated.15:09
v1k0d3nand that was in my clip i think.15:09
hrwv1k0d3n: https://bugs.launchpad.net/kolla/+bug/1811020 you linked is gpg keys related15:09
openstackLaunchpad bug 1811020 in kolla "Grafana image build failed" [Undecided,New]15:09
v1k0d3nwell, because it was along the same vein of thought :)15:10
v1k0d3nwas running into that on build, and then the mariadb issue with not starting. there were two issues i was having...both related to reading the install guides literally (rather than taking time to think there may be an update to pip stable).15:11
v1k0d3nfirst i tried installing kolla-ansible...ran into mariadb issue. thought it might be an issue issue, so i was going to rebuild using kolla...ran into second issue with gpg keys.15:11
hrwv1k0d3n: there are things which move behind us. like repo updates15:12
v1k0d3nautocorrect got me "thought it might be a build issue"15:12
serhatdmnasiadka: btw could not run ovsdb-tool convert while openvswitch-db running. Error message is: ovsdb-tool: /var/lib/openvswitch/conf.db: failed to lock lockfile15:12
serhatdmnasiadka: What about check for converting openvswitch database at bootstrap task?15:12
mnasiadkaserhatd: yeah, will think how to do it best15:13
hrwv1k0d3n: you use rocky, right?15:14
v1k0d3ncorrect. but only because, from the docs, it appeared that this was the latest/stable release.15:20
v1k0d3ni just needed something sort of quickly for development purposes (at least right now).15:20
hrwv1k0d3n: use git instead of 7.0.1 - grafana was fixed after release15:20
v1k0d3nyeah, i saw that actually.15:20
v1k0d3ndo you run upgrades off of git as well?15:21
hrwmgoddard: we need to release 7.0.2 as it has several fixes15:21
v1k0d3nor does that break for you?15:21
hrwv1k0d3n: stable branches gets only fixes, no new code15:21
hrwmgoddard: same for kolla-ansible15:22
mgoddardhrw: I'll propose some releases when all these version bumps go through15:22
hrwmgoddard: coolio15:23
hrwmgoddard: https://review.openstack.org/#/c/647390 etc?15:23
mgoddardhrw: yeah15:24
*** andrein has quit IRC15:24
hrwmgoddard: each has my +2. started rechecks15:24
mgoddardhrw: thanks15:24
*** andrein has joined #openstack-kolla15:26
*** shyamb has joined #openstack-kolla15:36
*** itlinux has quit IRC15:37
*** wolverineav has joined #openstack-kolla15:37
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Remove recurse: yes for owner/perms on /etc/kolla  https://review.openstack.org/64751415:39
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Fix up config file permissions on the host  https://review.openstack.org/64586115:43
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Check configuration file permissions in CI  https://review.openstack.org/64741015:43
v1k0d3nhave you folks seen a mariadb issue where there's a socket error (host-side) and inside the container logs state there's a permission issue on things like:15:46
v1k0d3nINFO:__main__:Setting permission for /var/lib/mysql/mysql-bin.00001115:46
v1k0d3nINFO:__main__:Setting permission for /var/lib/mysql/mysql-bin.00012715:46
v1k0d3nINFO:__main__:Setting permission for /var/lib/mysql/mysql-bin.00010915:46
v1k0d3nINFO:__main__:Setting permission for /var/lib/mysql/mysql-bin.00001615:46
mgoddardhrw: mnasiadka: we should get this merged before we do a rocky release: https://review.openstack.org/#/c/64519115:48
mgoddardv1k0d3n: I don't remember seeing that, could you give more info?15:51
v1k0d3nsure, what would be most helpful to provide?15:51
mgoddardv1k0d3n: what error are yuo seeing?15:52
v1k0d3nthis is an AIO deployment, and the mariadb container restarts constantly with the following error...15:52
v1k0d3nhttps://www.irccloud.com/pastebin/JY3TP4Jx/15:53
*** hamzaachi has joined #openstack-kolla15:55
*** tolisbar has quit IRC15:57
mgoddardmnasiadka: https://launchpad.net/kolla-ansible/rocky15:59
mnasiadkamgoddard: great, thanks ;)15:59
mgoddardv1k0d3n: how about /var/log/kolla/mariadb/mariadb.log?15:59
*** unicell has joined #openstack-kolla15:59
mgoddardv1k0d3n: is it a fresh deploy or upgrade?15:59
v1k0d3nah good call mgoddard here it is....16:00
v1k0d3nhttps://www.irccloud.com/pastebin/daIoTI0u/16:00
v1k0d3nit's a fresh install16:00
v1k0d3nso i can clean if i need to16:00
mgoddardv1k0d3n: can you remove the mariadb docker volume and run the deploy again? My guess is it failed part way through bootstrapping the DB16:01
v1k0d3nsound about right. sure, i'll do that and try again.16:02
*** wolverineav has quit IRC16:09
*** baha has quit IRC16:16
*** luksky has quit IRC16:23
*** ivve has quit IRC16:26
*** tolisbar has joined #openstack-kolla16:27
*** shyamb has quit IRC16:35
*** unicell has quit IRC16:44
*** andrein has quit IRC16:50
*** janki has quit IRC16:54
v1k0d3nmgoddard: update...thank you for your help. that seemed to clear this one up. i really appreciate your time. :)16:57
*** ktibi has quit IRC16:58
hrwmgoddard: +217:01
*** Sravan has joined #openstack-kolla17:10
*** unicell has joined #openstack-kolla17:10
*** Sravan has quit IRC17:15
*** andrein has joined #openstack-kolla17:16
*** Sravan has joined #openstack-kolla17:16
*** Sravan has quit IRC17:18
*** hamzaachi has quit IRC17:19
*** Sravan has joined #openstack-kolla17:21
*** baha has joined #openstack-kolla17:32
*** wolverineav has joined #openstack-kolla17:35
*** hamzaachi has joined #openstack-kolla17:35
*** wolverineav has quit IRC17:42
*** wolverineav has joined #openstack-kolla17:48
*** ivve has joined #openstack-kolla17:48
klindgrenmgoddard, re: the deploy-containers work that I was starting on, one thing I don't get, is you said that setting changed=true would cause all the containers to get restarted even if nothing changed.  However, my understanding - unless I am wrong, is that the notify restart actions would only have if compare_container returns changed=true.  Otherwise the handlers shouldn't get called.  IE no changes, no restarts?17:58
*** gfidente is now known as gfidente|afk18:06
*** spiette has quit IRC18:07
*** spiette has joined #openstack-kolla18:09
openstackgerritMark Goddard proposed openstack/kolla stable/rocky: Fix MariaDB 10.3 upgrade  https://review.openstack.org/64519118:11
*** hamzaachi has quit IRC18:13
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Check configuration file permissions in CI  https://review.openstack.org/64741018:19
mgoddardv1k0d3n: no problem, glad to have helped18:22
mgoddardklindgren: ah yes, you're right there actually, good point18:23
mgoddardklindgren: we're currently in feature freeze for the stein release, so no rush - this should be a feature in the Train release18:24
*** luksky has joined #openstack-kolla18:26
*** itlinux has joined #openstack-kolla18:26
*** goldyfruit has quit IRC18:45
openstackgerritMerged openstack/kolla stable/queens: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644118:55
*** ivve has quit IRC18:57
*** wolverineav has quit IRC19:01
openstackgerritMerged openstack/kolla stable/pike: Replace openstack.org git:// URLs with https://  https://review.openstack.org/64644019:04
*** ktibi has joined #openstack-kolla19:05
*** hamzaachi has joined #openstack-kolla19:10
openstackgerritMerged openstack/kolla stable/pike: Ignore undefined values in customizable filter  https://review.openstack.org/62750319:11
openstackgerritKris Lindgren proposed openstack/kolla-ansible master: Add a job that *only* deploys updated containers  https://review.openstack.org/64596619:25
openstackgerritKris Lindgren proposed openstack/kolla-ansible master: Add a job that *only* deploys updated containers  https://review.openstack.org/64596619:30
*** ktibi_ has joined #openstack-kolla19:32
*** ktibi has quit IRC19:34
v1k0d3nhey question for the kolla folks...what's the easiest way to create the octavia certs for development purposes? i know kolla doesn't create them, but expects them as part of the deployment. do you guys have something that makes it easier to spin up and take down for development of kolla deployments?19:38
*** wolverineav has joined #openstack-kolla19:40
*** wolverineav has quit IRC19:45
openstackgerritKris Lindgren proposed openstack/kolla-ansible master: Add a job that *only* deploys updated containers  https://review.openstack.org/64596619:48
*** wolverineav has joined #openstack-kolla20:02
*** wolverineav has quit IRC20:07
klindgrenmgoddard, I think the latest patchiest fixes up some of the things, I also refactored, to use a common check tasks.  Instead of copy the check tasks from config.yml into deploy-containers.yml.  I am curious about the question on splitting deployments.  I was not actually thinking about that specific issue.  I was more thinking about the issue where we have updated code that we want out there, but no config changes need to be made.20:14
klindgrenCurrently we use the deploy action, but with ~150 hosts it takes about 30 minutes for both nova and neutron to do their work, because of all the time evaluating conditionals for config generation, which seems to be the responsibility of the main python thread and not each fork.20:15
klindgrenWhich means that deployments are slow, and rollbacks if needed are slow as well.  So that's really the use case I was trying to address.20:16
*** wolverineav has joined #openstack-kolla20:17
klindgrenSorry when I say 30 minutes for both.  I mean nova takes 30 minutes, and neutron takes 30mins, for an hour total.20:18
klindgrenBut the "pull" task for the same set of servers (including pulling the images" takes 9 minutes, with most of the time being image pulls.  So just trying to get back to that level of speed.20:19
*** itlinux has quit IRC20:20
*** zbr has quit IRC20:22
*** zbr has joined #openstack-kolla20:22
*** pcaruana has quit IRC20:23
*** tosky has joined #openstack-kolla20:23
*** itlinux has joined #openstack-kolla20:27
*** itlinux has quit IRC20:27
*** tolisbar1 has joined #openstack-kolla20:36
*** tolisbar has quit IRC20:39
*** tolisbar1 is now known as tolisbar20:39
*** goldyfruit has joined #openstack-kolla20:44
*** k_mouza has joined #openstack-kolla20:58
*** tolisbar1 has joined #openstack-kolla21:07
openstackgerritKris Lindgren proposed openstack/kolla-ansible master: Add a job that *only* deploys updated containers  https://review.openstack.org/64596621:09
*** tolisbar has quit IRC21:10
*** tolisbar1 is now known as tolisbar21:10
*** mjturek has quit IRC21:17
*** hamzaachi has quit IRC21:26
*** baha has quit IRC21:31
*** k_mouza has quit IRC21:34
*** ktibi_ has quit IRC21:41
*** hamzaachi has joined #openstack-kolla21:45
*** itlinux has joined #openstack-kolla21:54
*** wolverineav has quit IRC22:02
*** mchlumsky has quit IRC22:02
*** wolverineav has joined #openstack-kolla22:04
*** gfidente|afk has quit IRC22:06
*** wolverineav has quit IRC22:08
*** wolverineav has joined #openstack-kolla22:13
*** munimeha1 has quit IRC22:20
*** hamzaachi has quit IRC22:21
*** goldyfruit has quit IRC22:24
*** ivve has joined #openstack-kolla22:25
*** serhatd has quit IRC23:25
*** k_mouza has joined #openstack-kolla23:37
*** k_mouza has quit IRC23:41
openstackgerritKris Lindgren proposed openstack/kolla-ansible master: Add a job that *only* deploys updated containers  https://review.openstack.org/64596623:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!