Wednesday, 2019-05-22

*** hamzy has joined #openstack-kolla00:10
*** altlogbot_1 has quit IRC00:10
*** altlogbot_0 has joined #openstack-kolla00:11
*** Chaserjim has quit IRC00:15
*** igordc has quit IRC00:36
*** goldyfruit has quit IRC01:06
*** k_mouza has joined #openstack-kolla01:08
*** k_mouza has quit IRC01:12
*** gregoryo has joined #openstack-kolla01:28
*** happyhemant has quit IRC01:28
*** goldyfruit has joined #openstack-kolla01:31
*** k_mouza has joined #openstack-kolla02:10
*** unicell has joined #openstack-kolla02:11
*** unicell has quit IRC02:11
*** k_mouza has quit IRC02:15
*** unicell has joined #openstack-kolla02:32
*** unicell has quit IRC02:32
*** k_mouza has joined #openstack-kolla02:52
*** goldyfruit has quit IRC02:54
*** Chaserjim has joined #openstack-kolla02:56
*** k_mouza has quit IRC02:56
*** Chaserjim has quit IRC03:04
*** whoami-rajat has joined #openstack-kolla03:20
*** itlinux has joined #openstack-kolla03:55
*** Sravan has joined #openstack-kolla03:59
*** ivve has quit IRC04:26
*** radeks_ has joined #openstack-kolla04:59
*** radeks_ has quit IRC05:05
*** itlinux has quit IRC05:06
*** radeks_ has joined #openstack-kolla05:06
*** dpawlik has joined #openstack-kolla05:18
*** Sravan has quit IRC05:21
*** rwellum has quit IRC05:27
*** portdirect has quit IRC05:27
*** mwhahaha has quit IRC05:30
*** rm_work has quit IRC05:31
*** portdirect has joined #openstack-kolla05:33
*** mwhahaha has joined #openstack-kolla05:33
*** rwellum has joined #openstack-kolla05:34
*** rm_work has joined #openstack-kolla05:34
*** ivve has joined #openstack-kolla05:39
*** guozijn has joined #openstack-kolla05:44
*** ivve has quit IRC05:54
*** Luzi has joined #openstack-kolla06:00
*** guozijn_ has joined #openstack-kolla06:05
*** guozijn has quit IRC06:07
*** guozijn_ is now known as guozijn06:08
*** unicell has joined #openstack-kolla06:15
*** cah_link has joined #openstack-kolla06:17
*** dpawlik has quit IRC06:27
*** janki has joined #openstack-kolla06:27
*** unicell has quit IRC06:29
*** Luzi_ has joined #openstack-kolla06:33
*** skramaja has joined #openstack-kolla06:38
*** Luzi_ has quit IRC06:41
openstackgerritJakub Bielecki proposed openstack/kolla-ansible master: [WIP] bugfix hostname for kolla-ansible repository  https://review.opendev.org/66031407:06
mnasiadkamorning07:09
*** pcaruana has joined #openstack-kolla07:17
*** guozijn_ has joined #openstack-kolla07:27
*** guozijn has quit IRC07:27
*** guozijn_ is now known as guozijn07:27
*** luksky has joined #openstack-kolla07:29
*** guozijn has quit IRC07:32
*** lemko has joined #openstack-kolla07:38
*** guozijn has joined #openstack-kolla07:42
*** luksky has quit IRC07:42
*** luksky has joined #openstack-kolla07:57
*** dpawlik has joined #openstack-kolla08:08
*** guozijn_ has joined #openstack-kolla08:11
*** guozijn has quit IRC08:14
*** guozijn_ is now known as guozijn08:14
*** hamzaachi has joined #openstack-kolla08:14
*** dougsz has joined #openstack-kolla08:15
*** devep has joined #openstack-kolla08:19
*** gregoryo has quit IRC08:20
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798908:24
*** gfidente|afk is now known as gfidente08:33
*** luksky has quit IRC08:48
mgoddardmnasiadka: didn't go for upstream ceph in the end?08:51
mnasiadkamgoddard: I wanted to have it as a separate commit - this one should work as well on CentOS Storage SIG packages08:52
mgoddardmnasiadka: oh ok08:52
mnasiadkamgoddard: second thing is - if we moved to upstream ceph packages - we could unify Ceph version on all distros...08:52
mgoddardmakes sense08:52
mgoddardmnasiadka: that's true08:53
mgoddardmnasiadka: could be a multi-version upgrade for ubuntu though08:53
mgoddardunless we went for mimic in stein and nautilus in train08:53
mnasiadkamgoddard: well, if we upgrade from Luminous - we should be fine08:53
mnasiadkamgoddard: according to the docs :)08:53
mgoddardok08:54
*** k_mouza has joined #openstack-kolla08:54
mgoddardlet's hope so08:54
mgoddardshould we consider releasing centos SIG packages for stein? any idea how long the switch to upstream will take?08:55
*** priteau has joined #openstack-kolla08:55
mnasiadkamgoddard: I'll push that change also today, and there will most probably be one in kolla-ansible as well08:56
mnasiadkamgoddard: might be we want to stay with CentOS SIG packages for Stein, and see how Ceph packages work in master08:56
*** sm806 has quit IRC08:59
mnasiadkamgoddard: or not :D08:59
*** sm806 has joined #openstack-kolla09:00
openstackgerritJakub Bielecki proposed openstack/kolla-ansible master: [WIP] bugfix hostname for kolla-ansible repository  https://review.opendev.org/66031409:00
mgoddardmnasiadka: it's probably quite a risk to switch to upstream for all distros at this stage of stein09:03
mnasiadkamgoddard: that's what I'm thinking as well09:04
*** k_mouza has quit IRC09:04
mgoddardmnasiadka: maybe just for centos, but could even be subtle differences with centos packages09:04
mnasiadkamgoddard: let's first do it in master, and then see how it evolves09:11
mgoddardmnasiadka: sure09:11
*** luksky has joined #openstack-kolla09:22
mnasiadkamgoddard: any reason why this is not +w? https://review.opendev.org/#/c/652606/509:25
mgoddardmnasiadka: busy atm, will take a look shortly09:27
mnasiadkamgoddard: asking because it will impact most probably the bump up to nautilus, and we'll need to rebase and fix I guess :)09:27
mnasiadkamgoddard: take your time09:28
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Add support for ceilometer custom pipeline.yaml  https://review.opendev.org/66061709:39
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Add support for ceilometer custom pipeline.yaml  https://review.opendev.org/66061709:41
openstackgerritMerged openstack/kolla stable/queens: Install ceph-mds in ceph-base on ubuntu/debian  https://review.opendev.org/66008810:11
openstackgerritMark Goddard proposed openstack/kolla stable/pike: Install ceph-mds in ceph-base on ubuntu/debian  https://review.opendev.org/66062310:14
mgoddardmnasiadka: done10:20
hrwbbl - replacing mainboard in desktop machine10:23
*** hrw has quit IRC10:23
*** guozijn has quit IRC10:29
*** erolg has joined #openstack-kolla10:30
*** skramaja has quit IRC10:34
*** k_mouza has joined #openstack-kolla10:38
*** k_mouza has quit IRC10:44
*** luksky has quit IRC10:47
*** k_mouza has joined #openstack-kolla10:53
*** k_mouza has quit IRC10:57
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798910:57
*** k_mouza has joined #openstack-kolla11:05
*** k_mouza has quit IRC11:06
*** k_mouza has joined #openstack-kolla11:07
*** guozijn has joined #openstack-kolla11:17
*** luksky has joined #openstack-kolla11:18
mnasiadkamgoddard: it's very interesting that the ceph jobs fail on running prechecks... with shared connection to x.x.x.x closed11:18
mgoddardmnasiadka: how can you tell? I can't see any logs11:19
mnasiadkamgoddard: we have ara logs :)11:19
mnasiadkamgoddard: http://logs.openstack.org/89/657989/6/check/kolla-ansible-centos-source-ceph/7ab211c/ara-report/result/838f1547-81ec-422d-babb-39787b7772a6/11:19
mgoddardoh, hadn't spotted tha11:19
mgoddardmnasiadka: it does seem to have got to deploy11:21
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Add support for ceilometer custom pipeline.yaml  https://review.opendev.org/66061711:21
mnasiadkamgoddard: either I'm blind again, but I see deploy.sh being run, and inside that kolla-ansible -vvv prechecks11:21
mnasiadkamgoddard: and somewhere there the host dies :)11:22
mgoddardmnasiadka: tools/kolla-ansible -i /etc/kolla/inventory -vvv deploy11:22
openstackgerritMerged openstack/kolla master: Fix multiple mon deployment failures  https://review.opendev.org/65260611:22
mnasiadkamgoddard: ah, right - blind again11:22
*** k_mouza has quit IRC11:23
mnasiadkamgoddard: either way - server dies, god knows why11:23
mnasiadkamgoddard: and we don't even know in which state :)11:23
mnasiadkastage11:23
mgoddardmnasiadka: maybe try removing the > ${LOGS}/ansible/deploy11:23
mgoddardthen you'll get logs in ara11:24
*** kplant has joined #openstack-kolla11:24
mnasiadkamgoddard: well, maybe we need to change it to tee ;)11:24
*** k_mouza has joined #openstack-kolla11:24
mgoddardmnasiadka: could do, but we tried to keep the main ansible log readable11:25
mnasiadkamgoddard: yeah, makes sense - let me see if it fails again in the same manner - if it does I'll do that.11:25
*** k_mouza has quit IRC11:25
*** k_mouza has joined #openstack-kolla11:26
openstackgerritPierre Riteau proposed openstack/kolla-ansible master: Add ansible_nodename (system hostname) to /etc/hosts  https://review.opendev.org/66046311:27
*** k_mouza has quit IRC11:27
*** k_mouza has joined #openstack-kolla11:28
*** k_mouza has quit IRC11:29
*** k_mouza has joined #openstack-kolla11:30
openstackgerritJakub Bielecki proposed openstack/kolla-ansible master: bugfix hostname for kolla-ansible repository  https://review.opendev.org/66031411:30
*** k_mouza has quit IRC11:32
*** k_mouza has joined #openstack-kolla11:32
*** sshnaidm|afk is now known as sshnaidm11:35
*** henriqueof has joined #openstack-kolla11:43
openstackgerritMerged openstack/kolla-ansible stable/stein: Set external web URL for Prometheus services  https://review.opendev.org/65556211:46
*** k_mouza has quit IRC11:48
*** skramaja has joined #openstack-kolla11:49
*** hjensas is now known as hjensas|afk11:50
*** k_mouza has joined #openstack-kolla11:50
*** k_mouza has quit IRC11:55
*** k_mouza has joined #openstack-kolla11:58
*** itlinux has joined #openstack-kolla12:12
*** itlinux has quit IRC12:12
*** hrw has joined #openstack-kolla12:15
*** janki has quit IRC12:35
*** k_mouza has quit IRC12:35
*** k_mouza has joined #openstack-kolla12:38
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Add support for ceilometer custom event yaml  https://review.opendev.org/66070212:42
*** k_mouza has quit IRC12:42
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Add support for ceilometer custom event yaml  https://review.opendev.org/66070212:45
*** itlinux has joined #openstack-kolla12:47
*** k_mouza has joined #openstack-kolla12:47
*** k_mouza has quit IRC12:51
dougszhttps://review.opendev.org/#/c/656295/12:57
dougsz^ Easy one to bump monasca log API version following a new release12:57
*** itlinux has quit IRC12:57
*** k_mouza has joined #openstack-kolla13:01
*** k_mouza has quit IRC13:06
*** igordc has joined #openstack-kolla13:06
*** goldyfruit has joined #openstack-kolla13:07
*** priteau has quit IRC13:09
*** itlinux has joined #openstack-kolla13:13
*** itlinux has quit IRC13:20
*** dave-mccowan has joined #openstack-kolla13:23
*** hjensas|afk is now known as hjensas13:25
*** k_mouza has joined #openstack-kolla13:26
*** k_mouza has quit IRC13:27
*** k_mouza_ has joined #openstack-kolla13:27
mnasiadkadougsz: finally not 404?;)13:29
dougsz?13:30
*** graysonh_ has joined #openstack-kolla13:30
dougszah - yes, finally got the release merged13:30
mnasiadkadougsz: I checked it today? And there was 404 errors when building.13:30
dougszthanks for checking!13:30
*** k_mouza_ has quit IRC13:31
*** k_mouza has joined #openstack-kolla13:32
*** k_mouza has quit IRC13:39
*** dave-mccowan has quit IRC13:40
*** k_mouza has joined #openstack-kolla13:43
*** k_mouza has quit IRC13:43
*** k_mouza has joined #openstack-kolla13:43
*** igordc has quit IRC13:51
*** iclon_ has joined #openstack-kolla14:00
*** iclon__ has quit IRC14:03
*** hrw has quit IRC14:04
*** cah_link has quit IRC14:04
*** JangwonLee has quit IRC14:09
*** k_mouza has quit IRC14:09
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug Ceph gates  https://review.opendev.org/66072514:10
*** k_mouza has joined #openstack-kolla14:11
*** k_mouza has quit IRC14:12
*** k_mouza has joined #openstack-kolla14:13
*** k_mouza has quit IRC14:16
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug Ceph gates  https://review.opendev.org/66072514:19
*** Luzi has quit IRC14:19
*** k_mouza has joined #openstack-kolla14:27
*** k_mouza has quit IRC14:27
*** itlinux has joined #openstack-kolla14:29
*** dpawlik has quit IRC14:29
*** itlinux has quit IRC14:30
*** itlinux has joined #openstack-kolla14:31
*** k_mouza has joined #openstack-kolla14:33
*** k_mouza has quit IRC14:35
openstackgerritMerged openstack/kolla stable/rocky: Use latest release of Monasca Log API  https://review.opendev.org/65629514:36
*** k_mouza has joined #openstack-kolla14:40
*** hamzaachi has quit IRC14:41
*** hamzaachi has joined #openstack-kolla14:41
*** luksky has quit IRC14:43
*** itlinux has quit IRC14:44
*** k_mouza has quit IRC14:45
*** henriqueof has quit IRC14:45
mgoddardmeeting in 10 minutes mgoddard mnasiadka hrw egonzalez14:51
*** Sravan has joined #openstack-kolla14:54
*** itlinux has joined #openstack-kolla14:54
*** k_mouza has joined #openstack-kolla14:56
*** itlinux has quit IRC14:56
*** k_mouza has quit IRC14:57
*** itlinux has joined #openstack-kolla14:57
*** k_mouza has joined #openstack-kolla14:57
*** itlinux has quit IRC14:57
*** Sravan has quit IRC14:58
*** k_mouza has quit IRC15:00
*** k_mouza has joined #openstack-kolla15:00
*** Wasaac has joined #openstack-kolla15:01
*** skramaja has quit IRC15:03
mgoddardanyone going to join the meeting today?15:04
*** itlinux has joined #openstack-kolla15:10
*** guozijn has quit IRC15:19
*** dciabrin_ is now known as dciabrin15:28
*** itlinux has quit IRC15:28
*** itlinux has joined #openstack-kolla15:30
mnasiadkamgoddard: it's getting less and less people :)15:31
mgoddardmnasiadka: seems so :/15:31
*** Wasaac has quit IRC15:31
mgoddardpeople still seem to want to use it, but not so many want to help build it15:32
mnasiadkamgoddard: well, there are some people pushing PRs, but not many attending the meeting :)15:34
mgoddardmeetings are boring :)15:35
mnasiadkatrue :)15:35
*** stingrayza has left #openstack-kolla15:40
*** stingrayza has joined #openstack-kolla15:40
openstackgerritPierre Riteau proposed openstack/kolla-ansible master: Make "Check if firewalld is installed" task quieter  https://review.opendev.org/66076515:40
*** hamzaachi has quit IRC15:42
*** bline has quit IRC15:42
*** itlinux has quit IRC15:47
*** k_mouza has quit IRC15:50
*** k_mouza has joined #openstack-kolla15:53
*** itlinux has joined #openstack-kolla15:55
*** k_mouza has quit IRC15:57
*** itlinux has quit IRC15:59
*** itlinux has joined #openstack-kolla16:01
*** hamzaachi has joined #openstack-kolla16:04
*** henriqueof has joined #openstack-kolla16:05
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Simplify handler conditionals  https://review.opendev.org/64769916:08
*** unicell has joined #openstack-kolla16:08
*** jbadiapa has quit IRC16:10
*** igordc has joined #openstack-kolla16:12
*** Sravan has joined #openstack-kolla16:13
*** itlinux has quit IRC16:20
*** itlinux has joined #openstack-kolla16:21
*** jbadiapa has joined #openstack-kolla16:24
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: CI Test: Support being source of a Ceph upgrade  https://review.opendev.org/65816316:27
*** lemko has quit IRC16:28
*** hamzaachi has quit IRC16:34
*** devep has quit IRC16:35
*** itlinux has quit IRC16:42
*** itlinux has joined #openstack-kolla16:48
*** unicell has quit IRC16:49
kplantmgoddard: why don't you have the meetings in here?16:49
kplantyou might get more attendees by accident :>16:50
*** dougsz has quit IRC17:03
*** Sravan has quit IRC17:06
*** itlinux has quit IRC17:13
*** luksky has joined #openstack-kolla17:23
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: Adds Qinling Ansible role  https://review.opendev.org/66078917:24
*** devep has joined #openstack-kolla17:35
*** jonaspaulo has joined #openstack-kolla17:35
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: Adds Qinling Ansible role  https://review.opendev.org/66078917:37
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Add --force to enabling ceph dashboard  https://review.opendev.org/66079217:41
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798917:42
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798917:46
*** itlinux has joined #openstack-kolla17:47
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798917:50
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: Adds Qinling Ansible role  https://review.opendev.org/66078918:11
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: Adds Qinling Ansible role  https://review.opendev.org/66078918:20
*** gfidente is now known as gfidente|afk18:35
*** dpawlik has joined #openstack-kolla18:36
*** itlinux has quit IRC18:39
*** itlinux has joined #openstack-kolla18:40
*** itlinux has quit IRC18:40
*** itlinux has joined #openstack-kolla18:44
*** sshnaidm is now known as sshnaidm|afk18:45
*** Sravan has joined #openstack-kolla18:46
*** Sravan_ has joined #openstack-kolla18:48
*** Sravan_ has quit IRC18:50
*** Sravan has quit IRC18:51
*** itlinux has quit IRC18:51
*** Sravan has joined #openstack-kolla18:52
klindgrenSo, we noticed that when we added another ~150 servers to our koala-ansible install is consuming significantly more memory than expected.  So much so that we had to drop the number of forks to keep it from oomkilling ansible-playbooks.  Looking at open shift they had a similar problem and addressed it via:19:25
klindgrenhttps://access.redhat.com/errata/RHBA-2018:060119:25
klindgrenThe node scale-up and installation Ansible playbooks may have consumed more memory on the control host (the system where you run the playbooks from) than expected due to the use of `include_tasks` in several places. The majority of these instances have now been converted to `import_tasks` calls, which do not consume as much memory. After this change, memory consumption on the control host should be below 100MiB per host; for large environments (119:26
klindgren00+ hosts), a control host with at least 16GiB of memory is recommended. (BZ#1558672)19:26
klindgrenSo kolla-ansible pretty much uses include_tasks everywhere.19:27
*** spiette_ has quit IRC19:36
*** spiette has joined #openstack-kolla19:38
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: Adds Qinling Ansible role  https://review.opendev.org/66078919:40
*** kplant has quit IRC19:41
*** Sravan has quit IRC19:55
*** itlinux has joined #openstack-kolla20:11
*** pcaruana has quit IRC20:20
*** itlinux has quit IRC20:21
*** Sravan has joined #openstack-kolla20:26
klindgrenGoing to try to see if changing some of the roles over to using import_tasks vs's include_tasks results in a memory decrease and perf improvement, as apparently referenced by a few bug reports20:30
*** Sravan has quit IRC20:30
*** whoami-rajat has quit IRC20:34
*** dpawlik has quit IRC20:51
*** Sravan has joined #openstack-kolla20:59
*** Talion has joined #openstack-kolla21:05
*** stingrayza has quit IRC21:06
*** stingrayza has joined #openstack-kolla21:11
*** henriqueof has quit IRC21:36
*** pomac has quit IRC21:40
*** gfidente|afk has quit IRC21:43
*** pomac has joined #openstack-kolla21:48
*** hrw has joined #openstack-kolla21:48
*** Chaserjim has joined #openstack-kolla21:53
*** iclon__ has joined #openstack-kolla22:05
*** iclon_ has quit IRC22:08
*** itlinux has joined #openstack-kolla22:21
*** hamzaachi has joined #openstack-kolla22:21
*** hrw has quit IRC22:24
*** cah_link has joined #openstack-kolla22:24
*** goldyfruit_ has joined #openstack-kolla22:28
*** goldyfruit has quit IRC22:30
*** goldyfruit_ has quit IRC22:33
*** hamzaachi has quit IRC22:36
*** goldyfruit has joined #openstack-kolla22:42
*** itlinux has quit IRC22:43
*** Sravan has quit IRC22:46
*** cah_link has quit IRC22:47
*** itlinux has joined #openstack-kolla22:49
*** igordc has quit IRC23:01
*** graysonh_ has quit IRC23:08
*** goldyfruit has quit IRC23:21
*** itlinux has quit IRC23:28
*** itlinux has joined #openstack-kolla23:32
*** itlinux has quit IRC23:46
*** goldyfruit has joined #openstack-kolla23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!