Thursday, 2019-08-01

*** Wasaac has joined #openstack-kolla00:00
*** jrist has quit IRC00:01
*** nafiux has quit IRC00:02
*** jrist has joined #openstack-kolla00:03
*** jrist has quit IRC00:04
*** Wasaac has quit IRC00:05
*** nafiux has joined #openstack-kolla00:06
*** jrist has joined #openstack-kolla00:10
*** nafiux has quit IRC00:14
*** jrist has quit IRC00:15
*** jrist has joined #openstack-kolla00:17
*** ivve has quit IRC00:24
*** nafiux has joined #openstack-kolla00:35
*** kplant has quit IRC00:46
*** gregoryo has joined #openstack-kolla00:47
*** nafiux has quit IRC01:14
*** radeks_ has joined #openstack-kolla01:19
*** nafiux has joined #openstack-kolla01:21
*** radeks_ has quit IRC01:24
*** igordc has quit IRC01:31
*** happyhemant has quit IRC01:35
*** nafiux has quit IRC01:38
*** Wasaac has joined #openstack-kolla02:01
*** Wasaac has quit IRC02:06
*** kevinz has quit IRC02:09
*** kevinz has joined #openstack-kolla02:09
*** kevinz has quit IRC02:10
*** BjoernT has joined #openstack-kolla02:33
*** sm806 has quit IRC02:58
*** sm806 has joined #openstack-kolla02:58
*** gkadam has joined #openstack-kolla03:29
*** gkadam has quit IRC03:34
*** gkadam has joined #openstack-kolla03:35
*** whoami-rajat has joined #openstack-kolla03:41
*** hongbin has joined #openstack-kolla03:45
*** hongbin has quit IRC03:46
*** Wasaac has joined #openstack-kolla04:02
*** Wasaac has quit IRC04:06
*** Luzi has joined #openstack-kolla04:25
*** BjoernT has quit IRC04:43
*** skramaja has joined #openstack-kolla04:45
*** goldyfruit has joined #openstack-kolla04:59
*** gfidente has joined #openstack-kolla05:02
*** goldyfruit has quit IRC05:04
*** nafiux has joined #openstack-kolla05:11
*** kevinz has joined #openstack-kolla05:12
*** dpawlik has joined #openstack-kolla05:26
*** kevinz has quit IRC05:31
*** radeks has joined #openstack-kolla05:52
openstackgerritMerged openstack/kolla-ansible stable/stein: Ingest non-standard Monasca logs  https://review.opendev.org/67107705:52
openstackgerritMerged openstack/kolla-ansible stable/stein: Increase log coverage for Monasca  https://review.opendev.org/67107005:57
yoctozeptomorning05:59
*** Wasaac has joined #openstack-kolla06:03
*** radeks has quit IRC06:08
*** gdwornicki has quit IRC06:09
*** Wasaac has quit IRC06:09
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: ceph: Fix issues with ganesha-nfs  https://review.opendev.org/67316506:14
*** henriqueof has joined #openstack-kolla06:17
mnasiadkamorning06:20
*** janki has joined #openstack-kolla06:22
*** hamdyk has joined #openstack-kolla06:24
*** kevinz has joined #openstack-kolla06:25
mnasiadkayoctozepto: I see the change is getting bigger and bigger :)06:32
yoctozeptomnasiadka: debugging atm06:34
yoctozeptobut yeah, the fixes are coming in :D06:34
mnasiadkayoctozepto: the cap mds thing probably comes because of my change to do "allow *" instead of "allow"06:38
mnasiadkayoctozepto: but it was required for nautilus...06:38
yoctozeptoyeah, exactly, no idea though why the module tried doing this change that way06:38
yoctozeptoit should do update06:38
yoctozeptonot get-or-create06:39
yoctozeptonow debugging06:39
yoctozeptoit's one of those situations06:39
yoctozeptowhere fixing one bug triggers another06:39
yoctozeptomost probably timing stuff06:39
yoctozeptosomething not being waited for properly etc.06:39
yoctozepto[errno 110] error connecting to the cluster06:55
yoctozeptojust what I imagined06:55
yoctozeptowe should check what the error is and not proceed as if it was just user missing eh06:55
hrwmorning06:58
*** mrunge_ is now known as mrunge07:04
*** pcaruana has quit IRC07:12
*** rpittau|afk is now known as rpittau07:13
mnasiadkayoctozepto: so you're in deep shit, just as me with ansible bump up, needs rewriting keystone stuff in all of our roles :D07:15
mnasiadkaI might even try to get rid of kolla_keystone_* modules...07:16
yoctozeptomnasiadka: we're paying off someone's technical debt :-(07:17
mnasiadkayoctozepto: well, just as in real life :)07:17
*** NOXOiD has quit IRC07:19
yoctozeptomnasiadka: yeah, since every country is in debt :-)07:19
hrw;D07:21
*** nafiux has quit IRC07:23
*** Wasaac has joined #openstack-kolla07:31
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Change kolla_keystone modules to upstream  https://review.opendev.org/67394907:38
*** shyamb has joined #openstack-kolla07:42
shyambHi07:42
*** pcaruana has joined #openstack-kolla07:42
shyambWe are doing upgrade from queens to rocky on centos platform07:42
shyambusing kolla-ansible07:42
openstackgerritMichal Nasiadka proposed openstack/kolla master: Bump up pkg versions in kolla-toolbox  https://review.opendev.org/64241607:43
shyambupgrade is failing at "rabbitmq : Find gospel node" task07:43
shyambAny thoughts about this issue?07:43
*** lemko has joined #openstack-kolla07:48
*** ivve has joined #openstack-kolla07:51
*** gregoryo has quit IRC07:51
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Change kolla_keystone modules to upstream  https://review.opendev.org/67394908:01
mnasiadkashyamb: tried running the command manually? https://opendev.org/openstack/kolla-ansible/src/branch/stable/rocky/ansible/roles/rabbitmq/tasks/upgrade.yml#L1908:03
*** goldyfruit has joined #openstack-kolla08:04
shyambno08:07
shyambIt's my test env08:07
shyambjust destroyed and will redeploy08:07
shyambsome people faced this issue08:08
shyambthey are saying /etc/hosts may have incorrect entries08:08
mnasiadkashyamb: and you're running kolla-ansible upgrade, not deploy when you install rocky version of kolla-ansible package?08:08
*** goldyfruit has quit IRC08:09
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: ceph: Fix issues with ganesha-nfs  https://review.opendev.org/67316508:09
*** happyhemant has joined #openstack-kolla08:09
*** gdwornicki has joined #openstack-kolla08:23
yoctozeptomnasiadka: https://review.opendev.org/671684 <- I think by now we know it should be abandoned ;-)08:27
yoctozeptohrw: https://review.opendev.org/673006 <- make me happy :-)08:28
mnasiadkayoctozepto: yeah, let me abandon that08:28
hrwyoctozepto: why you gave +2 for own change? :D08:28
*** Wasaac has quit IRC08:28
yoctozeptohrw: because I like it very much :D08:29
*** Wasaac has joined #openstack-kolla08:29
hrwyoctozepto: you sent patch == you like it08:29
yoctozeptohrw: then it shouldn't let me like it more08:29
*** ivve has quit IRC08:30
yoctozeptoit's like the heart eyes you can react with on messenger08:30
hrw;D08:30
mnasiadkayoctozepto: the agreed on "true" thing - do we have a style guide? :)08:32
yoctozeptomnasiadka: we should have, it was more like me discussing it with mgoddard ;-)08:33
yoctozeptothough perceptually "true" is more common08:33
mnasiadkayoctozepto: well, my problem is that in inventory it must be True, while in playbooks you can do both :)08:34
yoctozepto2081 vs 843 in our code08:34
mnasiadkayoctozepto: and I have like... asperger's disease08:34
yoctozeptoplus ansible docs seem to prefere true08:34
yoctozeptolet's settle on TRUE then08:34
yoctozepto;D08:35
mnasiadkahaha08:35
yoctozeptomust it be "True" in inventory? I  think I have slapped some yes/noes in there08:35
hrwTrue, False, File_not_found08:35
mnasiadkayoctozepto: inventory is ini file, not yaml, so yes - True :D08:35
yoctozeptoFiLe_nOt_fOuNd08:35
yoctozeptomnasiadka: ah yes, I forgot I outsourced the vars to yaml vars08:36
yoctozeptoyou are right08:37
*** ivve has joined #openstack-kolla08:40
shyambmnasiadka: yes08:44
shyambI am running upgrade08:44
mnasiadkagood, just wanted to make sure08:46
*** priteau has joined #openstack-kolla08:53
*** shyamb has quit IRC09:06
*** shyamb has joined #openstack-kolla09:07
*** janki has quit IRC09:08
*** janki has joined #openstack-kolla09:09
openstackgerritMerged openstack/kolla-ansible stable/stein: Don't drop unmatched Kolla service logs  https://review.opendev.org/67107609:12
*** shyamb has quit IRC09:12
*** jchhatbar has joined #openstack-kolla09:17
*** k_mouza has joined #openstack-kolla09:17
*** janki has quit IRC09:17
*** brtknr has joined #openstack-kolla09:22
*** jchhatbar has quit IRC09:25
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: ceph: Fix issues with ganesha-nfs  https://review.opendev.org/67316509:26
yoctozeptohrw: https://review.opendev.org/67389709:27
hrwdone09:28
openstackgerritMerged openstack/kolla-ansible master: CI: fix checks for upgrade and multinode jobs  https://review.opendev.org/67300609:33
*** ivve has quit IRC09:37
*** Wasaac has quit IRC09:39
*** Wasaac has joined #openstack-kolla09:39
*** priteau has quit IRC09:43
*** priteau has joined #openstack-kolla09:44
*** priteau has quit IRC09:50
*** priteau has joined #openstack-kolla09:52
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/stein: CI: fix checks for upgrade and multinode jobs  https://review.opendev.org/67398909:53
*** goldyfruit has joined #openstack-kolla09:55
*** shyamb has joined #openstack-kolla09:55
*** kevinz has quit IRC09:57
openstackgerritMerged openstack/kolla-ansible stable/stein: Format internal Fluentd logs  https://review.opendev.org/67107810:00
*** goldyfruit has quit IRC10:00
stingrayzamorning all :)10:04
*** rpittau is now known as rpittau|bbl10:04
*** kevinz has joined #openstack-kolla10:08
*** lemko has quit IRC10:11
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: ceph: Fix issues with ganesha-nfs  https://review.opendev.org/67316510:29
openstackgerritMerged openstack/kolla-ansible stable/stein: Fix fluentd monasca pos path for Debian/Ubuntu x86_64  https://review.opendev.org/67389710:45
*** dpawlik has quit IRC10:58
*** gdwornicki has quit IRC11:04
*** dpawlik has joined #openstack-kolla11:07
*** ivve has joined #openstack-kolla11:07
*** goldyfruit has joined #openstack-kolla11:14
*** hamdyk has quit IRC11:17
*** hamdyk has joined #openstack-kolla11:20
*** shyamb has quit IRC11:20
*** kplant has joined #openstack-kolla11:21
*** goldyfruit has quit IRC11:21
*** shyamb has joined #openstack-kolla11:21
*** dpawlik has quit IRC11:27
*** hamdyk has quit IRC11:35
*** ivve has quit IRC11:37
yoctozeptomnasiadka: yeah, managed to get what it hangs at: docker exec ceph_mon ceph osd df11:46
yoctozeptoceph -s passes fine11:46
*** priteau has quit IRC11:46
yoctozeptooutput: https://logs.opendev.org/65/673165/17/check/kolla-ansible-ubuntu-source-ceph/a31c7ab/primary/logs/kolla/ceph/ceph_s.txt11:46
*** shyamb has quit IRC11:46
yoctozeptoseems without mgr (or maybe without osds ;-) ) osd commands never finish11:47
*** shyamb has joined #openstack-kolla11:47
yoctozeptono output was recorded, full hang11:47
yoctozeptohttps://tracker.ceph.com/issues/2130011:48
yoctozeptoseems they traded kill for a hang ;p11:49
yoctozeptowhat's worse11:49
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: ceph: Fix issues with ganesha-nfs  https://review.opendev.org/67316511:50
*** k_mouza has quit IRC11:54
*** k_mouza has joined #openstack-kolla11:55
*** mloza has quit IRC11:55
*** dpawlik has joined #openstack-kolla11:57
*** k_mouza has quit IRC12:00
*** dpawlik has quit IRC12:01
*** gkadam has quit IRC12:10
mnasiadkayoctozepto: without mgr is the key12:12
*** dpawlik has joined #openstack-kolla12:29
*** Luzi has quit IRC12:55
*** rpittau|bbl is now known as rpittau12:57
*** k_mouza has joined #openstack-kolla13:05
openstackgerritjacky06 proposed openstack/kolla-cli master: Move sphinx-build into tox  https://review.opendev.org/67387113:08
*** skramaja has quit IRC13:10
*** shyamb has quit IRC13:11
openstackgerritjacky06 proposed openstack/kolla-cli master: Move sphinx-build into tox  https://review.opendev.org/67387113:13
*** mchlumsky has joined #openstack-kolla13:13
openstackgerritjacky06 proposed openstack/kolla-cli master: Move sphinx-build into tox  https://review.opendev.org/67387113:18
*** k_mouza has quit IRC13:22
*** priteau has joined #openstack-kolla13:28
*** mchlumsky has quit IRC13:28
*** mchlumsky has joined #openstack-kolla13:30
openstackgerritjacky06 proposed openstack/kolla-cli master: Move sphinx-build into tox  https://review.opendev.org/67387113:32
*** kevinz has quit IRC13:51
*** priteau has quit IRC13:53
*** k_mouza has joined #openstack-kolla14:00
*** priteau has joined #openstack-kolla14:01
*** BjoernT has joined #openstack-kolla14:01
*** lemko has joined #openstack-kolla14:08
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Change kolla_keystone modules to upstream  https://review.opendev.org/67394914:08
*** BjoernT_ has joined #openstack-kolla14:09
*** k_mouza has quit IRC14:10
*** BjoernT has quit IRC14:10
openstackgerritjacky06 proposed openstack/kolla-cli master: Move sphinx-build into tox  https://review.opendev.org/67387114:14
*** k_mouza has joined #openstack-kolla14:22
*** Wasaac has quit IRC14:27
*** Wasaac_ has joined #openstack-kolla14:27
yoctozeptomnasiadka: though osd tree works despite mgr being gone, the sooner we move to ceph-ansible, the better ;p14:33
yoctozeptobut still, checking things like that will require more effort from us14:40
yoctozeptoeh :/14:41
*** dpawlik has quit IRC14:43
mnasiadkayoctozepto: don't think ceph-ansible works like my little pony14:45
*** priteau has quit IRC14:56
openstackgerritjacky06 proposed openstack/kolla-cli master: Move sphinx-build into tox  https://review.opendev.org/67387115:01
mnasiadkayoctozepto: why do we even need osd df? I would rather vote for gather osd tree and versions, and of course health status along with some warnings/error logs15:32
mnasiadka*gathering15:32
*** drwahl has left #openstack-kolla15:35
*** Wasaac_ is now known as Wasaac15:38
*** stingrayza has quit IRC15:50
*** priteau has joined #openstack-kolla15:50
*** whoami-rajat has quit IRC15:51
*** stingrayza has joined #openstack-kolla15:51
*** dpawlik has joined #openstack-kolla15:52
*** gkadam has joined #openstack-kolla15:57
*** dpawlik has quit IRC15:59
*** gkadam has quit IRC16:01
yoctozeptomnasiadka: I have no idea, probably because someone wanted to check whether image creation issues are related to lack of space?16:08
*** igordc has joined #openstack-kolla16:08
yoctozeptore: ceph-ansible - I know, I opted for my in-house script for deploying my ceph cluster (shared between os and other services) instead of ceph-ansible because it was doing something funny to centos back then and I did not bother to play with it further16:09
yoctozepto;p16:09
*** ksdean has joined #openstack-kolla16:12
ksdeanHi There could someone tell me whats going on with kolla and ceph i see a warning about it being deprecated ?16:12
yoctozeptoksdean: yeah, we are deprecating the deployment of ceph cluster via kolla, external ceph cluster is and will be supported16:15
ksdeanwhy is that ?16:16
yoctozeptothe thing is ceph is not part of openstack per se and they have the dedicated tooling for it - ceph-ansible - no need to duplicate the work16:16
ksdeanahh ok, what release will it be deprecated in as i was about to deploy it using kolla16:16
yoctozeptono need to worry, we have deprecated it in stein but we are supporting it still in stein and train for sure16:17
ksdeanok, I was about to deploy into a production environment hence why im asking here16:18
yoctozeptoatm I'm working on some ceph fixes16:18
ksdeanthanks for putting my mind at ease and I will take note16:18
yoctozeptono problem16:18
yoctozeptohonestly, the deprecation is to block adding new features/integrations around ceph16:19
yoctozeptoand advertise ceph-ansible as U release will most likely be migrated to it16:20
ksdeanok, cool. One quick question. is it better to install openstack core then add services to it such as manila etc ?16:21
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/stein: Add 'allow *' to getting ceph mds keyring  https://review.opendev.org/67332016:30
openstackgerritRadosław Piliszek proposed openstack/kolla master: ceph mds cap 'allow *' in extend_start  https://review.opendev.org/67408816:36
yoctozeptoksdean: it is better in the sense that you feel better because something already works16:37
yoctozepto:-)16:37
yoctozeptoit works from zero to hero as well16:37
*** rpittau is now known as rpittau|afk16:40
*** nafiux has joined #openstack-kolla16:45
*** lemko has quit IRC16:47
*** dougsz has joined #openstack-kolla16:50
*** k_mouza has quit IRC16:52
*** ivve has joined #openstack-kolla16:55
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: ceph: fixes to deployment and upgrade  https://review.opendev.org/67316516:57
*** Wasaac has quit IRC16:58
*** dougsz has quit IRC17:00
*** nafiux has quit IRC17:02
*** k_mouza has joined #openstack-kolla17:02
*** igordc has quit IRC17:03
*** kdean has joined #openstack-kolla17:04
*** priteau has quit IRC17:05
*** ksdean has quit IRC17:06
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: ceph: fixes to deployment and upgrade  https://review.opendev.org/67316517:08
yoctozeptomnasiadka: ^ the only known issue with ceph left now is the one where upgrade does not wait for all services to raise their versions17:09
*** whoami-rajat has joined #openstack-kolla17:15
*** igordc has joined #openstack-kolla17:27
*** igordc has quit IRC17:30
*** Wasaac has joined #openstack-kolla17:34
*** Wasaac has quit IRC17:40
*** nafiux has joined #openstack-kolla17:41
*** igordc has joined #openstack-kolla17:42
*** gfidente has quit IRC18:03
*** k_mouza has quit IRC18:45
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Get rid of EPEL repo on target hosts  https://review.opendev.org/67411419:16
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Get rid of EPEL repo on target hosts  https://review.opendev.org/67411419:17
*** jschluet has joined #openstack-kolla19:21
*** jschlueter has quit IRC19:21
*** dpawlik has joined #openstack-kolla19:32
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Do not require EPEL repo on RHEL-based target hosts  https://review.opendev.org/67411419:32
*** Wasaac has joined #openstack-kolla19:35
*** nafiux has quit IRC19:38
*** Wasaac has quit IRC19:40
openstackgerritRadosław Piliszek proposed openstack/kolla master: DNM: test dropping epel repo  https://review.opendev.org/67412319:44
*** Luzi has joined #openstack-kolla20:06
*** kplant has quit IRC20:29
*** iniazi has joined #openstack-kolla20:59
*** cloudnull has quit IRC21:06
*** cloudnull has joined #openstack-kolla21:07
*** nafiux has joined #openstack-kolla21:13
*** Wasaac has joined #openstack-kolla21:36
*** Wasaac has quit IRC21:39
*** Wasaac_ has joined #openstack-kolla21:39
*** Wasaac_ has quit IRC21:45
*** Luzi has quit IRC21:46
*** henriqueof has quit IRC22:07
*** henriqueof has joined #openstack-kolla22:07
*** BjoernT_ has quit IRC22:17
*** mnasiadka_ has joined #openstack-kolla22:30
*** mnasiadka has quit IRC22:37
*** mnasiadka_ is now known as mnasiadka22:37
*** whoami-rajat has quit IRC22:55
*** nafiux has quit IRC23:00
*** ivve has quit IRC23:05
*** nafiux has joined #openstack-kolla23:06
*** igordc has quit IRC23:10
*** Wasaac has joined #openstack-kolla23:38
*** Wasaac has quit IRC23:43

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!