Friday, 2019-08-23

*** owalsh has joined #openstack-kolla00:13
*** nde has joined #openstack-kolla00:30
ndeHi all!  It seems like no matter what I do my deploy crashes on "remove restart policy from slave mariadb".  No mariadb containers are ever deployed.00:35
*** keper7 has joined #openstack-kolla00:35
ndeDoes anyone have any ideas to point me in the right direction perhaps?00:35
*** nde has quit IRC00:56
*** keper7 has quit IRC01:27
*** keper7 has joined #openstack-kolla01:30
*** gregoryo has joined #openstack-kolla02:32
*** dave-mccowan has joined #openstack-kolla02:35
*** keper7 has quit IRC02:58
*** goldyfruit has quit IRC03:01
*** KeithMnemonic1 has quit IRC03:06
*** KeithMnemonic has joined #openstack-kolla03:12
*** gkadam has joined #openstack-kolla03:19
*** rohit02 has joined #openstack-kolla03:37
*** zhangfei has joined #openstack-kolla03:48
*** skramaja has joined #openstack-kolla04:55
*** janki has joined #openstack-kolla05:08
*** dave-mccowan has quit IRC05:16
*** jbadiapa has quit IRC05:47
*** jbadiapa has joined #openstack-kolla05:48
*** gregoryo has quit IRC05:56
*** skramaja has quit IRC06:08
*** skramaja has joined #openstack-kolla06:11
yoctozeptomorning06:36
osmanlicilegigood morning06:36
yoctozeptonde: https://review.opendev.org/677453 this fixes the problem, though only partial deploys are affected, fresh installations should work fine either way06:38
yoctozeptomnasiadka: https://review.opendev.org/677453 - hi prio06:39
yoctozeptomorning osmanlicilegi06:39
*** gkadam has quit IRC06:43
mnasiadkamnasiadka: thanks for waking up :) fire in the hole ;)06:47
*** shyamb has joined #openstack-kolla06:49
yoctozeptomnasiadka: you replied to yourself06:51
yoctozeptomaybe you got some decaf instead of real coffee06:52
yoctozepto:D06:52
mnasiadkayoctozepto: you see? I should sleep :)06:52
yoctozeptonighty night06:52
yoctozeptoit did not finish publishing the latest images, so I will recheck the fluentd change later06:53
mnasiadkaok07:13
*** gfidente|afk is now known as gfidente07:20
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Fix periodic jobs for Ironic and Zun  https://review.opendev.org/67817007:38
*** shyamb has quit IRC07:51
openstackgerritMerged openstack/kolla-ansible stable/stein: Don't stop mariadb slaves if they aren't running  https://review.opendev.org/67745307:51
openstackgerritMerged openstack/kolla stable/rocky: Fixes wrong local_setting.py file issue for Ubuntu binary deployment.  https://review.opendev.org/67768007:53
*** dr_feelgood has joined #openstack-kolla08:03
*** dougsz has joined #openstack-kolla08:14
*** rpittau|afk is now known as rpittau08:14
*** k_mouza has joined #openstack-kolla08:18
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Use Docker healthchecks  https://review.opendev.org/67638908:24
*** ivve has joined #openstack-kolla08:26
*** rohit02 has quit IRC08:33
*** shyamb has joined #openstack-kolla08:43
mnasiadkayoctozepto: https://review.opendev.org/#/c/677957/ - can we +w it, or what's the problem? :)08:44
yoctozeptomnasiadka: Mark almost never slaps w+1 before v+108:47
mnasiadkayoctozepto: makes sense, I can do that myself ;)08:47
yoctozeptono need, I did08:48
*** janki has quit IRC08:48
*** dr_feelgood has quit IRC08:52
*** shyamb has quit IRC08:56
*** rohit02 has joined #openstack-kolla08:59
*** shyamb has joined #openstack-kolla09:04
*** owalsh is now known as owalsh|away09:16
mnasiadkayoctozepto: so, let's keep it straight, users should copy client.openstack or client.cinder ceph key as client.nova in k-a? :)09:22
mnasiadka(I'll make it a variable later, first we need to update the docs and backport)09:23
yoctozeptomnasiadka: client.cinder is enough09:25
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: docs: ceph-ansible usage, add note around ceph keys  https://review.opendev.org/67793909:27
mnasiadkayoctozepto: thought so, change updated - after it's merged I'll do the variables thing09:28
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637609:34
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Remove Nova legacy upgrade  https://review.opendev.org/67819009:37
yoctozeptomnasiadka: that stops ppl asking which key09:37
mnasiadkayoctozepto: that's my approach, the variables thing is rather a new feature, and we shouldn't backport it all the way to pike ;)09:38
yoctozeptoindeed, mnasiadka, indeed09:42
openstackgerritMerged openstack/kolla-ansible stable/stein: Add --force to ceph mgr dashboard enablement  https://review.opendev.org/67795709:53
*** shyamb has quit IRC09:57
*** hjensas has quit IRC10:00
*** dr_feelgood has joined #openstack-kolla10:01
*** shyamb has joined #openstack-kolla10:22
*** roukoswarf has quit IRC10:24
*** rouk has joined #openstack-kolla10:24
yoctozeptodougsz: I wanted to do some reviews for kayobe but got stuck at https://review.opendev.org/676907 - I seem unable to retrieve the releated story/task - could it be that it was made private? it makes little sense to link to it in the releasenotes if it can't be read...10:24
dougszyoctozepto, right you are, I will see if I can fix it10:27
dougszthanks for taking a look :)10:27
dougszArgh, I think Storyboard may be borked, I get immediately logged out with error code: invalid_grant10:29
*** dr_feelgood has quit IRC10:29
*** dr_feelgood has joined #openstack-kolla10:31
*** rohit02 has quit IRC10:32
*** rohit02 has joined #openstack-kolla10:35
openstackgerritDoug Szumski proposed openstack/kolla master: Remove Nova legacy upgrade  https://review.opendev.org/67819610:36
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Remove Nova legacy upgrade  https://review.opendev.org/67819010:37
dougszyoctozepto: I have pinged #storyboard10:42
*** shyamb has quit IRC10:44
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Use Docker healthchecks  https://review.opendev.org/67638910:46
*** dr_feelgood has quit IRC10:52
yoctozeptodougsz: thanks :-)10:53
*** dr_feelgood has joined #openstack-kolla10:55
*** shyamb has joined #openstack-kolla11:04
*** k_mouza has quit IRC11:05
*** k_mouza_ has joined #openstack-kolla11:05
jovial[m]does a recheck rerun the gate?11:09
yoctozeptojovial[m]: yes11:09
yoctozeptofirst all checks11:10
yoctozeptothen gate11:10
yoctozeptoif w+111:10
jovial[m]ok, thanks11:10
yoctozeptodougsz: now that sb works for you, can you access that story?11:10
yoctozeptoor is it gone?11:10
yoctozeptolong gone....11:10
dougszyoctozepto, was just looking at that, I still get `404: GET /api/v1/stories/2006409: Story 2006409 not found`, same for the task11:11
yoctozeptoso gone11:11
*** hjensas has joined #openstack-kolla11:11
dougszI've tried searching for the issue but haven't found it yet, although I never have much luck with Storyboard search11:11
yoctozeptomgoddard has created it for himself ;D11:11
dougsz:)11:12
yoctozeptodougsz: it's no joke, it's possible in storyboard11:12
yoctozeptothe software of wonders11:12
dougszI am definitely a launchpad person, it seems so much easier to find bugs11:13
*** goldyfruit has joined #openstack-kolla11:14
yoctozeptoyeah, for that reason I am into launchpad more too11:15
*** k_mouza_ has quit IRC11:15
yoctozeptothough sb looks more flexible11:15
yoctozeptoyet it feels clumsy in the basic stuff11:15
yoctozeptocan't really explain but you probably feel very same11:15
*** k_mouza has joined #openstack-kolla11:15
*** skramaja has quit IRC11:16
dougszyeah, it doesn't help that the search seems so slow.11:17
*** k_mouza has quit IRC11:20
yoctozeptodougsz: or you enter the dashboard and it happily greets you with 'there is nothing' to load the things a few seconds later11:20
*** tonythomas has quit IRC11:25
*** rohit02 has quit IRC11:28
*** zhangfei has quit IRC12:00
*** dave-mccowan has joined #openstack-kolla12:06
*** k_mouza has joined #openstack-kolla12:12
*** shyamb has quit IRC12:21
*** shyamb has joined #openstack-kolla12:25
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637612:27
mnasiadkayoctozepto: storyboard always is borked? :)12:27
*** hjensas has quit IRC12:27
*** shyamb has quit IRC12:41
openstackgerritScott Solkhon proposed openstack/kolla master: Add effective and permitted capability to blackbox exporter  https://review.opendev.org/67715612:45
*** roukoswarf has joined #openstack-kolla12:59
*** rouk has quit IRC13:00
openstackgerritDoug Szumski proposed openstack/kolla master: Remove Nova legacy upgrade  https://review.opendev.org/67819613:02
openstackgerritMerged x/kayobe stable/rocky: Disable chrony container by default  https://review.opendev.org/67664313:05
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Remove Nova legacy upgrade  https://review.opendev.org/67819013:08
*** keper7 has joined #openstack-kolla13:08
osmanlicilegiyoctozepto: are you there?13:13
osmanlicilegiyoctozepto: i'm having issue while scaling, please see https://hastebin.com/amucexatew.php. can it be caused by https://review.opendev.org/#/c/673165/ ?13:21
*** ivve has quit IRC13:24
*** BjoernT has joined #openstack-kolla13:27
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Remove invalid reference to Placement API  https://review.opendev.org/67822513:28
yoctozeptoosmanlicilegi: the fix for your issue is here: https://review.opendev.org/67003513:30
osmanlicilegiyoctozepto: didn't see it, thanks!13:35
*** keper7 has quit IRC13:42
*** spiette has quit IRC13:46
mnasiadkayoctozepto: it has been backported, but I think we didn't do a release of subsequent branches for some time13:48
*** spiette has joined #openstack-kolla13:48
yoctozeptomnasiadka: yes, we did not13:48
yoctozeptothough all backports look the same13:49
yoctozeptoit was a simple change13:49
mnasiadkamaybe we need to start flagging changes as a release candidate = fixing some common deployment problems14:05
yoctozeptomnasiadka: yeah, I also miss a mechanism for flagging as backport candidate14:06
*** dpawlik has quit IRC14:20
*** keper7 has joined #openstack-kolla14:44
*** keper7 has quit IRC14:44
openstackgerritWill Szumski proposed x/kayobe master: Improve detection of base path  https://review.opendev.org/65586414:47
*** dpawlik has joined #openstack-kolla15:00
openstackgerritWill Szumski proposed x/kayobe-config stable/rocky: Add kayobe as a dependency  https://review.opendev.org/67825115:09
openstackgerritDoug Szumski proposed openstack/kolla master: Remove Nova legacy upgrade  https://review.opendev.org/67819615:10
openstackgerritDoug Szumski proposed openstack/kolla master: Remove Nova legacy upgrade  https://review.opendev.org/67819615:15
openstackgerritWill Szumski proposed x/kayobe-config master: Add kayobe as a dependency  https://review.opendev.org/65380315:19
*** dave-mccowan has quit IRC15:20
*** rpittau is now known as rpittau|afk15:32
*** keper7 has joined #openstack-kolla15:34
openstackgerritJason Anderson proposed openstack/kolla-ansible master: [blazar] Use database_address as default DB addr  https://review.opendev.org/67825815:38
*** k_mouza_ has joined #openstack-kolla15:51
*** k_mouza_ has quit IRC15:53
*** k_mouza has quit IRC15:54
*** keper7 has quit IRC15:54
*** k_mouza has joined #openstack-kolla15:54
*** hamzy has quit IRC15:59
*** keper7 has joined #openstack-kolla16:06
*** dpawlik has quit IRC16:13
*** dr_feelgood has quit IRC16:24
*** dpawlik has joined #openstack-kolla16:29
*** dpawlik has quit IRC16:34
*** k_mouza has quit IRC16:44
*** dougsz has quit IRC16:54
*** keper7 has quit IRC16:57
openstackgerritMerged x/kayobe stable/rocky: Fix inspector iPXE boot with UEFI compute nodes  https://review.opendev.org/67793216:59
*** igordc has joined #openstack-kolla17:09
*** keper7 has joined #openstack-kolla17:11
*** keper7 has quit IRC17:24
*** hamzy has joined #openstack-kolla17:35
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Fix Zun Docker remote API host  https://review.opendev.org/67827617:40
*** jonaspaulo has joined #openstack-kolla17:41
yoctozeptogoldyfruit: yup, you probably figured out I'm trying to up Zun support in k-a17:47
goldyfruityoctozepto, :)17:47
yoctozeptounfortunately not every bit is testable nicely in CI17:48
yoctozeptoespecially the "wrong address used"17:48
goldyfruityoctozepto, you will have some fun with Zun and Capsule :p17:48
goldyfruithttps://docs.openstack.org/zun/latest/contributor/capsule.html17:49
yoctozeptogoldyfruit: fortunately users are unlikely to want capsules atm :-)17:49
yoctozeptomain reason why not going the magnum kubernetes route17:50
goldyfruitDifferent usage17:50
goldyfruitMagnum it's ~10-15 minutes to have a cluster17:51
yoctozeptoindeed17:51
goldyfruitZun it's ~1 minute17:51
goldyfruitto get the container UP17:51
yoctozeptoit might be needed at some point though17:51
yoctozeptoso keep the support ongoing ;p17:51
yoctozeptoyeah, exactly17:51
yoctozeptoand the direct console is a nice thing17:51
goldyfruityep17:53
goldyfruitAnd thanks for that :)17:53
yoctozeptoyw17:55
yoctozeptoI could say thanks for using kolla ;D17:55
yoctozeptocontainers all around :D17:56
yoctozeptohttps://imgflip.com/i/38q7sr :D17:59
goldyfruitPimp My Ride \o/18:01
yoctozeptopimp my docker :P18:02
goldyfruitDocker is my ride :p18:02
yoctozeptohahahaha18:02
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for self-signed SSL certificates  https://review.opendev.org/66451718:14
openstackgerritKris Lindgren proposed openstack/kolla-ansible master: Add a job that *only* deploys updated containers  https://review.opendev.org/64596618:28
NOXOiDhas anyone seen issues with rbd exclusive locks not being released after a node failure?18:39
NOXOiDeven just kill -9'ing the qemu process is enough to replicate it which makes the volume unusable until the lock is manually released18:40
goldyfruitNOXOiD, maybe a configuration in Ceph about the watcher/listener ?18:48
NOXOiDactually i just checked18:48
NOXOiDits only volumes, not instance root images18:48
NOXOiDim thinking it may be related to the fact that the ceph nova user has permission to blacklist osds but cinder does not?18:48
goldyfruitNOXOiD, I think they both should have this permission18:48
NOXOiDill test and compare with what kolla sets up by default. i dont believe i changed cinders caps but i may have18:49
goldyfruitNOXOiD, nop it's not https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/cinder/defaults/main.yml#L7318:50
NOXOiDbeat me to it lol18:51
NOXOiDyea ill add that cap and re-test18:51
NOXOiDif that fixes it ill file a bug report18:51
goldyfruitNOXOiD, Thanks18:51
NOXOiDconfirmed, that fixed it18:57
NOXOiDhttps://bugs.launchpad.net/kolla-ansible/+bug/176006519:00
openstackLaunchpad bug 1760065 in kolla-ansible "Ceph client.cinder insufficient privileges for blacklist op" [Undecided,New]19:00
NOXOiDoops, already reported19:00
*** hamzy has quit IRC19:01
*** k_mouza has joined #openstack-kolla19:06
openstackgerritMerged openstack/kolla-ansible master: Use fluentd image labels  https://review.opendev.org/67641319:09
*** hamzy has joined #openstack-kolla19:26
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: [cinder] Fix missing blacklist permission.  https://review.opendev.org/67829619:54
goldyfruitNOXOiD, just did  https://review.opendev.org/67829619:54
goldyfruitLet see if it will be merged19:54
NOXOiDhah and i was going through the contributor documentation :P20:00
NOXOiDappreciated20:00
*** ivve has joined #openstack-kolla20:03
goldyfruitNOXOiD, np20:08
*** weshay is now known as weshay_MOD20:10
*** igordc has quit IRC20:37
openstackgerritRafael Weingärtner proposed openstack/kolla-ansible master: Remove ``oslo_messaging_notifications`` that are not needed  https://review.opendev.org/67830620:52
*** igordc has joined #openstack-kolla21:45
*** Blinkiz has quit IRC22:00
*** Blinkiz has joined #openstack-kolla22:01
*** Blinkiz has quit IRC22:02
*** Blinkiz has joined #openstack-kolla22:06
*** KeithMnemonic has quit IRC22:13
*** ivve has quit IRC22:17
*** BjoernT has quit IRC22:18
*** igordc has quit IRC23:00
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for self-signed SSL certificates  https://review.opendev.org/66451723:09
*** jonaspaulo has quit IRC23:23
*** k_mouza has quit IRC23:51
*** k_mouza has joined #openstack-kolla23:51
*** k_mouza has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!