Thursday, 2018-01-25

*** rstarmer has joined #openstack-ansible00:02
*** cjloader has quit IRC00:12
*** acormier has joined #openstack-ansible00:24
*** acormier has quit IRC00:25
*** acormier has joined #openstack-ansible00:25
*** yifei1 has joined #openstack-ansible00:33
*** yifei1 has left #openstack-ansible00:33
*** jbadiapa has quit IRC00:34
*** DanyC has quit IRC00:36
*** chyka has quit IRC00:37
*** pmannidi has joined #openstack-ansible00:38
*** dave-mccowan has joined #openstack-ansible00:43
*** yifei1 has joined #openstack-ansible00:49
*** yifei1 has left #openstack-ansible00:58
openstackgerritwangqiang-bj proposed openstack/openstack-ansible-os_ceilometer master: fix misspelling of 'configuration'  https://review.openstack.org/53419501:01
*** rstarmer has quit IRC01:03
*** acormier has quit IRC01:04
*** dave-mccowan has quit IRC01:22
*** woodard_ has joined #openstack-ansible01:25
*** woodard has quit IRC01:25
*** acormier has joined #openstack-ansible01:43
*** acormier has quit IRC01:43
*** acormier has joined #openstack-ansible01:44
*** acormier_ has joined #openstack-ansible01:46
*** acormier_ has quit IRC01:46
*** acormier_ has joined #openstack-ansible01:47
*** acormier has quit IRC01:49
*** acormier has joined #openstack-ansible01:51
*** dave-mccowan has joined #openstack-ansible01:54
*** acormier_ has quit IRC01:54
*** acormier has quit IRC01:54
*** chyka has joined #openstack-ansible02:05
*** chyka has quit IRC02:09
*** germs has quit IRC02:15
*** gkadam has joined #openstack-ansible02:15
*** yolanda has quit IRC02:22
*** acormier has joined #openstack-ansible02:26
*** acormier_ has joined #openstack-ansible02:29
*** acormier_ has quit IRC02:29
*** acormier_ has joined #openstack-ansible02:29
*** germs has joined #openstack-ansible02:31
*** acormier has quit IRC02:32
*** dave-mccowan has quit IRC02:33
*** acormier has joined #openstack-ansible02:41
cloudnullevenings02:43
*** acormier_ has quit IRC02:45
*** ianychoi has quit IRC02:49
*** ianychoi has joined #openstack-ansible02:49
*** acormier has quit IRC02:53
*** acormier has joined #openstack-ansible02:53
*** acormier_ has joined #openstack-ansible02:55
*** dave-mccowan has joined #openstack-ansible02:56
*** acormier has quit IRC02:59
*** acormier_ has quit IRC03:09
*** acormier has joined #openstack-ansible03:09
*** cjloader has joined #openstack-ansible03:17
*** chyka has joined #openstack-ansible03:19
*** chyka has quit IRC03:23
*** acormier has quit IRC03:23
*** vnogin has joined #openstack-ansible03:26
*** vnogin has quit IRC03:30
*** openstackgerrit has quit IRC03:33
*** woodard_ has quit IRC04:04
*** poopcat1 has quit IRC04:15
*** poopcat has joined #openstack-ansible04:17
*** poopcat1 has joined #openstack-ansible04:22
*** dave-mccowan has quit IRC04:24
*** poopcat has quit IRC04:24
*** bhujay has joined #openstack-ansible04:28
*** gkadam_ has joined #openstack-ansible04:49
*** gkadam has quit IRC04:50
*** yifei has quit IRC04:54
*** yifei1 has joined #openstack-ansible04:54
*** bhujay has quit IRC04:54
*** bhujay has joined #openstack-ansible04:55
*** yifei1 is now known as yifei04:56
*** cjloader has quit IRC04:58
*** poopcat1 has quit IRC05:09
*** woodard has joined #openstack-ansible05:10
*** chyka has joined #openstack-ansible05:10
*** bhujay has quit IRC05:11
*** bhujay has joined #openstack-ansible05:12
prometheanfireyo05:12
*** woodard has quit IRC05:14
*** chyka has quit IRC05:15
*** gkadam_ has quit IRC05:18
*** askb has quit IRC05:21
*** gkadam has joined #openstack-ansible05:21
*** germs has quit IRC05:22
*** gkadam has quit IRC05:29
*** gkadam has joined #openstack-ansible05:30
*** askb has joined #openstack-ansible05:33
*** taseer1 has joined #openstack-ansible05:37
*** Taseer has quit IRC05:38
*** taseer2 has joined #openstack-ansible05:38
*** hybridpollo has quit IRC05:38
*** taseer1 has quit IRC05:42
*** m|y|k has joined #openstack-ansible05:43
*** m|y|k has quit IRC05:43
*** indistylo has joined #openstack-ansible05:44
*** taseer2 is now known as Taseer05:44
Taseerevrardjp: can you please let me know, which ip should I check the congress api ? localhost and the current ip are not working => http://logs.openstack.org/91/522491/50/check/openstack-ansible-functional-ubuntu-xenial/66a6124/job-output.txt.gz#_2018-01-24_17_38_48_34688705:46
*** bhujay has quit IRC05:53
*** woodard has joined #openstack-ansible05:57
*** woodard has quit IRC06:02
*** bhujay has joined #openstack-ansible06:03
*** bhujay has quit IRC06:08
*** ianychoi has quit IRC06:08
*** ianychoi has joined #openstack-ansible06:09
*** gkadam has quit IRC06:13
*** openstackgerrit has joined #openstack-ansible06:17
openstackgerritMerged openstack/openstack-ansible stable/pike: Update repo_build/lxc_hosts role SHA  https://review.openstack.org/53734206:17
*** indistylo has quit IRC06:21
*** indistylo has joined #openstack-ansible06:21
*** indistylo has quit IRC06:26
*** indistylo has joined #openstack-ansible06:26
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249106:28
*** bhujay has joined #openstack-ansible06:29
*** indistylo has quit IRC06:45
*** mbuil has joined #openstack-ansible06:49
*** indistylo has joined #openstack-ansible06:52
*** markvoelker has quit IRC06:52
*** markvoelker has joined #openstack-ansible06:53
*** indistylo has quit IRC06:55
*** indistylo has joined #openstack-ansible06:55
*** bhujay has quit IRC06:57
*** markvoelker has quit IRC06:57
*** arbrandes has joined #openstack-ansible07:01
*** threestrands_ has quit IRC07:02
*** bhujay has joined #openstack-ansible07:03
*** arbrandes1 has quit IRC07:03
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-os_nova master: Avoid explicit requirement for groups to exist  https://review.openstack.org/53777407:14
*** acormier has joined #openstack-ansible07:14
*** acormier has quit IRC07:14
*** acormier has joined #openstack-ansible07:15
*** chyka has joined #openstack-ansible07:21
*** chyka has quit IRC07:26
*** vnogin has joined #openstack-ansible07:26
*** indistylo has quit IRC07:29
*** aruns has joined #openstack-ansible07:29
*** DanyC has joined #openstack-ansible07:30
*** vnogin has quit IRC07:30
*** acormier has quit IRC07:31
*** indistylo has joined #openstack-ansible07:34
*** aruns has quit IRC07:37
*** peri has joined #openstack-ansible07:39
*** indistylo has quit IRC07:40
evrardjpTaseer: maybe check in congress logs, and check with the congress team?07:42
evrardjpThis looks like a functional issue to me, which is proving the test is doing fine :)07:42
*** DanyC has quit IRC07:43
*** armaan has joined #openstack-ansible07:47
openstackgerritsun cheng proposed openstack/openstack-ansible-os_sahara master: Fix the typo in doc files of openstack-ansible-os_sahara  https://review.openstack.org/53779207:55
*** pcaruana has joined #openstack-ansible07:55
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Add experimental tests of new ansible versions  https://review.openstack.org/53730207:58
Taseerevrardjp: I have set ignore_errors to True, so it is not failing. Just to check the output of the states08:02
Taseerlet nme check with congress team08:02
*** yifei1 has joined #openstack-ansible08:02
*** yifei has quit IRC08:02
*** yifei1 is now known as yifei08:02
Taseerevrardjp: How about this => https://github.com/Taseer94/openstack-ansible-os_blazar08:02
Taseerdo you think it is ready to merge with OSA ?08:03
*** woodard has joined #openstack-ansible08:12
evrardjpI would wait for the first one to actually pass tests before thinking about another one.08:14
evrardjp:D08:14
evrardjpbut what's good is you now have the experience to do it again :)08:14
*** woodard has quit IRC08:16
*** bhujay has quit IRC08:21
*** pmannidi has quit IRC08:23
*** pmannidi has joined #openstack-ansible08:24
*** gkadam has joined #openstack-ansible08:28
*** pmannidi has quit IRC08:29
*** bhujay has joined #openstack-ansible08:42
*** hamza21 has joined #openstack-ansible08:48
*** DanyC has joined #openstack-ansible08:50
*** shardy_afk is now known as shardy08:51
*** DanyC_ has joined #openstack-ansible08:51
*** shardy has quit IRC08:51
*** DanyC_ has quit IRC08:53
*** DanyC_ has joined #openstack-ansible08:53
*** markvoelker has joined #openstack-ansible08:54
*** DanyC has quit IRC08:54
*** shardy has joined #openstack-ansible08:55
*** bhujay has quit IRC08:56
*** pbandark has joined #openstack-ansible08:56
*** chyka has joined #openstack-ansible09:10
*** woodard has joined #openstack-ansible09:10
*** jafeha has joined #openstack-ansible09:12
*** jbadiapa has joined #openstack-ansible09:13
*** jafeha__ has quit IRC09:14
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: DNM - Testing ARA upgrades  https://review.openstack.org/53736009:14
*** Tyrantelf has quit IRC09:15
*** woodard has quit IRC09:15
*** chyka has quit IRC09:15
*** Tyrantelf has joined #openstack-ansible09:16
hwoaranggood morning09:22
hwoarangevrardjp: can i tempt you to get https://review.openstack.org/#/c/537411/ in so the galera client role stops breaking jobs?09:23
*** shardy has quit IRC09:23
*** chyka has joined #openstack-ansible09:24
*** vnogin has joined #openstack-ansible09:26
*** shardy has joined #openstack-ansible09:26
evrardjpgood morning hwoarang09:26
evrardjpI missed that part.09:27
evrardjpI am sorry.09:27
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249109:27
evrardjphwoarang: could I get a vote on this in exchange? That's fixing the upgrade job for cinder: https://review.openstack.org/#/c/535791/09:27
*** markvoelker has quit IRC09:27
hwoarangevrardjp: no worries i forgot to commit the client patch at the same time as the server :(09:28
*** chyka has quit IRC09:28
hwoaranghmm i thought i voted for that but it may have been swift09:29
evrardjpwhen the ugrades tests are passing, I'll prepare the milestone 3 release. Short on time, but we can make it in time!09:32
*** vnogin has quit IRC09:34
*** vnogin has joined #openstack-ansible09:35
hwoarangdo we need some specific patches to pass before that happens so i can keep an eye on them?09:35
*** hamza21 has quit IRC09:35
*** hachi has joined #openstack-ansible09:36
ionihello evrardjp09:36
ioniback in december there was a discussion in openstakc-operators related to MTU vxlan09:36
ionidid that guy contacted you guys ?09:37
evrardjpgood morning ioni09:40
evrardjpI think we discussed indeed.09:41
evrardjpDavid Young, right?09:41
ioniyes09:41
ioniis there a proposed fix that I can apply?09:42
evrardjpI have a full thread of mails with him, sometimes I forgot to reply the list.09:42
ionido you mind if you share with me the details of the fix?09:44
evrardjpisn't everything in the operators list? (When you check my content inside David's answers)09:44
evrardjpthere is no fix, so far it's just a configuration issue09:44
evrardjpyou configure what you need to have09:44
evrardjpIf I am not mistaken09:44
evrardjpI have not written that actions should be taken09:44
ionilast time it was a problem with the bridge coming up09:44
ioniand the discussion halted in that point09:44
evrardjpbut it was not linked to his mtu09:45
evrardjpwhat is you rissue?09:45
evrardjpI mean09:46
evrardjpdo you have an issue with a mtu? Can we debug this together?09:46
ionii still didn't made those changes, I wanted first to be sure that everything is working with those configurations09:47
*** woodard has joined #openstack-ansible09:50
openstackgerritMerged openstack/openstack-ansible-galera_client master: SUSE: Switch to upstream MariaDB repository  https://review.openstack.org/53741109:53
*** SmearedBeard has joined #openstack-ansible09:53
*** woodard has quit IRC09:55
*** vnogin has quit IRC09:55
*** electrofelix has joined #openstack-ansible10:01
*** bhujay has joined #openstack-ansible10:03
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Reinitialize the virtualenv  https://review.openstack.org/53581510:07
*** Smeared_Beard has joined #openstack-ansible10:08
*** SmearedBeard has quit IRC10:10
*** CloudToko has joined #openstack-ansible10:15
*** vnogin has joined #openstack-ansible10:17
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249110:18
gokhan_hi folks, when I try to run setup-hosts.yml multiple times, ı am getting diskquota exceeded error on lxc-hosts role. this is error log: http://paste.openstack.org/show/653356/10:19
gokhan_how can I solve this problem ?10:19
*** vnogin has quit IRC10:21
*** gameon has joined #openstack-ansible10:24
*** yifei1 has joined #openstack-ansible10:24
hwoaranggokhan_: you can increase the quotas10:24
*** markvoelker has joined #openstack-ansible10:25
*** yifei has quit IRC10:25
*** yifei1 is now known as yifei10:25
hwoaranggokhan_: you can override the lxc_host_machine_volume_size variable to something higher than 16 (gigabytes)10:26
evrardjpioni: so10:26
gokhan_hwoarang, ok thanks I am trying now10:27
openstackgerritMerged openstack/openstack-ansible-tests master: ARA: Prepare the ARA plugin for all the tests  https://review.openstack.org/53735910:27
evrardjpfor tenant networks that are encapsulated by vxlan to appear 1500, you have to have the carrying interfaces to be at least 1550. Which means that container_mtu needs to be set for those networks (external/vxlan) to 1550.10:27
evrardjpyou need to regenerate your config for your lxc containers.10:28
evrardjpand then start them up again10:28
evrardjpthey need to show up as 1550.10:28
evrardjpI know someone did a few changes in mtu while I was off last year, I hope it won't have an impact on you.10:29
ionievrardjp where do i need to set up container_mtu ?10:29
ioniin openstack_user_config ?10:29
evrardjpyes.10:30
evrardjpin your provider_networks.10:30
evrardjpthat's just for starters.10:30
evrardjpthen you need neutron to be smart.10:30
evrardjp:D10:30
gameonHi all10:30
evrardjpbut let's start with that part.10:30
gameonCould someone tell me what projects are created by default? I have deleted some projects, leaving just the admin one and now I have a serious issue where I cannot list any O/S services in Horizon10:31
evrardjpgameon: I think openstack needs a few tenants by default, probably admin/service(s?)10:31
evrardjpmaybe default10:32
evrardjpI can't remember10:32
evrardjpredeploy :p10:32
evrardjpbeing admin gives you full responsibilities!10:32
*** acormier has joined #openstack-ansible10:32
gameonevrardjp : Thanks - I really don't want to redeploy as I've spent quite a bit of time testing :(10:33
evrardjpcan't you redeploy on top of what you did, without destroying?10:34
evrardjpWith clouds you must be prepared of rebuilding10:34
evrardjpnot be afraid :)10:34
gameonLeave the containers and redeploy as is?10:35
evrardjpyeah, just re-run all the playbooks, no change.10:35
gameonsetup-openstack - the infra should be OK as is AFAIK10:35
*** stuartgr has joined #openstack-ansible10:35
evrardjpyeah.10:35
evrardjpI think if you broke your keystone, that's a big deal.10:36
gameonthanks evrardjp, I'll see what I can do and possibly redeploy10:36
gameonI haven't changed the config - I did delete the service project though !!!10:36
evrardjpyeah just try to re-run openstack-ansible setup-openstack.yml10:36
*** acormier has quit IRC10:37
*** vnogin has joined #openstack-ansible10:40
gameonThanks evrardjp: I'll begin with the keystone deployment as I suspect it will fix it10:41
ionievrardjp so i don't need to set up lxc_net_mtu and lxc_container_default_mtu to 1550?10:41
evrardjpfun stuff in translations scenario periodics -- we hit, two days in a row, the same error: "worker in dead state" -- which is an ansible strategy issue, (which we ofc load in our own strategy...)10:42
evrardjpioni: no10:42
evrardjpioni: I don't think os10:42
evrardjpso*10:42
evrardjpgameon: I don't think keystone would be enough, all your services need to be in that tenant.10:42
evrardjpproper uuid and all.10:43
evrardjpBut I might be wrong, and it might work directly.10:43
ionievrardjp ok, https://paste.xinu.at/LDPGj/10:43
evrardjpyep10:44
ionisetup hosts playbook10:44
ioniis there any other playbook that i need to run?10:44
*** vnogin has quit IRC10:44
evrardjpioni: is that prod?10:45
evrardjpor is that a test environement?10:45
evrardjpenvironment10:45
ionievrardjp production :D10:45
evrardjpdarn10:45
evrardjpdon't do that10:45
evrardjptry on a test env first.10:46
evrardjpyou're gonna break the env, because you need to shut down the containers10:46
evrardjplike really shut them down10:46
evrardjpI'd delete them.10:46
ioniok10:47
evrardjpioni: when all of that is done, I think you can re-deploy openstack with the appropriate neutron overrides.10:51
ionievrardjp that ones that i have sent you?10:51
evrardjpprobably https://paste.xinu.at/wGH5/#n4710:52
evrardjphttps://paste.xinu.at/wGH5/#n5010:52
evrardjpyou have to check the neutron docs for that10:52
evrardjpto be 100% sure.10:52
evrardjpdepending on your branch10:52
ionii'm still on newton10:52
ionigoing ocata and then newton in a month or so10:52
*** pylot has quit IRC10:53
*** jbadiapa has quit IRC10:55
gokhan_hwoarang, I tried it and set it to 32GB again failed. but when I run machinectl image-status result is root@infra1:/opt/openstack-ansible/playbooks# machinectl image-status10:56
gokhan_            Path: /var/lib/machines10:56
gokhan_           Usage: 2.7G10:56
gokhan_           Limit: 32.0G10:56
gokhan_this is more logs http://paste.openstack.org/show/653386/10:56
hwoarangare you sure that btrfs is not runnign out of useful disk space?10:57
*** markvoelker has quit IRC10:58
evrardjpjmccrory: the first ansible 2.5 failures! http://logs.openstack.org/02/537302/2/experimental/openstack-ansible-deploy_with_ansible_version-aio-ubuntu-xenial/01745e0/job-output.txt.gz#_2018-01-25_08_17_28_23104110:58
*** indistylo has joined #openstack-ansible10:58
*** vnogin has joined #openstack-ansible10:59
*** aruns has joined #openstack-ansible11:01
*** indistylo has quit IRC11:02
gokhan_hwoarang, I don't have enough information about btrfs. How can I test it ? I have'nt seen this error anytime. First I had problem about lxc-cache-prep timeout and then I applied this commit https://github.com/openstack/openstack-ansible-lxc_hosts/commit/f6afac77870f7c2d386e0522c084c68e9ad342d011:02
gokhan_after that ı run setup-hosts.yml again and got disk quota exceeded error11:03
*** vnogin_ has joined #openstack-ansible11:04
*** vnogin has quit IRC11:04
*** woodard has joined #openstack-ansible11:05
*** DanyC has joined #openstack-ansible11:07
*** bhujay has quit IRC11:07
hwoaranggokhan_: there are some info here https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_I_ran_out_of_disk_space.21 about checking the btrfs disk space11:07
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Fix upgrade test  https://review.openstack.org/53579111:08
*** DanyC has quit IRC11:09
*** woodard has quit IRC11:09
*** DanyC_ has quit IRC11:09
*** DanyC has joined #openstack-ansible11:10
gameonevrardjp: Indeed, keystone isn't enough. I'm deploying now. Thanks for the help11:12
*** bhujay has joined #openstack-ansible11:12
*** chyka has joined #openstack-ansible11:13
evrardjpgameon: hope it will get fixed that way.11:18
*** chyka has quit IRC11:18
gokhan_hwoarang, I run 'btrfs fi df /var/lib/machines' and see System, DUP: total=8.00MiB, used=16.00KiB may be this is problem. also when I run 'btrfs check /var/lib/machines' get Superblock bytenr is larger than device size. these are command results:http://paste.openstack.org/show/653389/11:19
Taseerevrardjp: is this alarming => http://logs.openstack.org/91/522491/53/check/openstack-ansible-functional-ubuntu-xenial/ed87fae/logs/openstack/congress1/syslog.txt.gz#_Jan_25_10_51_1711:19
*** Mr_Smurf has joined #openstack-ansible11:19
evrardjpTaseer: no, but what's after is alarming :p11:20
evrardjpTaseer: http://logs.openstack.org/91/522491/53/check/openstack-ansible-functional-ubuntu-xenial/ed87fae/logs/openstack/congress1/syslog.txt.gz#_Jan_25_10_51_1811:21
Mr_SmurfI have some issues with the new ubuntu 4.4.0.112 kernel and LXC containers, i get CPU lockups when the containers start.. No issues when i use 4.4.0.10911:21
evrardjpoh funsies11:22
evrardjpMr_Smurf: ^11:22
evrardjpMr_Smurf: do you have the changelog? It would be good to know if we are hitting the same issues in gates too.11:23
hwoaranggokhan_: for whatever reason that subvolume is very small11:23
Mr_Smurfevrardjp: I get a bunch of "NMI watchdog: Watchdog detected hard LOCKUP on cpu X" on the console when the containers start and then the system locks up11:24
hwoarangcan you post the output of the mount command?11:24
armaanevrardjp: Any news regarding O->P upgrade?11:24
hwoarangi want to see if that's a real subvolume or a mounted sparse file11:24
evrardjparmaan: working on it, found a new bug.11:25
evrardjpfixing one, finding another.11:25
evrardjpnow it's a certain kind of inventory issue.11:25
evrardjpI don't know what your status where.11:25
armaanevrardjp: oh ok, thanks for letting me know.11:25
Taseerevrardjp: Do I need to create a task to make the /var/log/congress/ directory ?11:26
evrardjpWell today was a bad day, pretty much not grean this morning.11:26
evrardjpnot green*11:26
evrardjpI like my dashboards green.11:26
armaanevrardjp: We have a customer asking for magnum but magnum no worky in Ocata, so i have to upgrade the prod environment to Pike.11:26
evrardjpTaseer: yes.11:26
Taseerok11:26
Mr_Smurfevrardjp: do you also have issues with the new kernel or is it some other issue?11:26
evrardjparmaan: I will make sure next release for O and P are working in terms of upgrades.11:26
armaanevrardjp: thanks a lot :)11:27
evrardjpso N to O to P should be working, at least on ubuntu with or without ceph.11:27
evrardjpthe magnum part I can't promise, as I told you, we don't have a test for it. You can implement it if you like, but I am not sure people are actively looking to it :D11:27
evrardjpbut we can probably try it together.11:27
evrardjpor a manual test, when the rest is green.11:28
evrardjpMr_Smurf: I don't know.11:28
armaanevrardjp: I am going to destroy magnum service in Ocata, upgrade to Pike and install magnum again ;)11:28
evrardjpMr_Smurf: what are you experiencing?11:28
gokhan_hwoarang, sorry which is the mount command ?11:28
evrardjpMr_Smurf: Is there a way we have automated tests for that? This way we can catch it up in gates?11:28
evrardjpLet's make sure magnum deploys in pike then :D11:29
hwoaranggokhan_: the regular 'mount' command on your system11:29
hwoarangjust type mount ;p11:29
Mr_Smurfevrardjp: NMI watchdog: Watchdog detected hard LOCKUP on cpu X messages on the console then the server frezzes on the controller nodes, the compute nodes seems to be running fine with 4.4.0.11211:30
gokhan_hwoarang, sorry I can't understand it :( this is result: http://paste.openstack.org/show/653393/11:30
evrardjparmaan: here is the activity for magnum: https://review.openstack.org/#/q/project:openstack/openstack-ansible-os_magnum11:31
Mr_Smurfevrardjp: as long as I don't start the lxc containers on the controller node it stays alive11:31
evrardjpMr_Smurf: oh you mean, as soon as you start a lxc container, your host freezes?11:31
evrardjpI can have a look at the kernel we run on xenial11:32
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249111:32
armaanevrardjp: sad, not a lot of activity11:32
Mr_Smurfevrardjp: no, it starts a few then something triggers it.. Do not know what yet I am still trying to figure out what11:32
hwoaranggokhan_: could you file a bug report with more information about your distribution, how to reproduce it etc?11:35
evrardjpMr_Smurf: we are running 4.4.0-112-generic in gates.11:35
hwoarangyour /var/lib/machines.raw file is very small and it shouldn't be like that11:35
Mr_Smurfevrardjp: ok, and you have no issues?11:35
evrardjpMr_Smurf: oh I just saw your NMI watchdog deteced lockups. Yeah that's bad11:36
Mr_Smurfevrardjp: it's bad but I'm not alone now :)11:37
evrardjpI don't see that in gates, but I am not on the machine directly, so hard to tell11:37
gokhan_hwoarang, Mr_Smurf I am olso using ubuntu new kernel 4.4.0.112. Maybe my problem is about new kernel11:37
evrardjpMr_Smurf: no I meant I just saw that on your message above11:37
evrardjpYou're still alone :(11:37
Mr_Smurfoh :(11:37
Mr_SmurfIt only displays the error on console so if you don't have a console log then you will not see it11:38
Mr_SmurfI will continue to try and isolate the problem..11:39
gokhan_hwoarang, if I delete machines.raw file and again run setup-hosts.yml can it be problem ?11:42
evrardjpisn't that a hardware issue?11:42
evrardjpMr_Smurf: ^11:42
Mr_Smurfevrardjp: I have three identical dell R630 servers all with the same issue.. But yes it could be hardware related..11:43
evrardjpMr_Smurf: else maybe it's worth determining the threshold value ?11:43
evrardjplike defining the timeout11:43
evrardjphwoarang: how far are your kernel developer debugging skills? :D11:44
hwoarangwe will see...11:45
evrardjpMr_Smurf: quick question, same cpu for compute and infra nodes?11:45
evrardjphwoarang: :D11:45
hwoaranggokhan_: i dont know maybe but i would suggest to open a bug so others can look at it11:45
*** mbuil has quit IRC11:45
openstackgerritPeriyasamy Palanisamy proposed openstack/openstack-ansible master: Make Opendaylight as the BGP speaker using Quagga  https://review.openstack.org/52390711:46
Mr_Smurfevrardjp: CPU E5-2670 v3 on compute E5-2630 v3 on infa11:46
gokhan_hwoarang, ok I am writing now.11:47
*** indistylo has joined #openstack-ansible11:47
*** aruns has quit IRC11:47
*** markvoelker has joined #openstack-ansible11:55
*** smatzek has joined #openstack-ansible12:02
*** woodard has joined #openstack-ansible12:04
*** jwitko has quit IRC12:06
*** woodard has quit IRC12:08
*** jwitko_ has joined #openstack-ansible12:11
gokhan_hwoarang, I opened the bug.  https://bugs.launchpad.net/openstack-ansible/+bug/174536112:15
openstackLaunchpad bug 1745361 in openstack-ansible "Failed to create subvolume /var/lib/machines/ when run 'openstack-ansible setup-hosts.yml' multiple times" [Undecided,New]12:15
Mr_Smurfevrardjp: if I don't start the neutron_agents_container then it looks like it works so I guess it could be a ethernet card bug12:23
evrardjpor feature offloads?12:26
Mr_Smurfevrardjp: yes, but it looks network related12:27
dcdamienevrardjp, odyssey4me: I want to deploy ELK for OpenStack needs. I'm already using OSA and found `rpc-openstack` which you maintain. How about using it to employ ELK? Any caveats?12:27
Mr_Smurfevrardjp: I have a 10-Gigabit X540-AT2 (rev 01) card on the infra nodes and rev 03 on the compute nodes12:27
evrardjpdcdamien: I think you should speak to d34dh0r53 :D12:28
evrardjpMr_Smurf: which branch are you running on?12:28
evrardjplatest pike?12:28
*** markvoelker has quit IRC12:29
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249112:29
Mr_Smurfevrardjp: no, newton 14.2.1212:30
evrardjpMr_Smurf: we have this in gates: ETHTOOL_OPTS="-K ${DEVICE} sg off"12:31
dcdamienevrardjp: thx12:31
evrardjpMr_Smurf: I don't know if sg offload could be the cause. but check any offloading you'd be using maybe?12:31
evrardjpalso check your nic isn't used twice, and things like that12:32
dcdamiend34dh0r53: I'll be grateful for any guidance12:32
evrardjpMr_Smurf: xenial?12:32
Mr_Smurfevrardjp: strange however that it works with 109 but not 11212:32
Mr_Smurfevrardjp: yes12:32
evrardjpyou never know :D12:32
Mr_Smurfethtool -K enp3s0f0 sg off triggerd the lockup12:38
*** woodard has joined #openstack-ansible12:41
evrardjpMr_Smurf: you're welcome?12:44
evrardjp:p12:44
Mr_Smurfevrardjp: I am still trying to verify it.. seting the sg to off after a reboot did not trigger it, so it might be that it triggers it only if the interface is in use12:45
*** vnogin_ has quit IRC12:47
*** vnogin has joined #openstack-ansible12:49
*** vnogin has quit IRC12:49
*** vnogin has joined #openstack-ansible12:49
evrardjpworth trying to set it off at boot, and see if things are fine after that12:50
Mr_Smurfwell, after a reboot it did not trigger anything12:56
*** JasonCL has joined #openstack-ansible12:59
*** jbadiapa has joined #openstack-ansible13:01
*** stuartgr has quit IRC13:03
*** bhujay has quit IRC13:03
*** dave-mccowan has joined #openstack-ansible13:05
*** acormier has joined #openstack-ansible13:16
*** acormier has quit IRC13:19
*** mbuil has joined #openstack-ansible13:20
*** armaan has quit IRC13:24
*** markvoelker has joined #openstack-ansible13:26
*** aruns__ has joined #openstack-ansible13:31
*** indistylo has quit IRC13:31
*** yolanda has joined #openstack-ansible13:34
*** armaan has joined #openstack-ansible13:39
mhaydenmorning13:48
*** acormier has joined #openstack-ansible13:51
mhaydeni went spelunking into the repo build logic last night to figure out the openstack client shenanigans13:51
mhaydenit seems like we could make a list of available clients and subtract the ones we don't need13:52
mhaydenbut it would mean that a deployer who adds a new service, like monasca, would have to rebuild the repo server, re-deploy utility, and deploy monasca13:53
mhaydenrebuilding the repo is already required13:53
mhaydenbut deploying the utility container is really fast13:53
*** acormier has quit IRC13:54
*** rromans has quit IRC13:54
*** esberglu has joined #openstack-ansible13:55
*** bhujay has joined #openstack-ansible13:56
*** markvoelker has quit IRC13:59
*** woodard has quit IRC14:01
*** woodard has joined #openstack-ansible14:01
*** hamza21 has joined #openstack-ansible14:03
*** rromans has joined #openstack-ansible14:04
evrardjpmhayden: morning14:05
evrardjpmhayden: fine for me, but I thought we already had conditional behavior14:05
mhaydenwell, we do, but heat drags in all of the clients14:06
*** hachi has quit IRC14:06
mhaydeneven if those clients will never be used14:06
mhaydeni don't really want to put in a special catch for heat in the jinja2 template ... :/14:06
evrardjpisn't that what doug has referred as something WE should do?14:06
evrardjplike the []14:06
mhaydenyeah14:06
evrardjpnah not for heat, but maybe do something smart if []?14:07
mhaydeni'm thinking we could match on '^python-(\w+)client$' regex and have special handling14:07
mhaydenso we only build python-monascaclient if monasca_all host group exists14:07
*** esberglu has quit IRC14:08
*** vnogin has quit IRC14:09
evrardjpwhat if we do something completely different?14:10
* mhayden is open to suggestions ;)14:11
evrardjplike if you're deploying monasca, you're not using this mechanism at all14:11
evrardjpso the monasca client isn't deployed14:11
evrardjpin the utility container14:11
evrardjpOR14:11
evrardjpit's deployed during the install of monasca14:11
evrardjpby delegating to the utility14:11
evrardjpbut that kinda forces us to do the developer mode14:11
mhaydentrue, things get wonky14:12
evrardjpis that something publicly available?14:12
evrardjpthe wheel14:12
mhaydenthere are two challenges: 1) keeping clients out of the pip wheel build process and 2) preventing the venv prep process from looking for those wheels14:12
evrardjpyeah so14:12
evrardjpjesse's patch is working on not building wheels at all14:13
mhaydenthis is the one i'm trying to avoid -> https://pypi.python.org/pypi/cassandra-driver/14:13
mhaydenit takes 4-6 minutes to build14:13
evrardjpyeah14:13
mhaydenoh, where's odyssey4me's patch for that? i'd be curious14:14
evrardjphttps://review.openstack.org/#/c/537015/14:14
Mr_Smurfwill openstack ansible for queens be available when queens is released?14:14
mhaydenohi think i found it14:14
evrardjpMr_Smurf: our deadline is two weeks after other projects have released.14:14
*** santacloud has joined #openstack-ansible14:15
Mr_Smurfevrardjp: ok, is it "safe" to upgrade from newton to queens? In multple steps..14:15
evrardjpin multiple steps yes.14:15
Mr_Smurfok, good.. That is my plan :)14:15
evrardjpRight now I have seen a few issues in the upgrades process, I am trying to fix.14:16
santacloudHi guys14:16
evrardjpnext release should get those fixes for all the branches.14:16
mhaydenMr_Smurf: queens greenfield deployments work pretty well right now14:16
evrardjphello santa!14:16
evrardjp:D14:16
evrardjpearly this year.14:16
santacloud;)14:16
mhaydenoh perhaps very late14:16
evrardjphohoho14:16
evrardjp(Waiting for your huehuehue gif)14:16
santacloudI'm looking haproxy configuration for ha deployment14:17
evrardjpmhayden: I guess the problem will arise for external git things14:17
evrardjpsantacloud: yes?14:17
*** vnogin has joined #openstack-ansible14:17
santacloudwhy timout are 5000s for galera?14:18
santacloudwhen I run HA test (for instance, I stop one of my infra1 nodes), I can see requests destinated to infra1 during all this timeout14:20
santacloudmy openstack is not really operationnal during this time.14:20
Taseerevrardjp: it did not execute the functional test => https://review.openstack.org/#/c/522491/14:20
Taseershould I rebase it ?14:21
*** phalmos has joined #openstack-ansible14:22
evrardjpsantacloud: I guess people complained about values. We should probably leave smart people chose their value.14:22
santacloudok, tests with 120s timeout seem ok, but I am not a expert either about this topic :s14:23
evrardjpTaseer: one moment14:23
*** smatzek has quit IRC14:24
evrardjpsantacloud: it all depends on your case. It's probably due to replaication issues or something14:24
evrardjpmaybe a git blame can help you14:24
evrardjpmhayden: https://bugs.launchpad.net/openstack-ansible/+bug/174538214:24
openstackLaunchpad bug 1745382 in openstack-ansible "O-P upgrade fails on inventory" [Undecided,New]14:24
evrardjpfun stuff.14:24
*** smatzek has joined #openstack-ansible14:24
*** smatzek has quit IRC14:24
mhaydenegads14:24
*** esberglu has joined #openstack-ansible14:24
mhayden-EDONOTWANT14:24
-openstackstatus- NOTICE: We're currently experiencing issues with the logs.openstack.org server which will result in POST_FAILURE for jobs, please stand by and don't needlessly recheck jobs while we troubleshoot the problem.14:24
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: container_host can be replaced by physical_host on is_metal  https://review.openstack.org/53792114:25
*** cjloader has joined #openstack-ansible14:25
evrardjpmhayden: ^ I don't know what I am doing.14:25
evrardjpseem to work though.14:26
evrardjpwill add jmccrory and cloudnull14:28
evrardjpsantacloud: apparently 5000s has been there for a while.14:29
santacloudevrardjp: yes I know, that's why I ask... I don't know the reason14:30
santacloudevrardjp: when my infra nodes are up, no problem14:31
*** jwitko has joined #openstack-ansible14:31
evrardjpsantacloud: it's for galera clustering I guess.14:31
evrardjpbut now we've changed the status reporting system14:32
santacloudevrardjp: yes it is14:32
*** markvoelker has joined #openstack-ansible14:34
*** cjloader has quit IRC14:38
*** cjloader has joined #openstack-ansible14:39
Adri2000hello14:41
Adri2000is anyone working on the heat-dashboard split from horizon?14:41
mgariepycan i have some review on this one ? https://review.openstack.org/#/c/537641/14:42
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249114:43
*** hamza21 has quit IRC14:43
mhaydenmgariepy: so if the variable is not set during upgrade, what happens? bind mounts are left unchanged?14:54
mgariepythe bind-mount option should only be changed when upgrading to the release that changed the default.14:55
mgariepyto prevent the restart of container and taking the cloud down in a messy way :)14:56
mgariepyif it's present in the user_variable you do not want to change it. the deployer probably knows what he is doing :) haha14:58
*** vnogin has quit IRC14:58
*** aruns__ has quit IRC14:58
armaanfolks, i just installed cinder-backup and it is running on compute nodes. The cinder community recommends to run the c-bac  on non compute nodes because it can consume a lot of cpu & network14:59
*** vnogin has joined #openstack-ansible14:59
armaanfirst question: How do i uninstall it :(14:59
*** smatzek has joined #openstack-ansible15:00
mgariepymhayden, does that answer your question?15:00
*** smatzek has quit IRC15:00
*** smatzek has joined #openstack-ansible15:01
*** RandomTech has joined #openstack-ansible15:01
*** phalmos_ has joined #openstack-ansible15:04
santacloudbye15:07
santacloudsee you soon15:07
*** phalmos has quit IRC15:07
mhaydenmgariepy: i see what you mean15:17
*** pcaruana has quit IRC15:20
*** vnogin has quit IRC15:20
mhaydenapparently gate logs are filling the disks for infra15:21
mhaydenhttps://review.openstack.org/#/c/537929/2/roles/upload-logs/tasks/main.yaml15:21
mgariepyhmm15:25
*** phalmos_ has quit IRC15:30
*** dave-mccowan has quit IRC15:31
evrardjpAdri2000: not yet, I wasn't really aware of it, but I am not surprised.15:33
evrardjpdidn't I make this super easy to adapt? I don't remember15:33
*** dave-mccowan has joined #openstack-ansible15:33
*** phalmos has joined #openstack-ansible15:34
evrardjpdo you have docs somewhere?15:34
evrardjpAdri2000: ^15:34
evrardjpmhayden: mmm that's bad15:38
evrardjpdid we cause inode breakage again?15:38
Adri2000evrardjp: bug I've filed: https://bugs.launchpad.net/openstack-ansible/+bug/1745400, horizon release note: https://docs.openstack.org/releasenotes/horizon/unreleased.html#upgrade-notes15:41
openstackLaunchpad bug 1745400 in openstack-ansible "Heat dashboard missing in Queens" [Undecided,New]15:41
Adri2000evrardjp: looks like it's easy to integrate heat-dashboard into horizon using the horizon real. but we need to have heat-dashboard installed first (might be easy, I'm not ure)15:41
Adri2000s/real/role/15:41
evrardjpyeah I think I just made it simpler in this cycle15:44
evrardjpAdri2000: do you want to patch it?15:44
evrardjpwrong question15:44
evrardjpWill you patch it?15:44
*** RandomTech has quit IRC15:46
*** yolanda has quit IRC15:47
mgariepyhmm. small issue with the upgrade from Newton > Ocata, running haproxy before galera breaks the galera cluster.15:49
mgariepyor more, the access to it.15:49
mhaydenevrardjp: sounds like infra has a disk outage of some sort15:49
*** jbadiapa has quit IRC15:50
evrardjpmmm15:51
Adri2000evrardjp: if you can give me some hints regarding the "install heat-dashboard" part, yes. I'm going to call it a day now, but we can definitely talk about it tomorrow :)15:53
evrardjpAdri2000: doing it15:54
evrardjpyeah, I'd be happy if you could at least test it15:54
evrardjpI am quickly hacking it15:54
Adri2000ok, perfect :) just ping me!15:55
evrardjpAdri2000: omg we can so much improve this role15:56
evrardjplet's do a quick first iteration15:56
-openstackstatus- NOTICE: logs.openstack.org is stabilized and there should no longer be *new* POST_FAILURE errors. Logs for jobs that ran in the past weeks until earlier today are currently unavailable pending FSCK completion. We're going to temporarily disable *successful* jobs from uploading their logs to reduce strain on our current limited capacity. Thanks for your patience !16:00
*** smatzek has left #openstack-ansible16:02
*** phalmos has quit IRC16:02
*** CloudToko has quit IRC16:03
*** CloudToko has joined #openstack-ansible16:03
CobHeadAnyone got the link to the documentation over all the variables that can be set in openstack_user_config.yml?16:04
CobHeadI can't find it anywhere16:04
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249116:05
mgariepyCobHead, https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.example16:06
mgariepyin the file itself :)16:06
CobHeadI remember seeing a different doc with every single option that you can put in the config.16:07
CobHeadThere is no way that file contains examples for all the overrides possible.16:07
CobHeade.g I want to put compute nodes in a different AZ16:08
CobHeadThere is *nothing* in the docs about this16:08
CobHeadThere's this in Cinder: Option: cinder_storage_availability_zone (optional, string)16:09
CobHeadWhat's the Nova equivalent16:09
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249116:09
*** vnogin has joined #openstack-ansible16:14
*** phalmos has joined #openstack-ansible16:15
*** jafeha has quit IRC16:17
openstackgerritMarc Gariépy (mgariepy) proposed openstack/openstack-ansible stable/ocata: Fix upgrade order, align docs with script  https://review.openstack.org/53795216:17
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_horizon master: Add heat dashboard to horizon  https://review.openstack.org/53795616:19
openstackgerritMerged openstack/openstack-ansible-os_sahara master: Fix the typo in doc files of openstack-ansible-os_sahara  https://review.openstack.org/53779216:19
evrardjpAdri2000: https://review.openstack.org/#/c/537956/116:20
evrardjpjmccrory: that would be great if you could have a look at https://review.openstack.org/#/c/537921/16:21
CobHead"nova_default_schedule_zone" is useless, as that doesn't allow me to put compute hosts in different availability zones16:22
*** gameon has quit IRC16:23
logan-CobHead: probably 'default_availability_zone'16:26
logan-in nova config overrides16:26
logan-you could also add the computes to availability zones using host aggregates16:27
logan-without needing to make any config changes16:27
CobHeadAgain, that override is useless as that would only pick the default availability zone for new resources to launch on.16:27
CobHeadNot putting compute nodes directly into their respective AZs in OSA16:27
logan-gotcha16:28
logan-well your best best to find the option is not in OSA docs, it is in the nova configuration reference: https://docs.openstack.org/ocata/config-reference/compute/config-options.html16:28
CobHeadNo dice there either16:29
CobHeadApparently Nova nodes must be put in their respective AZs after OSA deployment has been completed.16:29
logan-yeah i couldn't tell you offhand. for our cloud we use host aggregates to assign availability zone membership16:29
logan-i thought there was a nova conf option for it though16:30
CobHeadKinda lacking from OSA's point of view, imho. Putting 100 nodes over 5 AZ's should be done by OSA, not by setting it up from OpenStack Client16:30
*** SerenaFeng has joined #openstack-ansible16:32
openstackgerritMarc Gariépy (mgariepy) proposed openstack/openstack-ansible master: Remove default_bind_mount_logs from config changes.  https://review.openstack.org/53764116:32
mgariepyhwoarang, adjusted the commit message ^^16:33
*** chyka has joined #openstack-ansible16:35
*** DanyC has quit IRC16:38
hwoarangawesome thanks16:38
*** DanyC has joined #openstack-ansible16:38
logan-CobHead: you might take a look at https://docs.ansible.com/ansible/latest/os_nova_host_aggregate_module.html if nova doesn't support a conf option16:39
logan-openstack client might not be the most efficient way to do it through the API16:39
CobHeadI was more thinking along the lines like you can do with cinder in openstack_user_config.yml - Option: cinder_storage_availability_zone (optional, string)16:40
CobHeadWhy there is no option for that under compute_hosts, I have no idea :]16:40
*** bhujay has quit IRC16:41
odyssey4meCobHead if nova.conf has an option for it, then you can apply it using https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/app-advanced-config-override.html16:41
*** openstackstatus has quit IRC16:41
odyssey4meWe specifically are trying to avoid having an ansible var for every possible OpenStack .conf var because it's unmaintainable.16:41
*** openstackstatus has joined #openstack-ansible16:41
*** ChanServ sets mode: +v openstackstatus16:41
logan-looking at the config reference I don't even see a var for it, so I think you're SOL with config overrides unfortunately16:42
odyssey4meSo instead we give you the option per conf file to aplpy absolutely any config option you want.16:42
CobHeadExcept availability zones for compute nodes, apparently16:42
*** DanyC has quit IRC16:42
CobHeadBut that might well be an issue with Openstack itself, and not OSA.16:43
CobHeadOh, well.16:43
odyssey4meYes, if OpenStack doesn't allow it, then better to implement something for it using Ansible.16:43
CobHeadI guess I'll have to hack something together.16:43
CobHeadThanks for informing me anyways :]16:44
CobHead(And keep up the excellent work)16:44
odyssey4medcdamien There are plans to add the ELK stack setup from RPC into OSA in the next cycle. See https://github.com/openstack/openstack-ansible-specs/blob/master/specs/queens/elk-stack.rst - and yes, it's d34dh0r53 driving that.16:44
* d34dh0r53 hides16:45
odyssey4meCobHead If you put something together that you think would be useful for the OSA community at large, then the openstack-ansible-ops repo would appreciate the contribution. It can be playbooks, scripts or pretty much anything. Otherwise if it's just a simple procedure then add it to the operator docs!16:45
*** jafeha has joined #openstack-ansible16:46
CobHeadWill do :)16:46
odyssey4medcdamien otherwise, there is a set of tooling in https://github.com/openstack/openstack-ansible-ops/tree/master/elk_metrics already, but it's a bit rough around the edges ;)16:46
evrardjpodyssey4me: while you're here16:47
evrardjpif I want to add a file into something that's pip installable, what should I do into the setup.cfg ?16:47
evrardjpextra-files or something?16:47
evrardjpI can't rememeber16:47
odyssey4meah, you want to add data files?16:47
evrardjpyeah, policy files in this case.16:48
evrardjpI did another hack for now16:48
evrardjpbut if it's not accepted I'd like to know my options16:48
evrardjphttps://review.openstack.org/#/c/537963/16:48
*** chyka has quit IRC16:48
odyssey4meevrardjp see https://review.openstack.org/#/c/506210/3/setup.cfg16:49
*** hamzy has quit IRC16:49
*** chyka has joined #openstack-ansible16:49
odyssey4meah, neat idea - and a great follow on to https://bugs.launchpad.net/nova/+bug/171835616:50
openstackLaunchpad bug 1718356 in Zun "Include default config files in python wheel" [Undecided,In progress] - Assigned to Rajat Sharma (tajar29)16:50
odyssey4meI might just take a day or two off to convert us to using the fruits of that bug, seeing as most of the projects have accepted the patches :)16:50
evrardjpyeah that would be nice16:51
evrardjpI don't think it's a prio though16:51
evrardjpIf I could get more eyes on that inventory issue, that would be great, because it's blocking O to P upgrades :/16:52
odyssey4meno, but I'd like to convert at least keystone and the telemetry stack to using it, given that they currently have that horrible mish-mash which pulls it from git16:52
*** rpittau has quit IRC16:53
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements  https://review.openstack.org/53711216:54
dcdamienodyssey4me: thanks for the explanation, I'll look into mentioned docs and repo16:54
evrardjpmy concern for that dashboard is that we would have to do a very convoluted thing to get the policy16:54
evrardjpchange the heat deploy process to produce the policy on the deploy, then push it to heat and horizon16:54
evrardjpetcd is something we discuss more and more, but sometimes I am afraid of adding a new moving part.16:55
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_horizon master: Add heat dashboard to horizon  https://review.openstack.org/53795616:56
*** stuartgr has joined #openstack-ansible16:57
odyssey4meevrardjp yeah, what you've proposed is a fantastic way of solving the problem - way, way better!16:57
odyssey4meI've commented a suggested option in the review - hope that gets some uptake!16:57
odyssey4meEven if not in this cycle, hopefully the next!16:57
odyssey4meWith regards to using etcd, if you think about it we're actually wanting to use it to cut out moving parts and simplify things. Yes, it adds one more service but if we do things right we'll be ditching a bunch of code complexity and simplifying how things work quite dramatically.16:59
evrardjpyes.17:00
evrardjpI agree.17:00
evrardjpthe module is on its way for 2.5. I have 1 review already.17:00
odyssey4meIt'd be nice to have a sample or two of what we're thinking for a practical discussion at the PTG, perhaps we can hack them together in the two days early in the week.17:00
odyssey4meExcellent. :)17:01
odyssey4meAnyway, I'm on holiday today - just popped in to get some things done. time for me to get back to !computering and preparing the haggis, neeps and tatties for Burns Night!17:01
evrardjpyeah sorry17:01
evrardjpenjoy!17:02
odyssey4meCheerio all - have a fantastic day!17:02
evrardjpyou too17:02
mhaydenodyssey4me: enjoy sir17:02
hwoarangodyssey4me: re https://review.openstack.org/#/c/537387/ it's a real opensuse issue17:04
hwoaranglet me provide a fix17:04
*** gkadam has quit IRC17:06
*** electrofelix has quit IRC17:06
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-repo_build master: SUSE: Fix MariaDB development package  https://review.openstack.org/53797517:07
hwoarangodyssey4me: i am going to add a Depends-On on your package17:08
hwoarang*change17:08
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-repo_build master: Correct manifest index of openstack git folders  https://review.openstack.org/53738717:08
armaanlogan-: hello! have you ever played with cinder-bacup and ceph?17:13
logan-armaan: nope, have not used cinder-backup at all17:13
armaanlogan-: ok, thanks!17:13
logan-no problem. if you run into issues i might not be able to help but I will try!17:14
*** peri has quit IRC17:14
*** yolanda has joined #openstack-ansible17:15
cloudnullafternoons17:15
evrardjpcloudnull: good evening :)17:25
evrardjphow are you?17:25
cloudnulldoing well17:25
cloudnullfinally being productive today :)17:25
evrardjpcloudnull: FYI nova and glance are broken, I've discussed with rosmaita, and we're gonna fix it.17:25
cloudnullthe meetings have ended17:25
evrardjpI mean we have a fix17:25
evrardjpit's going for gates.17:25
cloudnullcool17:25
evrardjpthen the roles will be able to continue...17:25
cloudnullanything that needs to be reviewed ?17:26
evrardjpI have something I'd like your crasy brain on indeed.17:26
evrardjpBlockign the O to P upgrades.17:26
evrardjp(in the meantime the glance thing is fixed)17:26
evrardjpthe problem is there:17:27
evrardjphttps://slack-imgs.com/?c=1&url=http%3A%2F%2Friffsy.com%2Fimage%2Fvemp_tinygif.gif17:27
evrardjpso my fix is there:17:27
evrardjphttps://review.openstack.org/#/c/537921/17:27
cloudnullhahaha.17:27
cloudnullthat gif is amazing17:27
evrardjp:D17:27
evrardjpvery appropriate17:28
evrardjpanyway, it's a quick/safe hack, but I am not sure about the root cause17:28
cloudnullLGTM17:29
evrardjphaha17:29
cloudnullseems like the right fix17:29
evrardjpgogogo backport17:30
evrardjpwell it needs merge first I guess ...17:30
evrardjpthanks.17:30
*** hamza21 has joined #openstack-ansible17:33
*** shardy has quit IRC17:35
evrardjphwoarang: I am not sure omit can work there, but I will try, by curiosity. I could be wrong.17:41
*** esberglu has quit IRC17:42
*** Leo_m has joined #openstack-ansible17:42
Leo_mhowdy, just a quick question, the expected flavors for the AIO ansible install are tempest1 and tempest2 only?17:43
evrardjphwoarang: when not set, the env var would be defined to the following content:  "__omit_place_holder__612dd6b33b9ff6a1a520168e8cb18976fec6bafd"17:46
evrardjpwhich would basically break the thing17:46
evrardjpfyi17:47
evrardjphwoarang: http://paste.openstack.org/show/653467/17:48
evrardjpI didn't want to repeat the version used in the .zuul folder.17:49
cloudnullLeo_m: those flavors are created by tempest.17:51
*** SerenaFeng has quit IRC17:51
cloudnullyou could however create more traditional flavors as needed.17:51
cloudnullbut openstack does not, by default, provide any flavors17:52
Leo_mcloudnull how do I create them17:52
cloudnullyou can use the openstack api/horizon/etc17:52
Leo_mI see now where in horzon17:54
Leo_mguessing the flavors here https://docs.openstack.org/openstack-ansible/latest/admin/openstack-operations/cli-operations.html#list-flavors17:54
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Add experimental tests of new ansible versions  https://review.openstack.org/53730217:55
Leo_mwould be a good reference for creating my flavors in my AIO17:55
Leo_mcloudnull thanks!17:55
cloudnullyes that is a good reference17:55
cloudnullLeo_m: https://github.com/openstack/openstack-ansible-ops/blob/master/scripts/generate-flavors.sh17:55
cloudnullthat's a script we have to generate flavors17:56
Leo_mcool! will try it out17:56
Leo_mpozole?18:01
cloudnullI love pozole18:01
Leo_mheheh wrong window, me too!18:02
*** cxc99 has quit IRC18:02
cloudnullEl Pozole de Moctezuma, en DF is legit!18:03
spotzheheh18:04
*** DanyC has joined #openstack-ansible18:04
*** DanyC_ has joined #openstack-ansible18:05
*** DanyC has quit IRC18:09
*** cxc99 has joined #openstack-ansible18:11
*** germs has joined #openstack-ansible18:12
*** hamzy has joined #openstack-ansible18:13
*** germs1 has joined #openstack-ansible18:14
*** vnogin has quit IRC18:16
*** germs has quit IRC18:16
*** hamzy has quit IRC18:18
*** hamzy has joined #openstack-ansible18:21
evrardjpI love pozols too.18:21
evrardjpOk that was a terrible joke, so I am out.18:22
cloudnullhave a good night evrardjp18:23
*** hw_wutianwei_ has quit IRC18:24
mgariepycan I have some eyes on :  https://review.openstack.org/#/c/537952/18:24
*** esberglu has joined #openstack-ansible18:28
cloudnullmgariepy: +218:33
mgariepythanks a lot :)18:38
mgariepyit's hard to keep script + doc in the same ""state"" :(18:38
cloudnullyea, it'd be great to get that into a some self documenting state but IDK if it's worth the effort.18:39
mgariepyyeah18:40
*** armaan has quit IRC18:51
*** stuartgr has quit IRC18:51
*** mbuil has quit IRC18:58
*** poopcat has joined #openstack-ansible19:01
*** woodard has quit IRC19:01
*** germs1 is now known as germs19:01
*** woodard has joined #openstack-ansible19:01
*** poopcat has quit IRC19:06
*** poopcat1 has joined #openstack-ansible19:06
openstackgerritMerged openstack/openstack-ansible-os_gnocchi master: Updated from global requirements  https://review.openstack.org/53711019:07
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_neutron master: Fix LBaaSv2 config and package requirements  https://review.openstack.org/53380619:12
*** chyka has quit IRC19:12
*** chyka has joined #openstack-ansible19:13
*** jwitko_ has quit IRC19:13
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Updated from global requirements  https://review.openstack.org/53508519:14
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_cinder master: Updated from global requirements  https://review.openstack.org/53505219:15
openstackgerritMajor Hayden proposed openstack/openstack-ansible stable/pike: Stable keepalived priorities  https://review.openstack.org/53223019:15
*** chyka has quit IRC19:23
*** chyka has joined #openstack-ansible19:24
*** DanyC_ has quit IRC19:37
*** chyka has quit IRC19:40
*** chyka_ has joined #openstack-ansible19:41
*** hamza21 has quit IRC19:44
*** hamza21 has joined #openstack-ansible19:48
*** hamza21 has quit IRC19:55
mhaydenanyone else seeing this from glance in master? error: can't copy 'etc/glance-image-import.conf': doesn't exist or not a regular file19:56
*** esberglu has quit IRC19:57
mhaydenoh nice https://review.openstack.org/#/c/537453/219:58
mhaydenfix is inbound20:00
*** chyka_ has quit IRC20:03
*** chyka has joined #openstack-ansible20:04
*** esberglu has joined #openstack-ansible20:06
*** chason has quit IRC20:17
*** Jeffrey4l has quit IRC20:19
*** chyka has quit IRC20:21
*** kstev has joined #openstack-ansible20:21
*** chyka has joined #openstack-ansible20:21
*** cmart has quit IRC20:23
mnaserhttps://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/all/keystone.yml#L2720:23
*** chason has joined #openstack-ansible20:23
*** Jeffrey4l has joined #openstack-ansible20:24
mnaseri'm confused by the logic.  if keystone_service_adminuri_proto is https and (keystone ssl cert is undefined OR haproxy ssl cert is undefined) then its insecure20:24
mnaserhowever, if proto is https and haproxy ssl cert is set (but keystone ssl cert is not), then it still considers it as insecure?20:24
mnaserbut haproxy is doing ssl for the vip so .. it should be secure if it hits the vip?20:25
mgariepymnaser, it's indeed confused haha20:29
mgariepymnaser, it's indeed confusing** haha20:29
mnasermgariepy: im thinking it should be an "and" instead of the or20:30
*** guhcampos has joined #openstack-ansible20:30
mnaserso if its https AND (no ssl cert is defined) = insecure20:30
mgariepyyou can set the ssl cert for either keystone or haproxy20:31
mnaserya but if i have it set in haproxy, then it still considers it insecure20:31
*** hw_wutianwei1 has joined #openstack-ansible20:32
mnaserwhich results in the openrc file in the utility container generated with --insecure aliases20:32
*** hw_wutianwei has quit IRC20:34
*** hw_wutianwei1 is now known as hw_wutianwei20:34
mnaserhttps://github.com/openstack/openstack-ansible/commit/8387b680f7724b80ff9cf4b4577e6a7afbd1532220:34
mnaserwell i tracked it down but i doubt evrardjp remember this from 2 years ago :P20:34
mnaseroh there20:36
mnaserhttps://github.com/openstack/openstack-ansible/commit/2ffb77638f48d75a444c35e619355cd6f133907620:36
logan-yeah it is broken20:40
mnaserlogan-: ok ill push up something20:40
mnaseri think we can drop keystone_user_ssl_cert because it hits the vip anyways20:40
logan-no i think it is correct to include both of them in the eval, but either one being set should disable insecure20:41
mnaserfair enough, ill make it do that then20:41
logan-haproxy_* will do SSL termination in the LB, keystone_* will do ssl termination in the keystone container without any cert in the LB20:42
*** Smeared_Beard has quit IRC20:52
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Fix logic to check for insecure Keystone  https://review.openstack.org/53802920:55
mnaserlogan-: mgariepy ^ :>20:56
*** cmart has joined #openstack-ansible21:01
*** phalmos has quit IRC21:01
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Updated from global requirements  https://review.openstack.org/53711221:02
*** Jeffrey4l has quit IRC21:02
*** chason has quit IRC21:02
*** jbadiapa has joined #openstack-ansible21:08
cloudnullha logan- beat me to it. :)21:09
cloudnullre https://review.openstack.org/53802921:09
logan-:P21:11
*** sawblade6 has joined #openstack-ansible21:12
*** chason has joined #openstack-ansible21:13
*** Jeffrey4l has joined #openstack-ansible21:14
*** armaan has joined #openstack-ansible21:25
*** pramodrj07 has joined #openstack-ansible21:29
*** esberglu has quit IRC21:32
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Fix logic to check for insecure Keystone  https://review.openstack.org/53802921:39
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Fix logic to check for insecure Keystone  https://review.openstack.org/53802921:40
*** jheroux has joined #openstack-ansible21:40
*** jheroux has quit IRC21:43
*** hamzy has quit IRC21:43
openstackgerritMerged openstack/openstack-ansible-lxc_container_create master: Updated from global requirements  https://review.openstack.org/53710121:55
*** mbuil has joined #openstack-ansible21:56
*** mbuil has quit IRC21:56
*** jwitko_ has joined #openstack-ansible22:26
*** jwitko has quit IRC22:30
*** jwitko_ has quit IRC22:34
*** jwitko has joined #openstack-ansible22:58
openstackgerritDamian Cikowski proposed openstack/openstack-ansible-galera_server master: This patch changes group membership of /var/log/mysql_logs directory. It makes log files accesible for rsyslog daemon.  https://review.openstack.org/53805622:58
openstackgerritDamian Cikowski proposed openstack/openstack-ansible-galera_server master: This patch changes group membership of /var/log/mysql_logs directory. It makes log files accesible for rsyslog daemon.  https://review.openstack.org/53805623:00
*** Pramod has joined #openstack-ansible23:09
*** Aibot has joined #openstack-ansible23:10
*** Aibot has quit IRC23:10
*** Pramod has quit IRC23:10
*** pramodrj07 has quit IRC23:10
*** pramodrj07 has joined #openstack-ansible23:11
*** rstarmer has joined #openstack-ansible23:12
*** cmart has quit IRC23:13
openstackgerritMerged openstack/openstack-ansible-os_swift master: Updated from global requirements  https://review.openstack.org/53507223:17
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Updated from global requirements  https://review.openstack.org/53505223:22
*** chyka has quit IRC23:23
*** chyka has joined #openstack-ansible23:23
*** rstarmer has quit IRC23:24
*** rstarmer has joined #openstack-ansible23:25
*** ianychoi has quit IRC23:26
*** ianychoi has joined #openstack-ansible23:27
*** dave-mccowan has quit IRC23:28
*** chyka has quit IRC23:28
*** cjloader has quit IRC23:30
*** chyka has joined #openstack-ansible23:30
*** chyka_ has joined #openstack-ansible23:34
*** chyka has quit IRC23:35
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Fix LBaaSv2 config and package requirements  https://review.openstack.org/53380623:41
openstackgerritDamian Cikowski proposed openstack/openstack-ansible-galera_server stable/pike: This patch changes group membership of /var/log/mysql_logs directory. It makes log files accesible for rsyslog daemon.  https://review.openstack.org/53807123:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!