Wednesday, 2018-01-03

*** dave-mccowan has joined #openstack-ansible00:05
*** vnogin has quit IRC00:10
*** poul has joined #openstack-ansible00:11
*** vnogin has joined #openstack-ansible00:12
andymccrlogan-: happy new year! Are you gonna be in dublin? I'd like to do some ceph discussions around how we're approaching it in OSA (/cc hwoarang)00:12
*** poul has quit IRC00:16
*** vnogin has quit IRC00:17
*** vnogin has joined #openstack-ansible00:18
*** vnogin has quit IRC00:22
*** jamesdenton has joined #openstack-ansible00:26
*** poul has joined #openstack-ansible00:28
*** poul has quit IRC00:33
*** poul has joined #openstack-ansible00:37
*** poul has quit IRC00:42
*** poul has joined #openstack-ansible00:54
*** poul has quit IRC00:59
*** markvoelker has quit IRC01:08
*** poul has joined #openstack-ansible01:11
*** gkadam has quit IRC01:15
*** poul has quit IRC01:15
*** gkadam has joined #openstack-ansible01:16
*** esberglu has quit IRC01:26
*** poul has joined #openstack-ansible01:28
*** poul has quit IRC01:33
*** fatdragon has quit IRC01:33
*** fatdragon has joined #openstack-ansible01:34
*** dave-mccowan has quit IRC01:34
*** poul has joined #openstack-ansible01:37
*** fatdragon has quit IRC01:38
*** poul has quit IRC01:42
*** poul has joined #openstack-ansible01:54
*** dave-mccowan has joined #openstack-ansible01:55
*** gkadam has quit IRC01:56
*** gkadam has joined #openstack-ansible01:59
*** poul has quit IRC01:59
*** poul has joined #openstack-ansible02:03
*** markvoelker has joined #openstack-ansible02:05
*** poul has quit IRC02:08
*** dave-mccowan has quit IRC02:18
*** dave-mccowan has joined #openstack-ansible02:19
*** ANKITA has joined #openstack-ansible02:19
ANKITAHI02:19
ANKITAfatal: [infra1_repo_container-8a6f526f]: FAILED! => {"failed": true, "msg": "Failed to get information                                            on remote file (/etc/sudoers.d/nginx): MODULE FAILURE"}02:20
*** poul has joined #openstack-ansible02:20
ANKITAhas any one faced this issue?02:20
ANKITAfatal: [infra1_repo_container-8a6f526f]: FAILED! => {"failed": true, "msg": "Failed to get information on remote file (/etc/sudoers.d/nginx): MODULE FAILURE"}02:21
ANKITAinfra1_repo_container-8a6f526f : ok=6    changed=0    unreachable=0    failed=102:22
ANKITAinfra2_aodh_container-f95c75ad : ok=0    changed=0    unreachable=0    failed=1 infra2_ceilometer_api_container-cc5103cd : ok=0    changed=0    unreachable=0    failed=1 infra2_ceilometer_collector_container-d4efb6d8 : ok=0    changed=0    unreachable=0    failed=1 infra2_cinder_api_container-90624f0b : ok=0    changed=0    unreachable=0    failed=1 infra2_cinder_scheduler_container-5b94936b : ok=0    changed=0    unreacha02:22
ANKITAlog1_rsyslog_container-0eab7bf9 : ok=0    changed=0    unreachable=0    failed=102:23
*** poul has quit IRC02:25
*** poul has joined #openstack-ansible02:37
*** poul has quit IRC02:41
ANKITAhi02:48
*** yifei1 has joined #openstack-ansible02:54
*** poul has joined #openstack-ansible02:54
*** poul has quit IRC02:59
*** gkadam has quit IRC03:07
*** fatdragon has joined #openstack-ansible03:08
*** poul has joined #openstack-ansible03:11
*** poul has quit IRC03:16
*** fatdragon has quit IRC03:20
*** poul has joined #openstack-ansible03:20
*** m|y|k has joined #openstack-ansible03:21
*** m|y|k has quit IRC03:21
*** poul has quit IRC03:25
*** masber has quit IRC03:32
*** poul has joined #openstack-ansible03:37
*** poul has quit IRC03:41
*** fatdragon has joined #openstack-ansible03:45
*** yifei1 has quit IRC03:47
*** fatdragon has quit IRC03:50
*** fatdragon has joined #openstack-ansible03:51
*** poul has joined #openstack-ansible03:54
*** poul has quit IRC03:58
*** fatdragon has quit IRC04:02
*** poul has joined #openstack-ansible04:11
*** udesale has joined #openstack-ansible04:15
*** poul has quit IRC04:16
*** masber has joined #openstack-ansible04:21
*** poul has joined #openstack-ansible04:28
*** poul has quit IRC04:33
*** poul has joined #openstack-ansible04:46
*** poul has quit IRC04:50
*** indistylo has joined #openstack-ansible04:52
*** poul has joined #openstack-ansible04:55
*** fatdragon has joined #openstack-ansible04:59
*** poul has quit IRC04:59
*** poopcat1 has quit IRC05:01
*** chhavi has joined #openstack-ansible05:02
*** dave-mccowan has quit IRC05:06
masbergood afternoon, is openstack-ansible ready to configure SR-IOV integration in openstack?05:08
*** indistylo has quit IRC05:08
*** fatdragon has quit IRC05:09
*** poul has joined #openstack-ansible05:12
masbernever mind, I found the document05:17
*** poul has quit IRC05:17
masberbut can I use SR-IOV on linux containers deployment?05:17
*** poul has joined #openstack-ansible05:29
*** bhujay has joined #openstack-ansible05:29
*** poul has quit IRC05:33
prometheanfiremasber: nova runs outside of containers still and if it didn't I'm sure there are ways of getting pci devices exposed05:42
*** dmsimard has quit IRC05:42
*** dmsimard has joined #openstack-ansible05:42
masberprometheanfire, interesting so what is running in containers? the controller part or openstack?05:44
*** armaan has joined #openstack-ansible05:45
*** poul has joined #openstack-ansible05:46
prometheanfirejust about everything else05:46
prometheanfireswift and cinder are not as well iirc, but I'd need to confirm05:46
prometheanfireand for each of them it's only the parts that need to either for performance or maintence reasons (restarting a cinder container is a good wait to loose iscsi...)05:47
*** poul has quit IRC05:50
*** poul has joined #openstack-ansible05:55
*** poul has quit IRC05:56
*** poul has joined #openstack-ansible05:56
*** poul has quit IRC05:56
*** poul has joined #openstack-ansible05:57
*** indistylo has joined #openstack-ansible05:57
*** hybridpollo has quit IRC05:57
*** aruns has joined #openstack-ansible05:59
*** indistylo has quit IRC06:01
*** aruns__ has joined #openstack-ansible06:03
*** aruns has quit IRC06:05
*** masber has quit IRC06:13
*** eumel8 has joined #openstack-ansible06:19
*** fatdragon has joined #openstack-ansible06:25
*** markvoelker has quit IRC06:28
*** aruns has joined #openstack-ansible06:32
*** aruns__ has quit IRC06:35
*** fatdragon has quit IRC06:35
ANKITAinfra2_aodh_container-f95c75ad : ok=0    changed=0    unreachable=0    failed=1 infra2_ceilometer_api_container-cc5103cd : ok=0    changed=0    unreachable=0    failed=1 infra2_ceilometer_collector_container-d4efb6d8 : ok=0    changed=0    unreachable=0    failed=1 infra2_cinder_api_container-90624f0b : ok=0    changed=0    unreachable=0    failed=1 infra2_cinder_scheduler_container-5b94936b : ok=0    changed=0    unreacha06:35
ANKITAcontainers exist but services aren't installed...06:37
ANKITAany one faced similar issue?06:38
*** esberglu has joined #openstack-ansible06:40
*** esberglu has quit IRC06:40
*** esberglu has joined #openstack-ansible06:40
*** aruns__ has joined #openstack-ansible06:53
*** aruns has quit IRC06:55
*** nshetty has joined #openstack-ansible06:56
*** pmannidi has quit IRC06:58
*** pmannidi has joined #openstack-ansible07:08
*** pcaruana has joined #openstack-ansible07:09
*** fatdragon has joined #openstack-ansible07:10
*** vnogin has joined #openstack-ansible07:19
*** fatdragon has quit IRC07:19
*** vnogin has quit IRC07:23
*** pmannidi has quit IRC07:24
*** lihi has joined #openstack-ansible07:29
*** fatdragon has joined #openstack-ansible07:48
*** hamza21 has joined #openstack-ansible07:51
*** vnogin has joined #openstack-ansible07:56
*** fatdragon has quit IRC07:57
*** armaan has quit IRC08:04
*** SmearedBeard has joined #openstack-ansible08:04
*** armaan has joined #openstack-ansible08:05
*** mardim has joined #openstack-ansible08:05
evrardjpgood morning everyone, still catching up with all my emails and stuff08:07
evrardjpit's been a while08:07
*** epalper has joined #openstack-ansible08:10
evrardjpANKITA: more context would be interesting08:11
evrardjpmgariepy: SR-iov is supported08:11
evrardjpwoops08:11
evrardjpthat was for masber08:12
*** medy has joined #openstack-ansible08:15
*** medy has quit IRC08:16
*** medy has joined #openstack-ansible08:16
*** medy has quit IRC08:18
*** medy has joined #openstack-ansible08:18
*** medy has quit IRC08:19
*** medy has joined #openstack-ansible08:20
*** medy has quit IRC08:21
*** medy has joined #openstack-ansible08:21
*** medy has quit IRC08:22
*** medy has joined #openstack-ansible08:22
*** medy has quit IRC08:24
*** medy has joined #openstack-ansible08:24
*** medy has quit IRC08:25
*** medy has joined #openstack-ansible08:25
*** medy has quit IRC08:27
*** medy has joined #openstack-ansible08:27
*** medy has quit IRC08:28
*** medy has joined #openstack-ansible08:28
*** markvoelker has joined #openstack-ansible08:29
*** medy has quit IRC08:29
*** fatdragon has joined #openstack-ansible08:33
*** poul_ has joined #openstack-ansible08:42
*** poul has quit IRC08:42
*** fatdragon has quit IRC08:42
jafehagood morning everybody08:44
*** ThomasS has quit IRC08:45
*** armaan has quit IRC08:48
*** armaan has joined #openstack-ansible08:48
ANKITA@evrardjp hi had an issue with the installation in ocat08:51
ANKITAstable08:51
ANKITAwhile running setup scrpit08:52
ANKITA setup-infrastructure.yml08:52
*** aruns has joined #openstack-ansible08:54
evrardjpwhat's the detailed issue and your configuration?08:55
evrardjp(still catching up my emails)08:55
*** aruns__ has quit IRC08:57
ANKITAinfra3_nova_console_container-ec12305f : ok=0    changed=0    unreachable=0    failed=1 infra3_nova_scheduler_container-5fdac4c9 : ok=0    changed=0    unreachable=0    failed=1 infra3_rabbit_mq_container-b16ed1b2 : ok=0    changed=0    unreachable=0    failed=1 infra3_repo_container-82be5c9d : ok=0    changed=0    unreachable=0    failed=1 infra3_utility_container-24212ab2 : ok=0    changed=0    unreachable=0    failed=108:58
ANKITAthe services are not installed in containers whic exist08:58
ANKITAfor three servers infra2,3 & log08:58
ANKITAall the conmtainers are running but the services installation falis for multiple containers09:01
ANKITA@ evrardjp  ?09:02
evrardjpLike I said I am still busy, so I will have a look when i can :)09:03
evrardjpbut you have failures09:03
ANKITAok shall i clone into stable pike repo?09:03
evrardjpbut I don't see the failure09:03
evrardjpnor your changes09:03
*** markvoelker has quit IRC09:03
evrardjpjafeha: good morning09:04
evrardjpANKITA: your choice of Pike or ocata is completely up to you09:04
evrardjpthey are both stable09:04
ANKITAfatal: [infra1_repo_container-8a6f526f]: FAILED! => {"failed": true, "msg": "Failed to get information                                            on remote file (/etc/sudoers.d/nginx): MODULE FAILURE"}09:04
evrardjpso it looks like you have a sudoers failure on your node, we need to find the root cause09:05
evrardjpwhat did you change in that file?09:05
ANKITAnothing09:05
evrardjpwell we do not have that file by default IIRC09:05
ANKITAi followed the steps in documentation09:06
evrardjpoh we do09:06
evrardjpcould you show that file: /etc/sudoers.d/nginx ?09:06
evrardjpthe contents of that file*09:06
evrardjpyeah it seems you have a failure in your repo09:07
ANKITArepo container09:07
evrardjpxenial?09:07
ANKITAyes09:08
ANKITAno such file /etc/sudoers.d/nginx09:08
evrardjpour periodic succeeded09:09
evrardjpin your repo server container?09:09
evrardjpthat's wrong09:09
evrardjpre-run your setup-hosts then re-run setup-infrastructure09:09
evrardjptrack if you see a failure09:09
evrardjpthen let's see09:09
ANKITAok09:09
evrardjpthe full install of an openstack-ansible xenial on Ocata ran today: http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/stable/ocata/openstack-ansible-deploy-aio-ubuntu-xenial/833d557/job-output.txt.gz09:10
evrardjpno issue09:10
evrardjpyou can use that to track what's wrong on your environment09:10
evrardjpmaybe a network configuration issue or something09:10
ANKITAcan i check into the pike branch before running it now ?09:11
evrardjpif you try a new branch, you should delete everything on your machines first09:11
ANKITAam not using aio09:12
evrardjpso delete all the containers09:12
evrardjpwe have a playbook for that09:12
ANKITAok then i shall go with ocata09:12
ANKITAohh09:12
evrardjplxc-containers-destroy.yml09:12
ANKITAwhich play book?09:12
evrardjp^09:12
evrardjpopenstack-ansible lxc-containers-destroy.yml09:12
ANKITAonly conatainers09:12
evrardjpthat would destory your containers09:12
evrardjpyeah09:12
ANKITAto be deleted?09:12
evrardjpyeah you can start with that09:13
evrardjpwell to be honest09:13
evrardjpI'd not do that if I were you09:13
evrardjpnot right now09:13
evrardjpyou seem to have a failure on something that works09:13
*** sxc731 has joined #openstack-ansible09:13
evrardjpYou need to find out why, and that's probably a configuration issue09:13
evrardjpso I'd rather fix the configuration/networking issue first, then I'd redeploy with the knowledge acquired/configuration fix included.09:14
evrardjpelse you're just adding more moving parts09:14
evrardjpharder to diagnose09:14
ANKITAok am going with ocata have destroyed the containers though09:20
*** fatdragon has joined #openstack-ansible09:21
*** shardy has joined #openstack-ansible09:27
ANKITAconfiguration/networking issue cann't figure out the exact issue dunno where network went wrong as i had checked connectivity of VLAN &  briges exists too09:28
*** fatdragon has quit IRC09:29
*** Smeared_Beard has joined #openstack-ansible09:36
*** SmearedBeard has quit IRC09:40
ANKITAfatal: [infra2_cinder_api_container-90624f0b -> 10.40.100.22]: FAILED! => {"attempts": 3, "changed": true, "cmd": ["lxc-start", "--daemon", "--name", "infra2_cinder_api_container-90624f0b", "--logfile", "/var/log/lxc/lxc-infra2_cinder_api_container-90624f0b.log", "--logpriority", "INFO"], "delta": "0:00:05.529586", "end": "2018-01-03 01:43:36.405649", "failed": true, "rc": 1, "start": "2018-01-03 01:43:30.876063", "stderr"09:44
*** bhujay has quit IRC09:47
*** bhujay has joined #openstack-ansible09:48
*** hamza21 has quit IRC09:50
ANKITAfatal: [infra2]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to restart service haproxy: Job for haproxy.service failed because the control process exited with error code. See \"systemctl status haproxy.service\" and \"journalctl -xe\" for details.\n"} fatal: [infra3]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to restart service haproxy: Job for haproxy.service failed because the control09:54
*** Taseer has quit IRC09:55
*** taseer1 has joined #openstack-ansible09:55
*** markvoelker has joined #openstack-ansible10:01
*** ThomasS_ has joined #openstack-ansible10:03
*** pcaruana has quit IRC10:09
*** pcaruana has joined #openstack-ansible10:10
hwoarangandymccr: yep I will by in Dublin10:12
*** armaan has quit IRC10:12
hwoarangtalking about ceph would be nice10:12
hwoarangnot the best way to spend the time in Ireland though :D10:13
*** armaan has joined #openstack-ansible10:13
*** bhujay has quit IRC10:21
gokhanhi folks I am trying upgradecloudkitty role to pike version. but ı am getting error about  SQLAlchemy version. because on openstack requirements it is https://github.com/openstack/requirements/blob/stable/pike/upper-constraints.txt#L13510:21
gokhanbut on cloudkitty on requirements conflicts https://github.com/openstack/cloudkitty/blob/stable/pike/requirements.txt#L2610:22
gokhanhow can I override sqlalchemy version on role ?10:22
*** Trident has joined #openstack-ansible10:24
*** armaan has quit IRC10:29
*** armaan has joined #openstack-ansible10:30
*** mrch has joined #openstack-ansible10:30
*** markvoelker has quit IRC10:34
*** armaan has quit IRC10:43
*** armaan has joined #openstack-ansible10:44
*** stuartgr has joined #openstack-ansible10:49
*** fatdragon has joined #openstack-ansible11:13
*** fatdragon has quit IRC11:24
evrardjpgokhan: I am not sure cloudkitty works on pike11:27
evrardjpnot only for that.11:27
evrardjpgokhan: https://docs.openstack.org/openstack-ansible/latest/contributor/role-maturity.html it's still in incubated state, so we don't have any functional testing of it11:28
*** vnogin has quit IRC11:31
*** markvoelker has joined #openstack-ansible11:31
*** vnogin has joined #openstack-ansible11:31
*** vnogin has quit IRC11:36
*** chhavi has quit IRC11:40
*** fatdragon has joined #openstack-ansible11:51
*** vnogin has joined #openstack-ansible11:52
*** armaan has quit IRC11:54
*** armaan has joined #openstack-ansible11:54
*** fatdragon has quit IRC11:57
*** vnogin has quit IRC11:58
*** vnogin has joined #openstack-ansible11:59
*** markvoelker has quit IRC12:05
*** deadnull has joined #openstack-ansible12:05
*** fatdragon has joined #openstack-ansible12:05
*** smatzek has joined #openstack-ansible12:07
*** denta has joined #openstack-ansible12:09
*** nshetty has quit IRC12:11
*** nshetty has joined #openstack-ansible12:11
*** vnogin has quit IRC12:12
*** fatdragon has quit IRC12:14
*** vnogin has joined #openstack-ansible12:16
*** sxc731 has quit IRC12:22
gokhanevrardjp, yes I know. there is no cloudkitty pike role. we need cloudkitty service so I write cloudkitty pike role myself.12:35
*** Smeared_Beard has quit IRC12:38
*** SmearedBeard has joined #openstack-ansible12:39
*** fatdragon has joined #openstack-ansible12:41
*** fatdragon has quit IRC12:45
*** shardy is now known as shardy_mtg12:48
*** Jack_Iv has joined #openstack-ansible12:50
*** aruns__ has joined #openstack-ansible12:52
*** fatdragon has joined #openstack-ansible12:53
*** aruns has quit IRC12:54
gokhanevrardjp repo-build.yml skips the git clone task. how can ı prevent it ? Do I need remove anything about facts ?12:55
*** sxc731 has joined #openstack-ansible12:56
*** shardy_mtg is now known as shardy12:58
jafehahey everybody! i know the telemetry stack needs serious love, but has anyone a running setup with ceilometer + gnocchi? we're debugging warnings / errors, that gnocchi cannot find it's project in keystone and therefore fails to create resources. has anyone already encountered this issue?13:01
*** markvoelker has joined #openstack-ansible13:02
*** fatdragon has quit IRC13:03
evrardjpgokhan: you need to ensure the cloudkitty role is well downloaded first13:07
evrardjpwhen it's well downloaded, and you start the repo build, you should see a massive dictionary in the playbook that's given into the repo build role13:07
evrardjpthat dictionary should list all what needs to be processed.13:08
evrardjpHave a look if cloudkitty appears there13:08
evrardjpand with proper groups etc13:08
evrardjpjafeha: you said it yourself :)13:08
evrardjpneeds more love!13:09
jafehagna :D13:09
evrardjpanything in the logs?13:09
gokhanjafeha, ı am using them on pike13:09
evrardjpalso it's hard to know without the branches etc.13:09
evrardjpgokhan: you are adventurous!13:10
evrardjpgokhan: jafeha while we are here, the three of us ....13:10
jafehagokhan, we're trying it on pike too, but as i said, keystone auth seems to be broken13:10
evrardjpdo you think you could all find cycles to test telemetry in master?13:11
evrardjpso I'd start by creating a scenario, and we'll iterate on the this scenario to make it work13:11
evrardjpwe can call the scenario autoscaling or something like that, because it's both your use cases, right?13:11
evrardjpso we'd all work on a commit to test that into master, then when it's merged, we can think of backporting to pike13:12
evrardjpbut at least we'll know how to configure all of that, and make it work13:12
evrardjpI don't have cycles to fix all the telemetry myself, and on top of that, that's no priority of me... I can just help you getting things done.13:12
*** armaan has quit IRC13:13
evrardjpI thought it could be good for both of you to have this tested, this way you can both have production with those components13:13
*** armaan has joined #openstack-ansible13:13
gokhanjafeha, I installed ceilometer,aodh and pike with OSA 16.0.4 .  then I see that gnocchi does not see metrics so I realized tehere is a bug on OSA. activate cilometer container venv and need "ceilometer-upgrade"13:13
evrardjpThe only downsides are sometimes I am gonna ask you "Hey what do you think of that, does it break stuff" ? And you can say "Don't do it!" or "Yes, go ahead!"13:14
gokhanevrardjp, on these days ı am a tester boy :) ı see bug on ceilometer,gnocchi,ceph and magnum too. at least I solved them.  now only cloudkitty left.13:16
jafehai just asked my collegue wspthr who was working on a patch in december, we could sureley do some testing.13:17
*** Jack_Iv has quit IRC13:18
jafehawe have a lack of testing hw though. as our testing workstation is a quite specific setup with ovs and ceph13:18
*** Jack_Iv has joined #openstack-ansible13:19
*** markvoelker has quit IRC13:19
*** markvoelker has joined #openstack-ansible13:20
jafehathere are also some time / workload issues, but we really might be able to do some testing.13:20
evrardjpovs should also be a scenario13:20
jafehadefinitly13:20
evrardjpjafeha: do you think we can start with an ovs scenario?13:20
jafehai just had a huge laugh :D13:21
jafehaif we get wspthr to do telemetry with gokhan i can have a look at it.13:22
jafehabut tbh after finishing our deployment we're set to do some integration projects with our dev team. i'm not sure how much time we have until this cycle finishes. even though we have some new collegues13:23
jafehaovs testing scenario and also an aio ovs scenario would be a huge improvement13:25
*** yolanda__ has joined #openstack-ansible13:25
*** gfa has quit IRC13:26
jafehagokhan, thank you for your hint, i'll check out the ceilo upgrade path13:26
*** gfa has joined #openstack-ansible13:27
gokhanjafeha, and also restart ceilometer services. then on gnocchi, run gnocchi-upgrade. after that restart gnocchi metricd service it will work :)13:28
jafehasounds like magic :)13:28
*** yolanda has quit IRC13:29
jafehagokhan, on all the ceilo/gnocchi containers or just one?13:30
gokhanjafeha, on all ceilometer and gnocchi containers. but I am not sure may be one of them is enough13:31
gokhanI run on all gnocchi and ceilometer containers13:31
jafehathanks13:31
*** MikeW_ has joined #openstack-ansible13:31
gokhanjafeha, it would be great work on telemetry stack with you. we will install our production environment next week. after that I have good time to work :)13:33
MikeW_logan- cloudnull : Reporting back sorry about yesterday. I also found that bug, but at first didn't think it should matter since I wasn't updating from newton to ocata, but adding in ceilometer to my current ocata deploy. However, that is exactly what my issue was. Some qemu packages were marked as manual update only and when ansible updated the qmeu packages they became disjointed. Thanks for finding that wish I had seen i13:33
*** fatdragon has joined #openstack-ansible13:33
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Build everything on metal  https://review.openstack.org/50422413:33
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Build everything on metal  https://review.openstack.org/50422413:33
openstackgerritArno Kellermann proposed openstack/openstack-ansible-os_gnocchi master: fix gnocchi ceph name variable  https://review.openstack.org/52703713:35
*** fatdragon has quit IRC13:38
evrardjpMikeW_: which packages were not in sync for qemu packages?13:39
evrardjpare they not listed in the nova package?13:39
evrardjpMikeW_: https://github.com/openstack/openstack-ansible-os_nova/blob/master/vars/ubuntu-16.04.yml#L63-L7213:40
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Build everything on metal  https://review.openstack.org/50422413:43
jafehagokhan, looks much better now, thank you!13:43
gokhanjafeha, ok great :)13:47
MikeW_evrardjp I'll check and see which ones were marked for manual update only, and ping back. We were focused maybe too much on just getting the environment back up and ran a rather large update script on the compute hosts13:48
MikeW_Instead of downgrading the few that were upgraded13:48
jafehagokhan, resource types are getting created. still having some ceilo/gnocchi errors whining about a missing project in keystone, but i think get closer13:49
MikeW_evrardjp Actually my logic may be backwards. The ones that were set to automatic got updated: https://pastebin.com/EixDh36p13:50
*** dave-mccowan has joined #openstack-ansible13:50
jafehagokhan, can you tell me your gnocchi user, project_name, user_domain_id and project_domain_id?13:52
gokhanjafeha, also on OSA ceilometer service entity is created but we don't need it13:52
gokhanjafeha, you can delete it from keystone13:52
jafehainteresting13:53
gokhanjafeha, this is my gnocchi.conf http://paste.openstack.org/show/637407/13:55
ThomasS_hi13:56
MikeW_gokhan Do you have to have gnocchi for ceilometer to work? Or can I use cyclops?13:56
ThomasS_i used this guide to install openstack13:57
ThomasS_https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-config-prod-ceph.html13:57
ThomasS_works great13:57
ThomasS_only the aodh service is unavaibale13:57
ThomasS_the containers are running13:58
ThomasS_anything missing in the guide to fix this?13:58
ThomasS_i am on 16.0.513:58
wspthrThomasS_ the healthcheck of haproxy for aodh expects a 401 at / but gets a 200, we've changed the healthcheck to /v2 than it should work.14:00
gokhanMikeW_, sorry ı didn't know cyclops but we need gnocchi14:00
*** SerenaFeng has joined #openstack-ansible14:00
gokhanThomasS_, I also used this guide. on aodh wspthr is right. aodh will work like he said14:02
*** vnogin has quit IRC14:06
*** sxc731 has quit IRC14:07
*** fatdragon has joined #openstack-ansible14:12
*** sxc731 has joined #openstack-ansible14:12
*** sdake has quit IRC14:12
*** sdake has joined #openstack-ansible14:13
*** sdake has quit IRC14:13
*** sdake has joined #openstack-ansible14:13
ThomasS_thanks!14:16
ThomasS_where have you changed to healthcheck? i guess changing it manually in haproxy.conf is not the smartest thing riught?14:16
wspthrThomasS_ you are right it is not the smartest thing but afaik there is no patch/bug currently present, so it is the only way. I'm also not sure if it is smarter to change the check to expect the 200 or to change the path for the check. If a new api version gets introduced we will end up with the same issue.14:19
ThomasS_ok so where do i need to change it :-)14:20
TahvokWhen the new version bump coming? There are some fixes that were made I would like to see in the release. Currently using ocata branch because I need them14:22
*** fatdragon has quit IRC14:23
*** smatzek has quit IRC14:24
wspthrThomasS_ on your backend host in /etc/haproxy.cfg at the paragraph "backend aodh_api-back" at the line "option httpchk HEAD /" change it to "option httpchk HEAD /v2" and reload haproxy config (you need to do this on every backend host in your infra).14:24
*** smatzek has joined #openstack-ansible14:25
ThomasS_already found it!14:27
ThomasS_works like a charm14:27
ThomasS_hope this goes in the next release14:27
ThomasS_will this break when i run the haproxy playbook again? i guess so14:28
jafehaThomasS_, you're right, rerunning the playbook would overwrite that change14:29
ThomasS_ok so i need to write a "fix" playbook to alter the config afterwards14:29
*** smatzek has quit IRC14:29
*** vnogin has joined #openstack-ansible14:36
*** SerenaFeng has quit IRC14:49
*** ianychoi has quit IRC14:53
*** fatdragon has joined #openstack-ansible14:55
*** aruns__ has quit IRC14:55
*** vnogin has quit IRC15:01
*** vnogin has joined #openstack-ansible15:01
*** fatdragon has quit IRC15:06
*** smatzek has joined #openstack-ansible15:10
*** poul has joined #openstack-ansible15:11
*** vnogin has quit IRC15:12
*** nshetty has quit IRC15:12
*** udesale has quit IRC15:13
*** poul_ has quit IRC15:14
*** rromans has joined #openstack-ansible15:22
*** chhavi has joined #openstack-ansible15:29
*** fatdragon has joined #openstack-ansible15:32
*** fatdragon has quit IRC15:36
openstackgerritMerged openstack/openstack-ansible-galera_server master: Fix Galera socket for RedHat  https://review.openstack.org/52987915:37
*** mrch has quit IRC15:42
openstackgerritMerged openstack/openstack-ansible-tests master: Skip and echo out gzip log compression error  https://review.openstack.org/53078415:43
*** fatdragon has joined #openstack-ansible15:46
spotzevrardjp: You're back!:)15:47
*** poul has quit IRC15:48
*** poul has joined #openstack-ansible15:48
*** poul has quit IRC15:53
*** armaan has quit IRC15:55
*** armaan has joined #openstack-ansible15:55
*** sxc731 has quit IRC15:56
*** fatdragon has quit IRC15:58
evrardjpspotz: hello!16:02
evrardjpI am indeed!16:02
evrardjphappy new year to you and to everyone else that didn't see my first message :)16:02
*** poul has joined #openstack-ansible16:04
*** poul has quit IRC16:08
spotzevrardjp: Happy New Year16:08
evrardjphow was your end of year holidays?16:10
spotzevrardjp: You mean my trip to summit?:) Didn't have anything but the holidays off16:12
*** poul has joined #openstack-ansible16:13
*** poul has quit IRC16:17
*** Jack_Iv has quit IRC16:18
cloudnullmornings all16:19
*** openstackgerrit has quit IRC16:19
cloudnullo/ ANKITA16:19
cloudnulldo you happen to have more details on what's failing?16:19
cloudnullalso do is it truely failing, can you check to the exist status of the playbook run ?16:20
* cloudnull wonders if maybe we should have an exit status check in the openstack-ansible wrapper.16:20
evrardjpcloudnull: what do you mean?16:21
evrardjpwe just use rc of ansible16:21
evrardjp?16:21
cloudnullwe use blocks w/ rescue in quite a few places which reports failures, but nothing actually failed.16:21
evrardjpwe've removed that16:21
cloudnullits super confusing .16:21
cloudnullwe still have it in quite a few place.s16:22
evrardjpyeah it was16:22
evrardjpit will be fixed in 2.516:22
evrardjpwe can remove those tasks in the meantime16:22
evrardjpwell16:22
cloudnullI wonder if it'd help folks if the wrapper just had an exist status check16:22
evrardjpwe still have where it is meant as a rescue16:22
*** SerenaFeng has joined #openstack-ansible16:23
evrardjpnot sure where you want to go16:23
cloudnulluntil we can get to 2.516:23
evrardjpI haven't seen a false positive for a while16:24
*** dave-mccowan has quit IRC16:24
evrardjpbut I think we should have more healchecks16:24
evrardjphealthchecks16:24
*** dave-mccowan has joined #openstack-ansible16:25
*** deadnull has quit IRC16:25
*** sxc731 has joined #openstack-ansible16:25
*** vnogin has joined #openstack-ansible16:27
*** poul has joined #openstack-ansible16:30
*** ivve has quit IRC16:30
*** poul has quit IRC16:34
*** pcaruana has quit IRC16:35
*** openstackgerrit has joined #openstack-ansible16:35
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add a return code check to the osa wrapper  https://review.openstack.org/53095916:35
cloudnullevrardjp: ^ something like that16:35
cloudnullthat way there's a line which folks can see when the wrapper is use.d16:36
dmsimardare there ara reports in osa logs now ? can't find them16:36
dmsimardI see the zuul ones but not the OSA one16:36
cloudnulldmsimard: I think we're only running ara in the gate when there's a failure?16:37
cloudnullevrardjp: ^16:37
dmsimardhttps://review.openstack.org/#/c/530531/ has a few failures16:38
*** SerenaFeng has quit IRC16:41
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_build master: Add librados-dev to distro package list  https://review.openstack.org/53057016:42
*** ivve has joined #openstack-ansible16:42
cloudnulldmsimard: http://logs.openstack.org/18/523218/21/check/openstack-ansible-dir-centos-7/b150816/ara/16:42
cloudnulllooks like it's only there for the role jobs16:42
dmsimardcloudnull: that's the zuul report though16:43
cloudnullodyssey4me: evrardjp: do we know why we're not running ara for the integrated repo?16:43
dmsimardso there's this one big "command" task in the run.yaml playbook16:43
dmsimardI'm looking for a report with those 2000 tasks :D16:43
cloudnullsorry bad link16:43
cloudnullhttp://logs.openstack.org/18/523218/21/check/openstack-ansible-dir-centos-7/b150816/logs/ara/16:43
*** poul has joined #openstack-ansible16:44
evrardjpyeah I think our ara runs gets overriden in post job IIRC what odyssey4me said.16:49
*** poul has quit IRC16:49
evrardjpDidn't we move to another folder?16:49
evrardjpyeah that link works16:50
evrardjpbut that's a role test16:50
evrardjpyou want something for the integrated or ???16:50
dmsimardI don't know, I remember seeing ara reports everywhere before :/16:51
evrardjpyeah, we are kinda waiting for the middleware to be up :p16:56
spotzjamesdenton: If youo're Ford started can you comment on https://review.openstack.org/#/c/530146/ :)16:56
dmsimardworking on it T_T16:56
evrardjphaha joking, we should help you instead :p16:56
dmsimardit doesn't play nice with the htmlify middleware that's used to make clickable timestamps and colors in logs16:57
evrardjpwe just have 4 big fires to finish first: on metal, nspawn, docs improvements, all the rest.16:57
evrardjp:p16:57
evrardjpall the rests are testing scenarios for ovs, autoscaling, ... for example. Many ideas!16:58
*** poul has joined #openstack-ansible17:01
*** chhavi has quit IRC17:05
*** poul has quit IRC17:06
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add a return code check to the osa wrapper  https://review.openstack.org/53095917:06
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add a return code check to the osa wrapper  https://review.openstack.org/53095917:08
*** poul has joined #openstack-ansible17:10
cloudnullevrardjp: so I built an AIO using the all_metal patch, and it worked.17:11
cloudnull**using the all_metal patch and enabling the "all_metal" scenario**17:11
evrardjpso, I am not insane??17:12
cloudnullI didn't see that pesky keystone error17:12
evrardjpsame for me17:13
evrardjpit's been like one month now. maybe 2.17:13
evrardjpI am stuck stuck stuck17:14
evrardjpbut I will stop for today17:14
evrardjpif yours get to merge first, I will rebase :)17:14
evrardjplet me remove my negative vote. I think it's fair game.17:14
cloudnull:)17:15
*** poul has quit IRC17:15
cloudnullmaybe add a possitive  vote :P17:15
*** vnogin has quit IRC17:15
*** smatzek has quit IRC17:16
spotzheheh17:17
*** smatzek has joined #openstack-ansible17:17
evrardjpcloudnull: you're pushing your luck at the end of the day!17:18
evrardjpI will do it tomorrow :)17:18
evrardjpI will do a last attempt on my machine with the user variables overrides from gates.17:18
*** smatzek_ has joined #openstack-ansible17:19
evrardjpI know there is another file somewhere17:19
evrardjpwill use it17:19
evrardjpanyway17:19
evrardjpI am off for today!17:19
evrardjpto all of you: have a nice day! see you tomorrow for more fun adventures!17:19
*** smatzek__ has joined #openstack-ansible17:20
*** smatzek has quit IRC17:21
*** smatzek_ has quit IRC17:23
*** smatzek__ has quit IRC17:24
*** aruns__ has joined #openstack-ansible17:24
*** smatzek has joined #openstack-ansible17:25
*** vnogin has joined #openstack-ansible17:25
*** fatdragon has joined #openstack-ansible17:26
*** poul has joined #openstack-ansible17:27
*** fatdragon has quit IRC17:31
*** vnogin has quit IRC17:31
*** poul has quit IRC17:31
*** Jack_Iv has joined #openstack-ansible17:32
*** fatdragon has joined #openstack-ansible17:34
*** vnogin has joined #openstack-ansible17:36
*** vnogin has quit IRC17:41
*** sxc731 has quit IRC17:42
*** poul has joined #openstack-ansible17:44
*** smatzek has quit IRC17:47
*** poul has quit IRC17:49
*** smatzek has joined #openstack-ansible17:51
*** smatzek has quit IRC17:51
*** smatzek has joined #openstack-ansible17:51
*** vnogin has joined #openstack-ansible17:57
*** vnogin has quit IRC17:59
*** Jack_Iv has quit IRC18:03
*** armaan has quit IRC18:06
*** poul has joined #openstack-ansible18:09
*** shardy has quit IRC18:10
*** armaan has joined #openstack-ansible18:13
*** yolanda__ is now known as yolanda18:14
*** poul has quit IRC18:14
*** poul has joined #openstack-ansible18:18
*** ChanServ sets mode: +r 18:19
jamesdenton@spotz done18:20
spotzjamesdenton: Thank you sir, your tweet made me go out and start my truck, but it was already warmer:)18:20
jamesdentonI hadn't run mine in a few days, and it's been hovering between -10F and 10F here that last couple of days. I learned quickly.18:21
spotzjamesdenton: You move? I thought you were at Castle18:22
jamesdentonI was at Castle but relo'd to Ohio back in June18:22
spotzjamesdenton: Ahh. Yeah cause it hasn't been that cold. I try to drive my truck once a week though sometimes it's once a month if I don't need it18:23
*** poul has quit IRC18:23
spotzevrardjp: I agree with jamesdenton comment of a differnet file for this patch. How you want too do this?18:24
jamesdentonI'm usually out once a day or so, but I've had family here the last few days. School started back up today but was cancelled because the busses wouldn't start either. Not a good day for Diesels.18:24
*** epalper has quit IRC18:25
spotzNeed to look and see if my chevy does have the heated block it has pretty much everything18:25
jamesdentonI'm scheduled to have one installed later this week :D18:26
*** stuartgr has quit IRC18:28
spotzWell from looking at a description of another one mine should have it:)18:30
spotzShould we dip into freezing again I will go test, it'll be nice to know for sure18:30
*** sxc731 has joined #openstack-ansible18:43
*** aruns__ has quit IRC18:51
*** RandomTech has joined #openstack-ansible18:56
RandomTechHey is anyone here able to help figure out what went wrong with our19:03
RandomTechwith the ceph part of our install*19:03
*** gokhan is now known as Guest8946119:06
*** jhesketh has quit IRC19:08
*** jhesketh has joined #openstack-ansible19:11
*** sxc731 has quit IRC19:36
*** pcaruana has joined #openstack-ansible19:45
*** foutatoro has joined #openstack-ansible19:55
*** dave-mccowan has quit IRC19:56
*** pcaruana has quit IRC19:59
*** Jeffrey4l has quit IRC20:11
*** RandomTech has quit IRC20:14
*** sxc731 has joined #openstack-ansible20:15
*** cshen_ has joined #openstack-ansible20:19
*** cshen_ has quit IRC20:24
*** Jeffrey4l has joined #openstack-ansible20:24
*** cshen_ has joined #openstack-ansible20:25
*** cshen_ has quit IRC20:43
*** cshen_ has joined #openstack-ansible20:47
*** jbadiapa has quit IRC20:48
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Stop building a newton venv for LEAP upgrades  https://review.openstack.org/53101020:49
*** portdirect has quit IRC20:50
*** portdirect has joined #openstack-ansible20:50
*** foutatoro has quit IRC21:11
*** MikeW_ has quit IRC21:13
*** bgmccollum has quit IRC21:16
*** cshen_ has quit IRC21:18
*** bgmccollum has joined #openstack-ansible21:20
*** smatzek has quit IRC21:21
*** cshen_ has joined #openstack-ansible21:28
*** cshen_ has quit IRC21:54
*** sxc731 has quit IRC22:06
*** vnogin has joined #openstack-ansible23:00
*** vnogin has quit IRC23:04
*** eumel8 has quit IRC23:09
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add env vars to set the RAM for VMS in the mnaio  https://review.openstack.org/53103223:17
*** esberglu has quit IRC23:38

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!