Tuesday, 2018-04-10

*** logan- has quit IRC00:02
*** logan- has joined #openstack-ansible00:02
*** pmannidi has joined #openstack-ansible00:10
*** exodusftw has joined #openstack-ansible00:29
*** cjloader has joined #openstack-ansible00:32
*** cjloader has quit IRC00:33
*** cjloader has joined #openstack-ansible00:34
*** odyssey4me has quit IRC00:34
*** odyssey4me has joined #openstack-ansible00:34
*** tomtom001 has quit IRC01:01
*** tomtom001 has joined #openstack-ansible01:02
*** mhayden has quit IRC01:04
logan-iirc it doesnt shut down the lb endpoint theres some huge lua thing that starts/stop different services as they sync01:26
logan-in repo_server01:26
logan-oh my buffer is like 12 hours behind...01:26
logan-anyway mnaser in case you didn't find out by now, this is what manages the repo services going up/down during syncs: https://github.com/openstack/openstack-ansible-repo_server/blob/master/templates/lsyncd.lua.j201:28
mnaserlogan-: damn that's fancy01:47
logan-ya its pretty elaborate01:47
*** _moep_ has quit IRC02:05
*** _moep_ has joined #openstack-ansible02:06
*** dave-mccowan has joined #openstack-ansible02:12
*** pradiprwt has quit IRC02:13
dmsimardjust sharing something pretty cool I found: https://github.com/openshift/openshift-ansible/blob/4e5ef6d50f31eb51a249c3b4836141030f70b0de/roles/installer_checkpoint/callback_plugins/installer_checkpoint.py02:48
*** dave-mccowan has quit IRC02:48
dmsimardends up printing (and keeping an overlay?) on top of the playbook run http://paste.openstack.org/show/718777/02:49
*** savvas has joined #openstack-ansible03:05
cjloaderAnyone know what causes an 416 error with radosGW/Ceph03:07
cjloaderInvalidRange.: BackendException: Failed to add container to Swift.03:09
*** savvas has quit IRC03:10
*** cjloader has quit IRC03:16
*** cjloader_ has joined #openstack-ansible03:16
*** ianychoi has quit IRC03:16
*** cjloader_ has quit IRC03:19
*** indistylo has joined #openstack-ansible03:20
*** cjloader has joined #openstack-ansible03:20
*** ianychoi has joined #openstack-ansible03:20
openstackgerritMerged openstack/openstack-ansible master: Fix osa_toolkit dictutils import  https://review.openstack.org/55982303:26
*** armaan has quit IRC03:26
*** esberglu has quit IRC03:49
*** vincent has joined #openstack-ansible03:56
*** vincent is now known as Guest5786803:57
*** Administrator__ has quit IRC03:58
*** indistylo has quit IRC04:01
*** pcaruana has joined #openstack-ansible04:06
*** poopcat has quit IRC04:09
*** udesale has joined #openstack-ansible04:09
*** ianychoi has quit IRC04:14
*** ianychoi has joined #openstack-ansible04:16
*** pcaruana has quit IRC04:16
*** radeks has joined #openstack-ansible04:31
*** chhagarw has joined #openstack-ansible04:32
*** mgrewy has joined #openstack-ansible04:35
mgrewytempest-install is failing....upload image to glance is failing. glance-api.log has trace back -> http://paste.openstack.org/show/718779/04:36
*** indistylo has joined #openstack-ansible04:37
*** cjloader has quit IRC04:54
*** sar has quit IRC05:33
*** ianychoi has quit IRC05:47
*** ianychoi has joined #openstack-ansible05:49
*** mma has joined #openstack-ansible05:53
*** mma has quit IRC05:54
*** eumel8 has joined #openstack-ansible05:57
*** rodolof has joined #openstack-ansible06:02
*** mma has joined #openstack-ansible06:03
*** sar has joined #openstack-ansible06:12
*** cjloader has joined #openstack-ansible06:16
*** tomtom001 has quit IRC06:18
*** tomtom001 has joined #openstack-ansible06:18
*** cjloader has quit IRC06:19
*** lvdombrkr has joined #openstack-ansible06:28
*** armaan has joined #openstack-ansible06:30
*** pcaruana has joined #openstack-ansible06:40
*** gkadam has joined #openstack-ansible06:43
*** gkadam_ has joined #openstack-ansible06:50
*** gkadam has quit IRC06:50
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Fix osa_toolkit dictutils import  https://review.openstack.org/55992206:51
evrardjpgood morning06:53
*** marwel has quit IRC06:54
*** marwel has joined #openstack-ansible06:54
evrardjpdmsimard: it would be even better to combine the callback plugin with a strategy -- in case of upgrades, do things based on checkpoints06:56
evrardjpdmsimard: you're a bad person, you give me bad ideas.06:57
evrardjp:D06:57
evrardjpThe callback can definitely know if the playbook ran fine. Darn, we found a cleaner way to run upgrades!06:57
evrardjpno need for shell script anymore!06:58
evrardjplet me check if it's possible to force a strategy with an env var, just out of curiosity06:58
*** gkadam_ is now known as gkadam07:00
*** cshen has joined #openstack-ansible07:00
*** aruns has joined #openstack-ansible07:01
*** indistylo has quit IRC07:04
*** marwel has quit IRC07:07
*** marwel has joined #openstack-ansible07:11
openstackgerritMerged openstack/openstack-ansible-tests stable/queens: Extend linters time out  https://review.openstack.org/55979707:12
*** gkadam has quit IRC07:14
evrardjpexport DEFAULT_STRATEGY=upgrade07:16
evrardjphaha07:16
evrardjpthat could work07:16
*** radeks has quit IRC07:21
*** radeks has joined #openstack-ansible07:21
*** epalper has joined #openstack-ansible07:26
*** jhesketh_ has joined #openstack-ansible07:31
*** jhesketh has quit IRC07:37
ArchiFleKsodyssey4me:Hi, I have the same issue with nova cellv2 commit, what I don't understand is how the delegate work, on the first compute node the task is delegate to the first conductor, from a compute node I can resolve a nova conductor container with /etc/hosts but i cannot ssh into it, not with root or nova user07:41
*** rodolof has quit IRC07:45
*** rodolof has joined #openstack-ansible07:45
*** electrofelix has joined #openstack-ansible07:47
*** tomtom001 has quit IRC08:02
*** tomtom001 has joined #openstack-ansible08:03
*** mbuil has joined #openstack-ansible08:04
hwoaranggood morning08:05
mmaMorning.  Just to double check, there is no multi region support in OSA right?08:13
*** pmannidi has quit IRC08:17
*** mgrewy has quit IRC08:29
openstackgerritMerged openstack/openstack-ansible-tests stable/queens: Add Ceph experimental job for roles  https://review.openstack.org/55978108:32
*** vnogin has joined #openstack-ansible08:33
*** vnogin has quit IRC08:41
*** radeks has quit IRC08:50
*** radeks has joined #openstack-ansible08:50
*** madorn has quit IRC08:55
*** madorn has joined #openstack-ansible08:56
*** aruns has quit IRC09:00
*** pcaruana has quit IRC09:04
evrardjpmma: you can do multi region09:12
evrardjpthere is nothing "built in" because multi region means so many things for so many different ppl.09:12
evrardjpSwift for example, is planned for multi region09:12
evrardjpand you can do many thing depending on your overrides09:12
evrardjplogan-: odyssey4me maybe we should hack around a user story or user stories for multi regions09:13
evrardjpwhat do you think? That question came up like 2 times this week (and didn't come up the month before :p )09:14
mmaevrardjp: Ok, a story or guide would be awesome for sure.09:15
evrardjpmma: it depends on what you want to achieve basically09:15
openstackgerritFabien Andrieux proposed openstack/openstack-ansible-specs master: Blueprint specs for openstack fds/fdio/vpp integration  https://review.openstack.org/55207909:17
mmaevrardjp: In this case we have an existing OSA based installation running. And now we are to install a new installation in a new datacenter but connect it to the existing keystone service.09:18
evrardjpok so not multi region federated keystone09:21
evrardjp:p09:21
openstackgerritFabien Andrieux proposed openstack/openstack-ansible-specs master: Blueprint specs for openstack fds/fdio/vpp integration  https://review.openstack.org/55207909:21
evrardjpare the two DC far away from each other?09:21
evrardjp(latency wise)09:21
mmaNo very close09:21
mmaMilliseconds09:22
*** DanyC has joined #openstack-ansible09:24
*** vnogin has joined #openstack-ansible09:27
*** DanyC has quit IRC09:29
evrardjpms is big09:30
evrardjp:p09:30
*** DanyC has joined #openstack-ansible09:30
evrardjpdo you really need multi region then?09:30
evrardjplooks like you could just separate nova cells09:31
*** vnogin has quit IRC09:32
hwoarangodyssey4me: mnaser hello. I have a question about the 'distro package work'09:35
hwoarangin defaults/main.yml we have keystone_bin: /openstack/blabla but that only makes sense when using venv. so i was thinking to override that in vars/main.yml if installing distro packages. but recursive stuff like keystone_bin: "{{ (keystone_install_method == 'source') | ternary(keystone_bin, '/usr/bin') }}" do not seem to work.09:37
evrardjphwoarang: I still haven't got the chance to read the release notes09:38
hwoarangevrardjp: what release notes09:38
odyssey4mehwoarang the *_bin vars need to currently be overridable via group_vars, as the tag/bin are passed to the role from there, so using vars/main won't be good unless we implement a different way of setting the path09:41
mmaevrardjp: I think it will have to be multiregion in this case due to regulatory reasons. In reality we are forced to create 2 standalone installations with "as little shared as possible", however the customer insists on joined keystone for managing accounts.09:41
hwoarangodyssey4me: ok thanks for that. so would using stuff like "{{ (keystone_install_method == 'source') | ternary("/openstack/blabla", "/usr/ bin") }}"09:42
hwoarangbe better?09:42
hwoarangin defaults/main.yml that is09:42
mmaSo what is the best way for this scenario from an OSA perspective? Do we add it like a "new" playbook installation and override keystone configuration to point it to the existing keystone? (and skip the keystone setup playbook for the new installation?)09:43
evrardjpmma: I think you should simply do two installs, and the second one should point to the first keystone09:43
evrardjphwoarang: lapsus sorry09:43
evrardjphwoarang: I meant your spec09:43
evrardjpI am talking about reno in ansible and my brain messed up :p09:44
hwoarang;p09:45
mmaevrardjp: Ok, and there is no special variable to control that? Or how do I get it to understand where the existing keystone service is so that it can setup endpoints, etc?09:46
mmaIf there is a guide or documentation for it that I can look at instead of asking please tell me :)09:47
odyssey4mehwoarang do you need to refer to keystone_bin? is there a way to avoid it perhaps?09:47
odyssey4memma there is not, you'll have to read the code and implement via trial and error - there are so many different possibilities09:48
mmaodyssey4me: Got it thanks :)09:48
odyssey4memma you could use a shared keystone ldap back-end, or use a shared database for keystone, or setup keystone/horizon in its own region09:49
evrardjpmma: logan- has done it09:50
mmaIn this case it will be a shared database. Thanks for help.09:50
evrardjpso you should maybe ask him :)09:50
odyssey4memma if you do not need a component in your environment - like keystone, for example, you'd need to provide override variables to tell other components where to find it... usually the inventory gives us that information, but in a case where there's no keystone it won't so you have to provide them09:50
hwoarangodyssey4me: well the role uses keystone_bin everywhere09:50
hwoarangto call binaries, configure templates etc09:50
hwoarangfind out where venv is etc09:50
*** armaan has quit IRC09:50
odyssey4memma each one of the services has a db connection string in the conf file - to use a common database you'll need to override part of that to point at the right db09:51
hwoarangi will post an rfc and then we will see what to do with it09:51
odyssey4methen with galera you should be able to set it up to have a mini cluster per region, but implement a tunnel between the regions for galera to replicate09:51
evrardjpafk lunch everyone09:51
*** armaan has joined #openstack-ansible09:51
mmaodyssey4me and evrardjp: Thanks for help and hits. Also enjoy your lunch evrardjp :)09:52
stuartgrHi.  bootstrap-ansible.sh for stable/queens installs python3 on the deployment host but then I get "NameError: name 'unicode' is not defined" when it runs /etc/ansible/roles/plugins/action/config_template.py which sounds like that plugin is expecting python2  ???09:52
odyssey4mehwoarang it may make sense to implement a dict similar to the keystone_services dict.... or perhaps just add to that dict?09:52
*** vnogin has joined #openstack-ansible09:52
hwoarangpossibly...09:54
hwoarangi will think about it09:54
*** armaan has quit IRC10:02
*** armaan has joined #openstack-ansible10:02
*** radeks has quit IRC10:03
*** cshen has quit IRC10:09
*** cshen has joined #openstack-ansible10:20
odyssey4mehwoarang evrardjp the more and more I think about it, the more I think we should slowly try to move towards roles being components with a well defined input and output, and we should possibly try to make use of more discrete roles where we can10:31
odyssey4meexamples where we could use other roles are for nginx, apache, uwsgi10:31
odyssey4meso the distro install thing puts us in a place where we really need to think about what should be internal to the role, and what needs to be external10:32
evrardjp[m]odyssey4me yes sounds good.10:32
odyssey4mefor example, we're currently passing both a tag and a binary path into the role - do we really need to pass both? why not just one?10:32
evrardjp[m]i agree with it, but we should be relying on the minimum amount of deps if possible too10:33
niraj_singhevrardjp: Please review this spec when you get time https://review.openstack.org/#/c/556756/10:33
evrardjp[m]agreed10:33
odyssey4mefor a source-based install and a package-based install they use that information in different ways - for example a source-based install can infer a path for the venv from the tag, while the package-based install doesn't need to but it might still set a local fact using the tag10:34
evrardjp[m]niraj_singh: will review today10:34
niraj_singhthanks10:34
evrardjp[m]i think it's wise to say "use package" or "use source" as input and infer things from that by default10:36
*** udesale has quit IRC10:45
*** radeks has joined #openstack-ansible10:49
openstackgerritMerged openstack/openstack-ansible stable/queens: Fix osa_toolkit dictutils import  https://review.openstack.org/55992211:10
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron master: DNM: Testing ODL gates  https://review.openstack.org/55906511:10
odyssey4mecores, need another review for https://review.openstack.org/559804 which needs to merge and be ported back to queens for ironic to be deployable11:12
odyssey4mehwoarang evrardjp[m] another approach that we could take to simplify package vs source installs is, when deploying via source, to implement either symlinks for the venv binaries into something like /usr/local/bin/ or alternatively adding the venv path into the PATH - that way the binary is executable from anywhere and the path doesn't have to be referenced11:15
*** ansmith has quit IRC11:22
evrardjp[m]if that doesn't mess up with inclusion that's quite elegant11:32
*** vnogin has quit IRC11:37
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Bug fix: Parenthesis needed when using ternary  https://review.openstack.org/55887111:37
*** vnogin has joined #openstack-ansible11:38
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add container/service clean up for ironic/trove  https://review.openstack.org/55999511:39
*** vnogin has quit IRC11:39
*** markvoelker has joined #openstack-ansible11:40
logan-mma: what you're trying to do is not too hard with separate osa installs/inventories. heres an example of how you might configure some user vras in your 2nd region in order to tie it to the 1st region's keystone: https://gist.github.com/Logan2211/d2c9548f8a405e663ea356adb83b6e9711:44
evrardjplogan-: forked! I will try to find time in the future to document this11:46
evrardjpthanks for the share!11:46
evrardjpodyssey4me: reviewin'11:46
evrardjpok I am too late11:46
evrardjp:p11:46
evrardjpniraj_singh: doing my review right now11:46
*** vnogin has joined #openstack-ansible11:47
odyssey4meevrardjp yeah, considering the number of queries we get for it, I think that should be another user story11:48
mmalogan-: Thanks11:48
*** vnogin has quit IRC11:48
niraj_singhevrardjp: ok11:49
evrardjpniraj_singh: sorry, last details!11:49
evrardjpthen for me we're good11:49
niraj_singhyes :)11:50
logan-np evrardjp mma11:51
niraj_singhevrardjp: we are going to start it from empty role. There is no role developed till now for masakari and masakari-monitor.11:53
*** vnogin has joined #openstack-ansible11:53
Guest94774hi folks, I tried to install octavia with openstack ansible.  ı run repo-build.yml for this but getting error on pip install role. error log is http://paste.openstack.org/show/718808/. what can be root of this problem ?11:54
ivvebah i deleted the log but branch=master doesn't deploy on 3x controller nodes failing on rabbit host error (im guessing its the old environment in lxc container stuff)11:54
ivveit doesn't detect correct hostname11:54
ivve(a redeploy with exact same config stable/pike works)11:55
ivvethank god for MaaS & ansible :)11:55
evrardjpaspiers: do you, by any chance, have a corosync/pacemaker role in ardana we could re-use?11:55
evrardjpniraj_singh: you don't have a PoC somewhere?11:55
aspiersnope, ardana doesn't use pacemaker (yet, anyway)11:55
evrardjpok11:56
evrardjpaspiers: you'd probably need it for masakari if I understand correctly niraj_singh  :)11:56
aspierscorrect11:56
niraj_singhevrardjp: yes i have, should i mention that link in the review comment11:56
evrardjpniraj_singh: yes, so we don't start from nothing, we'll start from your code I guess?11:57
niraj_singhevrardjp: yes11:57
odyssey4meGuest94774 updated your checkout and bootstrap-ansible again to bring in the latest packages, that issue was resolved in the latest release11:57
aspiersarchitecturally it's not mandatory but IMHO it makes the most sense right now11:57
ivvealso i think there is an issue with nova-scheduler filters but not sure. i can migrate single machines no problem but when using "nova host-evacuate-live" or the "migrate host" in horizon i get "no hosts available".. even with debuglogs in placement, scheduler and nova i can't really figure out the issue.. im redeploying this testcluster and will submit a bug11:57
aspiersin the future there might be other fencing mechanisms11:57
evrardjpaspiers: do you know if there are plans to use tempest testing soon? I haven't seen that into the open specs11:57
evrardjpaspiers: oh that would be interesting.11:58
evrardjp(the other fencing mechanisms)11:58
aspiersevrardjp: ardana uses tempest now11:58
evrardjpaspiers: I meant for masakari11:58
aspiersoh11:58
aspiers:)11:58
aspiersI don't know11:58
evrardjp:)11:58
openstackgerritFabien Andrieux proposed openstack/openstack-ansible-specs master: Blueprint specs for openstack fds/fdio/vpp integration  https://review.openstack.org/55207911:58
aspierssampath can probably answer that11:58
evrardjpwill ask in the channel11:59
evrardjpthanks11:59
Guest94774odyssey4me, thanks a lot :)11:59
*** odyssey4me has quit IRC12:00
*** odyssey4me has joined #openstack-ansible12:00
ivvetest environment is openstack hyperconverged on 3 blades with ceph backend fyi12:00
*** mhayden has joined #openstack-ansible12:02
*** openstackgerrit has quit IRC12:04
*** openstackgerrit has joined #openstack-ansible12:07
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Bug fix: fwaas_v2 was missing in the pip task  https://review.openstack.org/55887312:07
evrardjpcloudnull: you're heading to vancouver, right? Is there anyone else going there?12:08
*** mgrewy has joined #openstack-ansible12:08
mgrewyi am seeing this issue StoreAddDisabled - http://paste.openstack.org/show/718779/    any pointer to resolve it.  nfs image mount has rw access12:09
openstackgerritNiraj Singh proposed openstack/openstack-ansible-specs master: Add specs to create masakari role  https://review.openstack.org/55675612:09
*** cshen has quit IRC12:12
niraj_singhevrardjp: updated specs :)12:13
evrardjpcool12:13
odyssey4meevrardjp well, this is an interesting development - it appears that some ansible playbooks/roles are in-tree for tempest: http://git.openstack.org/cgit/openstack/tempest/tree/12:13
evrardjpniraj_singh: do you mind if I edit it quickly? :D12:14
evrardjpodyssey4me: yes, so ...12:14
niraj_singhevrardjp: np12:14
evrardjpodyssey4me: I started to discuss this with Andrea12:14
evrardjpI said we're gonna end up doing the same thing, please let's be in sync12:15
*** cshen has joined #openstack-ansible12:15
evrardjpI haven't got the chance to discuss with him more recently12:15
evrardjpbut I'd like to do it in vancouver if I can :)12:15
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible stable/pike: Add missing Ubuntu packages  https://review.openstack.org/56000312:15
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible stable/pike: Add missing Ubuntu packages  https://review.openstack.org/56000312:16
odyssey4meyeah, if the implementation could be a bit pluggable to allow devstack or other ansible config/execution methods then I think we'd be able to converge12:16
evrardjpodyssey4me: yeah I wanted to say:12:17
evrardjpuse our config template, make your life easier.12:17
evrardjpThen ship a playbook to run your things.12:18
evrardjpThen guess what, you have the os_tempest role12:18
evrardjp:p12:18
odyssey4meyeah, sort of - our resource stuff is something they do via devstack I think12:18
*** dave-mccowan has joined #openstack-ansible12:19
odyssey4methat's part of why I think the resource stuff should go into the service roles - it's a similar model to devstack, each service sets up its own things12:19
evrardjpthat should probably exercise the sdk, not a devstack shell script12:19
evrardjpodyssey4me: I am fine with that later12:20
evrardjpor now if someone does it12:20
mgrewyi am seeing this issue StoreAddDisabled - http://paste.openstack.org/show/718779/    any pointer to resolve it??  nfs image mount has rw access12:20
*** vnogin has quit IRC12:22
*** vnogin has joined #openstack-ansible12:24
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-specs master: Add specs to create masakari role  https://review.openstack.org/55675612:25
*** epalper has quit IRC12:26
*** epalper has joined #openstack-ansible12:27
evrardjpmgrewy: it says Error in store configuration.12:27
evrardjpprobably worth double checking your configuration :p12:27
mgrewyi am using example playbooks12:29
odyssey4membuil could you backport https://review.openstack.org/558871 and https://review.openstack.org/558873 to whichever branches need it?12:29
mgrewyconfigurations looks good to me :(12:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Fix osa_toolkit dictutils import  https://review.openstack.org/56000812:33
odyssey4memgrewy you might want to ask about that in the glance IRC channel, but it looks to me like it's validating some capabilities on the storage system and that's failing12:35
odyssey4meso it could be any number of issues which all depend on that store12:36
openstackgerritMerged openstack/openstack-ansible-os_tempest master: Allow for single tempest image upload  https://review.openstack.org/55914612:36
*** vakuznet has joined #openstack-ansible12:36
RossSchulman[m]This is a really basic question, but for nodes that are going to live in containers, I need to create the containers before running OSA right?12:37
odyssey4meRossSchulman[m] huh? I'm not sure what you're asking actually makes sense - what do you mean by 'node' and 'node thats going to live in containers'12:37
odyssey4meFYI OSA will create, name set  IP addresses for and setup networking for all LXC/nspawn containers that it needs12:38
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest stable/queens: Allow for single tempest image upload  https://review.openstack.org/56001012:39
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest stable/pike: Allow for single tempest image upload  https://review.openstack.org/56001112:39
evrardjpoh that merged12:39
evrardjpcool12:39
odyssey4meyeah, finally12:39
evrardjpocata ?12:39
evrardjpodyssey4me: did you recheck the other patches?12:40
evrardjpI will12:40
*** vnogin has quit IRC12:40
evrardjpodyssey4me: could you vote on those: https://review.openstack.org/#/q/topic:on-metal-voting+(status:open+OR+status:merged) ?12:40
*** vnogin has joined #openstack-ansible12:40
*** _david_ has joined #openstack-ansible12:41
*** mgrewy has quit IRC12:41
*** cshen has quit IRC12:42
evrardjpcores that would be nice if you could vote on   https://review.openstack.org/560010 and  https://review.openstack.org/56001112:42
*** mgrew_ has joined #openstack-ansible12:42
evrardjpthat would unblock nova12:42
jrosserevrardjp: we've had to remove the check for python3 from here to get anywhere at all with queens https://github.com/openstack/openstack-ansible/blob/3734ce44bfd6705b715349f520f29fc2fff1bca8/scripts/bootstrap-ansible.sh#L11112:44
evrardjpso what do you use by default jrosser ?12:44
*** pcaruana has joined #openstack-ansible12:44
jrossergood question. it gets installed previously in boostrap-ansible.sh, detected, then used..... as far as i can see12:45
RossSchulman[m]odyssey4me: by node I mean discrete infrastructure unit. So, keystone is going to live in a container, do I need to create that container or will OSA do it?12:45
jrosserevrardjp: unless i'm missing something it gets installed on the deploy host here https://github.com/openstack/openstack-ansible/blob/3734ce44bfd6705b715349f520f29fc2fff1bca8/scripts/bootstrap-ansible.sh#L8412:46
*** ansmith has joined #openstack-ansible12:49
*** epalper has quit IRC12:50
*** epalper has joined #openstack-ansible12:50
odyssey4meRossSchulman[m] ok, in OSA you allocate a service group to a host - by default the services for that group (mostly) will go into a container which is created on that host by OSA12:51
odyssey4meso no, you never need to create containers12:51
odyssey4meall you do is allocate the service group to the applicable hosts12:51
RossSchulman[m]OK, thanks.12:51
RossSchulman[m]I'm creating a temporary aio that I plan to expand as I get more servers.12:52
RossSchulman[m]So I wanted to understand how aio worked without using it. 😊12:52
openstackgerritMerged openstack/openstack-ansible stable/queens: Ensure package cache is updated appropriately  https://review.openstack.org/55972512:53
odyssey4meRossSchulman[m] I suggest that you have some way of rebuilding that host quite quickly in your early stages, because it takes a while to get the networking right.12:54
RossSchulman[m]Aside from just wiping it and reinstalling Ubuntu?12:54
odyssey4mejrosser that is quite odd - I wonder why we're not seeing issues in gating12:55
odyssey4mejrosser I've been doing upgrades from p2q, and they're fine... I guess I could try out a q greenfield too to see if the same issue shows up12:55
*** logan- has quit IRC12:55
*** logan- has joined #openstack-ansible12:56
jrosserodyssey4me: i was wondering if there was any differece in the way the deploy host got set up12:56
jrossertaking out that check for python3 makes it find python2 and it then starts running12:56
*** epalper has quit IRC12:56
*** epalper has joined #openstack-ansible12:57
jrosserodyssey4me: it might be a factor that our deploy host is never one of the targets?12:57
*** radeks has quit IRC12:58
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron stable/queens: Bug fix: Parenthesis needed when using ternary  https://review.openstack.org/56001612:58
*** radeks has joined #openstack-ansible12:58
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron stable/pike: Bug fix: Parenthesis needed when using ternary  https://review.openstack.org/56001712:59
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron stable/queens: Bug fix: fwaas_v2 was missing in the pip task  https://review.openstack.org/56002013:01
evrardjpI'd rather move wholesale to py313:01
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron stable/pike: Bug fix: fwaas_v2 was missing in the pip task  https://review.openstack.org/56002113:01
evrardjpbut it's not done yet13:02
evrardjpansible modules, openstack things...13:02
mbuilodyssey4me: done :)13:02
*** haad1 has joined #openstack-ansible13:02
odyssey4meawesome, thanks mbuil13:03
mnaserhmm13:03
mnaseris it possible to use br-mgmt for vxlan traffic?13:03
jrosserevrardjp: i'm a bit confused about you asking me which python i use by default - because bootstrap-ansible appears to have its own ideas about that and my (maybe wrong!) expectation is that it chooses the version it needs13:04
*** udesale has joined #openstack-ansible13:06
*** armaan has quit IRC13:13
*** udesale has quit IRC13:13
evrardjpmnaser: why not? A terrible idea13:13
evrardjpbut it should be possible13:13
*** udesale has joined #openstack-ansible13:13
mnaserwhy would it be terrible if you had 40gbps + qos on vms? unless i'm missing something13:14
mnaserbut thats a whole another discussion :)13:14
odyssey4memnaser you can still have all bridges go into a single interface if you want13:14
mnaseroh i see13:15
mnaseri'm just trying to follow the path that simplifies my config file and relies on the auto-configuration stuff13:15
odyssey4meOSA only cares about the bridge existing, not what's under it as far as I know13:15
odyssey4methe test environment example is a single interface, using vlans for segregation of traffic: https://docs.openstack.org/openstack-ansible/latest/user/test/example.html13:17
odyssey4meof course you don't *have* to use vlans, just like you don't *have* to segregate the traffic... but you probably should :p13:17
openstackgerritMerged openstack/openstack-ansible master: Add nova/ironic/glance cross-service vars to group_vars/all  https://review.openstack.org/55980413:18
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add nova/ironic/glance cross-service vars to group_vars/all  https://review.openstack.org/56002413:21
*** esberglu has joined #openstack-ansible13:26
openstackgerritMatt Thompson proposed openstack/openstack-ansible stable/pike: Remove tempest_git_* overrides  https://review.openstack.org/56002513:28
openstackgerritMatt Thompson proposed openstack/openstack-ansible stable/pike: Remove tempest_git_* overrides  https://review.openstack.org/56002513:29
*** shardy has quit IRC13:29
*** shardy has joined #openstack-ansible13:31
d34dh0r53mornings13:32
*** qwebirc23189 has joined #openstack-ansible13:34
mnasero/13:36
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron master: DNM: Testing ODL gates  https://review.openstack.org/55906513:37
*** throwsb1 has joined #openstack-ansible13:40
mgrew_odyssey4me  i do not see any user named as 'glance' on glance-container, is this correct ?13:41
*** cristicalin has joined #openstack-ansible13:42
*** lbragstad has joined #openstack-ansible13:42
*** vnogin has quit IRC13:44
*** kstev has joined #openstack-ansible13:44
qwebirc23189hi i have an issue with installing galera server when it connects to repo server for requirements_absolute_requirements.txt its an emtpy file . so it can't findout which version of my sql . could anyone help me please? is there any file in net that i can copy to repo server13:45
*** jwitko has joined #openstack-ansible13:45
*** cristicalin has quit IRC13:47
*** armaan has joined #openstack-ansible13:47
*** cristicalin has joined #openstack-ansible13:47
*** cshen has joined #openstack-ansible13:48
*** jwitko has quit IRC13:48
*** vnogin has joined #openstack-ansible13:48
mnaserinteresting13:49
*** jwitko has joined #openstack-ansible13:49
mnaserso i guess openstack-ansible doesn't install openvswitch?13:49
mnaseroh it does13:49
mnaserbut i guess selinux is not letting it access it :\13:50
odyssey4memnaser by default, no, because linuxbridge is the default - but there are options to use openvswitch13:50
*** serverascode has quit IRC13:50
mnaseryeah it actually installed it but yeah, selinux13:50
odyssey4memgrew_ it should exist, yes: https://github.com/openstack/openstack-ansible-os_glance/blob/master/tasks/glance_install.yml#L16-L3213:50
mnaser(yet another thing that would be solved by using binary packages :P)13:50
*** serverascode has joined #openstack-ansible13:51
odyssey4memgrew you may also want to assign a gid/uid - see https://github.com/openstack/openstack-ansible-os_glance/blob/master/defaults/main.yml#L51-L6813:51
*** Guest57868 has quit IRC13:53
*** Guest57868 has joined #openstack-ansible13:54
evrardjpmnaser: the troll is strong with this one.13:54
evrardjp:P13:54
mnaseroh13:54
mnaser:13:54
*** fdegir has quit IRC13:54
odyssey4memnaser oh, but binary packages are used to install openvswitch ;)13:54
evrardjpI thought people using packages were using setenforce 013:54
*** fdegir has joined #openstack-ansible13:54
evrardjphaha13:54
mnaserevrardjp: nope, openstack-selinux exists and is very well tested13:55
*** Guest57868 has quit IRC13:55
evrardjpmnaser: we removed it13:55
evrardjpbecause it was slow13:55
mnaseroh well yes it takes a little while to install13:55
evrardjpinstalling it was taking AGES13:55
mnasercorrection13:55
mnaseranything selinux takes AGES13:55
mnaserceph-selinux is just as bad too13:55
evrardjp:D13:55
*** Guest57868 has joined #openstack-ansible13:56
evrardjpmnaser: why centos, ovs, and selinux? You love your pain?13:56
evrardjp:D13:56
mnasercentos because i'd like to eventually roll out rdo and i like the packaging on centos way more13:56
mnaserovs because vxlan13:56
*** Guest57868 has quit IRC13:56
evrardjpjoke aside it's great you're working on it :D13:56
mnaserselinux because well, it worked just fine in packages :D13:56
mnasernah i get it, it's not the easiest route13:56
evrardjplxb supports vxlan13:56
mnaserwait really13:56
mnaserhow13:56
*** Guest57868 has joined #openstack-ansible13:57
evrardjpthat's in linux kernel for ages13:57
evrardjplitterally13:57
mnaseri thought linuxbridge was just the super super basic like13:57
mnaserbasic linux bridge and nothing more13:57
evrardjp3.1213:57
evrardjpkernel 3.1213:57
evrardjpomg13:57
evrardjpthat's older than what I thought!13:57
mnaseroh well13:57
mnasercentos runs 3.10 :p13:58
evrardjpI mean for unicast and multicast support13:58
evrardjpmaybe it's even older13:58
evrardjpmnaser: could you live in the present a little more ? :D13:58
mnaser:(13:59
mnaseri wanna experiment with this now13:59
mnaserive always ran ovs13:59
evrardjpahah check a little, I think it's in 3.10 that it included unicase :)13:59
evrardjpunicast14:00
odyssey4memnaser the linuxbridge in centos also supports vxlan, otherwise none of our tests would work14:00
mnaserso seems to me like ovs is only useful if you're going to do something like odl14:00
evrardjpand it's included in the kernel, so it must be prutty good14:00
evrardjpmnaser: I'd say so14:01
evrardjpor DVR if you're using a very old kernel14:01
evrardjpbut you have l3-ha anyway14:01
ivvei keep getting unreachable errors (ssh) on containers during deploy of stable/pike in ubuntu. its always during "Perform a cell_v2 discover" different containers in every deploy... anything you guys know of?14:01
jrosserovs is useful if you want bare metal performance without the downsides of SR-IOV, when paired with the right choice of NIC14:01
ivveand they are reachable... :P14:01
evrardjpjrosser: I found that linux bridge was quite good below 20Gbps14:02
*** ianychoi has quit IRC14:02
ivvethis time it was a nova-conductor, last time it was nova-api-placement container14:02
odyssey4meivve most recently that was changed up and appears to be more reliable in the new mechanism - what release are you using?14:02
ivvestable/pike14:02
ivveubuntu 16.0414:02
ivvelet me check git14:03
*** ianychoi has joined #openstack-ansible14:03
qwebirc23189hi i have an issue in checking wsrep is ready in galera server installation im usng 16.0.11 . could anyone help me to solve the issue14:03
odyssey4meivve do you have this commit in your tree: https://github.com/openstack/openstack-ansible/commit/5ca262641f5a8182d0f87c413f919c47b43b27ac14:04
ivve6f6e75f5496910724209929e32730f592737358514:04
ivvethis commit14:04
ivveone sec14:04
odyssey4meivve and did you bootstrap-ansible after updating your git tree to ensure the roles are updated14:04
evrardjpqwebirc23189: what's your mysql version?14:04
ivvealways bootstrap-ansible.sh14:04
ivvehowever it was running master before14:04
ivvebut i bootstrapped a stable/pike after removing dirs.. perhaps stuff is still around14:05
qwebirc23189evrardjp it's mysql  Ver 15.1 Distrib 10.1.30-MariaDB,14:06
evrardjpshould be fine14:06
ivvei do many deploys all the time, its a bit of a hassle to have a new deployment host each time, perhaps i should use containers14:06
odyssey4meivve I think you might have given the wrong SHA: https://github.com/openstack/openstack-ansible/commit/6f6e75f5496910724209929e32730f592737358514:06
evrardjpI don't know what's 15.1 there14:06
evrardjpbut the rest should be fine14:06
ivve5ca262641f5a8182d0f87c413f919c47b43b27ac wasn't in there14:07
evrardjpivve: I have a spec for that!14:07
evrardjpunpriviledged and self contained deploys14:07
odyssey4meivve the sha you gave was the first ever commit into the repo ;)14:07
evrardjpany help is welcome :)14:07
ivveoh crap14:08
evrardjp?14:08
qwebirc23189evrardjp so what shoud i do ?14:08
evrardjpivve: that will be fun to hear , go ahead :D14:08
evrardjpqwebirc23189: is that prod or dev?14:08
qwebirc23189prod14:08
*** ildikov has quit IRC14:08
evrardjpqwebirc23189: dev -- reinstall. Prod --read the docs : https://docs.openstack.org/openstack-ansible/latest/admin/index.html14:09
ivveevrardjp: just for the deployhost/vm not targets :P14:09
*** ildikov has joined #openstack-ansible14:09
evrardjpivve: my point precisely14:09
evrardjpyour prod users don't have the same credentials as your dev user14:10
evrardjpand if you contain into the dev user home directory all the plays/user space config that should do the trick.14:10
ArchiFleKsodyssey4me: I think I resolvedd my issue about the patch rearrange cell_v2, I think OSA often assume that the deployer is located on the same host as a controller, but this is not my case, so /etc/hosts in the deployer is not populated, when I copied over the content of /etc/hosts from a controller node it worked and the os_nova run successfully14:10
*** neith has quit IRC14:11
odyssey4meArchiFleKs aha, so that's a bug then which we should fix - could you register a bug for it?14:11
*** neith has joined #openstack-ansible14:11
*** chhavi__ has joined #openstack-ansible14:12
ArchiFleKsI'll remove the content of /etc/hosts and redeploy to be sure I can reproduce it14:12
evrardjpif anyone wants to use it, my "OpenStack Ansible" public calendar has been updated with the next work for this cycle14:13
evrardjpArchiFleKs: good catch14:14
ArchiFleKsyes it failed when I clear deployer hosts file14:14
ivveodyssey4me: 53663ba31bc186d2cba77c9bdf183e0820d8c65b was the latest i had14:14
*** chhagarw has quit IRC14:15
odyssey4meivve yeah, that's up to date14:15
*** mnaser has quit IRC14:15
ivveaye, this one is giving me trouble14:16
ivvefatal: [infra1]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname infra1_nova_conductor_container-5828429a: Name or service not known\r\n", "unreachable": true}14:16
*** mnaser has joined #openstack-ansible14:16
ivveduring os_nova : Perform a cell_v2 discover14:16
ivveonly thing i keep reusing is the deployment vm, rest is fresh from maas14:17
odyssey4meivve ok, so that task is delegated to a random conductor host - see https://github.com/openstack/openstack-ansible-os_nova/blob/stable/pike/tasks/main.yml14:17
ivvebut i did the bootstrap14:17
*** AndyWojo has quit IRC14:17
*** AndyWojo has joined #openstack-ansible14:17
odyssey4meand it uses some delegated facts: https://github.com/openstack/openstack-ansible-os_nova/blob/stable/pike/tasks/nova_db_post_setup.yml14:17
ivveaye strange14:17
ivveall of them answer on ssh14:18
*** markvoelker_ has joined #openstack-ansible14:18
odyssey4meso, if you're using limits - try including the conductor group in that limit14:18
*** rodolof has quit IRC14:18
odyssey4meyou might need to refresh the cached facts14:18
ivveno limits, this happend during setup-openstack.yml right after setup-infrastructure.yml (not using setup-everything)14:19
*** portdirect has quit IRC14:19
*** portdirect has joined #openstack-ansible14:19
*** rodolof has joined #openstack-ansible14:19
ivvearen't those stored in inventory.json?14:20
ivveopenstack_inventory.json*14:20
*** markvoelker has quit IRC14:21
odyssey4meivve https://docs.openstack.org/openstack-ansible/latest/admin/troubleshooting.html#cached-ansible-facts-issues14:21
*** zul has quit IRC14:22
*** zul has joined #openstack-ansible14:22
ArchiFleKshttps://bugs.launchpad.net/openstack-ansible/+bug/176274214:22
openstackLaunchpad bug 1762742 in openstack-ansible "Some delegated tasks are failing when deployer is node part of a deployment (eg. controller node)" [Undecided,New]14:22
ivveodyssey4me: gotcha!14:22
*** redondo-mk has quit IRC14:24
*** redondo-mk has joined #openstack-ansible14:24
*** alanmeadows has quit IRC14:26
*** alanmeadows has joined #openstack-ansible14:27
*** DanyC has quit IRC14:29
ivveodyssey4me: shouldn't that be a non-problem if my procedure is: removal of /etc/openstack_deploy & /opt/openstack-ansible/, clone <insert branch> to /opt/openstack-ansible/, copy /opt/openstack-ansible/etc/openstack_deploy to /etc/, run bootstrap-ansible.sh and then run setup-hosts/infra/openstack.yml files in seq?14:34
jrosserhow should i specify my local percona mirror now in light of this? https://review.openstack.org/#/c/535253/14:34
ivvei mean i shouldn't need to remove the facts after each playbook?14:35
*** jamespage has quit IRC14:36
*** jamespage has joined #openstack-ansible14:36
*** markvoelker has joined #openstack-ansible14:36
*** markvoelker_ has quit IRC14:39
ivvealright im guessing new interface might interfer14:41
ivveand that happens in previous plays14:41
*** BjoernT has joined #openstack-ansible14:42
*** markvoelker_ has joined #openstack-ansible14:42
*** markvoelker has quit IRC14:44
*** markvoelker has joined #openstack-ansible14:46
*** markvoelker_ has quit IRC14:49
*** markvoelker_ has joined #openstack-ansible14:49
*** markvoelker has quit IRC14:53
qwebirc23189evrardjp i still can't find how to solve the issue is there any alternatie way ? i'm trying to deploy my first openstack ansible !14:53
evrardjpqwebirc23189: if it's deploying your first OSA it's not prod :p14:56
*** markvoelker has joined #openstack-ansible14:56
evrardjpdelete your galera containers, rebuild them14:56
*** coreywright has joined #openstack-ansible14:57
evrardjpqwebirc23189: on which platform are you?14:57
evrardjpubuntu?14:57
qwebirc23189it's my first.but i should install in a production enviremont ! yes i'm using ubuntu.14:57
*** cjloader has joined #openstack-ansible14:57
evrardjpqwebirc23189: OSA is meant for prod14:58
*** markvoelker_ has quit IRC14:58
evrardjpqwebirc23189: what I meant was do you currently have users using it or now14:58
evrardjpwhich is not the case14:58
openstackgerritAndy Smith proposed openstack/openstack-ansible master: (WIP) Support oslo.messaging services for separate RPC and Notification  https://review.openstack.org/55861214:58
mgrew_odyssey4me  i have glance:glance created on glance container still image creation failing for permission error. any suggestion how to recover from this?14:59
qwebirc23189oh :)) im really sorry i miss understood14:59
evrardjpqwebirc23189: Cleanup your containers, and rebuild. Gather logs, and send that to us14:59
evrardjpwithout logs it's harder to know14:59
evrardjpas we don't have the issue in our environments14:59
evrardjpso I suspect it's a configuration issue14:59
qwebirc23189evrardjp ok !thanks. for deleting container should i just delete its file from openstack directory ?15:01
*** lvdombrkr has quit IRC15:02
spotzqwebirc23189: One of the docs has instructions let me see if I can find it real quick15:02
coreywrightivve, are you here discussing lp 1762742?15:03
openstackLaunchpad bug 1762742 in openstack-ansible "Some delegated tasks are failing when deployer is node part of a deployment (eg. controller node)" [Undecided,New] https://launchpad.net/bugs/176274215:03
spotzqwebirc23189: https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks/containers.html15:03
*** sar has quit IRC15:03
*** markvoelker_ has joined #openstack-ansible15:05
qwebirc23189spotz tnx a lot15:06
spotzqwebirc23189: No prob:)15:06
*** cshen has quit IRC15:06
*** markvoelker_ has quit IRC15:08
*** markvoelker has quit IRC15:09
*** markvoelker has joined #openstack-ansible15:10
*** markvoelker_ has joined #openstack-ansible15:10
evrardjpd34dh0r53: the issue on the sha bump isn't transient fyi15:14
d34dh0r53evrardjp: yeah, trying to recreate it now to see what's going on15:15
*** markvoelker has quit IRC15:15
d34dh0r53evrardjp: it built fine on my mnaio though so not sure what's going on15:15
evrardjpodyssey4me: did you see an issue with disk space recently?15:15
evrardjpDid we move back to overlayfs?15:15
qwebirc23189evrardjp i recreate the container the main issue is that the file requirements_absoloute_requirements.txt is empty. so in galera server installation it gives error. is there any requirement_absoloute_requirement.txt which can i replace with current one ?15:16
coreywrightivve, ArchiFleKs, odyssey4me: so the problem of ansible falling back to dns when randomly choosing a delegated host and the delegated host not being found (ie not in dns, not in /etc/hosts on non-deployed hosts) is because ansible lazily evaluates the delegated host twice: 1. when getting the delegated vars (ie ansible 2.3.3.0, ansible/vars/__init__.py, _get_delegated_vars()) 2. when setting the delegated host15:16
coreywright(ie ansible 2.3.3.0, ansible/playbook/play_context.py, set_task_and_variable_override()) and if the second time the chosen delegated host isn't the same as the first time, ie the delegated vars for the host are absent, then ansible falls back to dns and it fails because /etc/hosts on non-deployed hosts doesn't contain inventoried hosts15:16
qwebirc23189evrardjp the exact error is : fatal: [infra1_galera_container-455f45a2]: FAILED! => {"attempts": 5, "changed": false, "cmd": "/usr/local/bin/pip2 install -U --constraint http://172.29.236.11:8181/os-releases/16.0.11/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt  MySQL-python pycrypto", "failed": true, "msg": "stdout: Collecting MySQL-python\n\n:stderr:   Could not find a version that satisfies the require15:16
evrardjpqwebirc23189: that's from a clean machine?15:18
qwebirc23189yes!15:18
coreywrightrun ansible with ANSIBLE_DEBUG=1 and for that job (ie "os_nova : Perform a cell_v2 discover") when it fails you should see "no remote address found for delegated host %s\nusing its name, so success depends on DNS resolution"15:18
evrardjpqwebirc23189: could you reach that url from your infra1_galera_container-... ?15:19
evrardjpcoreywright: where is the "randomly choosing a delegated host" part in our code?15:20
*** cjloader has quit IRC15:20
*** rodolof has quit IRC15:20
coreywrightthe funny thing (and what drove me to dig into this late last week) was that it happens randomly (with 3 infra nodes, it failed 66% of the time), which made sense when i finally figured out the problem: if you randomly choose between three infra nodes two different times (1. populating delegate vars and 2. selecting the delegate) then there's only a 33% chance of the same node being selected twice15:20
*** vnogin has quit IRC15:20
*** cjloader has joined #openstack-ansible15:20
coreywrightevrardjp, random_conductor: "{{ groups[nova_services['nova-conductor']['group']] | random }}"15:21
evrardjpDescription :15:21
evrardjpDear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae,odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT,claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes,mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory,alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum,darrenc, JRobinson__, asettle,15:21
evrardjpcolinmcnamara, thorst, adreznec, eil397,qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok15:21
evrardjpwe have a meeting in 45 minutes, and I forgot to send the notification early15:21
qwebirc23189evrardjp yes! its an empty .txt15:21
d34dh0r53thanks evrardjp15:21
evrardjpqwebirc23189: that's not normal15:21
evrardjpqwebirc23189: rebuild the repo15:21
evrardjpthat's your problem15:21
evrardjpqwebirc23189: did you ignore a failure ?15:22
evrardjpif you re-deploy from scratch, I'd like to see the full log, please15:22
evrardjpmaybe there is something hidden15:22
evrardjpbut right now I can tell you it works15:22
evrardjphttp://zuul.openstack.org/builds.html?pipeline=periodic&project=openstack%2Fopenstack-ansible&branch=stable%2Fqueens15:23
evrardjpcoreywright: worth setting a fact at the beginning and using it?15:24
coreywrightevrardjp, that's what i was thinking was the easiest solution (compared to populating /etc/hosts on localhost like the deployment nodes) but i hadn't had a chance to test it15:25
evrardjpcoreywright: that would be lovely!15:25
evrardjpthanks for the analysis15:25
coreywrightevrardjp, (that is if facts have templates fully evaluated before storing as the fact; that was to be part of my testing)15:26
odyssey4meevrardjp I have not, and no we haven't changed anything15:26
evrardjpcoreywright: could you file a bug with the issue and your analysis? It would help when fixing things to be able to know the context15:26
coreywrightevrardjp, i'm following up on the recently filed 176274215:26
coreywright(as we speak/type)15:26
evrardjpcoreywright: the will be evaluated at run time, but the fact set will stay and won't change15:26
*** mgrew_ has quit IRC15:26
qwebirc23189evrardjp yes! i had two failures in repo build  installation one from installing wheel requirement second: waiting for venv build to complete !15:27
evrardjpcoreywright: great15:27
qwebirc23189evrardjp yes! i recreate repo container i will show you the logs .15:27
evrardjpqwebirc23189: a lot of ! in your messages :)15:27
qwebirc23189evrardjp tnx a lot15:27
evrardjpqwebirc23189: could it be possible you have a network connectivity issue?15:27
evrardjpthat would prevent installing wheel/building venv slowly?15:28
qwebirc23189evrardjp yes you are right :))) i use too much !!! .15:28
*** rodolof has joined #openstack-ansible15:28
evrardjpqwebirc23189: If I were you, I'd start from a clean machine where you're sure you have no connectivity issues15:28
evrardjpbecause all you are saying looks like transient network issues we cannot deal with -- it's not our fault :(15:29
evrardjpcoreywright: thank you for that work!15:29
coreywrightevrardjp, yw!15:29
qwebirc23189evrardjp i had network issue. but i resolved that and reinstall ubuntu on all machines and start from scratch today !!!15:29
evrardjpqwebirc23189: looks like it's not 100% solved :p15:30
evrardjpcontainers are behind a nat by default for outside network connectivity15:30
evrardjpthat might be the cause15:30
evrardjpqwebirc23189: but please start clean and post a full log -- that would help.15:31
evrardjpThat's how we are used to fix things :)15:31
evrardjpif anything need fixing15:31
qwebirc23189evrardjp i can ping evry host and also google.com from containers.15:32
qwebirc23189i don't kbow how to post full log :((( i'm too new.15:32
evrardjpqwebirc23189: try curling/wgetting a larger file from your container15:33
tomtom001cloudnull, thanks for the help yesterday.  sadly the chroot build is still die'ing during the apt-get -install -y process for no apparent reason.  I removed the machine and deleted all ansible facts, still same result.  I upgraded the underlying host with apt-get dist-upgrade as well and still the chroot dies.   I'm going to google around to see if I can figure out why this happens.15:33
qwebirc23189evrardjp it's the exact issue in wheel requirement :  Cloning file:///var/www/repo/openstackgit/cinder (to 181f90f4cf0bfce2bf4652ccb23222c96b6118b7) to /tmp/openstack-builder/cinder", "warning: You appear to have cloned an empty repository."]}15:34
evrardjpqwebirc23189: that's indeed not normal15:34
*** vnogin has joined #openstack-ansible15:35
*** armaan has quit IRC15:38
*** armaan has joined #openstack-ansible15:38
jrossererror during queens os-nova-install.yml http://paste.openstack.org/raw/718846/15:39
jrosserthat looks like an error from nova-manage itself?15:39
*** vnogin has quit IRC15:39
*** rodolof has quit IRC15:40
qwebirc23189evrardjp it's the exact log :http://paste.openstack.org/show/718847/15:40
evrardjpjrosser: is that master?15:41
evrardjpjrosser: I'd say the "proxy: host" shouldn't be there :p15:41
*** udesale has quit IRC15:41
*** udesale has joined #openstack-ansible15:42
evrardjpgit clone issue qwebirc2318915:42
evrardjpqwebirc23189: I told you it's not us :p the initial clone went wrong15:42
jrosserevrardjp: i wonder what that is from?!15:43
qwebirc23189evrardjp no i don't say you are the issue. i don't know what to do at all. here i just ask question to solve issues! the initial clone ?15:43
evrardjpjrosser: configuration?15:45
evrardjpjrosser: are you using master or queens?15:45
jrosserevrardjp: stable/queens15:45
jrosserin fact there is ": host" all over that paste for all the services15:46
evrardjpqwebirc23189: I suggest you to do the following. Re-install your node. Make sure the networking goes fine15:46
evrardjpqwebirc23189: then, follow our deploy guide15:46
evrardjpqwebirc23189: if there is an issue in setup-hosts, tell us directly and paste the log you can find in /openstack/ in a pastebin15:47
*** johnduarte has joined #openstack-ansible15:47
evrardjpqwebirc23189: don't Control-C things if you see they take some time15:47
evrardjpthat's normal15:47
evrardjpthere are actions that take time15:47
tomtom001if I cp the repo.keys file to /var/lib/machines/....../root/  and then run chroot /var/lib/machines/ubuntu-xenial-amd64 /usr/local/bin/cache-prep-commands.sh > /var/log/lxc-cache-prep-commands.log from the command line on the physical host the process completes without an issue.15:47
evrardjpis you're on centos it's even more normal it's slow15:47
evrardjpjrosser: mmm15:48
evrardjpjrosser: could you compare with one of our gate logs, to see if there is something different?15:48
qwebirc23189it's my forth time i re install everything! it's about two weeks i'm trying to deploy. i userd the simple test environment configs. exact same ips. and there is no issue in host-setup15:48
*** cristicalin has quit IRC15:52
jrosserevrardjp: i think the error message has poor formatting, looks like the service isnt present at all in the container15:52
*** vakuznet has quit IRC15:55
evrardjpqwebirc23189: you told me you had a network issue, and it still looks like a network issue -- I can't help you. Screaming doesn't help either :)15:56
odyssey4metomtom001 that's odd, because the role does that copy as part of the process15:56
*** haad1 has quit IRC15:57
odyssey4meqwebirc23189 it does rather sounds like you have some bad connectivity interfering - you can re-run any part of the repo build process using the flags repo_build_git_reclone/repo_build_wheel_rebuild/repo_build_venv_rebuild, for example: openstack-ansible repo-build.yml -e repo_build_git_reclone=yes15:58
*** vnogin has joined #openstack-ansible15:59
evrardjpok so16:00
evrardjplet's stop all these conversations16:00
evrardjpDear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae,odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT,claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes,mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory,alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum,darrenc, JRobinson__, asettle,16:00
evrardjpcolinmcnamara, thorst, adreznec, eil397,qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok16:00
evrardjpHere is our agenda for today:https://wiki.openstack.org/wiki/Meetings/openstack-ansible16:00
evrardjp#startmeeting openstack_ansible_meeting16:01
openstackMeeting started Tue Apr 10 16:01:03 2018 UTC and is due to finish in 60 minutes.  The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot.16:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:01
*** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)"16:01
openstackThe meeting name has been set to 'openstack_ansible_meeting'16:01
hwoarango/16:01
evrardjp#topic "What happened since last meeting"16:01
*** openstack changes topic to ""What happened since last meeting" (Meeting topic: openstack_ansible_meeting)"16:01
evrardjpevrardjp reminds we have specs that need some eyes.16:01
evrardjpevrardjp says the newton EOL is in process, and only the integrated/tests repo are to EOL now.16:01
spotzo/16:01
odyssey4meo/16:01
evrardjpevrardjp reminded the etherpad for vancouver forum: https://etherpad.openstack.org/p/YVR-openstack-ansible-brainstorming16:01
d34dh0r53o/16:01
evrardjpevrardjp said the next release will be queried according to plans, the 13 April.16:02
jmccroryo/16:02
evrardjpodyssey4me suggested to use novnc instead of spice by default, or to have more than one method at the same time.16:02
evrardjpodyssey4me suggested changes to the tempest role: http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/latest.log.html#t2018-04-10T12:19:33 (see also the conversation above this link for more context)16:02
prometheanfireo/16:02
evrardjpodyssey4me and hwoarang discussed the distribution package vs source install (http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/latest.log.html#t2018-04-10T09:35:54 )16:02
evrardjpTwo people asked for multi-region OSA deploys this week, and evrardjp proposed to add some user stories about it (http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/latest.log.html#t2018-04-10T09:13:45) using logan- 's example (http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/latest.log.html#t2018-04-10T11:44:11)16:02
evrardjpI let you a few minutes to process this, then let's move to bug triage16:02
evrardjpit also allows ppl to join :)16:02
evrardjpIf you think I forgot to highlight something, please put it on the wiki16:03
*** vnogin has quit IRC16:04
evrardjpok16:04
evrardjpI think it's time to continue16:04
evrardjp#topic Bug triage16:04
*** openstack changes topic to "Bug triage (Meeting topic: openstack_ansible_meeting)"16:04
openstackgerritMerged openstack/openstack-ansible stable/queens: repo-use.yml: Scope host groups properly  https://review.openstack.org/55972616:04
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176248316:05
openstackLaunchpad bug 1762483 in openstack-ansible "openstack-ansible rabbitmq-install.yml -e 'rabbitmq_upgrade=true' does not upgrade rabbitmq" [Undecided,New]16:05
hwoarangmakes sense16:06
evrardjpthat looks legit16:06
evrardjpsince we moved suse/centos to not use latest16:06
evrardjpthat's bad :p16:06
evrardjpThat's also why I said we should do it on case by case in each role16:07
evrardjpbut that thing got merged anyway16:07
hwoarangit's ok you only want that on upgrades16:07
evrardjpwe can change the upgrade docs/code16:07
evrardjplet's mark it as confirmed and medium?16:07
odyssey4meyeah16:07
odyssey4meI always thought that it made better sense to use present and to have packages upgraded through a different playbook which you can target as you wish.16:08
evrardjpok16:09
evrardjplet's do it this way then16:09
odyssey4meThen we can use it for minor/major upgrades too.16:09
evrardjpI'll mark it as high instead16:10
odyssey4mealthough in that role I suppose just tying the package state to whether the upgrade var is set or not will work as a workaround16:10
odyssey4meand will be back portable16:10
evrardjpodyssey4me: we can also use a flip switch to load some variables only when upgrades16:10
evrardjpI'd rather do it in the role, because the role knows16:11
evrardjpand we set a convention16:11
odyssey4meyeah16:11
evrardjplike "in_upgrade: False" by default in the role16:11
odyssey4meanyway, I'd say high because it impacts upgrades quite seriously16:11
evrardjpand if someone does is_upgrade: True ->boom!16:11
evrardjpodyssey4me: agreed!16:11
evrardjpok done deal16:11
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176245916:12
openstackLaunchpad bug 1762459 in openstack-ansible "inventory-manage.py broken on 17.0.x" [Undecided,New]16:12
evrardjpI guess that one is solved?16:12
odyssey4meyeah, should be16:12
evrardjpodyssey4me: it merged back in Q hasn't it?16:12
evrardjpok will mark it as fix commited16:12
odyssey4meone more review for the pike backport: https://review.openstack.org/#/q/I6055df3da794439fb65ad2244445135dafb1cecf16:12
evrardjpnext16:13
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176242116:13
openstackLaunchpad bug 1762421 in openstack-ansible "openstack-ansible-galera_server role needlessly removes and reinstalls MariaDB-server" [Undecided,New]16:13
evrardjpodyssey4me: thanks! :)16:13
odyssey4meyeah, apparently this one's back - someone's going to have to confirm and figure out a fix16:14
odyssey4mewasn't mnaser looking at it during the PTG?16:14
evrardjpomg we should test this16:14
evrardjpI have started to work on idempotency testing16:14
evrardjptaking the patch from mhayden16:14
evrardjpbut we should do it for galera_server16:14
evrardjpby running galera_client in the middle16:14
evrardjp:)16:14
hwoarangyeah the prob is that the MariaDB-server from mariadb repos defines 'mariadb-server' symbol16:15
hwoarangwhich collides with the one from the RDO repos16:15
hwoarangso you can't really tell if you are re moving the RDO package or the MariaDB package16:15
hwoarangideally we shouldn't be removing anything and let dnf or yum figure out what to do16:16
hwoarangthere must be a magic switch for that16:16
evrardjphwoarang: I agree there16:16
evrardjplet's not compile mariadb from sources.16:16
odyssey4meyeah, the package removal was only done because that was recommended by mariadb in their knowledgebase for the 5.5 -> 10 upgrade16:16
evrardjp:D16:16
evrardjpmaybe we can remove it now?16:16
hwoarangwe could try16:16
*** evin has joined #openstack-ansible16:16
odyssey4menow that we're beyond that, and their docs say to just install over the top of it, we should be able to just do that16:17
odyssey4meit'd make the install/upgrade process quite a bit simpler too16:17
*** cshen has joined #openstack-ansible16:17
evrardjpyeah I think we are good to do that16:18
evrardjpI will push up a patch tomorrow16:18
evrardjpconfirmed high?16:18
evrardjpnot critical16:18
odyssey4meyeah16:18
evrardjpbut it's very disruptive to uninstall it16:18
evrardjp:p16:19
evrardjpif someone else wants to fix it, very welcome, I already have so much on my plate.16:19
evrardjp:D16:19
odyssey4mewell, it'll only be doing it to one server at a time - which is fine in a cluster, but not when you only have one ;)16:19
*** vakuznet has joined #openstack-ansible16:19
evrardjpnext16:19
evrardjpodyssey4me: haha true16:19
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176205116:19
openstackLaunchpad bug 1762051 in openstack-ansible "no logrotation on in /var/log/gnocchi (in gnocchi container)" [Undecided,New]16:19
evrardjpthat looks legit16:19
evrardjpI never did it16:20
evrardjp:p16:20
odyssey4mewe should perhaps use ansible-role-logrotate for all our log rotation16:20
odyssey4mehttps://github.com/openstack/ansible-role-logrotate16:20
odyssey4mevery simple role16:20
evrardjpI thought it was here: https://github.com/openstack/openstack-ansible-rsyslog_client/blob/master/templates/logrotate.j216:21
evrardjpyeah maybe we should16:21
evrardjpconfirmed medium?16:22
odyssey4melooks like there should be log rotation done for gnocchi: https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-gnocchi-install.yml#L6116:22
odyssey4meit's in pike too: https://github.com/openstack/openstack-ansible/blob/stable/pike/playbooks/os-gnocchi-install.yml#L5816:22
odyssey4meyeah16:22
odyssey4mewell, actually, not confirmed - itlooks like there is, so we need proof of a bug16:22
odyssey4meneed more info16:22
evrardjpthat's interesting16:24
evrardjpmaybe it's not named .log ?16:25
odyssey4meit should be, if https://github.com/openstack/openstack-ansible-os_gnocchi/blob/master/templates/gnocchi-systemd-init.j2#L13 is doing what it should16:26
evrardjpok so no need to list it in rsyslog_client_log_files16:26
qwebirc23189odyssey4me thaaaaaaaanks a looooooooot everything worked properly with your help and there was no network issue thanks a lot. you are my hero.16:26
odyssey4meqwebirc23189 no worries, glad you're over that hurdle16:27
*** DanyC has joined #openstack-ansible16:27
evrardjpfor me you're right it's not confirmed16:27
openstackgerritAndy Smith proposed openstack/openstack-ansible-tests master: (WIP) Update to use oslo.messaging services for RPC and Notify  https://review.openstack.org/55912716:27
evrardjpasking for more details in the bug16:28
evrardjpmarking it as incomplete16:28
evrardjpnext16:28
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176178516:29
openstackLaunchpad bug 1761785 in openstack-ansible "Unbound listening on all interfaces" [Undecided,New]16:29
evrardjpConfirmed and low16:29
evrardjpok for everyone?16:29
odyssey4meyeah16:29
evrardjpnext16:30
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176172216:30
openstackLaunchpad bug 1761722 in openstack-ansible "inconsistency in documenting the use of eth12" [Undecided,New]16:30
odyssey4meyeah, that's very likely16:31
*** DanyC has quit IRC16:31
odyssey4meconfirmed, I'd say medium given that this stuff is a reference16:31
d34dh0r53yep16:32
ArchiFleKscoreywright: thanks for your analysis ;)16:32
spotzput notes in and I'll grab it16:32
jrosserthis is one where i don't feel able to write what it *should* say myself16:32
spotzjrosser: That's why I tell them to put notes in:)16:32
odyssey4meI couldn't clarify either :/16:32
spotznetworking is confusing.... love spotz16:33
jrosseranway historically lots of newcomers trip over this so it could do with a tidy16:33
coreywrightArchiFleKs, yw!  (and a quick work-around is to set `delegate_to: "{{ hostvars[random_conductor]['ansible_host'] }}"`16:33
evrardjpif you need wiring towards a container for your bridge ...16:34
ArchiFleKscoreywright: thanks, are planning on proposing a patch ? I'm not sure a totally understood the way you would fix it16:34
coreywrightArchiFleKs, instead of `delegate_to: "{{ random_conductor }}"`)16:34
evrardjpthat could be simplified16:35
evrardjpcoreywright: ArchiFleKs can you speak about that later please?16:35
evrardjpLet's mark it as confirmed and medium16:35
evrardjpand we should remove the documentation from this file, it's confusing everyone16:35
evrardjplet's put that into the documentation with examples16:35
evrardjpthat can be more explanatory16:35
evrardjpmore text than just comments16:36
evrardjplet's move on shall we?16:36
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176171816:36
openstackLaunchpad bug 1761718 in openstack-ansible "container networks docs not updated for bare metal l3 agent deploy" [Undecided,New]16:36
evrardjpconfirmed medium16:36
odyssey4meyeah16:37
evrardjpcloudnull: could you check on both bugs ?16:37
evrardjpkill 2 birds one stone?16:37
evrardjpjust explaining better what we did with the hyper converged?16:37
evrardjpI'm assigning it to you anyway :p16:38
evrardjpok16:38
evrardjplet's move on16:38
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176155016:38
openstackLaunchpad bug 1761550 in openstack-ansible "'Reload systemd deamon' handle not being called in neutron" [Undecided,New]16:38
evrardjpthat is bad16:39
odyssey4meif it's newton, then there's nothing we can do about it16:39
evrardjpP and O can be fixed though16:40
evrardjpif it's a problem in P and O16:40
odyssey4meyeah, ocata onwards does the daemon reload when it stops/starts the service16:40
d34dh0r53I'm reading that it's a problem in >p, but that could be me16:41
odyssey4menewton does not16:41
evrardjplooks like this sha is N16:41
evrardjp84ed68a4a57713c8a66aada1028dbbf8ffa1eb0016:41
d34dh0r53ohh16:41
d34dh0r53you're right16:41
spotzAnyway to get permission to patch?16:41
odyssey4menah, for RPC we'll have to fork and fix in the fork16:41
evrardjpto patch what?16:41
spotznewton16:41
evrardjpno16:41
odyssey4menewton in the openstack namespace is gone16:42
evrardjpso16:42
evrardjpfor me that's not clear if that's above N16:42
odyssey4meI don't think it is.16:42
evrardjpok16:42
*** DanyC has joined #openstack-ansible16:43
evrardjpok won't fix then16:43
odyssey4methe notifies in https://github.com/openstack/openstack-ansible-os_neutron/blob/newton-eol/tasks/neutron_init_systemd.yml look fine though16:43
evrardjpnext16:43
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/176087816:43
openstackLaunchpad bug 1760878 in openstack-ansible "galera_server: Reveals galera passwords in task: "Create galera users"" [Undecided,New]16:43
evrardjpwow16:43
evrardjpthat's bad16:43
evrardjpcritical16:43
evrardjpI will do it tomorrow16:43
odyssey4mehmm, I though I saw a patch already for that16:44
evrardjpI will see16:44
evrardjpno worries I handle it16:44
odyssey4mehttps://review.openstack.org/#/q/Ia170e4f25e35984e68b4cd47c3d964a034236d1116:45
evrardjpnext16:45
d34dh0r53confirmed16:45
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/175989616:45
openstackLaunchpad bug 1759896 in openstack-ansible "Placing certificate on keystone containers does not work" [Undecided,New]16:45
evrardjpodyssey4me: not the same task16:45
evrardjpmysql_user is the one16:45
evrardjpbut yeah16:45
odyssey4meah ok16:45
evrardjpwill do it16:45
evrardjplet's move on16:45
evrardjp15 minutes16:45
evrardjp:)16:45
odyssey4meyeah, ssl on the back-end doesn't really work16:46
odyssey4meconfirmed, medium16:46
evrardjpok16:46
odyssey4meI do wish someone would work on that16:46
evrardjpanyone to look at it?16:46
odyssey4meI think it'll be easier once the fastrouter is in though16:46
evrardjpok next16:47
evrardjp#link #link https://bugs.launchpad.net/openstack-ansible/+bug/175814416:47
openstackLaunchpad bug 1758144 in openstack-ansible "resolv.conf in containers set too late" [Undecided,New] - Assigned to Logan V (loganv)16:47
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/175814416:47
evrardjpit's assigned to logan let's move on16:47
d34dh0r53should it be set to confirmed?16:47
evrardjpexcept if logan- is there?16:47
evrardjpd34dh0r53: logan- has to confirm16:47
evrardjpWe discussed this last week16:47
evrardjpnext16:47
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/175582116:47
openstackLaunchpad bug 1755821 in openstack-ansible "config_template fails to parse template if it contains a comment with leading spaces" [High,New] - Assigned to Kevin Carter (kevin-carter)16:48
d34dh0r53ahh, ok16:48
evrardjpthat's still in new state because we wanted to make sure cloudnull hasn't forgotten16:48
evrardjpI guess he has, so let's move on16:48
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/174303216:48
openstackLaunchpad bug 1743032 in openstack-ansible "Galera cluster maintenance in OpenStack-Ansible" [Undecided,New] - Assigned to Kevin Carter (kevin-carter)16:48
evrardjpsame16:48
evrardjpnext16:48
evrardjp#link https://bugs.launchpad.net/openstack-ansible/+bug/173782716:48
openstackLaunchpad bug 1737827 in openstack-ansible "(ceph-client): setting 'nova_ceph_client' results in deployment where volumes can't be attached to VMs" [Undecided,New] - Assigned to Andy McCrae (andrew-mccrae)16:48
*** cristicalin has joined #openstack-ansible16:48
*** udesale has quit IRC16:49
evrardjpandymccr: ?16:49
andymccrahh i was waiting on response from the bug filer16:49
evrardjpok16:50
*** ssmith has joined #openstack-ansible16:50
evrardjpmarking it incomplet16:50
andymccrok cool16:50
andymccri'll take another look at it though16:50
evrardjpunassigning16:50
evrardjpok you can if you want :)16:50
evrardjpthanks16:50
evrardjpand thanks everyone for the triage16:50
evrardjp#topic open discussion16:50
*** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)"16:50
evrardjpbecause we have extra time16:50
evrardjpopen discussion16:51
evrardjpI have a topic!16:51
evrardjp234 BUGS16:51
evrardjpwe've beaten our record.16:51
evrardjpnothing to be proud off.16:51
spotzthat sounds bad:(16:51
evrardjpit is16:51
hwoarang:/16:51
evrardjpI have nothing else to say. We have so many bugs, so many things pending. I am glad we are bringing new features and all, but at some point we need to look back and make sure our project is reliable and the experience is good. Not everyone is an expert that can quickly fix things.16:54
d34dh0r53agreed16:54
odyssey4meyep, agreed16:54
*** DanyC has quit IRC16:55
evrardjpok16:55
evrardjpthat's all I had to say16:55
*** DanyC has joined #openstack-ansible16:56
evrardjpanyone has anything to say ?16:56
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add container/service clean up for ironic/trove  https://review.openstack.org/55999516:56
odyssey4mepatches speak louder than words ;)16:56
evrardjpindeed16:57
evrardjpthanks everyone!16:57
evrardjpclosing up16:57
evrardjp#endmeeting16:57
*** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs"16:57
openstackMeeting ended Tue Apr 10 16:57:41 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:57
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-04-10-16.01.html16:57
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-04-10-16.01.txt16:57
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-04-10-16.01.log.html16:57
*** mbuil has quit IRC17:00
evrardjpI'm out ttyl everyone17:01
*** osnaya has joined #openstack-ansible17:03
*** uiet has joined #openstack-ansible17:06
uiethow to forcefully run all tasks on existing deployment?17:07
uietany suggestion?17:12
*** _david_ has quit IRC17:17
*** DanyC_ has joined #openstack-ansible17:18
*** cristicalin has quit IRC17:21
*** epalper has quit IRC17:21
*** DanyC has quit IRC17:22
*** nicolasbock has joined #openstack-ansible17:24
uiethow to forcefully run all tasks on existing deployment?17:24
Miougeuiet: what do you mean forcefully? openstack-ansible playbook/setup-everything.yml?17:31
uieti want to run all tasks on a re-run17:32
uieti see that on re-run many of the tasks are skipped17:33
*** openstackgerrit has quit IRC17:34
odyssey4meuiet tasks are only skipped if they're supposed to be17:52
odyssey4menot all tasks are supposed to run17:52
*** openstackgerrit has joined #openstack-ansible17:56
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_nova master: (WIP) Update to use oslo.messaging services for RPC and Notify  https://review.openstack.org/55862017:56
*** BjoernT has left #openstack-ansible17:56
uietif i want to run a task to reconfigure something ex. glance dir mount...can i do that17:57
*** tomtom001 has quit IRC18:05
*** tomtom001 has joined #openstack-ansible18:06
*** electrofelix has quit IRC18:09
*** poopcat has joined #openstack-ansible18:10
*** b1tsh1ft3r has joined #openstack-ansible18:10
*** osnaya has quit IRC18:12
*** ianychoi has quit IRC18:22
*** ianychoi has joined #openstack-ansible18:23
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-os_nova master: Update paste, policy and rootwrap configurations 2018-03-31  https://review.openstack.org/55807818:30
*** mma has quit IRC18:30
ivvecoreywright: sorry was away.. yes. to me it happend to conductor but also nova-api-placement18:32
evinHas anyone used haproxy_extra_services config successfully? It's supposed to allow specifying haproxy services that are not managed in the Ansible inventory by manually specifying their hostnames/IP Addresses. When I try it i get this error: "AnsibleUndefinedVariable: ansible.vars.hostvars.HostVars object has no element."18:39
*** ianychoi has quit IRC18:41
*** hamza21 has joined #openstack-ansible18:41
*** ianychoi has joined #openstack-ansible18:42
*** epalper has joined #openstack-ansible18:44
*** mma has joined #openstack-ansible18:46
*** mma has quit IRC18:48
*** cshen has quit IRC18:50
*** cshen has joined #openstack-ansible18:52
jrosserodyssey4me: do you have any advice on moving a config from P -> Q? i'm getting nova api containers created but a bunch of the conditional includes in the nova role are skipped - like these https://github.com/openstack/openstack-ansible-os_nova/blob/e3e7c1f00bd7155dc051ba093fc872751776f769/tasks/main.yml#L97-L9819:00
*** openstackgerrit has quit IRC19:04
*** bauruine has quit IRC19:08
odyssey4mejrosser being skipped on the nova_api container? (not the old ones)19:09
odyssey4methere're some docs and notes in https://review.openstack.org/#/q/topic:bp/hyperconverged-containers+(status:open+OR+status:merged)19:10
jrosserit's a fresh deploy, starting with the user_config we had for P19:11
jrosserwhich has pretty much worked ok up to nova19:11
odyssey4meoh, that's curious19:12
jrosserit's quite hard to follow the includes as they don't have a name:19:12
jrosserodyssey4me: is there a superquick fix for inventory_manage - i'd have compared the group contents with that...19:13
*** bauruine has joined #openstack-ansible19:15
*** cshen has quit IRC19:18
*** b1tsh1ft3r_ has joined #openstack-ansible19:19
odyssey4mejrosser you mean this fix which merged this morning? https://review.openstack.org/55992219:21
jrosserah yes thats it, thanks19:22
*** cshen has joined #openstack-ansible19:22
*** jafeha has joined #openstack-ansible19:26
b1tsh1ft3r_should i be able to just kill a horizon container and re-run the os-horizon-setup.yml ? Running into some major issues after re-running the horizon play after installing power dns and designate to get the designate dash plugin to work (part of the horizon install i believe)19:27
b1tsh1ft3r_Looks like im getting 400 errors every 2-3 attempts to hit the dashboard19:27
*** cshen has quit IRC19:27
*** jafeha__ has quit IRC19:28
*** jafeha__ has joined #openstack-ansible19:28
*** jafeha has quit IRC19:29
*** DanyC has joined #openstack-ansible19:29
*** b1tsh1ft3r has quit IRC19:29
*** epalper has quit IRC19:30
*** DanyC_ has quit IRC19:33
coreywrightivve, can you tell me where you've seen it fail for nova-api-placement containers (eg what task)?19:33
ivveexactly the same one19:34
ivvecell_v219:34
ivvei was deploying each playbook by it self, host, infra & openstack19:35
*** jafeha__ has quit IRC19:35
*** jafeha has joined #openstack-ansible19:35
ivveso cleaning out facts helped me out i think19:35
ivvenow i got some ceph-client error i've never seen before :)19:35
ivvereran with -vvvv19:35
ivvetbh i should wipe this deployment vm and install proper os ;)19:36
ivveits a centos7, should be ubuntu19:36
*** markvoelker_ has quit IRC19:41
*** markvoelker has joined #openstack-ansible19:41
*** markvoelker has quit IRC19:45
*** DanyC_ has joined #openstack-ansible19:50
*** DanyC has quit IRC19:53
*** markvoelker has joined #openstack-ansible19:54
b1tsh1ft3r_Alright... im a bit confused here. Found that HTTPD cant start on a new horizon container deployed from the os-horizon-install.yml play. Receiving the following error when attempting to deploy horizon to the container itself20:00
b1tsh1ft3r_http://paste.openstack.org/show/718858/20:00
*** pcaruana has quit IRC20:01
*** shardy has quit IRC20:02
*** cshen has joined #openstack-ansible20:07
*** openstackgerrit has joined #openstack-ansible20:08
openstackgerritAndy Smith proposed openstack/openstack-ansible-tests master: (WIP) Update to use oslo.messaging services for RPC and Notify  https://review.openstack.org/55912720:08
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_nova master: (WIP) Update to use oslo.messaging services for RPC and Notify  https://review.openstack.org/55862020:09
*** cshen has quit IRC20:12
openstackgerritMerged openstack/openstack-ansible-os_tempest stable/queens: Allow for single tempest image upload  https://review.openstack.org/56001020:13
*** DanyC_ has quit IRC20:14
*** markvoelker_ has joined #openstack-ansible20:17
*** markvoelker has quit IRC20:21
*** vnogin has joined #openstack-ansible20:21
*** hamza21 has quit IRC20:24
*** vnogin has quit IRC20:26
*** chhavi__ has quit IRC20:29
*** hamza21 has joined #openstack-ansible20:30
*** hw_wutianwei has quit IRC20:30
*** hw_wutianwei has joined #openstack-ansible20:32
*** evin has quit IRC20:32
*** b1tsh1ft3r_ has quit IRC20:41
*** ansmith has quit IRC20:46
*** johnduarte has quit IRC20:57
*** hamza21 has quit IRC21:13
*** cjloader has quit IRC21:19
*** haad1 has joined #openstack-ansible21:20
*** jafeha__ has joined #openstack-ansible21:30
*** jafeha has quit IRC21:31
*** admin0 has joined #openstack-ansible21:31
*** haad1 has quit IRC21:32
*** ansmith has joined #openstack-ansible21:34
admin0my setup-hosts fails on pip install because its looking for the repo RetryError: HTTPConnectionPool(host='int-dev.atl.domain.cloud', port=8181): Max retries exceeded with url: /os-releases/17.0.2/ubuntu-16.04-x86_64/ (Caused by ResponseError('too many 503 error responses .21:36
*** jwitko_ has joined #openstack-ansible21:36
admin0however setup-infra has not run yet ..21:36
admin0so where is this URL/repo hard-coded that pip is trying to connect but failign21:37
admin0or . whcih file has the pip proxy set .21:37
*** jwitko has quit IRC21:40
*** ssmith has quit IRC21:42
*** jwitko_ has quit IRC21:42
*** throwsb1 has quit IRC21:46
tomtom001so I've upgraded the OS and ansible but the lxc containers still never build and the process just dies in the middle of the upgrade from ocata to pike.  thats all the juice I have at this point, thinking of just wiping the entire system and re-installing,  the run-upgrade script just doesnt seem to work21:47
*** esberglu has quit IRC21:52
*** jwitko has joined #openstack-ansible22:02
*** kstev has quit IRC22:18
*** cshen has joined #openstack-ansible22:22
cloudnullhey tomtom001, sorry I've not been able to reply, crazy wrapped up at the moment.22:23
cloudnulldid you happen to have a look at the logs for the build ?22:23
cloudnullis there something failing in the process?22:23
cloudnullevrardjp: ++ thanks for keeping me assigned, should have something on those issues tonight22:24
cloudnullevrardjp: Also yes. I'm confirmed for the summit, WOOT! Whom else is going to be there?22:24
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update elk 6.x playbooks  https://review.openstack.org/56016822:26
*** cshen has quit IRC22:27
cloudnulltomtom001: if you can login to the build environment via `chroot` can you execute the build script manually ?22:28
cloudnulldoes it fail in interesting ways, or is it able to connect to the outside world ?22:29
*** lbragstad has quit IRC22:42
*** dave-mccowan has quit IRC22:49
*** cshen has joined #openstack-ansible22:52
*** cshen has quit IRC22:57
*** lbragstad has joined #openstack-ansible23:09
*** cjloader has joined #openstack-ansible23:20
*** radeks has quit IRC23:21
*** cjloader has quit IRC23:24
*** cmart has joined #openstack-ansible23:44
cmarthi. has anyone had trouble accessing the Keystone Admin API after upgrading from Newton to Ocata/Pike? getting "empty reply from server"23:44
*** cshen has joined #openstack-ansible23:46
*** cshen has quit IRC23:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!