Thursday, 2017-06-08

*** TxGirlGeek has quit IRC00:37
*** lbragstad has joined #openstack-ansible00:37
*** cuongnv has joined #openstack-ansible00:47
*** abhishekk has quit IRC00:52
*** abhishekk has joined #openstack-ansible00:52
*** perniciouscaffei has quit IRC01:01
*** jamesdenton has joined #openstack-ansible01:07
*** cNilesh has joined #openstack-ansible01:08
*** fyu has joined #openstack-ansible01:11
*** smatzek has joined #openstack-ansible01:14
*** thorst_afk has joined #openstack-ansible01:14
*** piliman974 has quit IRC01:15
*** piliman974 has joined #openstack-ansible01:17
*** thorst_afk has quit IRC01:19
*** vnogin has quit IRC01:21
*** jamesdenton has quit IRC01:25
*** smatzek has quit IRC01:33
*** lbragstad has quit IRC01:42
*** schwicht has joined #openstack-ansible01:43
*** thorst_afk has joined #openstack-ansible01:47
*** thorst_afk has quit IRC01:47
openstackgerritMerged openstack/openstack-ansible-os_cinder stable/ocata: Perform an atomic policy file change  https://review.openstack.org/47036501:57
*** albertcard1 is now known as poopcat01:57
*** dxiri has joined #openstack-ansible01:58
*** dxiri_ has joined #openstack-ansible02:01
*** jamielennox is now known as jamielennox|away02:04
*** dxiri has quit IRC02:04
*** schwicht has quit IRC02:10
*** thorst_afk has joined #openstack-ansible02:12
*** thorst_afk has quit IRC02:12
*** jamielennox|away is now known as jamielennox02:15
openstackgerritMerged openstack/openstack-ansible-os_cinder stable/ocata: Filter services dictionary per host  https://review.openstack.org/47068602:16
*** dxiri has joined #openstack-ansible02:16
*** dxiri_ has quit IRC02:18
*** schwicht has joined #openstack-ansible02:38
*** schwicht has quit IRC02:42
openstackgerritVan Hung Pham proposed openstack/monitorstack master: Remove support for py34  https://review.openstack.org/47202603:09
openstackgerritVan Hung Pham proposed openstack/monitorstack master: Remove support for py34  https://review.openstack.org/47202603:10
*** thorst_afk has joined #openstack-ansible03:13
openstackgerritMerged openstack/openstack-ansible-os_searchlight master: Updated from global requirements  https://review.openstack.org/47196803:15
openstackgerritMerged openstack/openstack-ansible-os_molteniron master: Updated from global requirements  https://review.openstack.org/47196503:17
openstackgerritMerged openstack/openstack-ansible-os_octavia master: Updated from global requirements  https://review.openstack.org/47196703:18
openstackgerritMerged openstack/openstack-ansible-os_monasca-ui master: Updated from global requirements  https://review.openstack.org/47196603:20
*** aludwar has quit IRC03:22
*** thorst_afk has quit IRC03:32
*** schwicht has joined #openstack-ansible03:39
*** gkadam has joined #openstack-ansible03:42
*** schwicht has quit IRC03:45
*** poopcat has quit IRC04:00
*** piliman974 has quit IRC04:12
*** thorst_afk has joined #openstack-ansible04:29
*** pcaruana has joined #openstack-ansible04:31
*** thorst_afk has quit IRC04:37
*** NWOutlier_ has quit IRC04:44
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Connect using physical host's ansible_host var  https://review.openstack.org/47206104:47
*** pcaruana has quit IRC04:57
*** penguintamer has quit IRC04:57
*** admin0 has joined #openstack-ansible04:58
*** jamielennox is now known as jamielennox|away05:12
*** jamielennox|away is now known as jamielennox05:19
*** dxiri_ has joined #openstack-ansible05:19
*** dxiri has quit IRC05:22
*** udesale has joined #openstack-ansible05:28
*** dxiri_ has quit IRC05:35
*** flg has joined #openstack-ansible05:42
*** jamielennox is now known as jamielennox|away05:56
*** tobberydberg has joined #openstack-ansible05:56
*** admin0 has quit IRC06:00
*** jamielennox|away is now known as jamielennox06:03
*** Oku_OS-away is now known as Oku_OS06:10
*** schwicht has joined #openstack-ansible06:19
*** oanson has quit IRC06:20
*** oanson has joined #openstack-ansible06:21
*** schwicht has quit IRC06:24
*** penguintamer has joined #openstack-ansible06:28
*** thorst_afk has joined #openstack-ansible06:29
*** admin0 has joined #openstack-ansible06:30
*** thorst_afk has quit IRC06:34
*** esberglu has joined #openstack-ansible06:49
*** manheim has joined #openstack-ansible06:50
*** manheim has quit IRC06:54
*** esberglu has quit IRC06:54
*** manheim has joined #openstack-ansible06:55
*** charcol has quit IRC06:57
*** pcaruana has joined #openstack-ansible06:57
*** manheim has quit IRC06:59
*** piliman974 has joined #openstack-ansible07:01
*** arbrandes has joined #openstack-ansible07:01
*** Hosam has quit IRC07:03
*** arbrandes1 has quit IRC07:04
*** shardy has joined #openstack-ansible07:14
*** markus_z has joined #openstack-ansible07:14
*** oanson has quit IRC07:20
*** Hosam has joined #openstack-ansible07:20
*** oanson has joined #openstack-ansible07:21
*** admin0 has quit IRC07:21
*** gkadam is now known as gkadam-afk07:30
*** thorst_afk has joined #openstack-ansible07:30
*** oanson has quit IRC07:31
*** piliman974 has quit IRC07:31
*** oanson has joined #openstack-ansible07:31
*** thorst_afk has quit IRC07:35
neithanyone heard of cisco switch not forwarding ARP requests to LACP ports?07:53
kysseaccess or trunk07:55
neithkysse: trunk07:56
kysseand you have necessary vlans tagged?07:57
kysseserver and switch07:58
neithkysse: I do07:59
*** flg has quit IRC08:01
*** manheim has joined #openstack-ansible08:08
*** gkadam-afk is now known as gkadam08:16
*** manheim has quit IRC08:22
*** manheim has joined #openstack-ansible08:23
*** admin0 has joined #openstack-ansible08:26
*** flg has joined #openstack-ansible08:26
openstackgerritJuan Vidal Allende proposed openstack/openstack-ansible-specs master: Blueprint for OpenDaylight integration in OSA  https://review.openstack.org/46834008:29
*** thorst_afk has joined #openstack-ansible08:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Simplify multi arch repo build  https://review.openstack.org/47214508:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/newton: Simplify multi arch repo build  https://review.openstack.org/47214608:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder stable/ocata: Optimise the distro package installation  https://review.openstack.org/47214708:32
*** marcoarildo has quit IRC08:35
*** esberglu has joined #openstack-ansible08:37
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder stable/newton: Optimise the distro package installation  https://review.openstack.org/47215208:38
*** manheim has quit IRC08:39
*** yifei has quit IRC08:41
*** manheim has joined #openstack-ansible08:42
*** yifei has joined #openstack-ansible08:42
*** esberglu has quit IRC08:42
*** electrofelix has joined #openstack-ansible08:47
odyssey4me_andymccr evrardjp ping - you around?08:49
*** odyssey4me_ is now known as odyssey4me08:49
mpranjicmorning guys :)08:49
odyssey4meo/ mpranjic how're you doing?08:50
*** thorst_afk has quit IRC08:50
mpranjicodyssey4me: automating my ass off :)08:50
mpranjicso another good day I'd say08:50
odyssey4methat's a good place to be :)08:50
mpranjictrue :)08:50
mpranjicodyssey4me: you gonna have another code review rampage today? :D08:51
*** winggundamth has joined #openstack-ansible08:53
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Switch to Cryptography over pycrypto  https://review.openstack.org/47166408:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic master: Switch to Cryptography over pycrypto  https://review.openstack.org/47166108:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Switch to Cryptography over pycrypto  https://review.openstack.org/47166508:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer master: Switch to Cryptography over pycrypto  https://review.openstack.org/47165208:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_gnocchi master: Switch to Cryptography over pycrypto  https://review.openstack.org/47165808:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Switch to Cryptography over pycrypto  https://review.openstack.org/47165708:58
*** winggundamth has quit IRC08:58
odyssey4mempranjic heh, I guess it looks that way08:58
*** flg has quit IRC09:00
*** winggundamth has joined #openstack-ansible09:00
mpranjiclol09:00
andymccrodyssey4me: pong!09:01
odyssey4meah, good morning andymccr09:01
mpranjicandymccr: morning :)09:02
andymccrhey mpranjic09:02
odyssey4meso, I tested with both ways of restarting services - one where we remove the pip_install/galera_client meta deps and one where we don't use include_role, but instead use some tasks to restart the services... both results appear to come back with the same performance profile09:02
odyssey4meso I'm going to go with the least invasive option - I think we need to discuss the meta-deps thing at the next PTG, but for now I'll just avoid it09:03
*** coolkil has joined #openstack-ansible09:03
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder master: Implement data migrations for rolling upgrades  https://review.openstack.org/47040809:03
andymccrodyssey4me: ok makes sense - that is a pretty big change to make so yeah its probably safer. although probably the best in the long run - id just like more buy in09:04
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement rolling upgrades for cinder  https://review.openstack.org/46995509:05
andymccrodyssey4me: do you remember why we picked to keep using the "neutron:" module we built vs using the os_network: shade module.09:08
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement rolling upgrades for cinder  https://review.openstack.org/46995509:09
andymccrthere was a reason, and the trove role is using it - although struggling to get it to work09:09
odyssey4meandymccr historically we had problems with shade building due to incompatibilities with the u-c09:09
andymccrahh ok - not it builds fine, so its probably just me being an idiot when using it :P09:10
andymccri can solve that one (in theory)09:10
odyssey4meif we add shade, we should pin the version that works as expected and builds along with the rest of the things... otherwise we'll be back in that world of pain09:10
odyssey4meeither that or we need to use a venv for it09:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Add haproxy install playbook  https://review.openstack.org/47180509:15
*** manheim has quit IRC09:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder stable/newton: Optimise the distro package installation  https://review.openstack.org/47215209:22
*** manheim has joined #openstack-ansible09:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Update paste, policy and rootwrap configurations 2017-06-02  https://review.openstack.org/47030109:24
*** Hosam has quit IRC09:29
*** Hosam has joined #openstack-ansible09:30
*** cNilesh has quit IRC09:30
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-plugins master: connection: ssh: Clear environment when connecting to LXC containers  https://review.openstack.org/47181009:30
*** cNilesh has joined #openstack-ansible09:30
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: [WIP] Add translations test-site deployment option  https://review.openstack.org/46722209:31
*** Hosam has quit IRC09:34
*** esberglu has joined #openstack-ansible09:35
odyssey4meandymccr have you done a role pin revert yet for master?09:35
*** cNilesh has quit IRC09:36
andymccrodyssey4me: not yet09:36
odyssey4mein other news, looking for reviews of this pretty urgent fix for newton: https://review.openstack.org/47127109:36
*** cNilesh has joined #openstack-ansible09:36
andymccri'll push that up - but +2 on that patch09:37
openstackgerritMerged openstack/openstack-ansible-pip_install master: SUSE: Add the Cloud:OpenStack:Ocata repository  https://review.openstack.org/47137409:37
coolkilmarkus_z: you arround?09:38
markus_zcoolkil: yep, I'm here, how are you?09:39
coolkilI'm fine how ar you?09:40
*** esberglu has quit IRC09:40
coolkilI got some advice for you if you want to run openstack-ansible on s390x09:41
markus_zcoolkil: same old, same old09:41
markus_zcoolkil: Cool, I'm all ears :)09:42
coolkilthere is one issue that is a big problem.09:44
coolkilthe linuxone has no support for spice and vnc09:44
markus_zcoolkil: Ah, yeah :) You need the "serial console" feature as a replacement.09:44
*** vnogin has joined #openstack-ansible09:44
coolkilso you need to setup a serial-console however openstack-ansible currently has no means to deploy this09:44
kysseHas anyone had odd behaviour with iptables driver?09:45
kysseIt seems that when someone deletes instance it does something weird to the same hosts itables rules09:45
markus_zcoolkil: Is it in terms of configuration or that the "nova-serialproxy" service isn't yet enabled in OSA?09:46
evrardjpodyssey4me: yes?09:46
kyssesomething to do with OUTPUT rules09:47
coolkilwell all the info is available. openstack ansible downloads all the files needed for the serialproxy but is not configuring them09:47
*** cNilesh has quit IRC09:47
*** thorst_afk has joined #openstack-ansible09:47
*** cNilesh has joined #openstack-ansible09:47
odyssey4mecoolkil that's a new feature since we last added any consoles, so I expect that we'd like to see that added as an option for easy implementation09:48
markus_zcoolkil: I thought I saw a way how one can specify things which differ from the defaults, let me look it up.09:48
odyssey4meif you can register a bug with the details of how to configure it, and perhaps the circumstances under which is must be enabled, then we can work up a patch for it09:48
*** vnogin has quit IRC09:49
*** vnogin has joined #openstack-ansible09:49
coolkilodyssey4me: I will register a bug with all available information i got.09:50
odyssey4meevrardjp I was just updating about the tests with regards to the upgrade process - see the transcript here: http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2017-06-08.log.html#t2017-06-08T09:02:5309:51
markus_zcoolkil: Let me know the number, maybe I can add some information.09:51
odyssey4mecoolkil thanks :)09:51
coolkilwill do markus_z09:51
markus_zcoolkil: Thank! FWIW, here's my attempt to document some z specifics for Devstack: https://review.openstack.org/#/c/468887/3/doc/source/configuration.rst09:51
*** pmannidi has quit IRC09:51
*** thorst_afk has quit IRC09:51
*** pmannidi has joined #openstack-ansible09:52
evrardjpodyssey4me: ok09:52
odyssey4mecoolkil FYI thanks for raising the multi-arch bug which evrardjp pushed up a patch for, it's merged in master and the backports are up for review: https://review.openstack.org/#/q/I36c3033b8a932a81d98bd85d6cfb738e77147192,n,z09:52
coolkilcool!! thnx a lot!!09:53
* markus_z ^ will take a look too09:53
evrardjpthanks for the backport odyssey4me09:53
*** cNilesh has quit IRC09:57
*** flg has joined #openstack-ansible09:59
openstackgerritMerged openstack/openstack-ansible-os_trove master: Deprecate rpc_backend option  https://review.openstack.org/46979510:00
openstackgerritMerged openstack/openstack-ansible-os_sahara master: Deprecate rpc_backend option  https://review.openstack.org/46979310:06
*** stuartgr has joined #openstack-ansible10:09
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Deprecate rpc_backend option  https://review.openstack.org/46978610:09
*** woodard has quit IRC10:11
*** woodard has joined #openstack-ansible10:12
openstackgerritMerged openstack/openstack-ansible-os_ironic master: Switch to Cryptography over pycrypto  https://review.openstack.org/47166110:18
kysseso is there any way to affect on what order neutron-agent installs iptables rules?10:18
kysseit seems that iptables rules go borken randomly when one deletes instance10:19
kyssehttps://ptpb.pw/J8o8/diff10:19
kyssenewton10:19
*** piliman974 has joined #openstack-ansible10:20
coolkilmarkus_z: https://bugs.launchpad.net/openstack-ansible/+bug/1696701 ther you go10:21
openstackLaunchpad bug 1696701 in openstack-ansible "Serial-console proxy missing" [Undecided,New]10:21
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: [WIP] Add translations test-site deployment option  https://review.openstack.org/46722210:23
*** markvoelker has quit IRC10:27
*** markvoelker has joined #openstack-ansible10:27
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: DNM - Testing lxc-attach with clear environments  https://review.openstack.org/47181410:28
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update vars to match latest keepalived role  https://review.openstack.org/47219310:30
*** cristicalin has joined #openstack-ansible10:31
*** cristicalin has quit IRC10:33
openstackgerritMerged openstack/openstack-ansible-os_nova master: Switch to Cryptography over pycrypto  https://review.openstack.org/47166410:36
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: [WIP] Add translations test-site deployment option  https://review.openstack.org/46722210:39
*** vnogin has quit IRC10:43
*** vnogin has joined #openstack-ansible10:49
openstackgerritChenjun Shen proposed openstack/openstack-ansible-haproxy_server master: Fix bug/1696703: add some new variables of haproxy default for overriding.  https://review.openstack.org/47220010:50
*** smatzek has joined #openstack-ansible10:53
cshenHi all, the last proposal 472200 from me is to fix  https://bugs.launchpad.net/openstack-ansible/+bug/169670310:57
openstackLaunchpad bug 1696703 in openstack-ansible "hard coded http timeout in haproxy_server role template" [Undecided,New]10:57
*** udesale has quit IRC11:06
*** flg has quit IRC11:10
*** Hosam has joined #openstack-ansible11:21
*** pbandark has joined #openstack-ansible11:21
*** Hosam_ has joined #openstack-ansible11:25
*** Hosam has quit IRC11:28
*** Hosam has joined #openstack-ansible11:29
markus_zcoolkil: I added my 2cents about the config options there.11:30
*** Hosam_ has quit IRC11:32
*** retreved has joined #openstack-ansible11:38
*** mikal has quit IRC11:41
*** mikal has joined #openstack-ansible11:43
*** cuongnv has quit IRC11:45
*** schwicht has joined #openstack-ansible11:46
*** thorst_afk has joined #openstack-ansible11:46
*** schwicht has quit IRC11:50
*** hwoarang_ is now known as hwoarang11:56
*** piliman974 has quit IRC12:05
*** piliman974 has joined #openstack-ansible12:07
*** flg has joined #openstack-ansible12:09
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder master: Implement data migrations for rolling upgrades  https://review.openstack.org/47040812:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement rolling upgrades for cinder  https://review.openstack.org/46995512:11
*** piliman974 has quit IRC12:16
*** piliman974 has joined #openstack-ansible12:17
coolkilthnx markus_z wil have a look at that12:18
*** jamesdenton has joined #openstack-ansible12:22
*** schwicht has joined #openstack-ansible12:22
odyssey4mekysse well, I guess you'd have to ask in #openstack-neutron about that12:22
*** winggundamth has quit IRC12:22
*** ianychoi has quit IRC12:27
*** ianychoi has joined #openstack-ansible12:28
*** schwicht has quit IRC12:29
*** vnogin has quit IRC12:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_neutron master: Add Neutron API availability test during upgrade  https://review.openstack.org/47000112:29
*** schwicht has joined #openstack-ansible12:32
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: [WIP] Add translations test-site deployment option  https://review.openstack.org/46722212:34
*** karimb has joined #openstack-ansible12:34
*** vnogin has joined #openstack-ansible12:40
*** jwitko_ has quit IRC12:42
*** jroll has left #openstack-ansible12:47
*** gkadam has quit IRC12:50
*** askb has quit IRC12:50
*** schwicht has quit IRC12:51
*** Hosam has quit IRC12:51
*** Hosam has joined #openstack-ansible12:52
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Revert role pin for further Pike development  https://review.openstack.org/47224812:54
odyssey4meandymccr ^ I've reverted the role pins.12:55
*** piliman974 has quit IRC12:56
*** Hosam has quit IRC12:56
*** aludwar has joined #openstack-ansible12:57
neithcloudnull: evrardjp so LACP trunk blocks ARP request12:57
neithfor an unkown reason12:58
neithwhen mac adresses are from instances, no problem with physical devices12:58
neithand I really don"t know why12:58
neithso my setup was perfectly good since day one! wannacry12:59
*** udesale has joined #openstack-ansible12:59
*** klamath has joined #openstack-ansible13:01
*** klamath has quit IRC13:02
*** klamath has joined #openstack-ansible13:03
*** cristicalin has joined #openstack-ansible13:03
*** cathrichardson has joined #openstack-ansible13:04
andymccrodyssey4me: ok cool13:04
*** schwicht has joined #openstack-ansible13:04
evrardjpneith: LACP shouldn't be blocking ARP :p13:06
*** esberglu has joined #openstack-ansible13:06
neithevrardjp: I know, but thats what happening13:07
evrardjpmac security? Are you sure your LACP is properly configured on all sides?13:07
evrardjpis that a server linked with lacp on switches?13:07
neithevrardjp: Digging in cisco docs13:07
evrardjpcheck in your logs13:07
evrardjpit should be in security13:07
neithevrardjp: all servers are linked with LACP on switchs13:07
neithevrardjp: will do13:08
evrardjpneith: you could temporarily change the bond mode to 0 on your hosts13:08
evrardjpmeant 113:08
neithevrardjp: on cisco side?13:08
*** woodard has quit IRC13:08
evrardjpnope, on the linux side13:09
evrardjpthis way you ensure osa is working fine13:09
evrardjpthen you have time to dig into your networking issues13:09
*** woodard has joined #openstack-ansible13:09
*** cathrichardson has quit IRC13:09
*** jamesdenton has quit IRC13:10
*** cathrichardson has joined #openstack-ansible13:10
neithevrardjp: I already switched to mode 1 server side13:10
neithevrardjp: thats how I discovered LACP was blocking13:11
neithnow checking the sec logs13:11
*** klamath_ has joined #openstack-ansible13:12
*** klamath has quit IRC13:13
evrardjpI'm a little rusty on that side, but could13:14
evrardjpshow port-sec int <whatever> help you ?13:15
neithevrardjp: I'm gonna read the doc13:15
neithevrardjp: I no nothing about cisco ios13:15
neith*know13:16
evrardjphaha13:16
neithevrardjp: thanks for support :P13:16
*** lucasxu has joined #openstack-ansible13:16
evrardjpalso check if you have configured ports as edge or portfast or something, and compare with best practices for your case13:16
*** cpuga has joined #openstack-ansible13:23
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Revert role pin for further Pike development  https://review.openstack.org/47224813:23
*** jwitko has joined #openstack-ansible13:24
*** cpuga has quit IRC13:25
*** cpuga has joined #openstack-ansible13:25
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone master: Optimise the distro package installation  https://review.openstack.org/47145113:27
*** smatzek has quit IRC13:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_neutron master: Normalise distro package installation  https://review.openstack.org/47226113:32
openstackgerritNolan Brubaker proposed openstack/openstack-ansible master: Fix Python 3 dictionary incompatibilities  https://review.openstack.org/47152913:34
*** vnogin has quit IRC13:36
*** vnogin has joined #openstack-ansible13:36
*** udesale has quit IRC13:38
*** lucasxu has quit IRC13:38
*** lucasxu has joined #openstack-ansible13:40
*** coolkil has quit IRC13:43
*** Jack_Iv has joined #openstack-ansible13:46
*** coolkil has joined #openstack-ansible13:46
*** smatzek has joined #openstack-ansible13:50
*** lucasxu has quit IRC13:51
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Add Neutron API availability test during upgrade  https://review.openstack.org/47000113:52
*** ianychoi_ has joined #openstack-ansible13:54
*** ianychoi has quit IRC13:57
*** ianychoi_ is now known as ianychoi13:57
*** jamesdenton has joined #openstack-ansible13:59
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Normalise distro package installation  https://review.openstack.org/47227213:59
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Normalise distro package installation  https://review.openstack.org/47227214:00
*** pmannidi has quit IRC14:00
*** pmannidi has joined #openstack-ansible14:00
*** lucasxu has joined #openstack-ansible14:02
*** galstrom_zzz is now known as galstrom14:03
*** flg has quit IRC14:04
*** phalmos has joined #openstack-ansible14:05
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: DNM - Testing lxc-attach with clear environments  https://review.openstack.org/47181414:05
*** phalmos_ has joined #openstack-ansible14:07
*** phalmos has quit IRC14:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Pass packages to install as a list  https://review.openstack.org/47227814:11
ArchiFleKsHi, I just updated the ocata branch to the latest SHA and I get this error with lxc-container-created : http://paste.openstack.org/show/611921/14:12
*** jamesdenton has quit IRC14:12
cloudnullArchiFleKs: does br-mgmt exist on that host ?14:13
cloudnulland is the container running  ?14:13
*** lbragstad has joined #openstack-ansible14:14
ArchiFleKscloudnull14:15
ArchiFleKsyes it was working before14:15
cloudnulland is the state of that container still running w/ that pid ?14:17
alextricity25odyssey4me:  ping14:17
odyssey4mealextricity25 yerp14:17
alextricity25Question about https://github.com/openstack/openstack-ansible-os_tempest/blob/master/tasks/tempest_run.yml. It seems like tempest xml subunit results are only built if the first tasks succeeds, should the "generate results" tasks have a block/aways around them so the results get built on a failure too?14:18
alextricity25i.e. a tempest test has failed, but we still want to build the results14:19
ArchiFleKscloudnull: it is the same issue for all the container, I can do a cat on  /proc ont the host it OK14:19
odyssey4mealextricity25 hmm, so the trick is going to be to execute things when the task fails, but still count it as a fail so that the gate fails14:20
ArchiFleKsthe pid is still the same14:20
alextricity25odyssey4me: I see what you mean. Could we put a fail task in the "always" block?14:20
alextricity25Right after the test result files have been built.14:21
alextricity25Wait...nvm that won't work14:21
alextricity25duh14:21
odyssey4meyou'll likely have to do a block/rescue, but then have a task to make it fail based on whether the initial task failed14:21
odyssey4meie register the result of tempest execution, have the rescue run if it fails, then assert that the tempest execution passed after the rescue14:22
odyssey4mebut then you also want it to execute if it passes14:22
alextricity25right14:23
alextricity25What you said will work with a block/always14:23
*** yolanda has quit IRC14:24
cloudnullArchiFleKs: are you able to work through that pid ?14:24
cloudnullsomething like `ls /proc/30782/root/etc/network/interfaces.d/`14:24
odyssey4mealextricity25 ah, I see that block/always should do it - you'll just need an assertion afterwards to cause it to fail if the task failed14:24
odyssey4mebut it must fail after the block14:24
*** jamesdenton has joined #openstack-ansible14:25
alextricity25:thumbs-up: I can submit a patch for that now14:26
ArchiFleKscloudnull: yes I can, i can do a cat on the file ansible is looking for, at first I thought it was because we were using static route in our configuration, but it seems to be an ansible failure right ?14:26
alextricity25I'll ping you for a review later odyssey4me. Thanks for the help!14:26
odyssey4mesure thing, thanks alextricity25 - good catch!14:26
*** yolanda has joined #openstack-ansible14:28
*** jamesdenton has quit IRC14:35
szaherandymccr: Hi14:40
*** cjloader_ has joined #openstack-ansible14:40
szaherI updated my patch here, we will be using only one repo for everything and we will use tags instead of 3 different repos https://review.openstack.org/#/c/469977/14:41
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts master: Optimise the distro package installation  https://review.openstack.org/47228014:46
*** tobberydberg has quit IRC14:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Normalise distro package installation  https://review.openstack.org/47227214:47
ArchiFleKscloudnull: i just re-boostraped ansible with old version 2.2.2.0 and it worked, this is the only difference14:52
hwoarangdoes anyone else notice that gate-openstack-ansible-openstack-ansible-aio-ubuntu-xenial fails with some  ""ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500)"  in /etc/ansible/roles/os_cinder/tasks/cinder_backends.yml:31 ?14:53
*** lazyGuy has joined #openstack-ansible14:59
lazyGuyhello all #openstack-ansible14:59
lazyGuyCan any anybody help me with "Remove known problem packages" error for centos-openstack-release-* AND centos-release-qemu-ev?15:01
*** galstrom is now known as galstrom_zzz15:03
*** lbragstad has quit IRC15:08
*** galstrom_zzz is now known as galstrom15:16
*** vnogin has quit IRC15:17
*** retreved has quit IRC15:21
spotzGreetings my fellow OSAers! cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, automagically, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, v1k0d3n15:21
spotz,15:21
spotzseverion, bgmccollum, darrenc, JRobinson__, asettle, colinmcnamara, thorst, adreznec, eil397, qwang,nishpatwa_, cathrichardson, drifterza, sc68cal Friendly reminder we will be meeting in 40 minutes in #openstack-meeting-4 Agenda is available at https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Agenda_for_next_meeting15:21
d34dh0r53cloudnull: seeing something similar on my multinode-aio15:22
d34dh0r53cloudnull: ArchiFleKs http://paste.openstack.org/show/611962/15:23
openstackgerritMiguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-os_tempest master: Add block/always to tempest result file tasks  https://review.openstack.org/47229415:24
d34dh0r53cloudnull: my containers aren't getting eth1.ini placed in their config properly15:24
d34dh0r53cloudnull: I can throw your key on there if you want to poke around15:24
alextricity25odyssey4me: : https://review.openstack.org/#/c/472294/115:25
odyssey4mealextricity25 LGTM, let's see the gate confirm15:25
cloudnulld34dh0r53: can you verify the version of ansible you have?15:26
*** manheim has quit IRC15:27
d34dh0r53cloudnull: 2.3.1.015:27
openstackgerritAndy McCrae proposed openstack/openstack-ansible-os_horizon master: Add Translations update steps.  https://review.openstack.org/47229615:28
*** cpuga has quit IRC15:28
*** cpuga has joined #openstack-ansible15:28
cloudnullArchiFleKs: ^ was that the same version you had too ?15:28
cloudnulld34dh0r53: you testing master btw?15:28
d34dh0r53cloudnull: yeah15:29
cloudnullok15:29
d34dh0r53cloudnull: this is relatively new, master was working last week15:29
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-rabbitmq_server master: Dynamically include install task file  https://review.openstack.org/47229815:29
*** vnogin has joined #openstack-ansible15:30
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Normalise distro package installation  https://review.openstack.org/47227215:30
*** woodard has quit IRC15:31
*** cpuga_ has joined #openstack-ansible15:32
ArchiFleKscloudnull: no, I had the issue with latest ansible from stable/ocata which is 2.2.3.0 and i reverted to 2.2.2.015:33
spotzandymccr: agenda update reminder:)15:34
*** deadnull has joined #openstack-ansible15:34
*** cpuga has quit IRC15:35
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Revert role pin for further Pike development  https://review.openstack.org/47224815:37
d34dh0r53ArchiFleKs: are you adding a new container to an existing deployment?15:37
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: [WIP] Add translations test-site deployment option  https://review.openstack.org/46722215:37
cloudnullArchiFleKs: interesting. I wonder if there's a bug that cropped up in 2.2.3?15:38
* cloudnull testing15:38
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Revert role pin for further Pike development  https://review.openstack.org/47224815:38
*** dxiri_ has joined #openstack-ansible15:38
*** dxiri has joined #openstack-ansible15:38
*** chyka has joined #openstack-ansible15:40
*** Jack_Iv has quit IRC15:41
*** Jack_Iv has joined #openstack-ansible15:41
*** piliman974 has joined #openstack-ansible15:41
*** Jack_Iv has quit IRC15:41
*** Jack_Iv has joined #openstack-ansible15:42
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: If repo build fails, force all hosts to fail  https://review.openstack.org/47230115:42
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: [WIP] Add translations test-site deployment option  https://review.openstack.org/46722215:42
*** cpuga_ has quit IRC15:42
*** Jack_Iv has quit IRC15:42
*** cpuga has joined #openstack-ansible15:43
*** Jack_Iv has joined #openstack-ansible15:43
*** dxiri_ has quit IRC15:44
*** dxiri has quit IRC15:44
*** dxiri has joined #openstack-ansible15:44
*** coolkil has quit IRC15:45
*** Jack_Iv has quit IRC15:47
*** scarlisle has joined #openstack-ansible15:51
ianychoiandymccr, thanks a lot for helping translation checksite with AIO - would you +1 for https://review.openstack.org/#/c/440825/ ?15:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-memcached_server master: Optimise the distro package installation  https://review.openstack.org/47230515:51
andymccrianychoi: ahh yeah of course15:51
*** clayton has quit IRC15:51
andymccrianychoi: robcresswell has fixed up the horizon bits, and I dropped you an email so I think we are almost ready with that.15:51
*** markus_z has quit IRC15:51
ianychoiandymccr, oh I am seeing your e-mail now thank you!15:53
*** manheim has joined #openstack-ansible15:54
*** clayton has joined #openstack-ansible15:54
*** tobberydberg has joined #openstack-ansible15:55
*** dxiri has quit IRC15:56
*** Mannheim has joined #openstack-ansible15:56
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Pass packages to uninstall/install as a list  https://review.openstack.org/47230615:57
andymccrianychoi: no problem - sorry it took a bit longer than I had hoped to get going!15:57
*** lbragstad has joined #openstack-ansible15:57
andymccrszaher: sorry i missed your message before!15:57
*** manheim has quit IRC15:58
andymccrspotz: thanks for the prod :) i updated it last week so i think we're good to go. there is not too much new there i think15:58
spotzandymccr: okie, 2 minute warning:)15:58
*** tobberydberg has quit IRC15:59
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-memcached_server master: Optimise the distro package installation  https://review.openstack.org/47230516:00
*** dxiri has joined #openstack-ansible16:00
*** Oku_OS is now known as Oku_OS-away16:00
spotzcloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, v1k0d3n, severion, bgmccollum, darrenc, JRobinson__, asettle16:00
spotz, colinmcnamara, thorst, adreznec, eil397, qwang,nishpatwa_, cathrichardson, drifterza, sc68cal - meeting time!16:00
asettlespotz: I'll be missing this one! Enjoy! o/16:01
spotzasettle sure sit out on our rapid meetingness:)16:01
asettleHhahahaha :p16:01
*** vnogin has quit IRC16:05
*** woodard has joined #openstack-ansible16:07
*** woodard has quit IRC16:07
*** woodard has joined #openstack-ansible16:08
*** retreved has joined #openstack-ansible16:09
*** schwicht has quit IRC16:09
*** cpuga has quit IRC16:12
*** gkadam has joined #openstack-ansible16:12
*** cpuga has joined #openstack-ansible16:13
*** retreved_ has joined #openstack-ansible16:13
*** retreved has quit IRC16:15
*** piliman974 has quit IRC16:16
*** jamesdenton has joined #openstack-ansible16:17
*** dxiri has quit IRC16:28
*** galstrom is now known as galstrom_zzz16:29
*** shardy has quit IRC16:31
*** shardy has joined #openstack-ansible16:32
*** ajmaidak has joined #openstack-ansible16:32
ArchiFleKsd34dh0r53: cloudnull no it is an existing container16:34
ArchiFleKsi'm trying to upgrade16:34
d34dh0r53ArchiFleKs: yeah, so my error goes away if I blow away all of the containers on a host and then re-run lxc-container-create, if there is an existing container it fails16:35
*** dxiri has joined #openstack-ansible16:36
odyssey4mewgat's the failure?16:38
odyssey4me*what16:38
*** piliman974 has joined #openstack-ansible16:39
openstackgerritAndy McCrae proposed openstack/openstack-ansible-os_horizon master: Add Translations update steps.  https://review.openstack.org/47229616:40
ajmaidakI think I'm seeing the same issue on 15.1.4. The play lxc_container_create : Drop container network file (interfaces) fails with a "MODULE FAILURE"16:41
ajmaidakLooking at the play it just a simple template call, but they way it does it is via jumping into the /proc/<container-pid>/root which I think ansible 2.2.3.0 does not like for some reason.16:42
odyssey4med34dh0r53 ArchiFleKs cloudnull ah, so the issue is that when you add another container to an existing deployment, the container create playbook fails?16:42
odyssey4meI think I saw that too when changing affinities.16:42
*** woodard has quit IRC16:42
d34dh0r53odyssey4me: yeah, that's what I'm seeing16:42
d34dh0r53odyssey4me: unless I blow away all of the containers on the host16:43
odyssey4meI think we should register a bug for that.16:43
*** woodard has joined #openstack-ansible16:43
odyssey4meif you can put a bug together with the details of how to replicate it, that'd help.16:43
ajmaidakI switched from ansible 2.2.3.0 to 2.2.2.0 it succeeds.16:43
*** clayton has quit IRC16:43
odyssey4meoh really? that's interesting16:44
ArchiFleKsodyssey4me: i did not recreate any container, it just an upgrade of the existing ones16:44
ArchiFleKsajmaidak: same16:44
odyssey4mehmm, cloudnull perhaps our network interface hack needs adjustment16:44
*** smatzek has quit IRC16:46
*** clayton has joined #openstack-ansible16:46
*** galstrom_zzz is now known as galstrom16:47
openstackgerritJimmy McCrory proposed openstack/openstack-ansible master: Support cidr_networks in L3 network environments  https://review.openstack.org/44885016:49
*** armaan has joined #openstack-ansible16:52
*** dxiri has quit IRC16:52
d34dh0r53odyssey4me: cloudnull ajmaidak ArchiFleKs https://bugs.launchpad.net/openstack-ansible/+bug/169680216:53
openstackLaunchpad bug 1696802 in openstack-ansible "lxc-containers-create fails when run on a host with existing containers" [Undecided,New]16:53
odyssey4meajmaidak ArchiFleKs please add any additional info or experiences, and workaround to the bug16:53
odyssey4methanks d34dh0r5316:53
d34dh0r53np16:54
*** dxiri has joined #openstack-ansible16:57
ajmaidakodyssey4me: not really.  That's pretty much the issue. I started to suspect ansible when I realized that vi /proc/56331/root/etc/network/interfaces.d/eth1.cfg; edit save fails. But cd /proc/56331/root/etc/network/interfaces.d/; vi eth1.cfg; edit/save works.  Something icky is going on with the filesystem namespacing I think.16:58
*** pcaruana has quit IRC17:00
*** gkadam has quit IRC17:05
*** lucasxu has quit IRC17:14
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Revert role pin for further Pike development  https://review.openstack.org/47224817:15
*** vnogin has joined #openstack-ansible17:15
*** vnogin has quit IRC17:17
*** vnogin has joined #openstack-ansible17:18
*** phalmos_ has quit IRC17:22
*** phalmos has joined #openstack-ansible17:22
*** schwicht has joined #openstack-ansible17:32
odyssey4med34dh0r53 I've marked it confirmed and critical considering the nature of it, and that I've personally just seen it.17:34
lbragstado/17:35
lbragstadcurious if http://lists.openstack.org/pipermail/openstack-dev/2017-June/118074.html is something openstack-ansible would be interested in17:35
odyssey4melbragstad yes17:36
lbragstadodyssey4me: sweet - i should write a spec then17:37
*** vnogin has quit IRC17:37
odyssey4meI think if we can ditch the complexities involved in using our current rsync mechanism for key distribution for something simpler, we're game.17:37
odyssey4meAs long as it remains optional, I think it'll please most operators.17:37
odyssey4mewe would likely end up making it the default because it's the best experience, then leave the current mechanism as an option for a few cycles.17:38
*** dxiri has quit IRC17:38
lbragstadodyssey4me: yeah - i don't think we'd remove the disk-based approach17:38
lbragstadsome folks might want that level of control, maybe? but i can see etcd having some advantages with the replications bits17:38
odyssey4meI find myself wondering whether it'd make sense to include barbican as a pluggable back-end.17:39
*** dxiri has joined #openstack-ansible17:39
lbragstadodyssey4me: for fernet keys?17:39
odyssey4melbragstad yep17:40
lbragstadi know folks have talked about it17:41
odyssey4mebarbican has a pluggable library, castellan I think, which does barbican or file17:41
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: DO NOT MERGE - test setting nova back to master  https://review.openstack.org/46994117:43
*** jamesdenton has quit IRC17:45
*** lazyGuy has quit IRC17:53
*** jbadiapa_ has joined #openstack-ansible17:56
*** stuartgr has quit IRC17:58
*** jbadiapa has quit IRC17:59
*** lucasxu has joined #openstack-ansible18:00
*** smatzek has joined #openstack-ansible18:07
*** dxiri has quit IRC18:07
*** pmannidi has quit IRC18:08
*** pmannidi has joined #openstack-ansible18:09
*** shardy has quit IRC18:10
*** jamesdenton has joined #openstack-ansible18:10
*** dxiri has joined #openstack-ansible18:11
*** cpuga has quit IRC18:19
*** cpuga has joined #openstack-ansible18:19
*** tobberydberg has joined #openstack-ansible18:22
*** electrofelix has quit IRC18:25
*** cjloader_ has quit IRC18:33
*** cjloader_ has joined #openstack-ansible18:34
*** jamesdenton has quit IRC18:34
*** jamesdenton has joined #openstack-ansible18:35
*** dxiri has quit IRC18:35
*** armaan has quit IRC18:37
*** dxiri has joined #openstack-ansible18:37
*** armaan has joined #openstack-ansible18:38
odyssey4mehuzzah, gate pass for https://review.openstack.org/47224818:38
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: If repo build fails, force all hosts to fail  https://review.openstack.org/47230118:39
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement rolling upgrades for cinder  https://review.openstack.org/46995518:39
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Fix Python 3 dictionary incompatibilities  https://review.openstack.org/47152918:41
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Fix Python 3 bytes vs strings differences  https://review.openstack.org/47153118:42
*** pbandark has quit IRC18:42
*** armaan has quit IRC18:42
*** tobberydberg has quit IRC18:45
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: DNM - Testing lxc-attach with clear environments  https://review.openstack.org/47181418:45
*** dxiri has quit IRC18:52
*** dxiri has joined #openstack-ansible18:53
*** tobberydberg has joined #openstack-ansible19:00
*** cristicalin has quit IRC19:00
*** vnogin has joined #openstack-ansible19:07
*** cpuga has quit IRC19:07
*** cpuga has joined #openstack-ansible19:09
*** Jack_Iv has joined #openstack-ansible19:09
*** flg has joined #openstack-ansible19:10
*** dxiri has quit IRC19:17
odyssey4meok, I'm out for the night - cheers all!19:18
*** dxiri has joined #openstack-ansible19:20
cloudnullnight odyssey4me19:33
*** ricardoas1 has joined #openstack-ansible19:37
openstackgerritMerged openstack/openstack-ansible-os_nova master: Update paste, policy and rootwrap configurations 2017-06-02  https://review.openstack.org/47030119:38
*** manheim has joined #openstack-ansible19:39
*** ricardoas has quit IRC19:40
*** dxiri_ has joined #openstack-ansible19:44
*** dxiri has quit IRC19:48
*** Mannheim has quit IRC20:01
*** Mannheim has joined #openstack-ansible20:02
*** ricardoas1 has quit IRC20:09
*** ricardoas has joined #openstack-ansible20:10
*** jgriffiths has quit IRC20:13
openstackgerritMerged openstack/openstack-ansible stable/newton: Update role SHA's to include keystone fixes  https://review.openstack.org/47127120:15
*** Jack_Iv has quit IRC20:17
*** Jack_Iv has joined #openstack-ansible20:18
*** manheim has quit IRC20:21
*** rromans has quit IRC20:22
*** Jack_Iv has quit IRC20:22
*** dxiri has joined #openstack-ansible20:22
*** dxiri_ has quit IRC20:23
*** dxiri_ has joined #openstack-ansible20:34
*** jgriffiths has joined #openstack-ansible20:35
*** dxiri has quit IRC20:35
*** dxiri_ has quit IRC20:35
*** dxiri has joined #openstack-ansible20:35
*** dxiri has quit IRC20:37
*** dxiri has joined #openstack-ansible20:38
*** hamzy has quit IRC20:38
*** dxiri has quit IRC20:41
*** dxiri has joined #openstack-ansible20:41
*** tobberydberg has quit IRC20:41
*** karimb has quit IRC20:42
*** jamesdenton has quit IRC20:42
*** dxiri has quit IRC20:46
*** piliman974 has quit IRC20:57
*** piliman974 has joined #openstack-ansible20:59
*** joelgriffiths has joined #openstack-ansible21:02
*** thorst_afk has quit IRC21:02
*** gadgetwiz has joined #openstack-ansible21:04
*** jgriffiths has quit IRC21:04
*** thorst_afk has joined #openstack-ansible21:04
*** jgriffiths has joined #openstack-ansible21:05
*** joelgriffiths has quit IRC21:07
*** gadgetwiz has quit IRC21:08
*** flg has quit IRC21:08
*** thorst_afk has quit IRC21:09
*** retreved_ has quit IRC21:10
*** lucasxu has quit IRC21:11
*** galstrom is now known as galstrom_zzz21:12
*** woodard_ has joined #openstack-ansible21:14
*** perniciouscaffei has joined #openstack-ansible21:16
openstackgerritMerged openstack/openstack-ansible-memcached_server master: Optimise the distro package installation  https://review.openstack.org/47230521:16
*** woodard has quit IRC21:17
*** woodard_ has quit IRC21:18
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: Optimise the distro package installation  https://review.openstack.org/47228021:20
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Pass packages to uninstall/install as a list  https://review.openstack.org/47230621:20
openstackgerritMerged openstack/openstack-ansible-plugins master: Python3 compatibility for osa-filters  https://review.openstack.org/47109621:21
*** esberglu has quit IRC21:27
*** thorst_afk has joined #openstack-ansible21:27
*** esberglu has joined #openstack-ansible21:27
*** vnogin has quit IRC21:27
*** vnogin has joined #openstack-ansible21:28
openstackgerritMerged openstack/openstack-ansible-os_cloudkitty master: Deprecate rpc_backend option  https://review.openstack.org/46976321:29
*** thorst_afk has quit IRC21:31
*** esberglu has quit IRC21:32
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server master: Dynamically include install task file  https://review.openstack.org/47229821:40
*** cjloade__ has joined #openstack-ansible21:42
*** smatzek has quit IRC21:42
*** cjload___ has joined #openstack-ansible21:44
*** cjloader_ has quit IRC21:45
*** cjloade__ has quit IRC21:46
openstackgerritMerged openstack/openstack-ansible stable/newton: Simplify multi arch repo build  https://review.openstack.org/47214621:48
openstackgerritMerged openstack/openstack-ansible stable/ocata: Simplify multi arch repo build  https://review.openstack.org/47214521:48
*** cjloader_ has joined #openstack-ansible21:49
*** cjload___ has quit IRC21:49
*** ricardoas1 has joined #openstack-ansible21:49
*** ricardoas has quit IRC21:49
*** askb has joined #openstack-ansible21:50
openstackgerritMerged openstack/openstack-ansible-os_tempest master: Add block/always to tempest result file tasks  https://review.openstack.org/47229421:51
*** cjloader_ has quit IRC21:53
*** craigs has joined #openstack-ansible22:04
craigshavign a problem on rabbitmq  3.6.9-1 - rabbitmq logs - Channel error on connection <0.3037.15> (x.x.x.x:59524 -> x.x.x.x:5671, vhost: '/nova', user: 'nova'), channel 1: operation basic.publish caused a channel exception not_found: no exchange 'reply_d6b5c1db63fa4a74a7959c0683541346' in vhost '/nova'22:06
craigsany ideas this has been happening all day. getting on nova and cinder - ERROR oslo.messaging._drivers.impl_rabbit [-] [deccea24-c590-4a7e-86ed-3b8c90cd7aab] AMQP server on x.x.x.x:5671 is unreachable: Too many heartbeats missed. Trying again in 1 seconds. Client port: 4369422:07
craigswe are on newton 14.2.422:07
*** Mannheim has quit IRC22:10
cloudnullcraigs: I've not seen that.22:14
cloudnullis the cluster up and not partitioned  ?22:14
*** jbadiapa_ has quit IRC22:14
craigswe are fighting a problem no vm's can be created right now22:14
craigsclsuter is up and partitioned, everythign was working but now nothing22:15
cloudnullcan you login to a rabbitmq node and do a cluster status ?22:15
craigsyep22:15
cloudnullit is partitioned ?22:15
cloudnullcoolj: ^22:16
cloudnullhave you seen that ^22:16
cloudnullYou might need to do a rolling restart on the rabbitmq cluster?22:16
cloudnull stop node 1 then 2, then 3. to bring it back up in reverse22:17
craigshttps://pastebin.com/dqZM1Z1r22:17
cloudnulls/to/then/22:17
*** pmannidi has quit IRC22:17
craigsI have restarted the container22:17
*** pmannidi has joined #openstack-ansible22:17
craigsrabbitmqctl cluster_status22:18
craigsCluster status of node 'rabbit@infr-infr01a-p18-rabbit-mq-container-fbf426ad' ...22:18
craigs[{nodes,[{disc,['rabbit@infr-infr01a-p18-rabbit-mq-container-fbf426ad',22:18
craigs                'rabbit@infr-infr02a-p19-rabbit-mq-container-522a7d22',22:18
craigs                'rabbit@infr-infr03a-p20-rabbit-mq-container-4aa77580']}]},22:18
craigs {running_nodes,['rabbit@infr-infr01a-p18-rabbit-mq-container-fbf426ad']},22:18
craigs {cluster_name,<<"openstack">>},22:18
craigs {partitions,[]},22:18
craigs {alarms,[{'rabbit@infr-infr01a-p18-rabbit-mq-container-fbf426ad',[]}]}]22:18
cloudnulloh looks good.22:19
cloudnulld34dh0r53: ^22:19
cloudnullhave you tried restarting the nova processes to see if it recovers?22:20
craigsI have not22:20
*** manheim has joined #openstack-ansible22:21
*** manheim has quit IRC22:21
*** manheim has joined #openstack-ansible22:22
cloudnullcraigs: you should be able to do something like https://cloudnull.io/2017/05/rolling-restart-using-openstack-ansible/ -- with just the two nova entries in their22:22
cloudnullI guess cinder too22:22
craigsack22:22
cloudnullidk if that solves the issue.. but its worth a shot to get you back online.22:23
openstackgerritMerged openstack/openstack-ansible master: Revert role pin for further Pike development  https://review.openstack.org/47224822:24
craigsthank you will let you know22:24
openstackgerritMerged openstack/openstack-ansible master: Fix Python 3 dictionary incompatibilities  https://review.openstack.org/47152922:28
openstackgerritMerged openstack/openstack-ansible master: Fix Python 3 bytes vs strings differences  https://review.openstack.org/47153122:28
openstackgerritMerged openstack/openstack-ansible stable/ocata: Update vars to match latest keepalived role  https://review.openstack.org/47219322:28
*** esberglu has joined #openstack-ansible22:29
*** jbadiapa_ has joined #openstack-ansible22:30
*** cpuga has quit IRC22:32
*** esberglu has quit IRC22:33
openstackgerritMiguel Alex Cantu (alextricity25) proposed openstack/openstack-ansible-os_tempest stable/ocata: Add block/always to tempest result file tasks  https://review.openstack.org/47246522:37
*** piliman974 has quit IRC22:39
*** schwicht has quit IRC22:40
*** piliman974 has joined #openstack-ansible22:41
*** klamath_ has quit IRC22:41
*** tobberydberg has joined #openstack-ansible22:42
craigs@cloudnull wondering if we have this issue coming back up... - https://ask.openstack.org/en/question/83391/rabbitmq-failover-issue-kilo/22:44
craigsrestarted all three rabbit containers it is working...22:45
craigsso far22:45
craigscloudnull: ^22:46
cloudnullwoot!22:46
* cloudnull reading os question22:46
*** tobberydberg has quit IRC22:46
cloudnullso setting an override for the delay may be all that's needed ?22:47
craigspossibly22:47
craigsif it comes up again  I will reset value to 3.0 and see if it works22:48
craigsI will let you know22:48
*** ajmaidak has quit IRC22:50
cloudnullcraigs: it looks like that option is still in newton22:50
cloudnullhttps://docs.openstack.org/newton/config-reference/common-configurations/rpc.html22:50
cloudnullyou could add it in as a config_template override22:50
craigsrgr22:50
craigsreading22:50
cloudnulladding something like http://paste.openstack.org/show/612063/22:51
cloudnullto your openstack_deploy/user_variables.yml file22:52
cloudnullthen running `openstack-ansible os-nova-install.yml --tags nova-config` should be all that's needed.22:52
cloudnullnot that you want to do all that right now22:52
cloudnullbut smoething to think about should it happen again22:52
craigshttps://ask.openstack.org/en/question/78519/cinder-rabbitmq-timeout-problem-with-multiple-rabbit-hosts/22:53
craigsyou should be able to add to nova.conf also22:53
cloudnullthat override syntax would do just that and inject the option into the nova.conf22:55
cloudnullwe have the config_template library so we're not needing to carry every potential option for all services.22:55
craigsack22:56
cloudnullwhich is useful when needing to poke a specific option in22:56
craigsawesome!22:56
cloudnullbut if it makes good sense to just have it be the deafult we can do that too22:56
craigsok very cool, lets see if it is needed. if so I will let you know after I test it.22:56
craigshey thanks for the help! @ cloudnull22:57
*** schwicht has joined #openstack-ansible22:59
*** manheim has quit IRC23:06
*** piliman974 has quit IRC23:11
cloudnullnp craigs23:12
cloudnullhave a good one.23:12
*** piliman974 has joined #openstack-ansible23:13
*** Hosam has joined #openstack-ansible23:14
*** vnogin has quit IRC23:15
*** charcol has joined #openstack-ansible23:16
*** d9k has quit IRC23:16
*** d9k has joined #openstack-ansible23:17
*** Jack_Iv has joined #openstack-ansible23:18
*** Jack_Iv has quit IRC23:22
*** scarlisle has quit IRC23:24
*** markvoelker has quit IRC23:34
*** admin0 has quit IRC23:43
*** schwicht has quit IRC23:45
*** schwicht has joined #openstack-ansible23:54
*** chyka has quit IRC23:56
*** jamielennox is now known as jamielennox|away23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!