Tuesday, 2016-06-21

*** thorst_ has joined #openstack-ansible00:00
*** karimb has quit IRC00:03
*** weezS has joined #openstack-ansible00:06
*** thorst_ has quit IRC00:09
*** pacoblox has joined #openstack-ansible00:13
openstackgerritMerged openstack/openstack-ansible-os_magnum: Correcting issue with Database and User Creation  https://review.openstack.org/33183800:19
*** rg___ has joined #openstack-ansible00:22
*** rg___ has quit IRC00:29
*** daneyon has joined #openstack-ansible00:38
*** thorst_ has joined #openstack-ansible00:42
*** sdake_ has quit IRC00:44
*** weezS has quit IRC00:45
*** markvoelker has quit IRC00:46
*** markvoelker has joined #openstack-ansible00:47
*** raddaoui has quit IRC00:47
*** pacoblox has quit IRC00:47
*** pacoblox has joined #openstack-ansible00:48
*** daneyon has quit IRC00:57
*** daneyon has joined #openstack-ansible01:00
*** daneyon__ has joined #openstack-ansible01:01
*** weezS has joined #openstack-ansible01:04
*** daneyon has quit IRC01:05
*** weshay has quit IRC01:07
*** klamath has quit IRC01:14
*** klamath has joined #openstack-ansible01:14
*** thorst_ has quit IRC01:18
*** pacoblox has quit IRC01:19
*** thorst_ has joined #openstack-ansible01:19
*** pacoblox has joined #openstack-ansible01:21
*** daneyon has joined #openstack-ansible01:25
*** daneyon_ has joined #openstack-ansible01:26
*** daneyon__ has quit IRC01:27
*** thorst_ has quit IRC01:28
*** weezS has quit IRC01:28
*** daneyon has quit IRC01:30
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-pip_install: Rename pip_packages_tmp variable  https://review.openstack.org/33190801:39
jmccroryto unblock integrated gate ^ stevelle mhayden automagically odyssey4me01:39
*** saneax is now known as saneax_AFK01:40
*** jthorne_ has joined #openstack-ansible01:42
*** jthorne has quit IRC01:42
*** klamath has quit IRC01:43
*** daneyon_ has quit IRC01:53
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-pip_install: Rename pip_packages_tmp variable  https://review.openstack.org/33190801:59
*** sacharya has joined #openstack-ansible01:59
*** sacharya has quit IRC01:59
*** sacharya has joined #openstack-ansible02:00
*** daneyon has joined #openstack-ansible02:06
*** daneyon_ has joined #openstack-ansible02:15
errrwhen using this: https://github.com/openstack/openstack-ansible-os_horizon/blob/master/tasks/horizon_post_install.yml#L51-L57  is there a way to make the dest dir be created if its not there?02:16
errrIm thinking no because ansible copy module doesnt have a flag I can tell02:18
*** daneyon has quit IRC02:18
*** daneyon_ has quit IRC02:23
*** daneyon has joined #openstack-ansible02:23
*** daneyon_ has joined #openstack-ansible02:24
errrI have a terrible workaround.02:25
*** thorst_ has joined #openstack-ansible02:26
*** daneyon has quit IRC02:28
mcardenerrr: I think you might use the file module like... file: path=/some/path state=directory02:29
errrmcarden: I need to do it using the current playbook, but I realized a terrible way to work around it using whats there02:31
errrerr role rather02:31
mcardenok02:31
*** thorst_ has quit IRC02:33
*** psilvad has quit IRC02:33
*** daneyon_ has quit IRC02:33
*** gregfaust has quit IRC02:45
*** gregfaust has joined #openstack-ansible02:45
*** pacoblox has quit IRC02:58
*** pacoblox has joined #openstack-ansible03:00
*** schwicht has joined #openstack-ansible03:04
*** schwicht has quit IRC03:08
*** pacoblox has quit IRC03:09
*** jorge_munoz has quit IRC03:15
*** jorge_munoz has joined #openstack-ansible03:19
*** weezS has joined #openstack-ansible03:22
*** zerda2 has joined #openstack-ansible03:30
*** thorst_ has joined #openstack-ansible03:32
*** thorst_ has quit IRC03:37
*** saneax_AFK is now known as saneax03:47
*** toanster is now known as toan03:49
*** toan is now known as toanster03:51
*** toanster is now known as toan03:51
*** weezS has quit IRC03:59
*** albertcard has quit IRC04:01
*** itlinux has joined #openstack-ansible04:04
*** dweaver` has quit IRC04:06
*** sguduru has joined #openstack-ansible04:08
*** dweaver has joined #openstack-ansible04:13
*** sguduru has quit IRC04:16
*** saneax is now known as saneax_AFK04:18
*** pacoblox has joined #openstack-ansible04:24
*** thorst_ has joined #openstack-ansible04:34
*** thorst_ has quit IRC04:42
*** shausy has joined #openstack-ansible04:46
*** adrian_otto has joined #openstack-ansible04:49
*** sguduru has joined #openstack-ansible04:55
*** adrian_otto has quit IRC04:58
*** daneyon has joined #openstack-ansible05:01
*** adrian_otto has joined #openstack-ansible05:01
*** daneyon_ has joined #openstack-ansible05:02
*** adrian_otto has quit IRC05:05
*** daneyon has quit IRC05:06
*** saneax_AFK is now known as saneax05:06
*** javeriak has joined #openstack-ansible05:13
*** galstrom_zzz is now known as galstrom05:15
*** galstrom is now known as galstrom_zzz05:20
*** pacoblox1 has joined #openstack-ansible05:26
*** pacoblox has quit IRC05:26
*** pacoblox has joined #openstack-ansible05:30
*** pacoblox1 has quit IRC05:30
*** javeriak has quit IRC05:31
*** pacoblox1 has joined #openstack-ansible05:34
*** pacoblox has quit IRC05:34
*** jorge_munoz has quit IRC05:34
*** thorst_ has joined #openstack-ansible05:40
*** javeriak has joined #openstack-ansible05:44
*** thorst_ has quit IRC05:47
*** schwicht has joined #openstack-ansible05:49
*** pacoblox1 has quit IRC06:05
*** pacoblox has joined #openstack-ansible06:05
*** javeriak has quit IRC06:14
*** pacoblox has quit IRC06:15
*** schwicht has quit IRC06:23
openstackgerritMerged openstack/openstack-ansible-pip_install: Address Ansible bare variable usage  https://review.openstack.org/33167606:26
*** bsv has joined #openstack-ansible06:27
*** markvoelker has quit IRC06:35
openstackgerritDarren Chan proposed openstack/openstack-ansible: [docs] Revise overview chapter in OSA install guide  https://review.openstack.org/33196606:37
*** thorst_ has joined #openstack-ansible06:45
*** javeriak has joined #openstack-ansible06:47
darrencmhayden automagically I've started working on the overview chapter in the OSA install guide: https://review.openstack.org/#/c/33196606:47
darrencbut will need your help on the technical details06:48
darrencI'll ping you guys US late afternoon/ AUS early morning06:48
*** jiteka has quit IRC06:49
*** agireud has quit IRC06:51
*** thorst_ has quit IRC06:52
*** pcaruana has joined #openstack-ansible06:52
*** jiteka has joined #openstack-ansible06:53
*** agireud has joined #openstack-ansible06:55
*** jiteka has quit IRC06:59
*** daneyon_ has quit IRC07:10
*** sacharya has quit IRC07:13
*** karimb has joined #openstack-ansible07:35
*** markvoelker has joined #openstack-ansible07:36
*** markvoelker has quit IRC07:41
*** mikelk has joined #openstack-ansible07:48
*** thorst_ has joined #openstack-ansible07:50
*** thorst_ has quit IRC07:57
*** karimb has quit IRC08:03
*** admin0 has joined #openstack-ansible08:05
evrardjpgood morning08:10
*** sacharya has joined #openstack-ansible08:14
odyssey4meo/08:14
*** sacharya has quit IRC08:19
javeriakmorning odyssey4me, quick question, are we still tagging liberty?08:19
*** agireud has quit IRC08:19
javeriakand is there an appox on when the next 12.0.15 i believe might come out08:20
odyssey4mejaveriak we tagged last week, so the next tag will be late next week08:24
odyssey4meand yes, liberty is still a supported branch08:24
*** karimb has joined #openstack-ansible08:26
javeriakodyssey4me right; i have a problem, im keeping some of the plumgrid templates in OSA and whenever our internal plugin team asks for changes i keep having to coordinate with your tags, which is even harder for older branches. Do you think there's any way for me to dynamically load variables into my templates so I that I dont keep having to change the upstream code?08:26
odyssey4mejaveriak is the resulting file an ini, json or yaml file?08:27
javeriakits an ini08:27
odyssey4methen just ensure that it's put down by config_template and make sure there's an override dict08:28
*** agireud has joined #openstack-ansible08:28
odyssey4meah, it already is using it08:28
javeriakyou found it, wow that was quick :)08:28
odyssey4methe var they should use is neutron_plumlib_ini_overrides and point them to http://docs.openstack.org/developer/openstack-ansible/liberty/install-guide/configure-openstack.html08:29
odyssey4methe template in repo should be as minimal as possible, and the override should be used to add any non-essential config elements the deployer wants08:29
odyssey4methat's exactly why we implemented config_template - we were dealing with the same silly repetitive requests to add another var to override08:30
javeriaklet me look through that, so basically i should be able to declare extra vars through these overides in my user files during deployment?08:30
odyssey4meyou may want to consume it for the plumgrid playbooks/roles outside of OSA too - you'll just need to use our plugins repo08:31
odyssey4meor vendor the lib/plugin08:31
pjm6good morning all :)08:31
odyssey4mejaveriak yep, you add the dict and it will merge that over the top of the template08:31
javeriakodyssey4me perfect, this takes away alot of headache; let me go update our logic for these bits; thanks alot08:32
odyssey4mejaveriak no problem :)08:32
*** karimb has quit IRC08:34
*** jiteka has joined #openstack-ansible08:34
ionihello08:35
odyssey4meo/ pjm6 ioni how're things today?08:40
openstackgerritDarren Birkett proposed openstack/openstack-ansible: Make ansible managed line in swift.conf static  https://review.openstack.org/33201108:40
ioniodyssey4me, good, what about you?08:40
pjm6odyssey4me, o/ fine thanks, and with you, how's going?08:40
ionii'm not sure were to place something, but not sure if is related to openstack-ansible08:41
odyssey4meioni what is it?08:41
ioniin order for qemu-agent to work, i managed to workaround it by placing /etc/apparmor.d/abstractions/libvirt-qemu the line /var/lib/libvirt/qemu/*.sock rw,08:42
odyssey4meioni pjm6 doing quite well - getting a little impatient at the new openstack-ci system giving false failures :/08:42
ioniit's an old bug in ubuntu package08:42
ionii'm not sure how to avoid this stupid bug08:43
odyssey4meioni do you have the bug ref handy?08:43
pjm6odyssey4me, openstack-ci of gerrit right?08:43
ionihttps://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/139384208:44
openstackLaunchpad bug 1393842 in libvirt (Ubuntu Trusty) "libvirt does not grant qemu-guest-agent channel perms" [High,Fix committed]08:44
ioniagain, i don't know if openstack-ansible should fix this08:44
odyssey4meioni a place pretty much anything can go is the openstack-ansible-ops repo - you can create a space there for a playbook to implement the fix and a readme to describe what it's for and to mention the bug08:44
*** greg_faust has joined #openstack-ansible08:45
odyssey4meioni I would definitely suggest that the bugfix is patched upstream though08:45
ioniyep08:45
odyssey4methe bug says that it's fixed though?08:46
ionibut is not fixed :)08:46
odyssey4meah I see that it may not be fixed, fixed :p08:46
*** permalac__ has quit IRC08:47
ionihmm08:47
ioniit says that is fixed in 1.2.2-0ubuntu13.1.1908:47
ionibut that didn't landed in repos08:47
*** gregfaust has quit IRC08:48
*** greg_faust is now known as gregfaust08:48
*** permalac has joined #openstack-ansible08:50
odyssey4meioni is it perhaps in ubuntu cloud archive? or maybe it's still in proposed?08:54
openstackgerritMerged openstack/openstack-ansible-pip_install: Rename pip_packages_tmp variable  https://review.openstack.org/33190808:54
ioniodyssey4me, it's in testing but the feedback is not postive08:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Introduce a playbook for deploying Gnocchi  https://review.openstack.org/32412508:56
*** tiagogomes has joined #openstack-ansible08:57
*** admin0 has quit IRC08:59
*** admin0 has joined #openstack-ansible08:59
openstackgerritMerged openstack/openstack-ansible: Fix typos in openstack-ansible/doc  https://review.openstack.org/33145809:02
*** javeriak has quit IRC09:04
*** javeriak has joined #openstack-ansible09:13
openstackgerritMerged openstack/openstack-ansible-os_rally: Remove pip_lock_down dependency  https://review.openstack.org/33181509:15
*** openstackgerrit has quit IRC09:18
*** openstackgerrit has joined #openstack-ansible09:19
*** admin0 has quit IRC09:27
*** admin0 has joined #openstack-ansible09:28
winggundamthodyssey4me: can you help me looking at this https://review.openstack.org/#/c/330397/. I have no idea why gate said it doesn't have keystone policy file here http://logs.openstack.org/97/330397/3/check/gate-openstack-ansible-os_horizon-ansible-func-ubuntu-trusty/4096f09/console.html09:29
*** vnogin has joined #openstack-ansible09:31
*** sacharya has joined #openstack-ansible09:37
*** markvoelker has joined #openstack-ansible09:37
*** admin0 has quit IRC09:39
*** sacharya has quit IRC09:41
*** admin0 has joined #openstack-ansible09:43
*** markvoelker has quit IRC09:43
*** admin0 has quit IRC09:46
*** winggundamth has quit IRC09:49
*** admin0 has joined #openstack-ansible09:55
*** winggundamth has joined #openstack-ansible10:23
*** johnmilton has quit IRC10:26
*** winggundamth has quit IRC10:28
*** admin0 has quit IRC10:38
*** markvoelker has joined #openstack-ansible10:38
*** winggundamth has joined #openstack-ansible10:40
*** admin0 has joined #openstack-ansible10:42
*** markvoelker has quit IRC10:43
*** sguduru has quit IRC10:53
*** johnmilton has joined #openstack-ansible11:06
*** smatzek has joined #openstack-ansible11:08
*** sguduru has joined #openstack-ansible11:09
*** sguduru_ has joined #openstack-ansible11:10
*** sguduru has quit IRC11:10
*** javeriak has quit IRC11:14
*** javeriak has joined #openstack-ansible11:18
*** markvoelker has joined #openstack-ansible11:39
*** deverter has joined #openstack-ansible11:41
*** weshay has joined #openstack-ansible11:43
*** markvoelker has quit IRC11:46
*** javeriak has quit IRC11:49
*** admin0 has quit IRC11:53
*** Zucan has quit IRC11:53
*** admin0 has joined #openstack-ansible11:55
*** sguduru_ has quit IRC12:00
*** psilvad has joined #openstack-ansible12:04
odyssey4mewinggundamth hrm, I can't quite figure that out - but I've added some review comments12:06
winggundamthodyssey4me: I already tried [0] with with_items. it does not works well with |default('') and causing error. that's why I'm using what you see right now.12:07
*** psilvad has quit IRC12:10
*** admin0 has quit IRC12:10
odyssey4mewinggundamth better for the role to have a list of dicts provided in a var than to have it fixed12:10
winggundamthodyssey4me: yep. I'll try it but I don't think that will fix the issue in jenkins gate check right now12:11
*** admin0 has joined #openstack-ansible12:12
mhaydenmornin'12:12
*** markvoelker has joined #openstack-ansible12:16
*** admin0 has quit IRC12:18
*** admin0 has joined #openstack-ansible12:19
*** psilvad has joined #openstack-ansible12:20
*** rg_ has joined #openstack-ansible12:23
*** shausy has quit IRC12:27
mhaydenodyssey4me: so the talk in austin about openstack-ansible went pretty well12:30
mhaydenlots of good questions about when/why to use it12:31
mhaydenand i gave them hughsaunders' home phone number if they found any bugs12:31
mhaydentold them he is really friendly and patient12:31
hughsaunders:)12:32
*** rg_ has quit IRC12:34
*** rg_ has joined #openstack-ansible12:35
*** kylek3h has joined #openstack-ansible12:35
*** deverter_ has joined #openstack-ansible12:36
*** deverter_ has quit IRC12:37
*** deverter_ has joined #openstack-ansible12:38
*** klamath has joined #openstack-ansible12:38
*** klamath has quit IRC12:39
*** deverter has quit IRC12:39
*** admin0 has quit IRC12:40
*** klamath has joined #openstack-ansible12:40
*** admin0 has joined #openstack-ansible12:47
*** rg_ has quit IRC12:53
*** jamesdenton has joined #openstack-ansible12:55
openstackgerritMerged openstack/openstack-ansible: Ignore Ansible .retry files  https://review.openstack.org/33172412:58
*** jthorne_ has quit IRC12:58
*** jthorne has joined #openstack-ansible12:58
*** Zucan has joined #openstack-ansible13:03
admin0i received a call ( overflow) vps66701 wants 1 extra core and 1 extra GB of ram till end of this month .. will send ticket soon  .. i checked owner information .. checks out13:05
admin0wrong window :D13:05
*** messy has joined #openstack-ansible13:09
*** afred312 has quit IRC13:11
*** sigmavirus24_awa is now known as sigmavirus2413:14
*** psilvad has quit IRC13:16
automagicallyMorning all13:18
openstackgerritMerged openstack/openstack-ansible-os_heat: Pin test-requirements to match OpenStack requirements  https://review.openstack.org/33062213:20
odyssey4memhayden awesome :) have you posted your slides yet?13:25
*** admin0 has quit IRC13:26
mhaydenodyssey4me: indeed http://www.slideshare.net/MajorHayden/flexible-simple-deployments-with-openstackansible13:26
*** pacoblox has joined #openstack-ansible13:26
mhaydeni don't think anyone in the room had heard of it before13:27
*** admin0 has joined #openstack-ansible13:27
mhaydenabout 1/4 to 1/3 of the room had ever used ansible13:27
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_neutron: Correct Openvswitch bridge_mappings configuration  https://review.openstack.org/33213913:27
openstackgerritMerged openstack/openstack-ansible-os_nova: Fix shell command usage  https://review.openstack.org/33148713:28
* mhayden scurries off to meetings13:28
openstackgerritMerged openstack/openstack-ansible-os_swift: Skip unavailable hosts when distributing keys  https://review.openstack.org/33185913:29
automagicallyNice slides mhayden13:30
odyssey4meautomagically ++ excellent slides13:32
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_neutron: Correct Openvswitch bridge_mappings configuration  https://review.openstack.org/33213913:34
*** zerda2 has quit IRC13:34
*** afred312 has joined #openstack-ansible13:35
*** sdake has joined #openstack-ansible13:36
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Remove remaining container_release properties  https://review.openstack.org/33186913:46
*** afred312 has quit IRC13:46
*** afred312 has joined #openstack-ansible13:47
*** openstackgerrit has quit IRC13:48
errrIs there a specific reason that this: https://github.com/openstack/openstack-ansible-os_horizon/blob/master/tasks/horizon_post_install.yml#L80-L90 is *not* the last task in that particular file?13:48
*** kjw3 has joined #openstack-ansible13:48
*** openstackgerrit has joined #openstack-ansible13:49
*** messy has quit IRC13:50
*** bsv has quit IRC13:51
*** afred312 has quit IRC13:51
*** kencjohnston has joined #openstack-ansible13:51
*** kencjohnston has quit IRC13:51
errrThe reason I ask is that I think it might be a bug.. lets say I use the customizations module which is last to monkey patch in some stuff to HORIZON_CONFIG like additional options that arent available like js_files or something else not currently in the local_settings.py In that case my js_files options wouldnt get picked up by the collectstatic13:52
errrs,is last,I use,13:53
*** messy has joined #openstack-ansible13:54
errrsince my customizations_module would get placed on the server after you run the collect static and compress13:54
*** pacoblox has quit IRC13:54
*** pacoblox has joined #openstack-ansible13:54
*** messy has joined #openstack-ansible13:54
odyssey4meerrr yep, it's likely not intended - logan- I think you might have added it?13:55
*** ametts has joined #openstack-ansible13:55
mrhillsmang'morning13:56
openstackgerritMerged openstack/openstack-ansible: Make ansible managed line in swift.conf static  https://review.openstack.org/33201113:56
*** oneswig has joined #openstack-ansible13:57
*** kencjohnston_ is now known as kencjohnston13:57
*** homerp has quit IRC13:57
*** sonus_ has quit IRC13:57
*** kencjohnston has quit IRC13:57
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-rsyslog_client: DOC - Adopting the common role documentation pattern  https://review.openstack.org/33171613:58
*** messy has quit IRC13:58
*** kencjohnston has joined #openstack-ansible13:58
*** sonus has joined #openstack-ansible13:59
*** homerp_ has joined #openstack-ansible13:59
*** messy has joined #openstack-ansible13:59
*** afred312 has joined #openstack-ansible14:00
*** sonus has quit IRC14:03
*** kencjohnston has quit IRC14:03
*** homerp_ has quit IRC14:03
*** homerp_ has joined #openstack-ansible14:03
*** kencjohnston has joined #openstack-ansible14:04
*** sonus has joined #openstack-ansible14:04
odyssey4mejmccrory have I ever told you that your rock?14:06
odyssey4medigging into why https://review.openstack.org/#/c/331079/ keeps failing, I find that jmccrory already registered https://bugs.launchpad.net/cinder/+bug/1594195 and has put in the patch we need to resolve it https://review.openstack.org/#/c/331786/14:06
openstackLaunchpad bug 1594195 in Cinder "db sync fails to create cinder.volumes" [Undecided,Fix released] - Assigned to Cao ShuFeng (caosf-fnst)14:06
*** sonus has quit IRC14:07
*** kencjohnston has quit IRC14:07
*** homerp_ has quit IRC14:07
*** afred312 has quit IRC14:08
*** homerp_ has joined #openstack-ansible14:08
*** afred312 has joined #openstack-ansible14:08
*** kencjohnston has joined #openstack-ansible14:08
*** sonus has joined #openstack-ansible14:09
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_neutron: Remove enable_tunneling config from openvswitch_agent.ini  https://review.openstack.org/33215714:10
vnoginHi All14:11
*** homerp_ has quit IRC14:11
*** sonus has quit IRC14:11
*** kencjohnston has quit IRC14:11
*** homerp_ has joined #openstack-ansible14:11
automagicallyo/ vnogin14:11
*** kencjohnston has joined #openstack-ansible14:12
*** sonus has joined #openstack-ansible14:12
openstackgerritMerged openstack/openstack-ansible: Gate: Restrict Ansible fact gathering to base subset  https://review.openstack.org/33141414:13
openstackgerritMerged openstack/openstack-ansible: DOC - Remove instructions to run haproxy-install.yml play  https://review.openstack.org/33172014:13
*** kencjohnston has quit IRC14:13
*** sonus has quit IRC14:13
*** homerp_ has quit IRC14:13
*** homerp_ has joined #openstack-ansible14:13
vnoginodyssey4me: hi Jesse. Look, I regarding the issue that I mentioned last Friday. Here is the problem https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/tasks/lxc_cache_preparation.yml#L73 . It needs sshd_config file however it doesn't exist14:14
*** kencjohnston has joined #openstack-ansible14:14
*** sonus has joined #openstack-ansible14:14
vnoginhere is errors http://paste.openstack.org/show/520945/14:15
vnoginif I create it manually - it works fine :)14:15
openstackgerritBjoern Teipel proposed openstack/openstack-ansible-os_keystone: Add support to tune the keystone apache MPM settings  https://review.openstack.org/33216214:15
*** sonus has quit IRC14:15
*** kencjohnston has quit IRC14:15
*** homerp_ has quit IRC14:15
*** homerp_ has joined #openstack-ansible14:16
*** kencjohnston has joined #openstack-ansible14:16
*** sonus has joined #openstack-ansible14:17
vnoginI've checked rootfs.tar.xz file and don't see sshd_config file there as well14:18
mhaydenautomagically / odyssey4me: thanks!14:18
mhaydenautomagically: i told everyone to watch more NBC shows while i was there :)14:18
*** Mudpuppy has joined #openstack-ansible14:20
odyssey4mevnogin ah, if that has happened then the packages inside your container didn't install14:21
odyssey4mevnogin ie the cache prep didn't complete execution - we should actually make that error properly if the apt-get install fails14:22
odyssey4mevnogin do you have a verbose output of the lxc-host play execution pastebin'd somewhere?14:22
vnoginodyssey4me: let me rerun playbook and I'll send it:)14:23
openstackgerritBjoern Teipel proposed openstack/openstack-ansible: Add support to tune the keystone apache MPM settings  https://review.openstack.org/33216514:24
*** cloader89 has joined #openstack-ansible14:28
*** KLevenstein has joined #openstack-ansible14:29
*** admin0 has quit IRC14:30
odyssey4meautomagically jmccrory evrardjp maybe we shouidn't be merging https://review.openstack.org/331786 (see my running commentary with myself)14:30
automagicallySaw it. -w for now til we hash it out14:31
*** admin0 has joined #openstack-ansible14:31
vnoginodyssey4me: here is output http://paste.openstack.org/show/520947/14:31
odyssey4meautomagically thanks14:31
automagicallyAgreed on the release note issue14:31
automagicallyAnd on the mysql_db module change14:31
automagicallyI think the upgrade concerns can be addressed in a separate patch14:32
openstackgerritMerged openstack/openstack-ansible-repo_build: Reorganize test playbooks  https://review.openstack.org/33085814:33
openstackgerritChristopher Hultin proposed openstack/openstack-ansible-os_magnum: Adding 'admin_user' to config file  https://review.openstack.org/33216914:33
odyssey4meyeah, although it'd probably not be too hard to execute a check for the current setting - then to change the table if it needs it?14:33
*** spotz_zzz is now known as spotz14:33
*** cloader89 has quit IRC14:33
odyssey4meautomagically that said, apparently this is only an issue for new db creation - not for existing DB's... also it only seems to be needed by cinder - so perhaps we should isolate this change only in the cinder role14:34
*** cloader89 has joined #openstack-ansible14:34
odyssey4mean example - we implement it in the cinder task that creates the db (in the integrated repo and in the role tests)14:34
automagicallyWell, but as jmccrory pointed out, the install guide suggests this setting everywhere14:35
odyssey4meapparently upgrades won't give the issue - so perhaps we just leave that alone and if it comes up as an issue for upgrades we implement a conversion14:35
odyssey4methere is that14:35
jmccroryodyssey4me automagically : the problem was cinder basically condensed their db migrations back to kilo a few days ago, and their starting point assumed configurations with a default utf8 charset14:36
jmccroryi'd think db migrations would handle altering to utf8 (and then fixing collation) for other services14:37
odyssey4mevnogin as you'll see in the log - the apt-get update is failing14:37
*** admin0 has quit IRC14:37
vnoginhere is full output https://docs.google.com/a/mirantis.com/document/d/1ShgZRN7gBR6xwMTeS9HS_cUKBg7AZvz7wpxJdRof7VM/edit?usp=sharing :)14:38
*** phalmos has joined #openstack-ansible14:38
*** jorge_munoz has joined #openstack-ansible14:42
*** afred312 has quit IRC14:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts: Ensure that cache prep failure bubbles up from the prep script  https://review.openstack.org/33218014:47
odyssey4mevnogin ^ not sure if that will help - it should be worth testing14:47
*** admin0 has joined #openstack-ansible14:48
*** daneyon has joined #openstack-ansible14:49
*** daneyon has quit IRC14:50
*** oneswig has quit IRC14:50
vnoginodyssey4me: added -e , checking14:51
vnoginnope :( the same output. as I said quick fix to do touch /var/lib/lxc/LXC_NAME/rootfs/etc/ssh/sshd_config on all problematic nodes :)14:52
odyssey4mevnogin it looks to me like https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/vars/ubuntu-14.04.yml#L55 isn't fully being executed14:53
*** raddaoui has joined #openstack-ansible14:53
odyssey4mevnogin yeah but the issue is actually that the openssh_server package set is not being installed14:53
odyssey4meit's the package that puts the file there14:53
odyssey4meso the real issue is that none of the required packages are implemented into the container cache14:53
odyssey4meie none of these: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/vars/ubuntu-14.04.yml#L71-L12214:54
*** admin0 has quit IRC14:54
*** weezS has joined #openstack-ansible14:56
vnoginyep, you are right, I don't see any package from the list in LXC_NAME container14:57
pjm6odyssey4me, tell me one thing, about rabbitmq the policy is just to "ha-mode": "all" to all vhost, right?14:57
pjm6wouldn't be necessary to use the "ha-sync-mode: automatic" ?14:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts: [WIP] Ensure that cache preparation errors if something fails  https://review.openstack.org/33218014:58
odyssey4mevnogin ^ try that change instead?14:58
*** admin0 has joined #openstack-ansible14:58
odyssey4mepjm6 I don't know what the difference is :)14:58
evrardjpcloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, automagically, spotz, vdo, jmccrory, alextricity25, jasondotstar, KLevenstein, admin0, michaelgugino, ametts, v1k0d3n, se14:59
evrardjpverion, bgmccollum, darrenc, JRobinson__, asettle, colinmcnamara, thorst, adreznec14:59
pjm6Well in the docs I saw "Queues can be set to automatically synchronise by setting the ha-sync-mode policy key to automatic"14:59
evrardjpI'd be happy if you could have a look at this: https://etherpad.openstack.org/p/osa-bugtriage14:59
mhaydenevrardjp: you rang, sir? :)14:59
admin0o/ evrardjp14:59
evrardjpI'm trying a new approach for the bug triage14:59
automagicallyevrardjp: Appreciate the pre-work reminder on bug traige14:59
evrardjpthis etherpad would allow people to leave comments for the bug triage without the need to attend, this way we can have proper discussion during the meeting15:00
pjm6idk but it seems that this will keep all the vHosts synced15:00
*** jorge_munoz_ has joined #openstack-ansible15:01
evrardjpSadly I have a tight schedule today15:02
*** jorge_munoz has quit IRC15:02
*** jorge_munoz_ is now known as jorge_munoz15:02
*** kstev has joined #openstack-ansible15:03
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Remove the AIO metadata checksum fix from run-playbooks  https://review.openstack.org/33038915:03
openstackgerritAshana Evans proposed openstack/openstack-ansible-os_cinder: These changes updates the cinder role to support Ubuntu 16.04 -Created a  ubuntu-16.04 file in vars -Updated meta/main.yml dependencies to include xenial as a verison of Ubuntu. -Added cinder-16.04-support.yaml in realesenotes  https://review.openstack.org/33218915:03
prometheanfirehi15:04
admin0evrardjp: i do not think i can work on any of those bugs :(15:10
evrardjpnp15:10
evrardjpI understand deadlines, it can be hard for everyone15:11
prometheanfireyay, roofers here15:11
*** saneax is now known as saneax_AFK15:16
*** TxGirlGeek has joined #openstack-ansible15:17
vnoginodyssey4me: ok, last fix fixed the issue on the controller node however the there is another issue on the compute nodes :) http://paste.openstack.org/raw/520954/15:17
spotzthanks evrardjp No imput for bug triage from me15:17
evrardjpok thanks spotz15:19
odyssey4mevnogin so it looks like the apt-get update keeps failing to fetch the indexes for the mariadb repo - any idea why?15:20
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts: [WIP] Ensure that cache preparation errors if something fails  https://review.openstack.org/33218015:22
odyssey4mevnogin maybe that change will help? ^15:22
*** TxGirlGeek has quit IRC15:24
openstackgerritAshana Evans proposed openstack/openstack-ansible-os_cinder: These changes updates the cinder role to support Ubuntu 16.04  https://review.openstack.org/33218915:25
*** javeriak has joined #openstack-ansible15:27
*** TxGirlGeek has joined #openstack-ansible15:29
*** jayc has joined #openstack-ansible15:30
vnoginodyssey4me: frankly - don't know why :( Tried the latest fix - the same issue with mariadb15:31
*** javeriak_ has joined #openstack-ansible15:31
*** admin0 has quit IRC15:31
odyssey4mevnogin on the host if you do an apt-get update does it fetch the index from that repo ok?15:31
odyssey4mevnogin do you perhaps have some sort of firewall/proxy setup that needs to be implemented?15:32
*** javeriak has quit IRC15:33
vnoginnope, manually I face this issue as well: W: Failed to fetch https://mirror.rackspace.com/mariadb/repo/10.0/ubuntu/dists/trusty/main/binary-amd64/Packages15:34
vnoginE: Some index files failed to download. They have been ignored, or old ones used instead.15:34
odyssey4mevnogin hmm - ok15:34
odyssey4mevnogin can you curl that URL?15:34
*** pcaruana has quit IRC15:35
vnoginif I install curl it works15:35
odyssey4mevnogin if you install curl?15:35
odyssey4mecurl wasn't installed?15:35
vnoginyep15:36
vnoginI've installed it and it started to work15:36
odyssey4mevnogin and once curl is installed, the apt-get update works?15:36
vnoginyep15:36
vnoginit seems that we need to install curl prior the apt-get update15:37
odyssey4meinteresting - I would have thought that apt itself would then have curl as a dep15:37
vnoginas you can see - nope :)15:37
odyssey4mevnogin oddly enough, curl is in the openstack_host package list: https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/vars/ubuntu-14.04.yml#L5915:38
odyssey4meso it should be there already15:38
vnoginyep, I saw it. however when I installed it manually it started to work15:39
odyssey4meand openstack-hosts is executed before lxc-hosts: https://github.com/openstack/openstack-ansible/blob/master/playbooks/setup-hosts.yml#L1615:39
*** vdo_ is now known as vdo15:39
*** vdo has joined #openstack-ansible15:39
odyssey4meso it seems that somehow curl wasn't there, but it should have been15:39
*** TxGirlGeek has quit IRC15:39
vnoginlet me check openstack-hosts-setup.yml playbook15:40
openstackgerritMerged openstack/openstack-ansible-plugins: Support users without projects in keystone library  https://review.openstack.org/33134315:40
*** jmckind has joined #openstack-ansible15:42
*** daneyon has joined #openstack-ansible15:42
vnoginas far as I openstack-hosts-setup.yml for the hosts and not for lxc (if I'm right)15:42
odyssey4mevnogin openstack-hosts touches all hosts, whereas lxc-hosts touches only the lxc hosts15:43
vnoginand when we talk about our issue it realted LXC_NAME conteiner, right?15:43
vnoginok checking playbook15:44
odyssey4mevnogin so if we look at the sequence an lxc host would have openstack_hosts executed against it, then lxc_hosts15:44
odyssey4mevnogin so I'm curious how you got into a state where that sequence was not honoured15:44
vnoginactually, as far as I see these playbooks applies only for hosts:15:48
vnogin TASK [openstack_hosts : Ensure ssh directory] **********************************15:48
vnoginok: [infra1]15:48
vnoginok: [compute1]15:48
vnoginok: [compute2]15:48
vnoginI don't see containers here15:48
odyssey4mevnogin yes, only hosts15:51
odyssey4mevnogin openstack-hosts applies to all hosts (not containers) where any openstack service will be installed15:52
*** v1k0d3n has joined #openstack-ansible15:52
odyssey4mevnogin lxc-hosts will apply to only lxc-hosts (those that will host containers)15:52
odyssey4mevnogin the lxc-hosts role implements the base 'cache' which is used to build the containers... but it doesn't actually build the containers15:52
odyssey4mevnogin the lxc-container-create playbook executes the actual container creation, which uses the cache15:53
*** afred312 has joined #openstack-ansible15:53
odyssey4meso the issue was not in containers at all - it was only on the lxc-hosts during the cache preparation process15:53
*** sacharya has joined #openstack-ansible15:53
vnoginodyssey4me: yep, agree.15:54
vnoginhowever one thing that needs to be done - we need install curl in LXC_NAME container on all hosts and it will work. I don't see see it when I start LXC_NAME when I attach it15:56
odyssey4mevnogin LXC_NAME should never get started - only the rootfs gets used15:57
odyssey4mevnogin hmm, wait - so when you installed curl on the host, the cache prep still failed?15:57
*** chris__hultin is now known as chris_hultin15:57
odyssey4mevnogin but you installed curl into the container (when chrooting or starting it) and then the apt-get update worked?15:57
vnoginroot@cz7828:~# lxc-attach -n LXC_NAME15:58
vnoginroot@LXC_NAME:~# dpkg -l | grep curl15:58
vnoginroot@LXC_NAME:~#15:58
vnoginyep, I 've started it manually and I know that we don't need to start it :)15:58
odyssey4meyeah, that makes sense - because curl is one of the packages it needs: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/vars/ubuntu-14.04.yml#L8015:58
vnoginit was started only to check that we don't have curl there15:58
*** KLevenstein_ has joined #openstack-ansible15:58
odyssey4mevnogin so the actual issue here is that the apt-get update failed, causing the apt-get install to never execute15:59
vnoginand if I install it - playbooks works fine after that and we don't need fixes that you have done :)15:59
vnoginyep, exactly15:59
odyssey4mehmm... I wonder what the best course of action is here16:00
odyssey4meobviously we can do 'apt-get update || true' but that's not ideal16:00
*** v1k0d3n has quit IRC16:00
evrardjpdear cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, automagically, spotz, vdo, jmccrory, alextricity25, jasondotstar, KLevenstein, admin0, michaelgugino, ametts, v1k0d316:00
evrardjpn, severion, bgmccollum, darrenc, JRobinson__, asettle, colinmcnamara, thorst, adreznec :16:00
evrardjpthe osa bug triage is about to start, Please  have a look at the bug list https://etherpad.openstack.org/p/osa-bugtriage16:01
*** KLevenstein has quit IRC16:01
*** KLevenstein_ is now known as KLevenstein16:01
evrardjpthank you16:01
evrardjpA first word before starting, this is an attempt to work differently, and prepare the bug triage as efficiently as possible16:02
evrardjpI'll try this format a few times, and I guess we can discuss about it later (community meeting/summit/...)16:02
vnoginodyssey4me: don't know... but it doesn't work properly in the current state :)16:02
evrardjpthe idea is to discuss about the bugs during the bug triage, and have more preparation16:02
odyssey4mevnogin I'll update the review - I think I have a plan.16:02
odyssey4mebug triage now16:03
evrardjpthis time the preparation came too late, so I'm sorry16:03
evrardjpanyway16:03
evrardjplet's start16:03
*** v1k0d3n has joined #openstack-ansible16:03
evrardjphere is the first one: https://bugs.launchpad.net/openstack-ansible/+bug/159458416:03
openstackLaunchpad bug 1594584 in openstack-ansible "Kilo nova extra migrations may not have successfully completed" [Undecided,New]16:03
vnoginodyssey4me: ok, cool, if you need my help, I'm ready :))16:03
d34dh0r53o/16:04
*** v1k0d3n has quit IRC16:04
-openstackstatus- NOTICE: Gerrit is being restarted now to apply an emergency security-related configuration change16:04
*** TxGirlGeek has joined #openstack-ansible16:04
jmccroryworked on another kilo->liberty upgrade last night and seems even force didnt catch everything needed, had to manually update db rows16:04
evrardjp:(16:05
evrardjpdo you think we should have a pre-check in Kilo?16:05
jmccroryliberty?16:05
openstackgerritMajor Hayden proposed openstack/openstack-ansible-rsyslog_server: Add CentOS 7 support  https://review.openstack.org/33222916:05
evrardjpand another question, does nova-manage now correctly returns non 0 on non-successful execution ?16:06
evrardjpliberty16:06
jmccroryit has to be done pre-liberty nova db sync16:06
evrardjpI meant liberty16:06
jmccroryno, its just a print statement so always 016:06
jmccrorycan get some more details in ticket in about 20 min16:06
evrardjpit's still like that, even on more recent versions of nova-manage?16:07
evrardjpthat's seems highly critical across branches then16:07
evrardjpwe rely on something that doesn't work as we expected16:07
jmccrorymigrate flavor data is only in kilo, and needs to be complete before dropping any liberty nova code16:07
evrardjpjmccrory: yes I understood that part, I was wondering if they were other parts that fall into the same category16:08
jmccrorysince the function is removed in liberty's nova-migtate16:08
jmccroryoh no sure16:08
*** mikelk has quit IRC16:08
evrardjpok16:08
jmccrorybut ran into this 2/2 clouds upgrading so far16:08
evrardjpso the impact could be less16:08
openstackgerritMajor Hayden proposed openstack/openstack-ansible-rsyslog_server: Add CentOS 7 support  https://review.openstack.org/33222916:08
evrardjpbecause you can get more details later, I propose to let the status as it is for now. Are you ok ?16:09
jmccroryyep16:09
*** adrian_otto has joined #openstack-ansible16:09
evrardjpnext one is: https://bugs.launchpad.net/openstack-ansible/+bug/159324716:10
openstackLaunchpad bug 1593247 in openstack-ansible "IPv6 breaks HA router because of keepalived version" [Undecided,New]16:10
evrardjpmy opinion was new - medium/high16:10
evrardjpHas someone confirmed this?16:11
evrardjpI think keepalived 1.2.7 is bad anyway...16:11
evrardjpFor v616:11
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts: Improve LXC cache preparation process  https://review.openstack.org/33218016:13
*** tiagogomes has left #openstack-ansible16:13
evrardjpNothing to say, I'll leave the status as it is for now - until someone talks16:15
evrardjpNext: https://bugs.launchpad.net/openstack-ansible/+bug/159307416:15
openstackLaunchpad bug 1593074 in openstack-ansible "Horizon Load Balancer V2 menu not showing" [Undecided,New]16:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts: Improve LXC cache preparation process  https://review.openstack.org/33218016:16
evrardjpwinggundamth: this was yours right?16:16
jmccroryhttps://review.openstack.org/#/c/319948/ looks like it added a var for that dashboard, change was master only and don't know if it's available in mitaka16:17
jmccrorymhayden? ^16:18
odyssey4mejmccrory evrardjp sorry to go back a bug - that migration issue, try and find out which kilo version they had in place... I think the flavor migration may have been implemented late in the kilo cycle or perhaps even in the liberty cycle when they figured something was missing16:18
mhaydenjmccrory: that's on my list of to-do's for today :)16:18
evrardjpthe doc is definitely not enough for this bug16:18
odyssey4mejmccrory so you may find the solution can be to upgrade to the latest kilo before upgrading to liberty16:19
evrardjpmhayden: so you confirm the issue, and I can assign it to you?16:19
jmccroryodyssey4me: think it was last or second to last tag, the migrate itself didn't do everything it needs to16:19
mhaydenevrardjp: i haven't confirmed that i see the same bug, but i plan to look at it today16:19
odyssey4mejmccrory ah, that sucks :/16:19
evrardjpok I'll document this mhayden16:20
jmccroryyeah...lot of late night troubleshooting. will put more details in ticket16:20
evrardjpthanks jmccrory and mhayden16:20
evrardjp(for the separate things, but the effort is done, that always deserve a large thank you!)16:21
evrardjpnext one is: https://bugs.launchpad.net/openstack-ansible/+bug/159250916:21
openstackLaunchpad bug 1592509 in openstack-ansible "Tempest fails to generate test list" [Undecided,New]16:21
*** tomjosekal has joined #openstack-ansible16:22
evrardjpmy opinion was: Confirmed - Medium/Low16:22
evrardjpI've seen that in the past, and so does hughsaunders16:22
odyssey4meyeah, we've stopped updating tempest in the stable branches because of this16:23
evrardjpA backport is needed for other branches16:23
odyssey4medoing that is introducing risks for us16:23
odyssey4meI think we'll need to actually fix this in liberty and mitaka16:23
odyssey4mekilo we leave alone16:23
vnoginodyssey4me: https://review.openstack.org/332180 it fixed the issues that I faced. Thanks.16:23
evrardjpThis is a good aproach16:23
hughsaundersodyssey4me: so backport to liberty and mitaka?16:23
evrardjpodyssey4me: you mean fixing without backporting right?16:24
odyssey4mehughsaunders yeah - my only concern here is that that one review may not resolve it all - but it's only tempest we're touching, I hope16:24
odyssey4meevrardjp no, I mean we need to backport this to the stable branches to actually fix it so that we can update the tempest sha's16:24
evrardjpIt's now clear16:25
*** KLevenstein has quit IRC16:25
odyssey4meas hughsaunders has mentioned in the bug comments, it seems to have been fixed through mattt's excellent work in master... but it now needs to be ported back to make it work for the other branches too16:25
odyssey4mehughsaunders would you be able to take that on?16:25
*** adrian_otto has quit IRC16:25
*** adrian_otto has joined #openstack-ansible16:25
hughsaundersodyssey4me: if backports will be accepted, I'll do them.16:26
odyssey4me.doit()16:26
evrardjp:D16:26
hughsaundersodyssey4me: I know mattt wasn't keen on backporting originally which is why it stayed in master onlyu16:26
evrardjpI'm triaging this them16:26
evrardjpthen*16:26
*** tomjosekal has quit IRC16:26
odyssey4mehughsaunders as long as the changes stay isolated to tempest and gate testing, then I don't see an issue with it16:26
odyssey4meGet it done for Mitaka first though - then we can look at Liberty.16:27
*** tomjosekal has joined #openstack-ansible16:27
odyssey4meie let's iron out all the bits needed first, then we know what we're in for and can make an informed decision for Liberty16:27
*** phalmos has quit IRC16:28
odyssey4mevnogin can you please vote & comment in the review to indicate that it worked for you?16:28
*** KLevenstein has joined #openstack-ansible16:29
evrardjpwhat about the importance? Should we decide it's medium?16:29
odyssey4meevrardjp I'd say high16:29
odyssey4methis is impacting the integrity of our tests16:29
evrardjpIf it's impacting the gates, it could even be considered as critical16:30
evrardjplet's say high16:30
odyssey4meyeah, but it's not a blocker right now16:30
evrardjpbecause the gates aren't broken16:30
odyssey4meyep16:30
evrardjpfine for me16:30
odyssey4mejust a little crippled16:30
evrardjpnext one: https://bugs.launchpad.net/openstack-ansible/+bug/158524316:30
openstackLaunchpad bug 1585243 in openstack-ansible "Kilo upgrade to Liberty died on repo build process" [Undecided,New] - Assigned to Cameron Loader (4-cameron)16:30
*** KLevenstein has quit IRC16:31
odyssey4meit would seem that cloader89 has confirmed it, so we can at least mark it as confirmed16:31
evrardjpthe commit we talked about last week doesn't ensure we have a fix for the bug until it merges16:31
evrardjpI agree16:32
mhaydencloader89 is in training at the moment16:32
odyssey4meI'm not yet sure that we have a proper triage done for it though - we don't know the root cause, but we do know a workaround to fix it16:32
jmccrorydidn't run into that, but it probably comes down to what path of tags is upgrade is done through16:32
odyssey4mejmccrory yeah, probably16:32
evrardjptoo bad bjoern isn't here16:32
odyssey4meit seems to have something to do with cruft left behind16:32
odyssey4meit's very possibly due to package version changes not properly removing bits16:33
odyssey4mebut it might also be bad packages in the repo itself16:33
evrardjpA rm of the /var/www/repo shoul dbe fine16:33
evrardjp:D16:33
odyssey4meevrardjp only if the offending bits are in the repo16:33
evrardjptrue16:33
evrardjpthat's what I read with the bug16:34
odyssey4meit seems more like the offending bits are installed into {system,site}-packages16:34
odyssey4methere's another similar bug which has more about that and I think they may be related16:34
vnoginodyssey4me: done, tnx again :)16:34
odyssey4methanks vnogin !16:34
cloader89I have proposed a fix -- will add more details16:34
evrardjpin any case we have an idempotency or a consistency issue for the repo16:34
evrardjpcloader89: thanks for the fix, could you add the Closes-Bug into the commit message too?16:35
*** tomjosekal has quit IRC16:35
evrardjpI'd be happy for the automatic triaging16:35
evrardjpWhat is the importance of this bug?16:36
cloader89evrardjp: I am in the training as mhayden said. I will look at it again in a bit, but sigmavirus24 and I were discussing final thoughts on it.16:37
evrardjpgood! thanks cloader8916:37
evrardjpsorry to interrupt you during your training :p16:37
odyssey4meevrardjp add a comment to the review to include that16:37
cloader89I just did a repo destroy/create16:38
cloader89as the fix16:38
cloader89since it was an issue among dependencies16:38
evrardjpok16:39
evrardjplet's set that to high priority, because it prevents the upgrade to any deployer following the same path ?16:39
d34dh0r53+116:40
evrardjpok next:16:40
evrardjphttps://bugs.launchpad.net/openstack-ansible/+bug/157991916:40
openstackLaunchpad bug 1579919 in openstack-ansible "msgpack pip issue after Juno to Kilo upgrade" [Undecided,New] - Assigned to Andrew Meserole (andrew-meserole)16:40
openstackgerritChristopher Hultin proposed openstack/openstack-ansible-os_magnum: Adding 'admin_user' to config file  https://review.openstack.org/33216916:43
evrardjpHas someone tried to follow the path that Bjoern had?16:44
evrardjpIf not, I think we should wait for Andrew Meserole to at least confirm the issue path16:44
evrardjpsigmavirus24: are you there?16:45
sigmavirus24evrardjp: hello16:45
evrardjpsigmavirus24: did you confirm this bug before assigning to Andrew?16:46
sigmavirus24Part of Andrew's task was to confirm the bug16:47
evrardjpok so I'm not mistaken16:47
evrardjpgood16:47
evrardjpI'll leave the status as is then16:47
*** saneax_AFK is now known as saneax16:47
evrardjpok next one: https://bugs.launchpad.net/openstack-ansible/+bug/158380416:48
openstackLaunchpad bug 1583804 in openstack-ansible "Security role should require an email to receive root's mail" [Wishlist,New]16:48
*** sacharya has quit IRC16:51
*** messy has quit IRC16:51
evrardjpmhayden: are you there?16:52
odyssey4memhayden why not just warn the user instead of halting the playbook16:52
odyssey4meor halting the task16:52
evrardjpor having a prompt?16:52
evrardjpcontinue y/n16:53
*** catintheroof has joined #openstack-ansible16:53
mhaydeni'm here16:53
* mhayden was buried in neutronland16:53
mhaydeni thought about having a preflight check for that16:53
odyssey4meyeah, if it's a hard requirement then a pre-flight check is nice16:53
mhaydenbut i don't want to it to derail someone when they run setup-hosts.yml16:54
odyssey4meanother option is to make the task prompt the user for input as evrardjp has suggested16:54
odyssey4meI wouldn't mind actually having some sort of pre-flight check framework...16:54
evrardjpor expect for ansible 2.x16:54
evrardjpI think pre-flight is a good idea, but I guess we can say this is a confirmed issue, and I can assign that to mhayden?16:55
*** cloader89 has quit IRC16:55
odyssey4memaybe a playbook that does a lookup in all the roles and finds the necessary things that need to be set based on something we can host in the repo... or even better dynamically determine.16:55
odyssey4mebut that's a story for another day16:55
*** tomjosekal has joined #openstack-ansible16:57
evrardjpI agree it's a story for another day16:58
*** tomjosekal has quit IRC16:58
evrardjpDo we keep the status as new?16:59
*** tomjosekal has joined #openstack-ansible16:59
*** dmsimard is now known as dmsimard|afk16:59
d34dh0r53if we agree that having an email is a requirement we can set it to confirmed16:59
evrardjpI'll move it as confirmed, because we know the possible issue, and it's verified in the code16:59
*** adrian_otto has quit IRC17:00
evrardjpimportance is already on whishlist, which is fine for me17:00
evrardjpok it's time to end this17:00
d34dh0r53yep17:00
evrardjplet's continue this next week17:00
evrardjpI hope I'll have more time to prepare17:00
d34dh0r53thanks evrardjp17:01
evrardjpthanks everyone for attending!17:01
evrardjpor would the word "assisting" be good here?17:02
*** TxGirlGeek has quit IRC17:06
automagically /me also buried in neutron land mhayden17:08
automagicallyI feel your pain17:08
*** BjoernT has joined #openstack-ansible17:10
spotznice job evrardjp17:10
evrardjppain? what are you talking about automagically? It's perfectly fine!17:11
evrardjpthanks spotz :D17:11
evrardjpBjoernT: hello!17:11
BjoernThey17:11
*** jmckind has quit IRC17:13
*** woodard has joined #openstack-ansible17:14
*** TxGirlGeek has joined #openstack-ansible17:15
jmccroryBjoernT odyssey4me : was able to reproduce https://bugs.launchpad.net/openstack-ansible/+bug/157991917:17
openstackLaunchpad bug 1579919 in openstack-ansible "msgpack pip issue after Juno to Kilo upgrade" [Undecided,New] - Assigned to Andrew Meserole (andrew-meserole)17:17
BjoernTjmccrory: thanks people probably thought I'm crazy with all the pip issue I see17:18
*** galstrom_zzz is now known as galstrom17:19
jmccroryheh was doing kilo-liberty upgrade all last night, pip issues are a nightmare17:20
jmccroryhad to destroy keystone containers17:20
*** jayc has quit IRC17:22
*** psilvad has joined #openstack-ansible17:22
BjoernTyeah those are my current workarounds too, going that far that I delete the repo containers and remove the contents of the bind mounts. Otherwise the repo build fails at 100% for me17:22
*** electrofelix has quit IRC17:23
*** sdake_ has joined #openstack-ansible17:24
jmccrorythink its the move from installing on system to venvs, and at some point should probably only support venv installs17:26
*** woodard has quit IRC17:26
*** sdake has quit IRC17:27
*** woodard has joined #openstack-ansible17:27
*** sacharya has joined #openstack-ansible17:29
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_nova: Replace network_api_class with use_neutron  https://review.openstack.org/33225917:29
*** jayc has joined #openstack-ansible17:34
*** mkrish004c has joined #openstack-ansible17:34
*** pcaruana has joined #openstack-ansible17:40
*** phalmos has joined #openstack-ansible17:40
*** sacharya has quit IRC17:41
*** arbrandes has quit IRC17:41
*** stack_ has joined #openstack-ansible17:43
odyssey4mejmccrory yeah, I'd actually like to remove the option of not using a venv this cycle17:44
odyssey4mewe should probably do a blueprint for that so that we can easily track the commits17:44
odyssey4mejmccrory automagically evrardjp a question regarding https://review.openstack.org/332180 ...17:45
automagicallyfire away, what’s the question17:45
odyssey4mecurrently the patch implements || true on apt-get update to cater for the situation where an apt repo can't be updated due to a missing package in the cache17:46
automagicallyyep, see that17:46
odyssey4meI think it might be better to remove /etc/apt/sourced.list/d from the copy list instead, so that only /etc/apt/sources.list is used in the cache prep.17:46
odyssey4methoughts?17:46
odyssey4meI'm not convinced that ignoring a failed apt-get update is a good idea.17:47
automagicallyMe either17:47
automagicallyAnd only taking some of the hosts apt sources seems a bit counter-intuitive at best17:47
odyssey4meAnd if a deployer does have extra sources they *want* in the cache then they are able to use https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/tasks/lxc_cache_preparation.yml#L39 to ensure that the desired extras are copied17:48
openstackgerritTom Cameron proposed openstack/openstack-ansible-os_neutron: Fix executor_thread_pool_size typo  https://review.openstack.org/33226617:49
odyssey4meyeah, that's a conundrum... until vnogin hit the issue with the mariadb repo though I didn't know that this was a possible problem17:49
odyssey4meeffectively the issue is that some apt repositories require curl to work, and curl is not installed in the downloaded base image17:49
*** itlinux has quit IRC17:49
odyssey4meI suppose I could propose that it is included in the base image upstream in the lxc repo, but it seems like a growing bloat. :/17:49
stack_hi all, i am running OSA liberty, when i run the play book json file is generated and throwing the attached error --> http://pastebin.com/XuBFWKjY17:51
odyssey4mepalendae ^ maybe you can help stack_ ?17:52
odyssey4mestack_ it's likely that your openstack_user_config is broken in some way... we've added a ton of protections in master, but they're pretty radical changes so they haven't been backported17:53
automagicallyodyssey4me: Dropped a comment on the review with what I think I’m most comfortable with17:53
*** messy has joined #openstack-ansible17:54
*** daneyon has quit IRC17:54
stack_odyssey4me, oh then i will validate my openstack_user_config.yml if there is any syntax error.17:54
*** daneyon has joined #openstack-ansible17:54
mrhillsmanstack_ is your /opt/openstack_deploy/user_variables.yml file all commented out?17:56
mrhillsmani will need to check mine again but iirc, i had that error when i left it untouched17:56
evrardjpfirst if curl is a dependency it should be listed as so :p17:56
odyssey4meevrardjp it is a package which gets installed into the cache if the indexes are up to date17:57
*** saneax is now known as saneax_AFK17:58
stack_mrhillsman, no i have haproxy and proxy url information there, both user_config.yml and user_variable.yml is validated. Non syntax error17:58
mrhillsmanok cool17:58
palendaestack_, what happens if you run `playbooks/inventory/dynamic_inventory.py --config /etc/openstack_deploy`?17:59
*** arbrandes has joined #openstack-ansible18:00
palendaeYou should get JSON output18:00
palendaeAlso, is /etc/openstack_deploy/openstack_inventory.json present?18:00
*** adrian_otto has joined #openstack-ansible18:01
palendaeSeems the inventory might not be generating consumable JSON18:01
*** admin0 has joined #openstack-ansible18:02
evrardjpheading home, will be back later online18:02
pabelangerodyssey4me: how is the amount of rechecks needed today?18:03
odyssey4meautomagically I've updated https://review.openstack.org/332180 with our agreed better option.18:03
stack_palendae, i have json file, it is generating JSON file if i remove the a;ready existing one..18:03
palendaestack_, Ok, and it should provide JSON to the stdout when you run it directly18:04
*** javeriak_ has quit IRC18:04
odyssey4mepabelanger I've had to do a few, and I've seen automagically do one or two too. I've been a little tied up in other activities.18:04
palendaeDon't remove the existing one18:04
palendaeAnsible actually wants the JSON output from stdout. the JSON file is written as a cache18:04
*** albertcard has joined #openstack-ansible18:04
*** admin0 has quit IRC18:06
*** BjoernT has quit IRC18:07
rackertomsorry for the branch name mess up odyssey4me18:07
odyssey4merackertom no worries :) thanks for the bug fix :)18:09
*** admin0 has joined #openstack-ansible18:11
*** adrian_otto has quit IRC18:14
*** woodard has quit IRC18:15
*** woodard has joined #openstack-ansible18:15
*** pacoblox has quit IRC18:15
*** pacoblox has joined #openstack-ansible18:15
*** gregfaust has quit IRC18:16
*** phalmos has quit IRC18:16
openstackgerritAshana Evans proposed openstack/openstack-ansible-os_cinder: [WIP] These changes updates the cinder role to support Ubuntu 16.04 and Systemd  https://review.openstack.org/33218918:16
*** tomjosekal has quit IRC18:17
*** javeriak has joined #openstack-ansible18:19
*** phalmos has joined #openstack-ansible18:19
*** pcaruana has quit IRC18:22
*** Drago has joined #openstack-ansible18:25
*** Drago has quit IRC18:26
*** adrian_otto has joined #openstack-ansible18:26
*** Drago has joined #openstack-ansible18:26
*** adrian_otto has quit IRC18:32
*** BjoernT has joined #openstack-ansible18:32
*** M00nr41n has joined #openstack-ansible18:33
*** cloader89 has joined #openstack-ansible18:36
mhaydenwent from 13.1.1 > 13.1.3 without any issues18:36
* mhayden struts18:36
cloader89evrardjp: I added the reference to the bug from the fix18:36
evrardjpcloader89: cool thanks!18:38
cloader89verify it was done correctly?18:38
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rsyslog_server: Add CentOS 7 support  https://review.openstack.org/33222918:38
*** adrian_otto has joined #openstack-ansible18:39
evrardjpsure I'll do that right away18:39
cloader89evrardjp: ty18:39
*** phalmos has quit IRC18:40
*** sacharya has joined #openstack-ansible18:41
*** rg_ has joined #openstack-ansible18:42
*** jmckind has joined #openstack-ansible18:42
*** adrian_otto has quit IRC18:42
evrardjpcloader89: did you submit another patchset?18:44
*** admin0 has quit IRC18:44
cloader89No, I modified the commit message to include the other bug. I will try to get one out later today18:45
*** adrian_otto has joined #openstack-ansible18:45
cloader89evrardjp:18:45
openstackgerritCameron Loader proposed openstack/openstack-ansible: Destroy repo containers before upgrade  https://review.openstack.org/32848518:46
*** sacharya has quit IRC18:47
openstackgerritChristopher Hultin proposed openstack/openstack-ansible-os_magnum: Adding repo_magnum.yml  https://review.openstack.org/33242618:48
openstackgerritMerged openstack/openstack-ansible-rsyslog_client: Add human readable logging callback plugin for func testing  https://review.openstack.org/33174418:48
*** KLevenstein has joined #openstack-ansible18:48
openstackgerritNolan Brubaker proposed openstack/openstack-ansible: Address low-hanging TODO items  https://review.openstack.org/32909818:49
*** admin0 has joined #openstack-ansible18:49
*** catintheroof has quit IRC18:51
*** catintheroof has joined #openstack-ansible18:51
*** adrian_otto has quit IRC18:51
*** weezS has quit IRC18:51
*** galstrom is now known as galstrom_zzz18:51
*** woodard has quit IRC18:53
mhaydenso i keep having an issue in mitaka where multiple ports are attached to an instance after it builds18:54
mhaydenfor example, i build an instance with one network attached, but four ports from that network get attached to the VM18:55
mhaydenso it gets four IP addresses18:55
bgmccollumthats the new HA mode :P18:55
evrardjplol18:55
evrardjpLA you mean18:55
mhaydend34dh0r53 suggested that it might be a broken rabbit queue perhaps, but that doesn't seem to be the case18:55
mhaydenit looks as if all four of my compute nodes are asking neutron for ports18:55
*** sdake has joined #openstack-ansible18:56
d34dh0r53mhayden: it's not rabbit?18:56
mhaydend34dh0r53: nah, rabbit looks okay18:56
odyssey4memhayden an easter egg?18:56
d34dh0r53mhayden: liar18:56
evrardjpmhayden: what do you see in nova?18:56
evrardjpor libvirt18:56
mhaydenevrardjp: nova looks fine -- but i see four POST requests to neutron-api18:56
mhaydenone POST per compute node18:56
mhaydenit's almost like all four computes are picking up the same message and asking neutron for a port18:57
*** javeriak has quit IRC18:57
evrardjpthat looks like a race condition and/or a scheduling issue18:57
mhaydenyeah, it's highly funky18:58
evrardjpbut I'm no expert on auto multinic :p18:58
*** admin0 has quit IRC18:59
*** sdake_ has quit IRC18:59
*** admin0 has joined #openstack-ansible18:59
openstackgerritMerged openstack/openstack-ansible-os_neutron: Correct Openvswitch bridge_mappings configuration  https://review.openstack.org/33213919:01
*** daneyon has quit IRC19:04
*** daneyon has joined #openstack-ansible19:04
*** rg_ has quit IRC19:05
*** johnmilton has quit IRC19:07
*** admin0 has quit IRC19:09
*** admin0 has joined #openstack-ansible19:10
*** weezS has joined #openstack-ansible19:11
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-memcached_server: Add human readable logging callback plugin for func testing  https://review.openstack.org/33244319:12
openstackgerritAshana Evans proposed openstack/openstack-ansible-os_cinder: [WIP] Updates the cinder role to support Ubuntu 16.04 and Systemd  https://review.openstack.org/33218919:13
*** adrian_otto has joined #openstack-ansible19:13
*** adrian_otto has quit IRC19:14
*** dmsimard|afk is now known as dmsimard19:15
*** adrian_otto has joined #openstack-ansible19:20
*** admin0 has quit IRC19:22
*** admin0 has joined #openstack-ansible19:22
*** adrian_otto has quit IRC19:23
*** permalac_ has joined #openstack-ansible19:27
*** deverter has joined #openstack-ansible19:29
*** jimbaker has quit IRC19:29
*** deverter_ has quit IRC19:29
*** arif-ali has quit IRC19:29
*** ioni has quit IRC19:30
*** permalac has quit IRC19:31
*** ioni has joined #openstack-ansible19:32
*** pacoblox has quit IRC19:33
*** ioni is now known as Guest2389919:33
stack_@palendae, i am not removing the json, i tried moving json file from that folder and checked. It is re creating. still the same error --> http://pastebin.com/XuBFWKjY19:33
palendaestack_, Did you try https://twitter.com/palendae/status/745295674728656897 ?19:34
palendaeEr19:34
palendaeRun `playbooks/inventory/dynamic_inventory.py --config /etc/openstack_deploy19:34
*** jimbaker has joined #openstack-ansible19:34
*** jimbaker has joined #openstack-ansible19:34
palendaeRunning that should produce JSON output on stdout19:35
*** arif-ali has joined #openstack-ansible19:35
d34dh0r53json is sort of like lizard speech19:36
*** phalmos has joined #openstack-ansible19:38
*** M00nr41n has quit IRC19:41
stack_@palendae, getting this error dynamic_inventory.py: error: unrecognized arguments: --config /etc/openstack_deploy19:42
palendaestack_, What version of openstack-ansible are you using?19:42
stack_@palendae, using 12.0.9, liberty19:43
palendaeI thought that changed already in liberty...but try --file /etc/openstack_deploy instead19:43
stack_palendae, yeah this is giving a JSON out,19:45
palendaeHm, ok19:45
palendaeSo it appears you're getting inventory generated correctly19:45
*** jayc has quit IRC19:46
stack_palendae, yeah i can see it is recreating if i move the file out of that directory. but when i run the basic openstack_setup_host.yml file still the same error19:48
*** admin0 has quit IRC19:48
palendaeAh,looks like it's fact caching19:50
palendaehttps://github.com/ansible/ansible/blob/v1.9.4-1/lib/ansible/cache/jsonfile.py#L65 is the error line19:51
palendaeShould be able to delete /etc/openstack_deploy/ansible_facts19:53
*** jmckind has quit IRC19:57
*** admin0 has joined #openstack-ansible19:58
*** pacoblox has joined #openstack-ansible19:59
*** jmckind has joined #openstack-ansible20:03
*** Zucan has quit IRC20:07
stack_@palendae, thanks a lot. just moved ansible_facts form that directory and i am able to proceed :) thanks20:08
palendaestack_, Welcome!20:08
openstackgerritLogan V proposed openstack/openstack-ansible: Allow Neutron deployment without overlay  https://review.openstack.org/33246220:15
*** vnogin_ has joined #openstack-ansible20:22
*** vnogin_ has left #openstack-ansible20:23
*** galstrom_zzz is now known as galstrom20:25
*** markvoelker has quit IRC20:27
*** smatzek has quit IRC20:28
*** woodard has joined #openstack-ansible20:29
*** vnogin_ has joined #openstack-ansible20:34
*** vnogin_ has quit IRC20:36
*** spotz is now known as spotz_zzz20:38
*** weezS has quit IRC20:41
*** weezS has joined #openstack-ansible20:43
*** Mudpuppy_ has joined #openstack-ansible20:46
*** Mudpuppy has quit IRC20:49
*** Mudpuppy_ has quit IRC20:50
*** cloader89 has quit IRC20:54
*** woodard has quit IRC21:00
*** messy has quit IRC21:00
*** woodard_ has joined #openstack-ansible21:00
*** messy has joined #openstack-ansible21:02
*** KLevenstein has quit IRC21:03
*** admin0 has quit IRC21:04
*** KLevenstein has joined #openstack-ansible21:04
*** admin0 has joined #openstack-ansible21:05
*** KLevenstein has quit IRC21:05
*** KLevenstein has joined #openstack-ansible21:07
*** psilvad has quit IRC21:08
*** schwicht has joined #openstack-ansible21:14
*** stack_ has quit IRC21:19
*** schwicht has quit IRC21:20
*** spotz_zzz is now known as spotz21:21
*** weezS has quit IRC21:21
*** markvoelker has joined #openstack-ansible21:21
*** schwicht has joined #openstack-ansible21:26
*** woodard_ has quit IRC21:30
*** woodard has joined #openstack-ansible21:30
*** schwicht has quit IRC21:31
*** KLevenstein has quit IRC21:31
*** KLevenstein has joined #openstack-ansible21:31
*** admin0 has quit IRC21:31
*** mkrish004c has quit IRC21:39
*** markvoelker has quit IRC21:39
*** jamesdenton has quit IRC21:42
*** admin0 has joined #openstack-ansible21:42
*** sacharya has joined #openstack-ansible21:44
*** sacharya has quit IRC21:49
*** markvoelker has joined #openstack-ansible21:50
*** catintheroof has quit IRC21:51
*** jmckind has quit IRC21:52
*** ametts has quit IRC21:54
*** deverter has quit IRC21:56
*** v1k0d3n has joined #openstack-ansible22:13
*** severion has joined #openstack-ansible22:14
*** phalmos has quit IRC22:14
*** v1k0d3n has quit IRC22:18
*** kstev has quit IRC22:22
*** javeriak has joined #openstack-ansible22:28
*** woodard has quit IRC22:30
*** woodard has joined #openstack-ansible22:31
darrencgood morning22:31
*** admin0 has quit IRC22:31
openstackgerritMerged openstack/openstack-ansible-os_neutron: Remove enable_tunneling config from openvswitch_agent.ini  https://review.openstack.org/33215722:33
*** markvoelker has quit IRC22:36
*** admin0 has joined #openstack-ansible22:37
*** KLevenstein has quit IRC22:38
*** rcarrillocruz has quit IRC22:39
*** rcarrillocruz has joined #openstack-ansible22:40
*** severion has quit IRC22:45
*** kylek3h has quit IRC22:46
*** weezS has joined #openstack-ansible22:49
*** galstrom is now known as galstrom_zzz22:50
*** messy has quit IRC22:55
*** javeriak has quit IRC23:01
*** admin0 has quit IRC23:05
*** TxGirlGeek has quit IRC23:07
*** spotz is now known as spotz_zzz23:10
*** weezS has quit IRC23:10
*** weezS has joined #openstack-ansible23:11
*** itlinux has joined #openstack-ansible23:14
*** daneyon has quit IRC23:14
*** daneyon has joined #openstack-ansible23:15
*** saneax_AFK is now known as saneax23:20
*** weezS has quit IRC23:21
*** Drago has quit IRC23:23
*** Drago has joined #openstack-ansible23:23
*** woodard has quit IRC23:24
*** scarlisle has quit IRC23:26
*** Drago has quit IRC23:28
openstackgerritNeill Cox proposed openstack/openstack-ansible-os_horizon: [WIP] Enable Horizon Ironic dashboard plugin  https://review.openstack.org/33250723:29
openstackgerritNeill Cox proposed openstack/openstack-ansible: Add Horizon Ironic dashboard plugin  https://review.openstack.org/33250823:31
openstackgerritNeill Cox proposed openstack/openstack-ansible: Add Horizon Ironic dashboard plugin  https://review.openstack.org/33250823:32
openstackgerritNeill Cox proposed openstack/openstack-ansible: [WIP] Add Horizon Ironic dashboard plugin  https://review.openstack.org/33250823:33
*** johnmilton has joined #openstack-ansible23:37
*** klamath has quit IRC23:44
*** sacharya has joined #openstack-ansible23:45
*** BjoernT has quit IRC23:46
*** sacharya has quit IRC23:49

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!