derekh | bandersnatch is now running again, so everything should start working in a while | 00:06 |
---|---|---|
derekh | only a ssh key from the rh1 rack was on the bandersnatch host, which makes logging in difficult (unless you know what key to use), slagle and dprince I've added your keys as I had them to hand | 00:09 |
*** Marga_ has quit IRC | 00:10 | |
*** Marga_ has joined #tripleo | 00:10 | |
*** derekh has quit IRC | 00:12 | |
*** chlong_ has joined #tripleo | 00:20 | |
*** david-lyle has quit IRC | 00:28 | |
*** adrianopetrich has quit IRC | 00:34 | |
*** rwsu has quit IRC | 00:54 | |
*** cwolferh has quit IRC | 01:10 | |
*** eghobo has quit IRC | 01:19 | |
*** mestery has quit IRC | 01:45 | |
*** fvollero has quit IRC | 02:14 | |
*** shardy_z has quit IRC | 02:16 | |
*** tserong has quit IRC | 02:22 | |
*** yamahata has quit IRC | 02:39 | |
*** tserong has joined #tripleo | 02:42 | |
*** spzala has quit IRC | 02:42 | |
*** weshay has quit IRC | 02:46 | |
*** sdake has quit IRC | 02:51 | |
*** tserong has quit IRC | 02:52 | |
*** tserong has joined #tripleo | 02:53 | |
*** tserong has joined #tripleo | 02:53 | |
*** untriaged-bot has joined #tripleo | 03:00 | |
untriaged-bot | Untriaged bugs so far: | 03:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1452400 | 03:00 |
openstack | Launchpad bug 1452400 in diskimage-builder "debian-minimal builds still busted, now on locale issues" [Undecided,New] | 03:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449852 | 03:00 |
openstack | Launchpad bug 1449852 in diskimage-builder "Buidling ramdisk with ironic-agent behind proxy fails" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 03:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449854 | 03:00 |
openstack | Launchpad bug 1449854 in diskimage-builder "Ironic agent ramdisk built using disk-image-create fails with iscsi_ilo driver" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 03:00 |
untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1452752 | 03:00 |
openstack | Launchpad bug 1452752 in tuskar "keystone_authtoken section is wrong in default shipped tuskar.conf.sample" [Undecided,Confirmed] | 03:00 |
*** untriaged-bot has quit IRC | 03:00 | |
*** yamahata has joined #tripleo | 03:03 | |
*** chlong has quit IRC | 03:05 | |
*** panda has quit IRC | 03:13 | |
*** panda has joined #tripleo | 03:14 | |
*** links has joined #tripleo | 03:30 | |
*** stendulker has joined #tripleo | 03:33 | |
*** alop has quit IRC | 03:47 | |
*** david-lyle has joined #tripleo | 03:48 | |
*** chlong_ has quit IRC | 03:52 | |
*** eghobo has joined #tripleo | 03:55 | |
*** stendulker_ has joined #tripleo | 04:05 | |
*** stendulker has quit IRC | 04:05 | |
*** tzumainn has quit IRC | 04:10 | |
*** eghobo has quit IRC | 04:18 | |
*** stendulker has joined #tripleo | 04:31 | |
*** stendulker_ has quit IRC | 04:31 | |
*** eghobo has joined #tripleo | 04:32 | |
*** lazy_prince has joined #tripleo | 04:43 | |
*** MasterPiece has quit IRC | 04:50 | |
*** jdob1 has joined #tripleo | 05:14 | |
*** adrianopetrich has joined #tripleo | 05:14 | |
*** funzo_ has joined #tripleo | 05:14 | |
*** SlickN1k has joined #tripleo | 05:14 | |
*** killer_prince has joined #tripleo | 05:19 | |
*** yog__ has joined #tripleo | 05:21 | |
*** lazy_prince has quit IRC | 05:22 | |
*** lazy_prince has joined #tripleo | 05:23 | |
*** chlong has joined #tripleo | 05:23 | |
*** killer_prince has quit IRC | 05:24 | |
*** funzo has quit IRC | 05:26 | |
*** jdob has quit IRC | 05:26 | |
*** ir2ivps10 has quit IRC | 05:26 | |
*** SlickNik has quit IRC | 05:27 | |
*** SlickN1k is now known as SlickNik | 05:27 | |
*** dasm|afk is now known as dasm | 05:27 | |
dasm | o/ | 05:28 |
*** masco has joined #tripleo | 05:31 | |
*** chlong has quit IRC | 05:34 | |
*** ir2ivps10 has joined #tripleo | 05:38 | |
*** chlong has joined #tripleo | 05:47 | |
*** rlandy has joined #tripleo | 05:52 | |
*** lazy_prince has quit IRC | 05:55 | |
*** lazy_prince has joined #tripleo | 05:57 | |
*** ukalifon has joined #tripleo | 06:01 | |
*** ukalifon has quit IRC | 06:06 | |
*** jrist has quit IRC | 06:14 | |
*** ukalifon has joined #tripleo | 06:15 | |
*** jprovazn has joined #tripleo | 06:20 | |
*** ishant has joined #tripleo | 06:20 | |
*** chlong has quit IRC | 06:24 | |
*** aufi has joined #tripleo | 06:24 | |
*** yog__ has quit IRC | 06:24 | |
*** jcoufal has joined #tripleo | 06:25 | |
*** chlong has joined #tripleo | 06:27 | |
*** aufi has quit IRC | 06:27 | |
*** killer_prince has joined #tripleo | 06:28 | |
*** aufi has joined #tripleo | 06:28 | |
*** lazy_prince has quit IRC | 06:31 | |
*** shardy has joined #tripleo | 06:32 | |
*** killer_prince has quit IRC | 06:33 | |
*** yog__ has joined #tripleo | 06:37 | |
*** jprovazn has quit IRC | 06:39 | |
*** jprovazn has joined #tripleo | 06:41 | |
*** lazy_prince has joined #tripleo | 06:42 | |
hewbrocca | dasm: morning! | 06:46 |
hewbrocca | dasm: glad you got through installation | 06:46 |
*** mmagr has joined #tripleo | 06:56 | |
openstackgerrit | Yanis Guenane proposed openstack-infra/tripleo-ci: Fix path issue to controller-config-pacemaker https://review.openstack.org/182553 | 07:00 |
*** adrianopetrich has quit IRC | 07:01 | |
*** mmagr has quit IRC | 07:08 | |
*** mmagr has joined #tripleo | 07:10 | |
*** yog__ has quit IRC | 07:11 | |
*** panda has quit IRC | 07:13 | |
*** adrianopetrich has joined #tripleo | 07:13 | |
*** panda has joined #tripleo | 07:14 | |
*** eghobo has quit IRC | 07:21 | |
*** ifarkas has quit IRC | 07:22 | |
*** ifarkas has joined #tripleo | 07:23 | |
*** Goneri has joined #tripleo | 07:27 | |
*** adrianopetrich has quit IRC | 07:30 | |
*** Marga_ has quit IRC | 07:31 | |
*** Goneri has quit IRC | 07:32 | |
dasm | hewbrocca: yeah. i didn't find where the problem was, but it's already fixed with new packages. | 07:33 |
marios | dasm: i've been saying that for years | 07:39 |
*** jtomasek has joined #tripleo | 07:44 | |
dasm | marios: hehe... "works for me, i don't know what are you talking about" :) | 07:48 |
*** jistr has joined #tripleo | 07:49 | |
*** derekh has joined #tripleo | 08:03 | |
*** athomas has joined #tripleo | 08:06 | |
*** jayg|g0n3 has quit IRC | 08:10 | |
spredzy | jistr, shardy https://review.openstack.org/#/c/182553/ for the broken CI in pacemaker scenario | 08:11 |
jistr | spredzy: ah yes, relative paths in resource registry. good catch :) | 08:12 |
jistr | i wonder if we have to copy over the whole resource_registry map (to set all keys), or if automatic merge of the keys will happen. We'll see whether CI will be happy. | 08:14 |
*** jayg|g0n3 has joined #tripleo | 08:15 | |
spredzy | jistr, this is a question for shardy but I think the automatic merge will happen, this is the whole point of adding 'a custom heat env' | 08:16 |
*** al has quit IRC | 08:16 | |
*** al has joined #tripleo | 08:17 | |
*** lucasagomes has joined #tripleo | 08:29 | |
*** adrianopetrich has joined #tripleo | 08:38 | |
spredzy | erf ... http://repos.fedorapeople.org/repos/openstack/openstack-kilo/f21/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found | 08:43 |
jistr | spredzy: where's that from? | 08:44 |
spredzy | in the CI job trace | 08:45 |
spredzy | jistr, http://logs.openstack.org/65/177765/14/check-tripleo/check-tripleo-ironic-overcloud-f20puppet-nonha/17c0adb/console.html | 08:45 |
jistr | spredzy: btw i'm testing the galera setup locally right now | 08:46 |
jistr | spredzy: hmm that doesn't look good | 08:46 |
jistr | spredzy: i mean the CI error | 08:46 |
spredzy | jistr, ahah it gave space for quiproquo | 08:46 |
spredzy | :p | 08:46 |
spredzy | For the CI error should we ask on #rdo ? | 08:47 |
jistr | yes | 08:47 |
spredzy | done | 08:48 |
jistr | spredzy: thx. i'll keep following it there | 08:49 |
spredzy | jistr, let me know how the galera testing go :) | 08:49 |
jistr | ack | 08:49 |
spredzy | jistr, ping https://review.openstack.org/#/c/182222/. I think I will drop this review and let everyone add a review per pacemaker service s/he is trying to bring in. I wrote it as is to demostrate that it would work and for people to grasp the idea | 08:52 |
*** Goneri has joined #tripleo | 08:52 | |
spredzy | but merging the review as suche is non-sense, as the review in t-h-t are not merged, so it should be a review in t-h-t with a Depends-On a review in puppet-tripleo | 08:53 |
jistr | spredzy: yeah doing the refactoring subsequently is fine by me. It would probably reduce the friction of getting the pacemaker bits in. | 08:55 |
*** dshulyak has joined #tripleo | 08:56 | |
*** untriaged-bot has joined #tripleo | 09:00 | |
untriaged-bot | Untriaged bugs so far: | 09:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1452400 | 09:00 |
openstack | Launchpad bug 1452400 in diskimage-builder "debian-minimal builds still busted, now on locale issues" [Undecided,New] | 09:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449852 | 09:00 |
openstack | Launchpad bug 1449852 in diskimage-builder "Buidling ramdisk with ironic-agent behind proxy fails" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 09:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449854 | 09:00 |
openstack | Launchpad bug 1449854 in diskimage-builder "Ironic agent ramdisk built using disk-image-create fails with iscsi_ilo driver" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 09:00 |
untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1452752 | 09:00 |
openstack | Launchpad bug 1452752 in tuskar "keystone_authtoken section is wrong in default shipped tuskar.conf.sample" [Undecided,Confirmed] | 09:00 |
*** untriaged-bot has quit IRC | 09:00 | |
*** gfidente has joined #tripleo | 09:12 | |
*** jrist has joined #tripleo | 09:16 | |
spredzy | marios, o/ Can I ask for a vote here https://review.openstack.org/#/c/182553/ | 09:19 |
spredzy | its currently blocking the CI for the pacemaker scenario | 09:19 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add metering agent as neutron-server plugin https://review.openstack.org/179370 | 09:21 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add neutron-lbaas as neutron-server plugin & neutron-lbaas-agent https://review.openstack.org/180535 | 09:21 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds neutron-server and agents as pacemaker resources https://review.openstack.org/181015 | 09:21 |
*** mcornea has joined #tripleo | 09:26 | |
jistr | spredzy: so murphy's laws do work. While deploying the overcloud with galera, my seed node's heat stopped responding, so i wanted to redeploy everything, but even though i ran devtest with -c it still started to do something with repos and failed because of the missing f21 repo... | 09:27 |
spredzy | jistr, /o\ | 09:28 |
spredzy | bad luck ;) | 09:28 |
jistr | yeah :) | 09:28 |
spredzy | jistr, quick q. if you do a df -H do you have enough space ? | 09:29 |
spredzy | Yesterday something similar happend to me (seed suddenly stopped), and it was due to the fact that I had no more space left on device | 09:29 |
*** killer_prince has joined #tripleo | 09:33 | |
*** athomas has quit IRC | 09:34 | |
jistr | spredzy: i don't have seed now but before i tried to redeploy, i did that check :) the same thing you describe happened to me before, but it wasn't the case today | 09:35 |
jistr | /dev/sda1 30G 12G 17G 43% / | 09:35 |
*** lazy_prince has quit IRC | 09:36 | |
spredzy | ack, then not sure what happend ;) | 09:37 |
*** mmagr is now known as mmagr|afk | 09:37 | |
*** lazy_prince has joined #tripleo | 09:37 | |
marios | spredzy: looking sorry, just saw this | 09:38 |
marios | spredzy: also, thanks for review, can you please look again when you have a sec @ https://review.openstack.org/#/c/180535/11/puppet/hieradata/controller.yaml | 09:39 |
spredzy | marios, ping https://review.openstack.org/#/c/180535/11/puppet/hieradata/controller.yaml. Is neutron_lbaas.services.loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver the same thing as neutron_lbaas.services.loadbalancer.drivers.haproxy.namespace_driver:HaproxyNSDriver | 09:39 |
spredzy | heheh was just looking at it | 09:40 |
*** killer_prince has quit IRC | 09:40 | |
marios | spredzy: i believe so | 09:42 |
*** ishant has quit IRC | 09:42 | |
marios | spredzy: the point is i'd rather not set it at all | 09:42 |
*** ishant has joined #tripleo | 09:42 | |
derekh | Puppet builds have been failing for the last few hours because a rdo repository vanished, the repo is back now so new ci jobs should start to pass again from now | 09:42 |
marios | spredzy: but what i get by default doesn't work (and this is intended to be the default) | 09:43 |
marios | spredzy: you can see for example @ http://docs.openstack.org/admin-guide-cloud/content/install_neutron-lbaas-agent.html | 09:43 |
marios | spredzy: device_driver = neutron.services.loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver | 09:44 |
marios | this is a lie | 09:44 |
spredzy | hehe | 09:44 |
marios | there is no such thing (anymore) "Error importing loadbalancer device driver: neutron.services.loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver" | 09:44 |
marios | ( https://github.com/openstack/neutron/tree/master/neutron/services/loadbalancer ) | 09:44 |
spredzy | and there is not entry point for the backward compat ? | 09:44 |
marios | spredzy: to be honest i didn't dig any further than these notes | 09:45 |
spredzy | derekh, thank you for letting us know | 09:45 |
marios | spredzy: i was chasing the setup... and once i saw there was already some upstream attention (bugs and reviews) kinda moved on to finish stuff | 09:45 |
spredzy | marios, ack no problem. I just thought that this commit https://github.com/openstack/neutron-lbaas/commit/3665e0cc7b69e47d24b838aa6c91e007d16a71d1 was suppose to maintain backward compat. But nevermind will remove the -1 if that work as is | 09:47 |
marios | spredzy: thanks for revoting, yeah i don't know either... we can always revisit once this lands if someone needs to change the driver etc | 09:48 |
*** jehb has quit IRC | 10:02 | |
*** yog__ has joined #tripleo | 10:06 | |
*** akrivoka has joined #tripleo | 10:08 | |
gfidente | jistr, spredzy I remember a review with some comments on how to pass clone params | 10:15 |
gfidente | was there an agreement on how to pass clone params to resources? | 10:15 |
spredzy | gfidente, I don't recall the comment | 10:16 |
spredzy | do you have the review at than ? | 10:17 |
spredzy | s/than/hand/ | 10:17 |
gfidente | marios, probably knows? I seem to remember was neutron related | 10:17 |
gfidente | jistr, I left two comments https://review.openstack.org/#/c/181067/3 , do they look valid? | 10:20 |
marios | gfidente: not sure what you mean, but i have constraints with clones @ https://review.openstack.org/#/c/181015/7 | 10:22 |
marios | gfidente: it might be become isn't a clone (comment on jistr review) | 10:24 |
marios | s/become/because | 10:24 |
gfidente | marios, see cwolfe comment here https://review.openstack.org/#/c/178743/3/puppet/manifests/overcloud_controller.pp | 10:24 |
gfidente | clone_params | 10:24 |
gfidente | we just don't support it yet it seems | 10:25 |
*** avozza has quit IRC | 10:25 | |
marios | gfidente: oh yeah. i was referring to the comment @ https://review.openstack.org/#/c/181067/3 | 10:25 |
marios | gfidente: was planning to cycle back to that once landed | 10:25 |
*** wendar has quit IRC | 10:25 | |
*** gus has quit IRC | 10:25 | |
*** wendar_ has joined #tripleo | 10:25 | |
*** aberdine has quit IRC | 10:26 | |
marios | gfidente: (cycle back, i mean the clone params) | 10:26 |
*** hewbrocc` has joined #tripleo | 10:26 | |
gfidente | marios, we can't pass clone params yet, ack on cycling it later | 10:26 |
marios | gfidente: exactly | 10:26 |
*** spredzy has quit IRC | 10:26 | |
gfidente | marios, I think this is something we should do for all services probably | 10:26 |
gfidente | marios, I actually like the idea of working on the HA config by refinement | 10:27 |
gfidente | also replied on jistr's change :) | 10:27 |
marios | ack | 10:28 |
*** hewbrocca has quit IRC | 10:29 | |
*** spredzy has joined #tripleo | 10:31 | |
*** zz_avozza has joined #tripleo | 10:32 | |
*** EmilienM has quit IRC | 10:33 | |
*** zz_avozza is now known as avozza | 10:33 | |
*** gus has joined #tripleo | 10:33 | |
*** aberdine has joined #tripleo | 10:34 | |
*** EmilienM has joined #tripleo | 10:35 | |
*** mmagr|afk is now known as mmagr | 10:44 | |
*** mmagr is now known as mmagr|afk | 10:50 | |
jistr | yeah re https://review.openstack.org/#/c/181067/3/puppet/manifests/overcloud_controller_pacemaker.pp | 10:50 |
jistr | the cinder-volume is active/passive, hence the absence of -clone | 10:50 |
*** mmagr|afk is now known as mmagr | 10:52 | |
*** gfidente is now known as gfidente|afk | 10:54 | |
*** lucasagomes is now known as lucas-afk | 10:57 | |
openstackgerrit | Yanis Guenane proposed openstack/tripleo-heat-templates: Add Redis as a Pacemaker resource when EnablePacemaker https://review.openstack.org/182617 | 11:03 |
*** adrianopetrich has quit IRC | 11:12 | |
*** panda has quit IRC | 11:13 | |
*** panda has joined #tripleo | 11:14 | |
*** weshay has joined #tripleo | 11:15 | |
*** yuanying has quit IRC | 11:16 | |
*** julim has quit IRC | 11:18 | |
*** thrash|g0ne is now known as thrash | 11:35 | |
*** stendulker has quit IRC | 11:38 | |
*** stendulker has joined #tripleo | 11:39 | |
*** hewbrocc` is now known as hewbrocca | 11:46 | |
*** yamahata has quit IRC | 11:48 | |
*** sdake has joined #tripleo | 11:54 | |
*** dprince has joined #tripleo | 11:54 | |
*** lazy_prince has quit IRC | 11:58 | |
*** sdake has quit IRC | 11:59 | |
*** sdake has joined #tripleo | 12:00 | |
openstackgerrit | Merged openstack/os-cloud-config: Remove pxe_ prefix from deploy_ramdisk and deploy_kernel params https://review.openstack.org/181354 | 12:02 |
*** stendulker has quit IRC | 12:02 | |
*** gfidente|afk is now known as gfidente | 12:02 | |
*** morazi has joined #tripleo | 12:05 | |
*** derekh_ has joined #tripleo | 12:08 | |
*** ishant has quit IRC | 12:11 | |
*** derekh has quit IRC | 12:12 | |
jistr | anyone seen this when building an image? | 12:12 |
jistr | Error: Package: python-nova-2015.2-dev282.fc21.noarch (delorean) | 12:12 |
jistr | Requires: python-rfc3986 | 12:12 |
jistr | hmm looks like the CI has seen it too :/ | 12:14 |
jistr | http://logs.openstack.org/22/182622/1/check-tripleo/check-tripleo-ironic-overcloud-f20puppet-nonha/86756d1/console.html | 12:14 |
jistr | so that's a real problem, not just my env | 12:14 |
jistr | i wonder if that's related to this morning's RDO Kilo repo disappearing and then restoration | 12:15 |
jistr | maybe the python-rfc3986 package was in the kilo repo but isn't anymore? | 12:15 |
jistr | because we have a pinned rdo-manager delorean in the upstream CI, so that couldn't have changed | 12:16 |
*** jayg|g0n3 is now known as jayg | 12:20 | |
* jistr proceeds to query #rdo | 12:21 | |
*** rhallisey has joined #tripleo | 12:23 | |
jistr | it indeed disappeared from the f21 kilo repo, apevec is working on a fix | 12:26 |
gfidente | jayg, you around? I just got your same error I think | 12:32 |
*** trown|outttypeww is now known as trown | 12:37 | |
openstackgerrit | Yanis Guenane proposed openstack/tripleo-heat-templates: Add Redis as a Pacemaker resource when EnablePacemaker https://review.openstack.org/182617 | 12:37 |
*** prad has joined #tripleo | 12:39 | |
*** mestery has joined #tripleo | 12:44 | |
*** dasm is now known as dasm|afk | 12:46 | |
jayg | gfidente: which one is that? | 12:51 |
jayg | no tables? rabbit fails? | 12:52 |
gfidente | the cib error | 12:52 |
gfidente | it doesn't seem to stop puppet though | 12:52 |
gfidente | and apparently the pacemaker config is also correct | 12:52 |
jistr | gfidente, jayg: yeah i've seen that too, it doesn't seem to affect anything though | 12:52 |
gfidente | jistr, one more thing I am seeing now is pcs returning unknown option -c on colocation resources | 12:53 |
jayg | yeah, that is what I saw as well, though it did seem to cause later steps to not happen for me | 12:53 |
*** julim has joined #tripleo | 12:53 | |
jayg | also, I noticed 'hiera step' always returns nil for me if I run it on a controller | 12:53 |
jayg | jistr: before I forget, are we ok to merge crag's puppet-pacemaker patch now? neutron-scale will not work until we do | 12:54 |
jayg | recall it may need a bit of rework on some of the calls to set up pacemaker resources | 12:55 |
jayg | gfidente: which patch causes the '-c' error you see? | 12:55 |
gfidente | jayg, that is an updated version of https://review.openstack.org/#/c/178152/ | 12:57 |
gfidente | I haven't pushed it yet as I was testing it | 12:57 |
gfidente | but again, this seems intermittent | 12:57 |
gfidente | so I will push meantime | 12:57 |
jistr | jayg: re puppet-pacemaker, it would break our current patches i think... i'm still on the fence wrt that /cc gfidente spredzy | 12:58 |
jistr | e.g. https://github.com/cwolferh/puppet-pacemaker/commit/59fe631609de3543cbb9082f631ea0a5a118a472#diff-db0fbf2c24e17f11202aa7143f37c44fR32 | 12:58 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Glance as a Pacemaker resource when EnablePacemaker https://review.openstack.org/178152 | 12:58 |
jayg | gfidente: ok, I'll take a look and see if I see any clues in the patch | 12:58 |
gfidente | jayg, updated now | 12:58 |
jayg | jistr: well the thing is, it has to go in, I htin the sooner the better, we already waited until after the demo | 12:59 |
jayg | gfidente: k, loking | 12:59 |
jayg | looking | 12:59 |
jistr | jayg: do you think there's much more for the resource changes than changing clone => true into clone_params => "interleave=true" ? | 13:00 |
jayg | gfidente: my first guess would be something with the interpolation of "${::glance::params::registry_service_name}-clone" | 13:00 |
jayg | jistr: let me look again to remember the details | 13:01 |
*** lucas-afk is now known as lucasagomes | 13:01 | |
jistr | jayg: yeah re hitting it sooner than later, that makes sense. I guess i'm mostly worried about updating the most important patches like galera (spredzy's) because that patch has the potential to get us to a green CI on HA job, and every delay hurts. | 13:02 |
spredzy | (I was hoping to have a green job back today ... but I don't think it will happen :/) | 13:03 |
spredzy | with current state of the CI | 13:03 |
jayg | jistr: understood | 13:03 |
jayg | it looks to me like that clone change would be the most common one, other than services that require extra params, or ocf agents (like galera) | 13:04 |
jistr | spredzy: apevec fixed the repos right now (see #rdo) | 13:05 |
jistr | so if everything's ok, recheck should do it | 13:05 |
jayg | spredzy: for galera, this is the part to look at for upcoming changes in puppet-pacemaker - https://github.com/redhat-openstack/puppet-pacemaker/pull/45/files#diff-761b9ccb2d93fc11a9f73b2555768c0eL1 | 13:08 |
*** masco has quit IRC | 13:08 | |
spredzy | jayg, jistr what it the plan though ? First we merge the PR in puppet-pacemaker and we all reabase on it ? | 13:12 |
jayg | spredzy: up to you guys, but we do need it soon, as astapor work for osp 7 also depends on it | 13:14 |
jayg | seems to me we either get current patches rebased plus a couple updates for existing stuff, or everything will be broken again and need followon ptches anyway | 13:15 |
jayg | and note that some neutron services, specifically neutron-scale will simply never work until this is in | 13:15 |
*** adrianopetrich has joined #tripleo | 13:16 | |
jistr | ok that might be a way to go, especially wrt astapor depending on it... | 13:16 |
spredzy | Well as you say doesn't matter if we do it no or then we will need to rebase anyway | 13:17 |
spredzy | That will only delay the pacemaker patches in tripelo-heat-templates again :D | 13:17 |
jistr | yeah... for the future it would be nice if we could get puppet-pacemaker to gerrit though... | 13:17 |
jayg | right, and the changes in calls should not be terrible, but the benefits are good | 13:17 |
spredzy | then fair enough | 13:17 |
jistr | ok +1 | 13:17 |
jayg | jistr: I don't think anyone would oppose that, though I have no idea what is involved | 13:18 |
*** radez_g0n3 is now known as radez | 13:19 | |
jistr | jayg: i guess EmilienM has experience with getting things to gerrit. The main benefit i'd see in having puppet-pacemaker in gerrit is that we can leverage cross-repository "Depends-On" headers in commit messages, and we could prepare the puppet-pacemaker patch along with the change to tripleo (and have CI test the tripleo change with a puppet-pacemaker commit which is not merged yet) and land both things +/- in parallell, keeping the breakage | 13:20 |
jistr | period on a minimum | 13:20 |
*** yog__ has quit IRC | 13:20 | |
jistr | ^^ that's for the future though, not necessarily for now | 13:20 |
jistr | (getting the repo to gerrit would slow us down further ATM) | 13:21 |
EmilienM | yes | 13:21 |
EmilienM | jistr: to openstack gerrit? | 13:21 |
jistr | EmilienM: yes that's what i meant. It's probably not an action we need to do right now though, as everything is in the air and moving puppet-pacemaker could make even more chaos :) | 13:22 |
jayg | moar chaos! | 13:26 |
spredzy | jistr, the recheck on galera is unfortuantely unecessary until the toci_gater.sh review gets merged | 13:26 |
EmilienM | jistr: I'm not sure it's a great idea | 13:26 |
jistr | duh | 13:26 |
* jistr searches for that | 13:26 | |
EmilienM | this module would not be supported by Puppet OpenStack at least | 13:26 |
*** mcornea has quit IRC | 13:28 | |
*** mcornea has joined #tripleo | 13:28 | |
*** rbrady has joined #tripleo | 13:29 | |
jistr | spredzy: ah i see... you could make it "depends-on" on the toci patch though... | 13:30 |
jistr | i'll do a recheck on the toci one | 13:31 |
spredzy | already done for the CI job | 13:31 |
spredzy | jistr, right for the Depends-on | 13:32 |
gfidente | jayg, the update actually worked | 13:32 |
spredzy | lemme fix that | 13:32 |
gfidente | jayg, so this seems intermittent as well | 13:32 |
jayg | gfidente: hmph, ok then | 13:34 |
gfidente | jistr, spredzy I was thinking to change the path to the yaml file instead of pasting the env file into the t-h-t repo dir actually | 13:34 |
openstackgerrit | Yanis Guenane proposed openstack/tripleo-heat-templates: Add Galera as a Pacemaker resource when EnablePacemaker https://review.openstack.org/177765 | 13:34 |
gfidente | jistr, spredzy but we can do this later | 13:34 |
*** tzumainn has joined #tripleo | 13:37 | |
*** sdake_ has joined #tripleo | 13:38 | |
*** jprovazn has quit IRC | 13:40 | |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Set up corosync using hostnames rather than IPs https://review.openstack.org/177735 | 13:42 |
*** sdake has quit IRC | 13:42 | |
jistr | | 0c867bc1-0758-4fc4-94a3-9053ec0a3ee0 | ov-traa3mc2at-0-paxw2ukbncyk-NovaCompute-quch5sbo6vkv | ACTIVE | - | Running | ctlplane=192.0.2.5 | | 13:45 |
jistr | | e65726af-30b3-4f3b-8f39-84abf585c1de | ov-uzg2h7hm5jr-0-jl5ycwvldmaj-Controller-echwwwyyr7ti | ACTIVE | - | Running | ctlplane=192.0.2.8 | | 13:45 |
jistr | | 59faef39-3b75-41ab-b74e-fd4e3f22b25a | ov-uzg2h7hm5jr-1-inamxksvekz3-Controller-ubzoixh5uhog | ACTIVE | - | Running | ctlplane=192.0.2.6 | | 13:45 |
jistr | | 224e0aae-e805-487f-92b3-7616aaaa510b | ov-uzg2h7hm5jr-2-cdq3wf6kxxpg-Controller-ue7yihyhru56 | ACTIVE | - | Running | ctlplane=192.0.2.7 | | 13:45 |
jistr | devtest completed including launching an instance in the overcloud and pinging it | 13:46 |
*** BadCub-Hiding is now known as BadCub | 13:46 | |
jistr | congrats spredzy, great job ;) | 13:46 |
hewbrocca | wooowooo+!!!!! | 13:46 |
hewbrocca | nice! | 13:46 |
spredzy | jistr, :) Do I get a beer from Brno ? :D | 13:47 |
*** links has quit IRC | 13:48 | |
jistr | spredzy: absolutely :)) hope i see you soon | 13:48 |
jistr | (i won't be at the summit) | 13:49 |
spredzy | :/ | 13:49 |
spredzy | hewbrocca, will arrange a team meeting :p so we can all get together ;) | 13:49 |
*** lazy_prince has joined #tripleo | 13:51 | |
hewbrocca | yes I will | 13:52 |
hewbrocca | so, wait | 13:52 |
hewbrocca | are we saying that devtest configures pacemaker correctly? | 13:52 |
jistr | hewbrocca: it already did configure pacemaker itself correctly before, but we didn't have galera, which caused everything else which needs a DB fail in the multi-controller scenario (DB sync races -> puppet error -> deployment stopping) | 13:54 |
jistr | hewbrocca: we're still missing a good chunk of the proper HA arch though | 13:54 |
jistr | hewbrocca: so in terms of being done with HA there's still some way to go, but we're now very close to having a green CI for 3-controller deployment, and whenever we add some service to pacemaker, we'd see if the patch breaks the 3 node scenario or not, rather than relying on manual testing or trust in visual review as it is now | 13:56 |
*** rbrady_ has quit IRC | 13:57 | |
jistr | hewbrocca: we have a breaking (but necessary) change to land in puppet-pacemaker soon, which will get the 3-controller scenario red again, but hopefully just for a short time | 13:57 |
hewbrocca | jistr: that is pure excellence | 14:00 |
hewbrocca | well done | 14:00 |
*** sdake has joined #tripleo | 14:01 | |
*** masco has joined #tripleo | 14:02 | |
spredzy | is there anyone already working on Ceilometer resource for Pacemaker ? jistr gfidente marios jayg jdob1 ? | 14:04 |
jistr | not me atm | 14:04 |
jdob1 | yes, its on our sprint | 14:04 |
*** sdake_ has quit IRC | 14:04 | |
hewbrocca | I hope it's not at the top of the list :) | 14:05 |
hewbrocca | (Ceilometer) | 14:05 |
jdob1 | hewbrocca: for my sprint, the priority is nova -> horizon -> ceilometer | 14:06 |
jdob1 | so its last on the stuff we've signed up for | 14:06 |
*** jdob1 is now known as jdob | 14:07 | |
hewbrocca | jdob1: that sounds like the right order | 14:07 |
jdob | thank morazi for that, I made him determine the order :) | 14:07 |
jdob | which is basically like a kid yelling "i need an adult" | 14:07 |
hewbrocca | LOL well sometimes that's the right approach anyway | 14:08 |
*** derekh_ changes topic to "Deploying OpenStack Using OpenStack | Summit proposed topics: https://etherpad.openstack.org/p/tripleo-liberty-proposed-sessions | https://wiki.openstack.org/wiki/TripleO" | 14:09 | |
openstackgerrit | Steven Hardy proposed openstack/tripleo-incubator: Adding stepped deployment to devtest_overcloud.sh https://review.openstack.org/162109 | 14:15 |
morazi | jistr, spredzy jdob awesome folks! | 14:15 |
jayg | spredzy: no, I am not, and agree nova should be in asap | 14:16 |
gfidente | jayg, yeah I have nova, I think everything rabbit-related merged now | 14:16 |
gfidente | including the cluster-related settings | 14:17 |
gfidente | I have the change which moves client from haproxy to list of nodes as well up | 14:17 |
gfidente | and the changes for haproxy merged as well, so looks clear to me | 14:17 |
jayg | nice! | 14:17 |
gfidente | except we don't have the two haproxy instances as described in the beekhof doc | 14:17 |
gfidente | does staypuft have that? | 14:17 |
jayg | 2? for what? | 14:18 |
gfidente | 2 haproxy instances? | 14:18 |
jayg | we have haproxy on all control nodes | 14:18 |
morazi | spredzy, I think if you have bandwidth re: ceilo I'd say go for it. Make sure jdob jdob1 knows though | 14:18 |
gfidente | jayg, I meant this: nova and horizon are exposed to the external LAN via an extra haproxy instance | 14:19 |
jayg | gfidente: oh, hmm, I have not seen that | 14:19 |
morazi | spredzy, and yea agree with the ordering from jdob in general but iirc you did a ton of previous work on ceilo so it might be quite trivial for you to know it out... | 14:19 |
jayg | gfidente: I'll search through the docs for that now | 14:20 |
spredzy | morazi, ack, will tackle the ceilometer part. /cc jdob | 14:20 |
jdob | kk, i'll add you to the card | 14:20 |
jdob | more than happy to have help on my sprint work :D | 14:20 |
openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Remove nova temprevert https://review.openstack.org/182695 | 14:25 |
jayg | gfidente: ok, I assume you are referring to a reference here - https://github.com/beekhof/osp-ha-deploy/blob/rdo-7-pcmk-phase-1/ha-openstack.md#assumptions | 14:26 |
gfidente | jayg, yeah | 14:26 |
*** bvandenh has joined #tripleo | 14:26 | |
jayg | I have not yet found where such an extra instance is set up yet though, or if that is expected to be something external to the pacemaker configuration | 14:27 |
gfidente | jayg, so we don't have that yet for now, pacemaker is only managing a single instance | 14:27 |
gfidente | jayg, I also investigated a little the colocation thing | 14:28 |
gfidente | jayg, ideally that turns useful when total number of instances of a service are less than number of cluster nodes | 14:28 |
*** mmagr is now known as mmagr|afk | 14:28 | |
jayg | I meant 'where it is set up in the ref arch docs', I only see that one general 'assumption' reference to it | 14:28 |
gfidente | jayg, only in the assumption yes | 14:28 |
jayg | gfidente: yes, that sounds correct | 14:28 |
jayg | so I think we'd need to ask andrew and perhaps fabio for more detail on what they mean there | 14:29 |
*** chlong has quit IRC | 14:30 | |
gfidente | so if astapor doesn't have it, we can skip it in the short term I think as well, jistr ^^ | 14:30 |
jayg | I woudl think so - also, instead of an extra instance to make any service available on a pulic network, we just add another ip to the existing haproxy config | 14:31 |
jayg | I have done this many times - in our setup, it is simply the public_vip, which we manage with pacemaker | 14:31 |
*** stendulker has joined #tripleo | 14:32 | |
jayg | so long as that vip is on an external network, you should be all set | 14:32 |
gfidente | jayg, vips are on pacemaker for us as well already | 14:32 |
gfidente | jayg, only two though, public and private | 14:32 |
jayg | right, but I think tripleo only does one network right now, doesnt it? | 14:33 |
jayg | ah, 2, ok | 14:33 |
gfidente | jayg, what about the sticky table? | 14:33 |
gfidente | jayg, is it meant to ensure connections are redirected to a single node at all times or is there more to it? | 14:33 |
*** mmagr|afk is now known as mmagr | 14:34 | |
gfidente | I am asking because with marios there was some 'shared thought' on improving the HA config by refinement | 14:36 |
jayg | yeah, the stick table basically makes connections 'stick' to one node unless there is a problem, though there are a few more configuations to it when needed | 14:36 |
gfidente | so it is probably a good time now to start tracking what are the refinements we need | 14:37 |
jayg | sure | 14:37 |
gfidente | and you and jistr probably know those pieces better | 14:37 |
jistr | gfidente, jayg: re skipping the 2 haproxies in short term -- i'd be on board with it, in general i think i'd first try to get parity with Astapor and then we can refine | 14:38 |
gfidente | jistr, yeah was trying to figure if that is in astapor already or not | 14:39 |
jayg | to be clear, it is most definitely not | 14:39 |
jayg | jistr gfidente spredzy: I am not sure if we reached consensus regarding timing on updates to puppet-pacemaker? | 14:41 |
spredzy | jayg, my opinion are do it whenever since we'll have to rebase anyway | 14:42 |
spredzy | my opinion is* | 14:42 |
gfidente | jayg, how much is it going to impact existing resources? | 14:42 |
jayg | gfidente: it is not huge, as we dont use a ton of params per pacemaker resource, but rabbit and galera would new a few param changes for sure | 14:43 |
*** mestery has quit IRC | 14:44 | |
jayg | spredzy: ok, my inclination is, the sooner the better, so we have more time to discover issues and make updates | 14:44 |
jistr | spredzy, jayg, gfidente: i'd be leaning towards getting the HA job green now and merge spredzy's patch if possible, then we can merge puppet-pacemaker and fix the job | 14:45 |
spredzy | jayg, so what I would prob do. is if next jenkins runs for galera gives green for ha job, merge galera (then merge the puppet-pacemaker stuff), so for testing people can stick to previous commit of puppet-pacemaker and move forward with their current work with a working job locally | 14:45 |
jistr | my reasoning is | 14:45 |
gfidente | I am with jistr, rabbit is already in, galera will turn the job green | 14:45 |
jistr | that the patch is big and it will be easier to review a new separate patch with the fixes | 14:45 |
spredzy | once the puppet-pacemaker is merged, we do a review that fixes Rabbit + Galera + whatever and we should be back on track by then | 14:45 |
jistr | rather than review a new patchset for the big patch | 14:45 |
gfidente | yeah so after it is green, we update the existing for it to get back working | 14:46 |
* jistr never knew why patches are composed of patch sets and not vice versa, but such is world | 14:46 | |
jistr | gfidente: +1 | 14:46 |
gfidente | and rebase only openstack service changes | 14:46 |
jistr | +1 | 14:46 |
gfidente | jistr we can't vote each other | 14:46 |
spredzy | ok then +1 | 14:47 |
spredzy | :D | 14:47 |
spredzy | gfidente, can I ? | 14:47 |
jayg | it sounds like you are all saying the same thing :) | 14:47 |
jayg | so, we have hope current jenkins run will be green with galera? | 14:47 |
jistr | jayg: yes. if for some reason we can't land the galera bit soon, then maybe we should go ahead with the puppet-pacemaker anyway to unblock astapor | 14:48 |
jistr | spredzy, gfidente: ^ | 14:48 |
jayg | ok, I'll watch for notification that the patch has passed then, and proceed from there | 14:48 |
spredzy | ok for ereything said above | 14:49 |
jayg | thanks for clearing that up for me guys :) | 14:49 |
jayg | one other question - spredzy, above you said 'local job', are you referring to a local jenkins setup, or manual testing (which is what I have been doing so far) | 14:50 |
spredzy | jayg, manual testing | 14:50 |
spredzy | running devtest_overcloud on your lcoal station | 14:50 |
jayg | ok, fair enough, thanks | 14:51 |
EmilienM | I think spredzy is going to patch all Puppet modules of the world to be able to disable services :P | 14:55 |
spredzy | EmilienM, ahahah no just the one we need ;) | 14:55 |
jistr | spredzy: which one is that? | 14:56 |
jistr | i've seen the $enabled param on many, but didn't check if all | 14:56 |
EmilienM | spredzy: I have a stupid question | 14:56 |
EmilienM | spredzy: have you ever checked puppet runs in tripleo are idempotent? | 14:56 |
spredzy | jistr, Redis, MongoDb, Memcached | 14:56 |
spredzy | EmilienM, they are not, it will return 2 all the time | 14:57 |
*** aufi is now known as aufi|ask | 14:57 | |
spredzy | but we are aware of it for now, and some work will be done after the current effort | 14:57 |
EmilienM | spredzy: ok. Do we know where it's not, | 14:57 |
*** killer_prince has joined #tripleo | 14:57 | |
*** aufi|ask is now known as aufi|afk | 14:57 | |
EmilienM | spredzy: it would be great to track what is not idempotent, so we can fix upstream modules (most of the case it's in upstream modules) | 14:57 |
spredzy | What I've seen so far is puppet-pacemaker and the execs (that have no onlyif, or unless) | 14:57 |
jayg | right, and we have a bug on that in github that jistr submitted | 14:58 |
*** spredzy is now known as spredzy|afk | 14:59 | |
*** Marga_ has joined #tripleo | 14:59 | |
*** untriaged-bot has joined #tripleo | 15:00 | |
untriaged-bot | Untriaged bugs so far: | 15:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1452400 | 15:00 |
openstack | Launchpad bug 1452400 in diskimage-builder "debian-minimal builds still busted, now on locale issues" [Undecided,New] | 15:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449852 | 15:00 |
openstack | Launchpad bug 1449852 in diskimage-builder "Buidling ramdisk with ironic-agent behind proxy fails" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 15:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449854 | 15:00 |
openstack | Launchpad bug 1449854 in diskimage-builder "Ironic agent ramdisk built using disk-image-create fails with iscsi_ilo driver" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 15:00 |
untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1452752 | 15:00 |
openstack | Launchpad bug 1452752 in tuskar "keystone_authtoken section is wrong in default shipped tuskar.conf.sample" [Undecided,Confirmed] | 15:00 |
*** untriaged-bot has quit IRC | 15:00 | |
*** lazy_prince has quit IRC | 15:00 | |
*** killer_prince has quit IRC | 15:02 | |
openstackgerrit | Yanis Guenane proposed openstack/tripleo-heat-templates: Add Redis as a Pacemaker resource when EnablePacemaker https://review.openstack.org/182617 | 15:03 |
*** Marga_ has quit IRC | 15:03 | |
*** Marga_ has joined #tripleo | 15:03 | |
*** panda has quit IRC | 15:13 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Update ci for F21 jenkins nodes https://review.openstack.org/182219 | 15:14 |
*** panda has joined #tripleo | 15:15 | |
*** rlandy has quit IRC | 15:15 | |
gfidente | jistr, jayg I was thinking looking at https://review.openstack.org/#/c/181067/3/puppet/manifests/overcloud_controller_pacemaker.pp,cm | 15:17 |
gfidente | is there a point in making colocation depend on base ? | 15:17 |
*** spredzy|afk is now known as spredzy | 15:18 | |
* jayg looks | 15:18 | |
gfidente | I see they do in astapor | 15:18 |
*** jistr is now known as jistr|mtg | 15:19 | |
*** adrianopetrich has quit IRC | 15:19 | |
jayg | gfidente: I am not positive, but I believe we were told the orderthese constraints are added in matters | 15:20 |
gfidente | jayg, I read that as, we should make colocation require base, correct? | 15:21 |
spredzy | gfidente, jayg jistr|mtg job is back to green for Galera https://review.openstack.org/#/c/177765/ | 15:24 |
jayg | spredzy: awesome! | 15:24 |
jayg | gfidente: I believe so, yes | 15:25 |
gfidente | spredzy, the sooner those two get merged the better | 15:25 |
gfidente | I still can't exercise +2 superpower though | 15:26 |
gfidente | slagle ^^ | 15:26 |
gfidente | ^_^ | 15:26 |
spredzy | +1 for giving gfidente +2 power ;) | 15:27 |
gfidente | derekh_, the broken pipe error, FWIW, was due to misconfiguration of rabbit servers list in ceilometer | 15:27 |
derekh_ | gfidente: ok, thanks for letting me know | 15:28 |
gfidente | derekh_, check the diffs https://review.openstack.org/#/c/181081/3..4 | 15:28 |
*** jistr|mtg is now known as jistr | 15:29 | |
derekh_ | opps | 15:29 |
*** pdb has quit IRC | 15:29 | |
gfidente | eheheh :) | 15:30 |
*** pdb has joined #tripleo | 15:30 | |
gfidente | but again, how that can cause job to fail with broken pipe ... : | 15:30 |
*** sdake has quit IRC | 15:30 | |
*** rhallisey has quit IRC | 15:30 | |
*** sdake has joined #tripleo | 15:30 | |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Glance as a Pacemaker resource when EnablePacemaker https://review.openstack.org/178152 | 15:33 |
*** yamahata has joined #tripleo | 15:33 | |
*** daneyon_ has joined #tripleo | 15:33 | |
EmilienM | spredzy: looking at https://review.openstack.org/#/c/177765/15/puppet/manifests/overcloud_controller_pacemaker.pp,cm | 15:33 |
EmilienM | the galera-ready, don't you have a PR somewhere to use a provider? | 15:34 |
jistr | spredzy, jayg: the dependency failed though /o\ https://review.openstack.org/#/c/182553/ | 15:34 |
jistr | spredzy, jayg: i'm sure it's unrelated to the patch | 15:35 |
slagle | gfidente: yea, it's been a week. i'll submit the update today | 15:35 |
spredzy | EmilienM, nop not for galera-ready since it runs a specific command | 15:35 |
EmilienM | spredzy: ok, just asking | 15:35 |
spredzy | I have it for mongodb, since its a tcp check | 15:35 |
spredzy | jistr, yeah I thought that, want to dive into precise ? :) | 15:36 |
jayg | jistr: you think still repo issues? | 15:36 |
*** yamahata has quit IRC | 15:36 | |
*** yamahata has joined #tripleo | 15:36 | |
jistr | jayg, spredzy: no it's ubuntu, so no RDO repo issues. gfidente gave it a recheck 30 minutes ago so we'll have the results in about 1 hour | 15:37 |
*** daneyon has quit IRC | 15:37 | |
openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Allow puppet jobs to also run on F21 https://review.openstack.org/182722 | 15:37 |
jayg | ok | 15:37 |
spredzy | jistr, jayg I saw that in the log ERROR: Parameter SwiftReplicas from deprecated/swift-deploy.yaml conflicts. | 15:37 |
*** alop has joined #tripleo | 15:37 | |
spredzy | not sure ho relevant it is here | 15:37 |
*** spredzy is now known as spredzy|afk | 15:39 | |
jistr | spredzy: that looks super weird if it should be an intermittent error, but on the other hand the job on the galera patch ran with the toci patch in place too (due to depends-on), and it succeeded, so it has to be intermittent somehow | 15:40 |
*** ukalifon has quit IRC | 15:40 | |
EmilienM | marios: hey, I dropped a comment on https://review.openstack.org/#/c/180535 (does not deserver -1 but need your attention) | 15:41 |
spredzy|afk | jistr, erf the worst one ... | 15:41 |
gfidente | jistr, on the cinder change, given previous chat with jayg I'm changing the require for constraint::colocation to be on constraint::base | 15:42 |
gfidente | for the glance patch | 15:42 |
jistr | gfidente, jayg: ack, i'll amend cinder too, thanks | 15:43 |
*** bvandenh has quit IRC | 15:43 | |
gfidente | I think these can go slower though as we'll have to merge changes to puppet-pacemaker first | 15:44 |
jistr | yeah | 15:44 |
gfidente | derekh_, in https://review.openstack.org/#/c/182722/1 , I remember f21 is the distro the slave uses right? | 15:45 |
*** rhallisey has joined #tripleo | 15:45 | |
derekh_ | gfidente: yes, that patch is about changing the version of the slave (f thats what you mean) | 15:48 |
gfidente | ack | 15:48 |
*** masco has quit IRC | 15:49 | |
*** shakamunyi has quit IRC | 15:51 | |
*** barra204 has quit IRC | 15:51 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add neutron-lbaas as neutron-server plugin & neutron-lbaas-agent https://review.openstack.org/180535 | 15:53 |
marios | EmilienM: thanks! | 15:53 |
jistr | tripleo-core folks, can we get another review here? it's a crucial patch that unblocks the world for us (gets HA CI green, and we're waiting for it to make further changes to puppet-pacemaker) https://review.openstack.org/#/c/177765/ | 15:53 |
openstackgerrit | James Slagle proposed openstack/tripleo-incubator: Remove Jon-Paul Sullivan from tripleo-cd-admins https://review.openstack.org/182737 | 15:53 |
openstackgerrit | James Slagle proposed openstack/tripleo-incubator: Remove Jon-Paul Sullivan from tripleo-cd-admins https://review.openstack.org/182737 | 15:54 |
slagle | gfidente: shardy : you both should be added to tripleo-core | 15:55 |
jistr | marios, dprince, jdob: possibly the patch i linked above is best reviewable for one of you, so if one of you found some time for it, that would be great :) | 15:55 |
slagle | it seems odd to make that change directly in gerrit...is the list of who's in tripleo-core in a git repo anywhere? | 15:55 |
jistr | gfidente: ^^ or you can switch from +1 to +2 now, which would solve the problem too :D | 15:56 |
jistr | thx slagle | 15:56 |
gfidente | slagle, isn't tripleo-core taken from launchpad? | 15:56 |
gfidente | asking as I assumed so but it does not seem to be? | 15:57 |
slagle | not that i know of | 15:57 |
gfidente | hehe | 15:57 |
*** dprince has quit IRC | 15:58 | |
marios | jistr: ok | 15:58 |
jistr | marios: thx! we'll wait for the CI on the dependency | 15:59 |
marios | exactly | 15:59 |
marios | EmilienM: i updated btw if you get a sec | 15:59 |
marios | https://review.openstack.org/#/c/180535 | 15:59 |
EmilienM | marios: +1 now | 16:00 |
marios | EmilienM: also clarification sought for the neutron-server one | 16:02 |
marios | https://review.openstack.org/#/c/181015/ when you get a sec | 16:02 |
EmilienM | marios: I think we should drop this code | 16:02 |
EmilienM | because Puppet don't manage this orchestration anymore, it's Pacemaker itself with constraints | 16:03 |
marios | EmilienM: so the constraints themselves should be enough ? | 16:03 |
EmilienM | yeah | 16:03 |
marios | EmilienM: ok, i can do that immediately | 16:03 |
marios | sec | 16:03 |
EmilienM | marios: maybe in separated patchset though | 16:03 |
EmilienM | because it's not really related, but... I don't know | 16:03 |
EmilienM | you can do as you like, I just suggest here :) | 16:04 |
jdob | GOD jistr, you're so needy | 16:04 |
marios | EmilienM: well, sure, i can make a new one under the... ok lemme see | 16:04 |
jdob | ... wait, which patch jistr ? | 16:04 |
jistr | jdob: ahahah nevermid, marios got us sorted out :P this one https://review.openstack.org/#/c/177765/ | 16:04 |
jdob | :D | 16:04 |
jistr | jdob: thx for answering the call though ;) | 16:05 |
jdob | now i'm gonna go -2 the shit out of it just to be obnoxious | 16:05 |
*** eghobo has joined #tripleo | 16:05 | |
jistr | :D | 16:05 |
jdob | (no idea why i'm so punchy right now) | 16:05 |
*** cwolferh has joined #tripleo | 16:06 | |
*** jcoufal has quit IRC | 16:08 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add metering agent as neutron-server plugin https://review.openstack.org/179370 | 16:08 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add neutron-lbaas as neutron-server plugin & neutron-lbaas-agent https://review.openstack.org/180535 | 16:08 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds neutron-server and agents as pacemaker resources https://review.openstack.org/181015 | 16:08 |
marios | EmilienM: done ^^^ | 16:08 |
EmilienM | marios: cool, nice | 16:09 |
marios | (fixed in place at the top review) | 16:09 |
EmilienM | the tripleo-heat-templates backlog is huge to review | 16:09 |
marios | ah damnit i ate the metadata agent | 16:10 |
marios | EmilienM: sorry, gimme sec | 16:10 |
marios | (https://review.openstack.org/#/c/181015/8/puppet/manifests/overcloud_controller_pacemaker.pp) | 16:10 |
marios | EmilienM: oh no that's fine sorry (it is defined earlier for pacemaker) | 16:11 |
* marios stops talking | 16:11 | |
*** rwsu has joined #tripleo | 16:13 | |
shardy | All, overcloud stepped deployment is all now working and IMO ready to land, check it out: | 16:17 |
shardy | https://review.openstack.org/#/c/162109/ | 16:17 |
shardy | all the patches are passing CI now, reviews would be awesome :) | 16:17 |
shardy | If you want to pull/test it you'll need to rebuild your seed to include latest heat and heatclient | 16:17 |
shardy | relatedly, https://review.openstack.org/#/c/163108/3 needs one more +2/A, it's been around for ages and enables you to easily just rebuild your seed without rebuilding all the other images | 16:18 |
jistr | jayg: i'll be heading home now, but both patches (https://review.openstack.org/#/c/177765/ and https://review.openstack.org/#/c/182553/) have the necessary +2s, so any tripleo-core can then just +A if the CI is green. If there's some problem then i think you can go ahead with the puppet-pacemaker change anyway to unblock Astapor, as we agreed before with gfidente and spredzy|afk | 16:19 |
jayg | jistr: awesome, sounds good, thanks! | 16:22 |
*** jistr has quit IRC | 16:25 | |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Glance as a Pacemaker resource when EnablePacemaker https://review.openstack.org/178152 | 16:26 |
gfidente | EmilienM, https://review.openstack.org/#/c/178152/11..12/puppet/manifests/overcloud_controller_pacemaker.pp,cm had a typo :( | 16:26 |
EmilienM | gfidente: ok | 16:27 |
derekh_ | Can we get a new release of os-cloud-config , we'll need it befor this can merge https://review.openstack.org/#/c/181357/4 | 16:28 |
openstackgerrit | Merged openstack/tripleo-incubator: Remove Jon-Paul Sullivan from tripleo-cd-admins https://review.openstack.org/182737 | 16:28 |
*** dprince has joined #tripleo | 16:29 | |
*** jprovazn has joined #tripleo | 16:31 | |
*** derekh_ has quit IRC | 16:37 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/178418 | 16:37 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/python-tuskarclient: Updated from global requirements https://review.openstack.org/181242 | 16:38 |
*** prad has quit IRC | 16:44 | |
*** mmagr has quit IRC | 16:45 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Fix path issue to controller-config-pacemaker https://review.openstack.org/182553 | 16:50 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Add Galera as a Pacemaker resource when EnablePacemaker https://review.openstack.org/177765 | 16:50 |
*** dshulyak has quit IRC | 16:54 | |
*** daneyon_ has quit IRC | 16:56 | |
*** daneyon has joined #tripleo | 16:56 | |
*** Marga_ has quit IRC | 16:57 | |
*** sdake has quit IRC | 17:00 | |
*** jcoufal has joined #tripleo | 17:01 | |
*** jcoufal has quit IRC | 17:02 | |
*** spredzy|afk is now known as spredzy | 17:03 | |
spredzy | marios, big thank you :) | 17:03 |
marios | spredzy: por qe? | 17:04 |
spredzy | por el last +2 that removed Galera from my review list ;) | 17:04 |
* spredzy can breath now | 17:04 | |
jayg | spredzy: awesome news! | 17:05 |
jayg | now we can break it again shortly with puppet-pacemaker changes ;) | 17:05 |
jayg | marios: fyi, you'll want those for your neutron patch, they will allow you to set up neutron-scale | 17:05 |
spredzy | ahaha true, one stone at a time | 17:05 |
jayg | patch is here, so you guys know what is likely coming- https://github.com/redhat-openstack/puppet-pacemaker/pull/46 | 17:06 |
jayg | we are assessing juno v kilo for ofi right now, but I hope to merge that patch by eod my time today | 17:06 |
marios | spredzy: np jistr was doing a fine job of pimping it | 17:07 |
*** erikmack has joined #tripleo | 17:08 | |
marios | jayg: why do you hate me? I mean what did i ever do to you? | 17:09 |
marios | jayg: getting ready to call it a day here, gonna look tomorrow. do i need to add sthing to my existing reviews you mean? | 17:10 |
marios | jayg: thanks for the heads up man anyway, really calling it now and i'm cutting it fine with catching little man. later | 17:11 |
jayg | marios: hate you? it helps you! :) seriously, it is totally needed to make the pacemaker stuff match ref arch, we just agreed earlier to wait to see if galera could get past the CI | 17:11 |
marios | jayg: yeah was joking man | 17:11 |
jayg | and not rushing you, just heads up :) | 17:11 |
jayg | how is the little man? | 17:12 |
*** dshulyak has joined #tripleo | 17:12 | |
marios | lol i will not bite | 17:12 |
* marios gone | 17:12 | |
jayg | haha | 17:12 |
jayg | night | 17:12 |
openstackgerrit | Yanis Guenane proposed stackforge/puppet-tripleo: Add Galera cluster check at the proxy level https://review.openstack.org/182762 | 17:13 |
*** dshulyak has quit IRC | 17:14 | |
*** mcornea has quit IRC | 17:14 | |
*** pelix has quit IRC | 17:16 | |
erikmack | Hey folks, I've been tasked with evaluating TripleO as an alternative to PackStack or RedHat's installer. Do you agree that devtest is the correct starting point? Please confirm: I'm meant to run devtest_setup.sh, etc, on a non-virtualized Ubuntu or Fedora machine, yes? And this creates the seed VM? Thanks for your help. | 17:17 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Align puppet Controller post-deploy Deployment names https://review.openstack.org/182381 | 17:22 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Add support for generic controller validations https://review.openstack.org/174150 | 17:22 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: overcloud stepped deployment environment https://review.openstack.org/161927 | 17:22 |
*** lucasagomes is now known as lucas-dinner | 17:25 | |
openstackgerrit | Ben Nemec proposed stackforge/puppet-tripleo: Allow use of ssl for public api endpoints https://review.openstack.org/175958 | 17:34 |
*** yamahata has quit IRC | 17:40 | |
*** jtomasek has quit IRC | 17:45 | |
*** dshulyak has joined #tripleo | 17:52 | |
*** yamahata has joined #tripleo | 18:05 | |
*** stendulker has quit IRC | 18:09 | |
*** gfidente has quit IRC | 18:13 | |
EmilienM | dprince: I don't see where we manage endpoints | 18:20 |
EmilienM | dprince: in tht | 18:20 |
dprince | EmilienM: You mean where we configure the keystone endpoints? | 18:22 |
EmilienM | dprince: yeah | 18:22 |
dprince | EmilienM: we have a setup-endpoints tool for that | 18:22 |
EmilienM | I don't find the pupuppet class | 18:22 |
EmilienM | dprince: where can I read that code? | 18:22 |
dprince | EmilienM: http://git.openstack.org/cgit/openstack/tripleo-incubator/tree/scripts/setup-endpoints | 18:23 |
EmilienM | dprince: why not using puppet? | 18:23 |
dprince | EmilienM: kind of crude actually | 18:23 |
dprince | EmilienM: we could, perhaps should even | 18:23 |
EmilienM | yes we should | 18:23 |
EmilienM | because I'm deploying and I see ceilometer is missing | 18:23 |
*** rlandy has joined #tripleo | 18:24 | |
EmilienM | dprince: maybe because CEILOMETER_PASSWORD is unset? | 18:24 |
dprince | EmilienM: That is likely becaues devtest just didn't include the --ceilometer-password maybe? | 18:24 |
dprince | EmilienM: exactly | 18:24 |
dprince | EmilienM: it should be supported... just have to have the right variables set | 18:25 |
*** Marga_ has joined #tripleo | 18:28 | |
*** ukalifon has joined #tripleo | 18:31 | |
*** spzala has joined #tripleo | 18:33 | |
*** Marga_ has quit IRC | 18:35 | |
*** Marga_ has joined #tripleo | 18:35 | |
*** ifarkas has quit IRC | 18:40 | |
*** jprovazn has quit IRC | 18:43 | |
*** dshulyak has quit IRC | 18:52 | |
*** dshulyak has joined #tripleo | 18:55 | |
openstackgerrit | Yanis Guenane proposed stackforge/puppet-tripleo: Remove database code from puppet-tripleo https://review.openstack.org/182789 | 18:57 |
*** sdake has joined #tripleo | 19:06 | |
*** dshulyak has quit IRC | 19:07 | |
*** dshulyak has joined #tripleo | 19:08 | |
openstackgerrit | Yanis Guenane proposed openstack-infra/tripleo-ci: Specify Redis Virtual IP https://review.openstack.org/182795 | 19:11 |
*** panda has quit IRC | 19:13 | |
*** panda has joined #tripleo | 19:14 | |
*** dshulyak has quit IRC | 19:16 | |
*** Goneri has quit IRC | 19:17 | |
*** dshulyak has joined #tripleo | 19:17 | |
openstackgerrit | Yanis Guenane proposed openstack/tripleo-heat-templates: Add Redis as a Pacemaker resource when EnablePacemaker https://review.openstack.org/182617 | 19:21 |
*** dshulyak has quit IRC | 19:28 | |
*** dprince has quit IRC | 19:57 | |
*** aufi|afk has quit IRC | 19:58 | |
*** aufi|afk has joined #tripleo | 20:04 | |
*** aufi|afk has quit IRC | 20:07 | |
*** aufi|afk has joined #tripleo | 20:07 | |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Add support for generic controller validations https://review.openstack.org/174150 | 20:07 |
*** aufi|afk is now known as aufi | 20:09 | |
shardy | EmilienM: ^^ I think that should work OK now | 20:10 |
shardy | Note you have to rebuild your overcloud images with heat-config-script element, if you want to enable the controller-validation.yaml template | 20:10 |
shardy | Or add it via NodeUserData I guess | 20:11 |
*** jayg is now known as jayg|g0n3 | 20:12 | |
*** aufi has quit IRC | 20:13 | |
openstackgerrit | Jay Dobies proposed openstack/tripleo-heat-templates: Add Keystone as a Pacemaker resource when EnablePacemaker https://review.openstack.org/178694 | 20:15 |
*** Goneri has joined #tripleo | 20:18 | |
*** bvandenh has joined #tripleo | 20:22 | |
*** dsneddon has quit IRC | 20:25 | |
*** dsneddon has joined #tripleo | 20:28 | |
*** bvandenh has quit IRC | 20:33 | |
*** julim has quit IRC | 20:34 | |
*** Goneri has quit IRC | 20:37 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Create split out neutron networks via Heat. https://review.openstack.org/177843 | 20:43 |
openstackgerrit | Merged openstack/diskimage-builder: Add packages required for iscsi extension in agent https://review.openstack.org/178476 | 20:47 |
*** radez is now known as radez_g0n3 | 20:52 | |
*** Goneri has joined #tripleo | 20:52 | |
*** thrash is now known as thrash|g0ne | 21:00 | |
*** untriaged-bot has joined #tripleo | 21:00 | |
untriaged-bot | Untriaged bugs so far: | 21:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449852 | 21:00 |
openstack | Launchpad bug 1449852 in diskimage-builder "Buidling ramdisk with ironic-agent behind proxy fails" [Undecided,In progress] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 21:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1449854 | 21:00 |
openstack | Launchpad bug 1449854 in diskimage-builder "Ironic agent ramdisk built using disk-image-create fails with iscsi_ilo driver" [Undecided,Fix committed] - Assigned to Ramakrishnan G (rameshg87) (rameshg87) | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1454803 | 21:00 |
openstack | Launchpad bug 1454803 in tripleo "puppet: Neutron is not configured with L2 population" [Undecided,New] | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1452752 | 21:00 |
openstack | Launchpad bug 1452752 in tuskar "keystone_authtoken section is wrong in default shipped tuskar.conf.sample" [Undecided,Confirmed] | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1454802 | 21:00 |
openstack | Launchpad bug 1454802 in tripleo "puppet: Neutron does not use Nova notifications" [Undecided,New] | 21:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1452400 | 21:00 |
openstack | Launchpad bug 1452400 in diskimage-builder "debian-minimal builds still busted, now on locale issues" [Undecided,New] | 21:00 |
*** untriaged-bot has quit IRC | 21:00 | |
EmilienM | shardy: ack | 21:04 |
*** isq_ has joined #tripleo | 21:04 | |
*** NobodyCa1 has joined #tripleo | 21:07 | |
*** GheRiver1 has joined #tripleo | 21:07 | |
*** Goneri has quit IRC | 21:08 | |
*** shardy has quit IRC | 21:08 | |
*** larsks has quit IRC | 21:09 | |
*** cody-somerville has quit IRC | 21:09 | |
*** gchamoul has quit IRC | 21:09 | |
*** NobodyCam has quit IRC | 21:09 | |
*** tserong has quit IRC | 21:09 | |
*** greghaynes has quit IRC | 21:09 | |
*** GheRivero has quit IRC | 21:09 | |
*** isq has quit IRC | 21:09 | |
*** greghayn1 has joined #tripleo | 21:09 | |
*** isq_ has quit IRC | 21:09 | |
*** isq_ has joined #tripleo | 21:09 | |
*** tserong has joined #tripleo | 21:10 | |
*** tserong has joined #tripleo | 21:10 | |
*** cody-somerville has joined #tripleo | 21:11 | |
*** cody-somerville has joined #tripleo | 21:11 | |
*** larsks has joined #tripleo | 21:11 | |
*** NobodyCa1 is now known as NobodyCam | 21:11 | |
*** Marga_ has quit IRC | 21:11 | |
*** gchamoul has joined #tripleo | 21:11 | |
*** Marga_ has joined #tripleo | 21:12 | |
trown | in order to customize a tripleo deployment would one edit the hieradata directly? i.e. if I wanted a different neutron::core_plugin than ml2 would I just change: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/hieradata/controller.yaml#L67 | 21:24 |
*** greghayn1 is now known as greghaynes | 21:29 | |
*** erikmack has quit IRC | 21:41 | |
*** trown is now known as trown|outttypeww | 21:46 | |
dsneddon | trown|outttypeww, That's an example of something that should have been converted to a parameter. In this case, since it's hardcoded, then yes. Ordinarily, that would be a parameter that you could override in a resource file or overcloud-without-mergepy.yaml. | 21:50 |
dsneddon | trown|outttypeww, I would comb through the templates looking for other places where 'ml2' might be hardcoded, though, or other references to the Neutron plugin type. | 21:51 |
*** rhallisey has quit IRC | 21:59 | |
*** lucas-dinner has quit IRC | 22:03 | |
*** rlandy has quit IRC | 22:08 | |
spredzy | when running devtest I hit : glance-api already found in services list. Check glance for duplicate service name. | 22:11 |
spredzy | and then the install stop | 22:11 |
spredzy | does that ring a bell to anybody ? | 22:11 |
*** Marga_ has quit IRC | 22:11 | |
*** Marga_ has joined #tripleo | 22:11 | |
*** julim has joined #tripleo | 22:20 | |
spredzy | never mind ... | 22:34 |
-openstackstatus- NOTICE: Gerrit and Zuul are going offline for reboots to fix a security vulnerability. | 22:42 | |
*** ChanServ changes topic to "Gerrit and Zuul are going offline for reboots to fix a security vulnerability." | 22:42 | |
*** openstackgerrit has quit IRC | 22:47 | |
*** openstackgerrit has joined #tripleo | 22:49 | |
*** spzala has quit IRC | 22:50 | |
*** alop has quit IRC | 22:51 | |
*** ChanServ changes topic to "Deploying OpenStack Using OpenStack | Summit proposed topics: https://etherpad.openstack.org/p/tripleo-liberty-proposed-sessions | https://wiki.openstack.org/wiki/TripleO" | 23:00 | |
-openstackstatus- NOTICE: Gerrit and Zuul are back online. | 23:00 | |
*** yuanying has joined #tripleo | 23:01 | |
*** akrivoka has quit IRC | 23:10 | |
*** panda has quit IRC | 23:13 | |
*** panda has joined #tripleo | 23:14 | |
*** russell_h has quit IRC | 23:18 | |
*** russell_h has joined #tripleo | 23:19 | |
*** russell_h has quit IRC | 23:19 | |
*** russell_h has joined #tripleo | 23:19 | |
*** Marga_ has quit IRC | 23:20 | |
*** erikmack has joined #tripleo | 23:21 | |
*** Marga_ has joined #tripleo | 23:21 | |
*** russell_h has quit IRC | 23:26 | |
*** russell_h has joined #tripleo | 23:27 | |
*** russell_h has quit IRC | 23:27 | |
*** russell_h has joined #tripleo | 23:27 | |
*** russell_h has quit IRC | 23:28 | |
*** russell_h has joined #tripleo | 23:28 | |
*** russell_h has quit IRC | 23:30 | |
*** russell_h has joined #tripleo | 23:30 | |
*** BadCub is now known as BadCub-Hiding | 23:38 | |
*** russell_h has quit IRC | 23:52 | |
*** russell_h has joined #tripleo | 23:53 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!