*** retr0h has quit IRC | 00:05 | |
*** mkrai has quit IRC | 00:05 | |
*** bachp has quit IRC | 00:05 | |
*** mkrai has joined #openstack-kolla | 00:13 | |
*** cduchesne has quit IRC | 00:15 | |
*** cduchesne has joined #openstack-kolla | 00:19 | |
*** bachp has joined #openstack-kolla | 00:20 | |
*** retr0h has joined #openstack-kolla | 00:20 | |
*** masber has joined #openstack-kolla | 00:21 | |
*** masber has quit IRC | 00:29 | |
*** dardelean has joined #openstack-kolla | 00:40 | |
*** dardelean has quit IRC | 00:44 | |
*** kbaegis has joined #openstack-kolla | 00:45 | |
*** caowei has joined #openstack-kolla | 00:57 | |
*** ljjjustin has joined #openstack-kolla | 01:01 | |
*** dave-mccowan has joined #openstack-kolla | 01:06 | |
*** yangyapeng has joined #openstack-kolla | 01:10 | |
*** masber has joined #openstack-kolla | 01:10 | |
*** tovin07_ has joined #openstack-kolla | 01:12 | |
*** unicell has joined #openstack-kolla | 01:19 | |
*** daidv has joined #openstack-kolla | 01:20 | |
*** daidv_ has joined #openstack-kolla | 01:20 | |
*** unicell1 has quit IRC | 01:21 | |
*** daidv_ has quit IRC | 01:24 | |
*** kiennt26 has joined #openstack-kolla | 01:36 | |
*** dardelean has joined #openstack-kolla | 01:42 | |
*** kbaegis has quit IRC | 01:53 | |
*** pengdake has joined #openstack-kolla | 01:58 | |
*** dardelean has quit IRC | 02:26 | |
*** dardelean has joined #openstack-kolla | 02:27 | |
*** dardelean has quit IRC | 02:31 | |
*** zhurong has joined #openstack-kolla | 02:46 | |
spsurya | morning all | 02:56 |
---|---|---|
*** dave-mccowan has quit IRC | 03:09 | |
*** duonghq has joined #openstack-kolla | 03:09 | |
xinliang | morning spsurya | 03:11 |
duonghq | morning guys | 03:11 |
xinliang | hi all, there is a building error on my patch: kolla-build-centos-source finger://ze06.openstack.org/dc0e1c05c0044d06ba9332c44040692d : RETRY_LIMIT in 1m 00s | 03:12 |
xinliang | anyone knows what's the problem? | 03:12 |
xinliang | The ci tells: kolla-build-centos-source finger://ze06.openstack.org/dc0e1c05c0044d06ba9332c44040692d : RETRY_LIMIT in 1m 00s | 03:13 |
xinliang | And can't see any log for that. | 03:13 |
*** kong has quit IRC | 03:17 | |
*** kong has joined #openstack-kolla | 03:17 | |
*** zshi has joined #openstack-kolla | 03:33 | |
*** ddurst has quit IRC | 03:46 | |
*** pengdake has quit IRC | 04:01 | |
*** janonymous has joined #openstack-kolla | 04:01 | |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration https://review.openstack.org/525999 | 04:02 |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support compute node bootstrap https://review.openstack.org/531640 | 04:02 |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support compute node bootstrap https://review.openstack.org/531640 | 04:05 |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration https://review.openstack.org/525999 | 04:05 |
*** ddurst has joined #openstack-kolla | 04:11 | |
*** zshi is now known as zshi_afk | 04:21 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2] https://review.openstack.org/531641 | 04:22 |
*** pengdake has joined #openstack-kolla | 04:27 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Remove httplog in defaults section for haproxy.cfg https://review.openstack.org/531644 | 04:32 |
*** genek_ has quit IRC | 04:40 | |
*** genek has joined #openstack-kolla | 04:43 | |
openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Remove extra configuration item in solum.conf.j2 https://review.openstack.org/472647 | 04:45 |
openstackgerrit | Merged openstack/kolla-ansible stable/pike: Remove the unnecessary register.yml from upgrade.yml for Nova https://review.openstack.org/528799 | 04:45 |
openstackgerrit | Merged openstack/kolla-ansible master: change kuryr default driver option name https://review.openstack.org/529042 | 04:45 |
*** zhurong has quit IRC | 04:52 | |
*** caowei has quit IRC | 04:52 | |
*** ArminderSingh has joined #openstack-kolla | 05:12 | |
*** yangyapeng has quit IRC | 05:16 | |
*** yangyapeng has joined #openstack-kolla | 05:17 | |
*** janki has joined #openstack-kolla | 05:28 | |
*** skramaja has joined #openstack-kolla | 05:30 | |
*** caowei has joined #openstack-kolla | 05:30 | |
*** mikal has quit IRC | 05:33 | |
*** zhurong has joined #openstack-kolla | 05:52 | |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration https://review.openstack.org/525999 | 06:02 |
*** janki has quit IRC | 06:17 | |
*** janki has joined #openstack-kolla | 06:18 | |
openstackgerrit | Xinliang Liu proposed openstack/kolla-ansible master: Change debian mariadb backup tool to mariabackup https://review.openstack.org/531656 | 06:22 |
*** strgout has quit IRC | 06:31 | |
*** tvignaud has joined #openstack-kolla | 06:37 | |
*** jbadiapa has joined #openstack-kolla | 06:47 | |
*** zhangfei has joined #openstack-kolla | 06:47 | |
*** zshi_afk is now known as zshi | 06:49 | |
*** coolsvap has joined #openstack-kolla | 06:51 | |
openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: Enable vitrage dashboard https://review.openstack.org/530455 | 06:53 |
*** genek has quit IRC | 06:58 | |
*** pcaruana has joined #openstack-kolla | 07:04 | |
openstackgerrit | Merged openstack/kolla master: Add vitrage dashbord into horizon image https://review.openstack.org/530453 | 07:05 |
*** genek has joined #openstack-kolla | 07:07 | |
*** egonzalez has joined #openstack-kolla | 07:11 | |
*** bachp has quit IRC | 07:13 | |
*** mgkwill has quit IRC | 07:13 | |
*** mgkwill has joined #openstack-kolla | 07:13 | |
*** bachp has joined #openstack-kolla | 07:13 | |
*** brad[] has quit IRC | 07:15 | |
*** mrunge has quit IRC | 07:31 | |
*** mrunge has joined #openstack-kolla | 07:32 | |
*** Arminder has joined #openstack-kolla | 07:33 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2] https://review.openstack.org/531641 | 07:34 |
*** dciabrin has joined #openstack-kolla | 07:36 | |
openstackgerrit | Merged openstack/kolla-ansible master: Add rsync env for account & container replicators https://review.openstack.org/527402 | 07:37 |
openstackgerrit | Merged openstack/kolla-ansible stable/pike: Fix the cinder backup action failed when use external ceph https://review.openstack.org/530222 | 07:40 |
*** mrunge_ has joined #openstack-kolla | 07:41 | |
*** mrunge has quit IRC | 07:44 | |
*** zhangfei has quit IRC | 07:44 | |
*** mrunge_ has quit IRC | 07:46 | |
hrw | morning | 07:51 |
*** zhangfei has joined #openstack-kolla | 08:00 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2] https://review.openstack.org/531641 | 08:01 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2] https://review.openstack.org/531641 | 08:01 |
*** wojdec has joined #openstack-kolla | 08:06 | |
*** b_bezak has joined #openstack-kolla | 08:07 | |
*** mrostecki1 is now known as mrostecki | 08:11 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3] https://review.openstack.org/531681 | 08:13 |
*** shardy has joined #openstack-kolla | 08:18 | |
*** trozet has quit IRC | 08:19 | |
*** wojdec1 has joined #openstack-kolla | 08:19 | |
*** honza has quit IRC | 08:20 | |
*** honza has joined #openstack-kolla | 08:20 | |
*** honza is now known as Guest14500 | 08:20 | |
*** trozet has joined #openstack-kolla | 08:20 | |
*** wojdec has quit IRC | 08:21 | |
*** wojdec1 has quit IRC | 08:22 | |
*** wojdec has joined #openstack-kolla | 08:23 | |
*** lvdombrkr has joined #openstack-kolla | 08:28 | |
*** kevzha01 has joined #openstack-kolla | 08:30 | |
*** kevzha01 is now known as kevinz | 08:30 | |
openstackgerrit | caoyuan proposed openstack/kolla-ansible master: Ensure docker running before execute the destroy action https://review.openstack.org/531688 | 08:34 |
*** ktibi has joined #openstack-kolla | 08:34 | |
*** wojdec1 has joined #openstack-kolla | 08:35 | |
openstackgerrit | caoyuan proposed openstack/kolla-ansible master: Ensure docker running before execute the destroy action https://review.openstack.org/531688 | 08:36 |
*** wojdec has quit IRC | 08:37 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3] https://review.openstack.org/531681 | 08:37 |
*** nathharp has joined #openstack-kolla | 08:40 | |
*** athomas has joined #openstack-kolla | 08:41 | |
openstackgerrit | Merged openstack/kolla-ansible master: Mount kolla_logs volume in bifrost_deploy https://review.openstack.org/530545 | 08:44 |
egonzalez | who's going to Dublin PTG? | 08:45 |
* hrw plans to | 08:45 | |
egonzalez | looking to ask for holidays, but wants to know if people will be there :) | 08:46 |
*** dalvarez has joined #openstack-kolla | 08:47 | |
*** gfidente has joined #openstack-kolla | 08:51 | |
*** gfidente has quit IRC | 08:51 | |
*** gfidente has joined #openstack-kolla | 08:51 | |
openstackgerrit | Merged openstack/kolla-ansible master: Ensure inspector uses internal ironic interface https://review.openstack.org/530544 | 08:52 |
openstackgerrit | Merged openstack/kolla-ansible master: Add ironic inspector to fluentd log collection https://review.openstack.org/530506 | 08:52 |
openstackgerrit | Pierre Hanselmann proposed openstack/kolla-ansible stable/pike: Fix dhcp_agent configuration for vmware DVS https://review.openstack.org/531691 | 08:54 |
*** skramaja_ has joined #openstack-kolla | 08:54 | |
*** skramaja has quit IRC | 08:54 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3] https://review.openstack.org/531681 | 08:55 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2] https://review.openstack.org/531641 | 08:55 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2] https://review.openstack.org/531641 | 08:55 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3] https://review.openstack.org/531681 | 08:55 |
*** skramaja has joined #openstack-kolla | 08:59 | |
*** skramaja_ has quit IRC | 08:59 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 4] https://review.openstack.org/531695 | 08:59 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 4] https://review.openstack.org/531695 | 09:00 |
*** jascott1 has quit IRC | 09:02 | |
*** sbezverk has quit IRC | 09:11 | |
*** Guest14500 is now known as honza | 09:14 | |
*** owalsh_ is now known as owalsh | 09:15 | |
*** mrunge has joined #openstack-kolla | 09:19 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Support squash docker images layers https://review.openstack.org/525871 | 09:21 |
hrw | can someone look at https://review.openstack.org/#/c/522712/ and get that merged? | 09:24 |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support compute node bootstrap https://review.openstack.org/531640 | 09:26 |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration https://review.openstack.org/525999 | 09:26 |
*** shardy has quit IRC | 09:27 | |
*** dardelean has joined #openstack-kolla | 09:30 | |
*** egonzalez has quit IRC | 09:32 | |
*** shardy has joined #openstack-kolla | 09:32 | |
*** dardelean has quit IRC | 09:36 | |
*** dardelean has joined #openstack-kolla | 09:36 | |
*** dardelean has quit IRC | 09:40 | |
*** egonzalez has joined #openstack-kolla | 09:43 | |
openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: Restructure the vpnaas roles https://review.openstack.org/530022 | 09:44 |
*** dardelean has joined #openstack-kolla | 09:45 | |
egonzalez | sdake, good luck with individual board election :) | 09:49 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 5] https://review.openstack.org/531702 | 09:51 |
spsurya | egonzalez: i am in plan.....but not 100% sure | 09:52 |
hrw | egonzalez, spsurya: https://review.openstack.org/#/c/522712/ - can you? it is now the most important aarch64 blocker in kolla-ansible. | 09:53 |
egonzalez | hrw, why not adding a docker_package_version: {% '..ce' if aarch64 else '..1.12' %} and set {{ docker_package_version }} in install package list? this avoid a duplicated file just for a package name change | 09:55 |
*** bootsha has joined #openstack-kolla | 09:56 | |
*** wojdec1 has quit IRC | 09:57 | |
hrw | egonzalez: can you write that in comment? I will look at doing that later today | 09:57 |
egonzalez | on ti | 09:57 |
egonzalez | it | 09:57 |
hrw | thx | 09:57 |
*** wojdec has joined #openstack-kolla | 09:58 | |
*** duonghq has quit IRC | 09:58 | |
*** wojdec1 has joined #openstack-kolla | 10:02 | |
*** ljjjustin has quit IRC | 10:02 | |
*** zhurong has quit IRC | 10:02 | |
*** jascott1 has joined #openstack-kolla | 10:03 | |
*** wojdec has quit IRC | 10:04 | |
spsurya | hrw: ok | 10:04 |
*** kiennt26 has quit IRC | 10:07 | |
*** dardelean has quit IRC | 10:08 | |
*** dardelean has joined #openstack-kolla | 10:09 | |
*** pbourke has quit IRC | 10:12 | |
*** pbourke has joined #openstack-kolla | 10:12 | |
*** dardelean has quit IRC | 10:13 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 6] https://review.openstack.org/531707 | 10:20 |
*** tovin07_ has quit IRC | 10:21 | |
-openstackstatus- NOTICE: zuul has been restarted, all queues have been reset. please recheck your patches when appropriate | 10:25 | |
*** bootsha_ has joined #openstack-kolla | 10:31 | |
*** pengdake has quit IRC | 10:31 | |
*** bootsha has quit IRC | 10:33 | |
*** bootsha_ has quit IRC | 10:36 | |
*** chandankumar has joined #openstack-kolla | 10:36 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 7] https://review.openstack.org/531711 | 10:36 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [Final] https://review.openstack.org/531712 | 10:48 |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 4] https://review.openstack.org/531695 | 10:50 |
*** sambetts|afk is now known as sambetts | 10:50 | |
*** bootsha has joined #openstack-kolla | 10:50 | |
*** nanliu_ has quit IRC | 10:51 | |
openstackgerrit | Dai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 5] https://review.openstack.org/531702 | 10:51 |
*** nanliu_ has joined #openstack-kolla | 10:52 | |
openstackgerrit | Chandan Kumar proposed openstack/kolla master: Include building python-tempestconf from source in Tempest https://review.openstack.org/531714 | 10:53 |
hrw | egonzalez: thanks for suggestion. rewrote. testing now | 10:58 |
*** caowei has quit IRC | 11:00 | |
*** numans has quit IRC | 11:06 | |
*** numans has joined #openstack-kolla | 11:08 | |
*** bootsha has quit IRC | 11:09 | |
hrw | hm. from +67, -10 to +28, -3 ;D | 11:19 |
hrw | +27, -3 | 11:23 |
*** bootsha has joined #openstack-kolla | 11:30 | |
*** bootsha has quit IRC | 11:35 | |
*** zhurong has joined #openstack-kolla | 11:35 | |
openstackgerrit | Merged openstack/kolla master: Fix MariaDB bootstrap for 10.1 version https://review.openstack.org/529199 | 11:36 |
hrw | yay! | 11:43 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: Handle newer Docker and support AArch64 https://review.openstack.org/522712 | 11:50 |
hrw | egonzalez: you should be happier with this version ;) | 11:50 |
*** egonzalez has quit IRC | 11:56 | |
*** jistr is now known as jistr|mtg | 11:58 | |
*** dave-mccowan has joined #openstack-kolla | 12:09 | |
*** bootsha has joined #openstack-kolla | 12:13 | |
*** rhallisey has joined #openstack-kolla | 12:17 | |
*** bootsha has quit IRC | 12:19 | |
*** bootsha has joined #openstack-kolla | 12:20 | |
*** janki has quit IRC | 12:24 | |
*** bootsha has quit IRC | 12:34 | |
*** bootsha has joined #openstack-kolla | 12:37 | |
*** bootsha has quit IRC | 12:43 | |
*** bootsha has joined #openstack-kolla | 12:44 | |
*** bootsha_ has joined #openstack-kolla | 12:47 | |
*** bootsha_ has quit IRC | 12:47 | |
*** dave-mccowan has quit IRC | 12:47 | |
*** bootsha_ has joined #openstack-kolla | 12:48 | |
*** dave-mccowan has joined #openstack-kolla | 12:49 | |
*** bootsha has quit IRC | 12:50 | |
*** zhurong has quit IRC | 13:01 | |
*** hieulq has quit IRC | 13:04 | |
*** skramaja has quit IRC | 13:05 | |
*** hieulq has joined #openstack-kolla | 13:05 | |
*** dardelean has joined #openstack-kolla | 13:10 | |
*** dardelean has quit IRC | 13:14 | |
*** bootsha_ has quit IRC | 13:14 | |
*** dardelean has joined #openstack-kolla | 13:14 | |
*** janonymous has quit IRC | 13:22 | |
*** pcaruana has quit IRC | 13:25 | |
*** pcaruana has joined #openstack-kolla | 13:25 | |
rwellum | jascott1: quick start looks interesting | 13:26 |
*** egonzalez has joined #openstack-kolla | 13:32 | |
*** gkadam has joined #openstack-kolla | 13:33 | |
*** jistr|mtg is now known as jistr | 13:35 | |
*** coolsvap has quit IRC | 13:36 | |
*** mchlumsky has joined #openstack-kolla | 13:39 | |
hrw | rwellum: hi | 13:44 |
rwellum | Hi hrw | 13:45 |
hrw | rwellum: have a moment to look at https://review.openstack.org/522712 patch? it adds support for newer docker so we can deploy on aarch64 | 13:45 |
rwellum | :wave hrw | 13:45 |
rwellum | Haha | 13:45 |
rwellum | Yes will look | 13:46 |
hrw | thanks | 13:46 |
hrw | I rewrote Tone's patch so it is now much shorter and easier to review | 13:46 |
rwellum | hrw: do we have docker-ce official support for non-AArch64? | 13:52 |
hrw | rwellum: docker-ce can be installed on x86-64 for sure. Just it was discussed that most of people here prefer to stay at 1.12 so I kept that as it was | 13:53 |
rwellum | I'm not sure if that's a preference or just it was stable and worked? | 13:56 |
hrw | rwellum: in other words: I kept 1.12 as it was and use docker-ce on aarch64 because this is version we (as Linaro) provide for debian/aarch64 ;d | 13:57 |
rwellum | hrw: so it works for kolla-k8s on debian/aarch64 ? | 13:58 |
hrw | rwellum: kolla-ansible | 14:00 |
hrw | rwellum: from what I know there are some guys working on k8s on aarch64. so same docker-ce for them | 14:00 |
rwellum | Ok cool | 14:00 |
openstackgerrit | Chandan Kumar proposed openstack/kolla master: Include building python-tempestconf from source in Tempest https://review.openstack.org/531714 | 14:02 |
hrw | rwellum: this patch is our main blocker for deploy now | 14:02 |
chandankumar | mandre: i think i have fixed all the comments please verify once you have time :-) | 14:03 |
rwellum | hrw: just a couple of nits | 14:09 |
sdake | egonzalez thx :) | 14:10 |
hrw | rwellum: thx | 14:11 |
*** yangyapeng has quit IRC | 14:11 | |
hrw | rwellum: that line in commit has 73 chars ;D | 14:11 |
mandre | chandankumar: I reviewed again, back to you :) | 14:12 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: Handle newer Docker and support AArch64 https://review.openstack.org/522712 | 14:13 |
hrw | rwellum, mandre: please take a look and vote | 14:14 |
*** zhangfei has quit IRC | 14:25 | |
*** lpetrut has joined #openstack-kolla | 14:31 | |
*** dardelean has quit IRC | 14:34 | |
*** dardelean has joined #openstack-kolla | 14:34 | |
*** hw_wutianwei has joined #openstack-kolla | 14:36 | |
*** kbaegis has joined #openstack-kolla | 14:42 | |
openstackgerrit | Chandan Kumar proposed openstack/kolla master: Include building python-tempestconf from source in Tempest https://review.openstack.org/531714 | 14:50 |
chandankumar | mandre: ^^ i think this time i have done that, please have a look :-) | 14:50 |
mandre | chandankumar: looks about right, let's wait for CI | 14:52 |
chandankumar | mandre: sweet, Thank you :-) | 14:53 |
*** yangyapeng has joined #openstack-kolla | 14:59 | |
*** sbezverk has joined #openstack-kolla | 15:09 | |
*** ktibi has quit IRC | 15:20 | |
openstackgerrit | Jianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration https://review.openstack.org/525999 | 15:27 |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Support squash docker images layers https://review.openstack.org/525871 | 15:27 |
*** SamYaple_ has quit IRC | 15:30 | |
*** SamYaple has joined #openstack-kolla | 15:30 | |
*** rmart04 has joined #openstack-kolla | 15:31 | |
*** b_bezak has quit IRC | 15:31 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Remove httplog and forwardfor in defaults section for haproxy.cfg https://review.openstack.org/531644 | 15:33 |
*** kbaegis has quit IRC | 15:40 | |
*** ktibi has joined #openstack-kolla | 15:41 | |
ktibi | anyone can check this issue ? https://bugs.launchpad.net/kolla-ansible/+bug/1740399 | 15:49 |
openstack | Launchpad bug 1740399 in kolla-ansible "Deployment fails with 3 Ceph mon" [Undecided,New] | 15:49 |
kolla-slack | <aslancimbom100> ktibi how many ceph monitors do you have? | 15:54 |
sdake | time to delete ceph from kolla ;) | 15:55 |
kolla-slack | <aslancimbom100> :+1: | 15:55 |
kolla-slack | <aslancimbom100> ceph-ansible is very good option for anyone who wants it | 15:56 |
ktibi | ceph-ansible can deploy in docker ? | 15:57 |
ktibi | aslancimbom100 I have 3 mon | 15:57 |
kolla-slack | <aslancimbom100> yes with or without docker | 15:57 |
*** SamYaple has quit IRC | 15:57 | |
*** SamYaple has joined #openstack-kolla | 15:57 | |
*** SamYaple has quit IRC | 15:58 | |
*** SamYaple has joined #openstack-kolla | 15:58 | |
*** TxGirlGeek has joined #openstack-kolla | 15:59 | |
ktibi | aslancimbom100 no doc about docker in http://docs.ceph.com/ceph-ansible | 15:59 |
sdake | aslancimbom100 the problem is upgrades.. | 16:00 |
ktibi | maybe kolla can use ceph-ansible ? | 16:01 |
*** hw_wutianwei has quit IRC | 16:02 | |
*** rmart04 has quit IRC | 16:03 | |
sdake | ktibi we have talked long and hard about it - and the issue always comes back to upgrades | 16:06 |
*** gfidente has quit IRC | 16:09 | |
*** gfidente has joined #openstack-kolla | 16:10 | |
*** gfidente has quit IRC | 16:10 | |
*** gfidente has joined #openstack-kolla | 16:10 | |
ktibi | sdake ok so If I talk ceph luminous in kolla ... | 16:11 |
sdake | ktibi i'm the wrong fella to ask - try inc0 :) | 16:11 |
kolla-slack | <aslancimbom100> ktibi yes documentation doesn't mention it, however if you take a look at github files docker options are obvious | 16:13 |
kolla-slack | <aslancimbom100> ie it says `site.yml.sample`...that would be `site-docker.yml.sample` | 16:14 |
ktibi | aslancimbom100 yes I can see, | 16:15 |
*** brad[] has joined #openstack-kolla | 16:18 | |
hrw | aslancimbom100: first I need to check how to build ceph-ansible images ;D | 16:20 |
ktibi | hrw so easy to build, it's same image for all | 16:23 |
ktibi | https://github.com/ceph/ceph-ansible/blob/4f7223d5fb5317e7fde9202d8b4f904a534420bf/docker/Linode.dockerfile | 16:23 |
hrw | ktibi: build for arm64 I mean ;D | 16:23 |
ktibi | aslancimbom100 I saw playbook and how ceph-ansible works, a lot of command like shell: cat .... , if docker ... i don't know if the project is very mature :/ | 16:25 |
kolla-slack | <aslancimbom100> its opensource project, it only goes so far | 16:26 |
kolla-slack | <aslancimbom100> or you could pay some company to install it for you ;) | 16:26 |
kolla-slack | <aslancimbom100> Trial and error, i installed it within few hours | 16:26 |
*** krtaylor has quit IRC | 16:27 | |
*** cah_link has joined #openstack-kolla | 16:27 | |
*** krtaylor_ has joined #openstack-kolla | 16:27 | |
ktibi | aslancimbom100 yes it's opensource but the project is on ceph official repo, so redhat behind. And I'm not fan about deploy one image and play exec command in the container :/ | 16:28 |
ktibi | I prefer kolla method | 16:29 |
*** jistr is now known as jistr|off|mtg | 16:33 | |
kfox1111 | morning. | 16:40 |
hrw | hi kfox1111 | 16:41 |
kfox1111 | hi. :) | 16:42 |
hrw | https://review.openstack.org/#/c/522712/ (handle newer docker) patch still waits for brave reviewers! sdake? egonzalez? mandre? | 16:42 |
*** cah_link has quit IRC | 16:42 | |
hrw | much shorter than earlier version, just 5 files changed, no new ones | 16:42 |
*** david-lyle has quit IRC | 16:47 | |
*** david-lyle has joined #openstack-kolla | 16:48 | |
*** pcaruana has quit IRC | 16:48 | |
*** kbaegis has joined #openstack-kolla | 16:51 | |
*** kbaegis has joined #openstack-kolla | 16:51 | |
*** kbaegis has joined #openstack-kolla | 16:52 | |
*** kbaegis has quit IRC | 16:52 | |
*** david-lyle has quit IRC | 16:52 | |
*** kbaegis has joined #openstack-kolla | 16:52 | |
kbaegis | Hi all. I'm seeing this: /var/lib/ansible/.my.cnf on my kolla-ansible deploy. | 16:52 |
sbezverk | kfox1111: if you want, I have steps to test csi hostpath driver, it is for a single node cluster though.. | 16:52 |
kfox1111 | sbezverk: interesting. yeah, I'd like to see that, as it will show what is to come. :) | 16:53 |
kbaegis | Anyone had that issue before? Looking for a quick resolution. | 16:56 |
sbezverk | kfox1111: Here is general doc https://github.com/kubernetes-csi/docs.. here is my yaml. You only need to change image location. http://paste.openstack.org/show/640979/ | 17:00 |
kfox1111 | cool. thanks. :) | 17:01 |
kfox1111 | sbezverk: just confirmed mount propigation works with crio. :) | 17:01 |
sbezverk | kfox1111: btw it also required for csi | 17:02 |
kfox1111 | yeah. kind of figured it would be. :) I'm glad they pushed for it being a requirement. | 17:02 |
kfox1111 | that should keep the door wide open. :) | 17:02 |
kfox1111 | sbezverk: interesting pastebin. plan on making a chart out of that? | 17:03 |
sbezverk | kfox1111: sorry, probably later, the main focus now is csi rbd driver | 17:04 |
kfox1111 | yeah, that would be very useful. | 17:05 |
kfox1111 | they have been stonewalling userspace rbd and cephfs for a while. | 17:06 |
kfox1111 | the user space versions often have more features then the kernel versions. | 17:06 |
*** egonzalez has quit IRC | 17:08 | |
SamYaple | ktibi: kolla method was first! | 17:08 |
SamYaple | ktibi: me and leseb were writting the kolla-ceph and ceph-asnible stuff at the same time. i maintain we got docker imeplemntation done first.... (but we both had working POC within a few hours of each other :P ) | 17:09 |
SamYaple | the ceph-docker stuff was actually pulled in from an external repo. and i agree, the layout is bad with all the shell/cat/if docker stuff | 17:10 |
*** TxGirlGeek has quit IRC | 17:11 | |
kfox1111 | sbezverk: how do you feel about https://review.openstack.org/#/c/458649/10 ? | 17:15 |
kfox1111 | rwellum has been using it for a bit and likes it. | 17:15 |
kfox1111 | I just got fluent-bit to work with an elasticsearch-operator managed cluster too. so I think that may be a very easy way to go. | 17:15 |
*** jascott1 has quit IRC | 17:17 | |
*** TxGirlGeek has joined #openstack-kolla | 17:17 | |
*** jascott1 has joined #openstack-kolla | 17:17 | |
sbezverk | it is long overdue to have some decent logging in kolla-k8s ;) | 17:18 |
kfox1111 | yeah. been waiting for something to manage es to come about. | 17:18 |
kfox1111 | finally got elasticsearch-operator to work this week. | 17:18 |
kfox1111 | well, this last week. | 17:18 |
kfox1111 | without some place to catch the log messages, it was kind of not worth focusing on. | 17:19 |
*** unicell has quit IRC | 17:20 | |
kfox1111 | I think now, you could probably use elasticserach-operator and rook and have everything inside k8s. | 17:20 |
kfox1111 | though I don't think they have fully fleshed out upgrades yet. | 17:20 |
kfox1111 | but would be great for a dev env I think. | 17:20 |
kfox1111 | deploy kube, use kube to deploy everything else. | 17:21 |
*** nathharp has quit IRC | 17:22 | |
*** itlinux_ has joined #openstack-kolla | 17:27 | |
inc0 | good mornign | 17:27 |
kfox1111 | inc0: good morning. | 17:28 |
inc0 | ktibi: so we talked with leseb in Denver about ceph-ansible - we've decided that for now we will keep kolla way in kolla - upgrade to L | 17:28 |
inc0 | but probably we'll want to move to ceph-ansible for next LTS, all depends on migration path | 17:28 |
*** itlinux_ has quit IRC | 17:30 | |
*** itlinux_ has joined #openstack-kolla | 17:30 | |
*** itlinux_ has quit IRC | 17:31 | |
*** itlinux_ has joined #openstack-kolla | 17:31 | |
*** david-lyle has joined #openstack-kolla | 17:34 | |
*** itlinux_ has quit IRC | 17:34 | |
*** jascott1 has quit IRC | 17:34 | |
*** itlinux_ has joined #openstack-kolla | 17:36 | |
*** itlinux_ has quit IRC | 17:37 | |
*** itlinux_ has joined #openstack-kolla | 17:38 | |
*** itlinux_ has quit IRC | 17:42 | |
*** itlinux_ has joined #openstack-kolla | 17:42 | |
*** itlinux_ has quit IRC | 17:43 | |
*** david-lyle has quit IRC | 17:47 | |
*** jascott1 has joined #openstack-kolla | 17:48 | |
*** itlinux_ has joined #openstack-kolla | 17:49 | |
*** itlinux_ has quit IRC | 17:49 | |
*** itlinux has joined #openstack-kolla | 17:50 | |
*** b_bezak has joined #openstack-kolla | 17:52 | |
rwellum | kfox1111: the patch is also great for folks using the infra but who aren't knowledgeable on containers etc. So yeah I like it. :) | 17:54 |
*** itlinux has quit IRC | 17:55 | |
*** kbaegis has quit IRC | 17:55 | |
*** itlinux has joined #openstack-kolla | 17:55 | |
*** b_bezak has quit IRC | 17:56 | |
*** bmace has joined #openstack-kolla | 18:01 | |
*** mdorman has joined #openstack-kolla | 18:06 | |
*** kristaps_ has joined #openstack-kolla | 18:06 | |
*** lvdombrkr has quit IRC | 18:09 | |
*** strgout has joined #openstack-kolla | 18:16 | |
strgout | howdy all, looking for some input for ceph. I have a 2 node setup with control/network/storage on one box and compute on a different one. | 18:17 |
strgout | Storage is currently a md raid10 with 12 JBOD disks. I'm going to nuke the raid 10 and start over so i can use Ceph. Any pointers on how to setup the disks? I was thinking 2 - 4 for OS just to keep the mysql database snappy as possible but that might be over kill. | 18:18 |
strgout | disks are 2TB 7200 SAS drives. | 18:19 |
strgout | BTW my compute node does have 4 900gb SAS drives also. Currently in raid 10 as well. Both boxes have 10 gig ports as well. | 18:20 |
jascott1 | strgout for kolla-ansible? | 18:22 |
strgout | yup | 18:22 |
*** Rodrigo_BR has joined #openstack-kolla | 18:23 | |
strgout | well my deploy host is different but ill be attempting this with kolla-ansible. | 18:23 |
strgout | I read the ceph dock i'm just not sure about journal drive(s?). I'm assuming cache teir is no go since i don't have SSD. | 18:24 |
kfox1111 | probably fine just using one disk per osd if you just have spinners. | 18:25 |
kfox1111 | ignore cache tier if you don't have ssd's. no real benifit in them, unless oou are playing with EC teirs or ssd's. | 18:25 |
strgout | ok what about a journal drive? and should there be one then one? | 18:26 |
strgout | err more then one | 18:27 |
*** shardy is now known as shardy_afk | 18:27 | |
strgout | and to be clear i want to use for for disk volumes for VMs as well. | 18:30 |
*** shardy_afk has quit IRC | 18:35 | |
*** shardy_afk has joined #openstack-kolla | 18:36 | |
inc0 | it would be highly beneficial to put journals on ssd | 18:38 |
inc0 | but all spindle doesn't really matter | 18:39 |
inc0 | in any case, you can put OSD on every drive with no additional confs imho | 18:39 |
inc0 | one thing to do is after deployment check your ceph | 18:40 |
inc0 | docker exec -it ceph_mon ceph -s | 18:40 |
inc0 | you'll probably see that number of pg/pgp is incorrect, fix it accordingly to size | 18:40 |
inc0 | of env | 18:40 |
inc0 | there are ceph docs about that | 18:41 |
*** lpetrut has quit IRC | 18:41 | |
*** gfidente is now known as gfidente|afk | 18:53 | |
*** TxGirlGeek has quit IRC | 18:55 | |
kfox1111 | back. | 18:59 |
kfox1111 | just be careful to have some redundancy if you use ssd's. | 18:59 |
*** TxGirlGeek has joined #openstack-kolla | 19:00 | |
kfox1111 | ceph is great because it lets you pull things out and not take out the whole system. | 19:00 |
kfox1111 | if you only put one ssd in or that kind of thing, then you break the model. | 19:00 |
strgout | hmm ok so should be looking at 2 SSD if i was going to go that way? | 19:00 |
strgout | i don't think i'll be doing SSD for a while FYI. | 19:01 |
strgout | CC is already on fire. | 19:01 |
strgout | bought a sound proof cabinet. :D | 19:02 |
inc0 | so nobody can hear you scream? | 19:04 |
*** TxGirlGeek has quit IRC | 19:04 | |
strgout | if i don't get this openstack setup up and running yes very much so. | 19:04 |
kfox1111 | yeah. make sure you have at least two ssd's or 0. you don't want one drive blowing out all your storage. | 19:07 |
strgout | well i have 0 covered at this point. | 19:07 |
kfox1111 | :) | 19:08 |
*** gfidente|afk has quit IRC | 19:11 | |
*** gfidente has joined #openstack-kolla | 19:12 | |
*** gfidente has quit IRC | 19:12 | |
*** gfidente has joined #openstack-kolla | 19:12 | |
*** ManoX has joined #openstack-kolla | 19:13 | |
*** unicell has joined #openstack-kolla | 19:15 | |
*** harlowja has joined #openstack-kolla | 19:19 | |
strgout | i don't know.. i may just stick with the iscis. I deployed with that but i'm seeing the compute isn't trying to open the tcp connection to the control box. Like nothing showing in netstat -anp | grep 3260 | 19:20 |
*** TxGirlGeek has joined #openstack-kolla | 19:21 | |
*** pramodrj07 has joined #openstack-kolla | 19:22 | |
*** pramodrj07 has quit IRC | 19:22 | |
*** Rodrigo_BR has quit IRC | 19:30 | |
*** lpetrut has joined #openstack-kolla | 19:41 | |
mwynne | Hi guys. What's the best process to restart containers after a host reboots? | 19:43 |
*** david-lyle has joined #openstack-kolla | 19:43 | |
SamYaple | mwynne: probably running the playbooks to ensure proper starting order | 19:45 |
openstackgerrit | Brian Haley proposed openstack/kolla-kubernetes master: Start using nova_metadata_host https://review.openstack.org/531892 | 19:46 |
mwynne | SamYaple: Which playbooks? | 19:51 |
mwynne | I also don't seem to have a '/var/log/kolla/` directory, which I think should exist, right? | 19:52 |
*** jgriffith_ is now known as jgriffith | 20:03 | |
*** strgout has quit IRC | 20:10 | |
*** ManoX has quit IRC | 20:11 | |
rwellum | mwynne: kolla-ansible or kolla-kubernetes? | 20:12 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support https://review.openstack.org/458649 | 20:13 |
mwynne | rwellum: Ansible. (I found the logs, just need to get my services and containers going again) | 20:13 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support https://review.openstack.org/458649 | 20:14 |
kfox1111 | rwellum: I updated the ps. I think it should be mergable. | 20:15 |
*** itlinux has quit IRC | 20:19 | |
*** itlinux has joined #openstack-kolla | 20:29 | |
*** openstack has quit IRC | 20:38 | |
*** openstack has joined #openstack-kolla | 20:41 | |
*** ChanServ sets mode: +o openstack | 20:41 | |
*** TxGirlGeek has quit IRC | 20:45 | |
rwellum | kfox1111: one basic qn about the PS: std_logger: true - so you're setting this to default - but it's over-writeable correct? | 20:52 |
*** TxGirlGeek has joined #openstack-kolla | 20:54 | |
*** TxGirlGeek has quit IRC | 20:55 | |
*** TxGirlGeek has joined #openstack-kolla | 20:56 | |
*** Angeh has joined #openstack-kolla | 20:56 | |
*** ktibi has quit IRC | 20:58 | |
*** mdorman has quit IRC | 21:01 | |
*** openstackgerrit has quit IRC | 21:03 | |
hrw | fsck. zuul died again | 21:11 |
kfox1111 | rwellum: yeah. | 21:12 |
kfox1111 | setting it to false sets it back to the old behavior. | 21:12 |
*** rhallisey has quit IRC | 21:14 | |
rwellum | kfox1111: ack - looking at it | 21:20 |
*** dardelean has quit IRC | 21:20 | |
*** dardelean has joined #openstack-kolla | 21:21 | |
mwynne | rwellum: Did you have suggestions for getting the containers started in the correct order? | 21:23 |
*** dardelean has quit IRC | 21:26 | |
mwynne | rwellum: I'm having issues getting a compute node back up and running. the libvirt and nova compute containers aren't cooperating. | 21:28 |
*** david-lyle has quit IRC | 21:28 | |
*** gfidente has quit IRC | 21:29 | |
mwynne | I haven't been able to find anything in the docs regarding this, but I'm probably just missing it, right? | 21:31 |
rwellum | mwynne: in general the docs aren't great regarding what to when things go wrong. But inc0 is better qualified to answer. | 21:41 |
inc0 | mwynne: how "not cooperating"? | 21:42 |
mwynne | inc0: I just rebooted my compute node and all the containers came up except for the nova-compute one. I started it with `docker start`, and the VM came up shortly after that. However, it couldn't get a DHCP address for some reason now. | 21:43 |
mwynne | inc0: What's the process like for restarting processes/containers in the event that things go wrong? | 21:44 |
inc0 | mwynne: kolla-ansible? | 21:44 |
mwynne | This was running just fine for me a few days ago, and now nothing is working. I haven't changed anything :S | 21:44 |
mwynne | inc0: Yes. | 21:44 |
inc0 | hmm that's strange, well dhcp issue might be due to neutron agent | 21:45 |
inc0 | if you're running multinode tho | 21:45 |
inc0 | mariadb hates being restarted | 21:45 |
inc0 | you need to perform manual recovery | 21:46 |
mwynne | inc0: I am running multinode | 21:46 |
inc0 | right | 21:46 |
inc0 | so probably mariab cries | 21:46 |
mwynne | inc0: How can I verify that? | 21:46 |
inc0 | check logs if nodes can handle mariadb | 21:47 |
inc0 | most of services should raise errors | 21:47 |
inc0 | http://galeracluster.com/documentation-webpages/restartingcluster.html this is doc that descrbes what needs to happen | 21:48 |
mwynne | inc0: Looks ok | 21:48 |
inc0 | soo...no errors anywhere? | 21:49 |
mwynne | Not that I've seen | 21:49 |
inc0 | and vms still doesn't worj? | 21:49 |
mwynne | neutron-server logs should be full of errors, right? | 21:49 |
mwynne | Well, the VM is running | 21:49 |
inc0 | yeha I guess | 21:49 |
mwynne | but it's not getting an IP from DHCP | 21:49 |
inc0 | is dhcp agent healthy? | 21:49 |
mwynne | Seems to be | 21:50 |
mwynne | logs look ok | 21:50 |
inc0 | soo I guess it's tcpdump time.. | 21:50 |
mwynne | Oh, maybe not. neutron.agent.ovsdb.native.vlog [-] 10000-ms timeout __log_wakeup /usr/lib/python2.7/dist-packages/ovs/poller.py:187 | 21:50 |
inc0 | you need to pinpoint where dhcp request/response is dropped | 21:50 |
*** TxGirlGeek has quit IRC | 21:50 | |
mwynne | inc0: is there a way I can just restart all containers, or similar? | 21:51 |
inc0 | also check docker ps -a if there are any failed containers that don't restart | 21:51 |
inc0 | restarting all of them is probably bad idea | 21:51 |
mwynne | inc0: Well, what happens if/when a server is rebooted? | 21:52 |
inc0 | it should just work tbh | 21:52 |
*** openstackgerrit has joined #openstack-kolla | 21:52 | |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: Zuulv3 iptables https://review.openstack.org/516727 | 21:52 |
inc0 | so if it doesn't in your case, something is off | 21:52 |
kfox1111 | sbezverk: inc0: https://review.openstack.org/#/c/516727/ | 21:52 |
*** itlinux has quit IRC | 21:52 | |
mwynne | So, after a reboot everything should just come back up? | 21:53 |
inc0 | well again, if you reboot ALL of mariadb nodes at once | 21:54 |
inc0 | it's going to need restoration | 21:54 |
mwynne | If there's only 1 node? | 21:55 |
inc0 | should just work | 21:55 |
mwynne | ok | 21:55 |
inc0 | only one node of mariadb? | 21:55 |
mwynne | yeah | 21:55 |
mwynne | I have a controller and 2 compute nodes | 21:55 |
inc0 | still should just work, but that's not multinode;) I guess it is in a way you have compute node and whatnot | 21:55 |
mwynne | controller has infra + os services and that's it | 21:56 |
inc0 | ok | 21:56 |
mwynne | then compute nodes have neutron agent and compute | 21:56 |
inc0 | yeah that's how it's supposed to work | 21:56 |
mwynne | it's pseudo-multinode ;) | 21:56 |
inc0 | I assumed you have more than 2 nodes of control | 21:56 |
inc0 | that's where mariadb cries when you restart cluster | 21:56 |
mwynne | Ah, no. It's not AIO, but it's not "multinode" by your definition. | 21:56 |
inc0 | right, hence misunderstanding | 21:56 |
inc0 | you're right ofc, it's multinode | 21:57 |
inc0 | just not HA;) | 21:57 |
mwynne | yes | 21:57 |
*** TxGirlGeek has joined #openstack-kolla | 21:58 | |
*** TxGirlGeek has quit IRC | 21:59 | |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support https://review.openstack.org/458649 | 22:00 |
kfox1111 | rwellum: let me know what you think plz. | 22:00 |
mwynne | inc0: I just needed to reboot all my nodes in order I guess... | 22:08 |
mwynne | Seems to have brought everything back up :S | 22:08 |
inc0 | good to hear | 22:10 |
rwellum | This is where k8s is far superior at keeping things alive, reboot etc. At least in theory. | 22:14 |
*** TxGirlGeek has joined #openstack-kolla | 22:14 | |
mwynne | rwellum: I might have to give kolla-kubernetes a shot. | 22:15 |
rwellum | mwynne: +1 | 22:17 |
rwellum | kfox1111: couple of reviews in. | 22:19 |
rwellum | Love the logging. Think I've said that a few times now! | 22:19 |
kfox1111 | glad its working for you. :) | 22:25 |
kfox1111 | I'm really hoping the recently added json log output from oslo will work well. | 22:25 |
kfox1111 | between the two, would make for some really powerful efk. | 22:25 |
*** lpetrut has quit IRC | 22:27 | |
*** dave-mccowan has quit IRC | 22:37 | |
*** TxGirlGeek has quit IRC | 22:41 | |
*** TxGirlGeek has joined #openstack-kolla | 22:46 | |
*** itlinux has joined #openstack-kolla | 22:51 | |
*** TxGirlGeek has quit IRC | 22:53 | |
openstackgerrit | Merged openstack/kolla-kubernetes master: Zuulv3 iptables https://review.openstack.org/516727 | 22:54 |
*** david-lyle has joined #openstack-kolla | 22:55 | |
*** kristaps_ has quit IRC | 22:59 | |
*** kristaps_ has joined #openstack-kolla | 22:59 | |
*** mchlumsky has quit IRC | 23:07 | |
*** david-lyle has quit IRC | 23:12 | |
*** david-lyle has joined #openstack-kolla | 23:13 | |
*** TxGirlGeek has joined #openstack-kolla | 23:14 | |
*** sambetts is now known as sambetts|afk | 23:22 | |
*** Angeh has quit IRC | 23:26 | |
*** dardelean has joined #openstack-kolla | 23:47 | |
*** dardelean has quit IRC | 23:52 | |
*** k_mouza has joined #openstack-kolla | 23:53 | |
*** itlinux has quit IRC | 23:54 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!