| *** manheim has quit IRC | 00:00 | |
| *** ductta___ has quit IRC | 00:08 | |
| *** yangyapeng has quit IRC | 00:14 | |
| *** yangyapeng has joined #openstack-kolla | 00:14 | |
| *** Pavo has quit IRC | 00:14 | |
| *** jamesbenson has joined #openstack-kolla | 00:16 | |
| *** yangyapeng has quit IRC | 00:18 | |
| *** jamesbenson has quit IRC | 00:21 | |
| *** klindgren has quit IRC | 00:32 | |
| openstackgerrit | zhongshengping proposed openstack/kolla master: Switch to Cryptography over pycrypto https://review.openstack.org/473702 | 00:34 |
|---|---|---|
| *** emccormick has quit IRC | 00:40 | |
| *** zhurong has joined #openstack-kolla | 00:43 | |
| *** vhosakot has quit IRC | 00:46 | |
| *** dixiaoli has joined #openstack-kolla | 00:53 | |
| *** dixiaoli has quit IRC | 00:55 | |
| *** jamesbenson has joined #openstack-kolla | 00:57 | |
| *** manheim has joined #openstack-kolla | 01:01 | |
| *** jamesbenson has quit IRC | 01:02 | |
| sdake | kfox1111 ceph is fantastic until it eats your babies the first time | 01:03 |
| *** tovin07_ has joined #openstack-kolla | 01:05 | |
| duonghq | inc0, sdake morning guys, I'm sorry, last afternoon I forgot disconnect from work-pc | 01:12 |
| larsks | /buffer #rhos-dev | 01:16 |
| *** caoyuan has joined #openstack-kolla | 01:16 | |
| larsks | Whoops. | 01:16 |
| *** yeyon has joined #openstack-kolla | 01:17 | |
| *** yangyapeng has joined #openstack-kolla | 01:20 | |
| yeyon | https://docs.openstack.org/developer/kolla/ this web page does not exist | 01:21 |
| yeyon | ? | 01:21 |
| *** zhubingbing_ has joined #openstack-kolla | 01:21 | |
| *** yangyapeng has quit IRC | 01:21 | |
| *** yangyapeng has joined #openstack-kolla | 01:21 | |
| *** itlinux has joined #openstack-kolla | 01:23 | |
| *** itlinux has quit IRC | 01:23 | |
| *** lyang__ has joined #openstack-kolla | 01:24 | |
| *** itlinux has joined #openstack-kolla | 01:24 | |
| larsks | yeyon: it does for me (it redirects to https://docs.openstack.org/kolla/latest/). I think maybe the docs site has been underoing some sort of maintenance recently, because I've also hit a numer of unexpected 404 errors. | 01:26 |
| dmsimard | Need a last +2 on a small review: https://review.openstack.org/#/c/479881/ | 01:29 |
| *** manheim has quit IRC | 01:35 | |
| *** unicell has quit IRC | 01:37 | |
| *** jamesbenson has joined #openstack-kolla | 01:39 | |
| Seyong | yeyon: try it again it works for me. https://docs.openstack.org/kolla/latest/ | 01:41 |
| *** jamesbenson has quit IRC | 01:44 | |
| *** caowei has joined #openstack-kolla | 01:49 | |
| sdake | query kfox1111 | 01:51 |
| sdake | kfox1111 see PM plz | 01:51 |
| sdake | duonghq can you ck dmsimard 's review plz | 01:52 |
| dmsimard | sdake: btw the use case | 01:52 |
| sdake | yeyon use the refresh button on your browser | 01:52 |
| dmsimard | sdake: tbh it's super awesome that kolla-build can build a million things super fast | 01:52 |
| dmsimard | sdake: I've also said before that troubleshooting that when *something* goes wrong usually sucks, a lot | 01:53 |
| dmsimard | sdake: I am trying a more granular approach while also trying to keep builds asynchronous and concurrent | 01:54 |
| dmsimard | But I found an edge case where it would try to build a layer that was already built | 01:54 |
| dmsimard | That granular approach would allow to easily profile and troubleshoot each individual image build easily | 01:54 |
| sdake | dmsimard you do know you can output the logs of each build separatey? | 01:55 |
| sdake | each container that is | 01:55 |
| *** eaguilar has joined #openstack-kolla | 01:56 | |
| dmsimard | sdake: been there done that, in our case things are in foreground and ran with ansible (thus available through ARA :p) | 01:57 |
| dmsimard | It's something I'll need to consider if this doesn't work out though, but it seemed like a sensible and easy improvement | 01:57 |
| sdake | ok well i dont understand why yu need it - but it doesn't seem to cause any harm | 01:57 |
| sdake | we are past the point where adding another API to the build is all that damaging | 01:57 |
| dmsimard | sdake: I'll share it if it works :) | 01:57 |
| sdake | dmsimard i guess if it doesn't you can revert the patch prior to release :) | 01:58 |
| dmsimard | sdake: bah, if you insist -- like I said, I still think this is useful. We have ephemeral build machines that are discarded after the containers are pushed to a docker registry. We aren't interested in even attempting to build layers that have already been built. | 01:59 |
| duonghq | sdake, dmsimard done, I just come back from meeting | 02:00 |
| sdake | duonghq thanks | 02:03 |
| duonghq | how about you these days? | 02:04 |
| sdake | duonghq was very busy with internal work + had couple falls because of clumsiness | 02:04 |
| sdake | the falls injured arms/hands, so took few weeks pto to recover | 02:04 |
| duonghq | oops, I'm sorry to hear that | 02:05 |
| masber | hi, I use kolla-ansible for deployment and I am trying to use ironic, where is the tftp location? | 02:05 |
| sdake | duonghq shit happens | 02:06 |
| sdake | dmsimard i see that makes sense | 02:06 |
| *** eaguilar has quit IRC | 02:07 | |
| *** eaguilar has joined #openstack-kolla | 02:08 | |
| *** jamesbenson has joined #openstack-kolla | 02:21 | |
| *** eaguilar has quit IRC | 02:22 | |
| *** jamesbenson has quit IRC | 02:25 | |
| *** goldyfruit has quit IRC | 02:33 | |
| *** goldyfruit has joined #openstack-kolla | 02:35 | |
| *** hieulq_ has joined #openstack-kolla | 02:52 | |
| *** lucasxu has joined #openstack-kolla | 02:53 | |
| *** hieulq_ has quit IRC | 02:58 | |
| *** hieulq_ has joined #openstack-kolla | 02:58 | |
| openstackgerrit | Merged openstack/kolla master: Add new --skip-existing option to kolla-build https://review.openstack.org/479881 | 02:58 |
| *** hieulq_ has quit IRC | 02:59 | |
| *** lucasxu has quit IRC | 02:59 | |
| *** jamesbenson has joined #openstack-kolla | 03:02 | |
| spsurya | morning | 03:05 |
| *** negronjl has quit IRC | 03:06 | |
| spsurya | sdake: sorry to hear that :( | 03:06 |
| sdake | spsurya shit happens | 03:06 |
| spsurya | sdake: hope you are fine now | 03:06 |
| sdake | 80-90% | 03:06 |
| sdake | catching up on backlog | 03:06 |
| spsurya | good to hear that | 03:06 |
| *** jamesbenson has quit IRC | 03:07 | |
| *** hieulq_ has joined #openstack-kolla | 03:07 | |
| spsurya | hmmm | 03:10 |
| openstackgerrit | Merged openstack/kolla-ansible master: Adds NTP sync process to account for ceph time sync requirements. Removes precheck portion of NTP. Corrects for redhat Fixes typo https://review.openstack.org/477605 | 03:11 |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla-ansible master: change fernet-key repository permissions to 0750 https://review.openstack.org/481388 | 03:14 |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Add vSphere support to nova_compute https://review.openstack.org/449529 | 03:30 |
| *** hieulq_ has quit IRC | 03:32 | |
| *** vhosakot has joined #openstack-kolla | 03:35 | |
| *** jamesbenson has joined #openstack-kolla | 03:44 | |
| *** MarginHu has joined #openstack-kolla | 03:47 | |
| *** jamesbenson has quit IRC | 03:48 | |
| *** jascott1 has joined #openstack-kolla | 03:55 | |
| *** goldyfruit has quit IRC | 04:03 | |
| *** unicell has joined #openstack-kolla | 04:25 | |
| *** jamesbenson has joined #openstack-kolla | 04:25 | |
| *** jamesbenson has quit IRC | 04:30 | |
| *** janki has joined #openstack-kolla | 04:35 | |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible https://review.openstack.org/459270 | 04:39 |
| *** lyang__ has quit IRC | 04:42 | |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible https://review.openstack.org/459270 | 04:42 |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Add vmware support to ceilometer https://review.openstack.org/462842 | 04:46 |
| *** Seyong has quit IRC | 04:47 | |
| *** lyang__ has joined #openstack-kolla | 04:55 | |
| *** dixiaoli has joined #openstack-kolla | 04:56 | |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Fix fwaas options https://review.openstack.org/463882 | 05:00 |
| *** jamesbenson has joined #openstack-kolla | 05:07 | |
| *** jamesbenson has quit IRC | 05:12 | |
| *** prateek has joined #openstack-kolla | 05:25 | |
| *** jamesbenson has joined #openstack-kolla | 05:28 | |
| *** jascott1 has quit IRC | 05:30 | |
| *** jascott1 has joined #openstack-kolla | 05:31 | |
| *** jamesbenson has quit IRC | 05:32 | |
| *** yangyape_ has joined #openstack-kolla | 05:33 | |
| *** yangyapeng has quit IRC | 05:35 | |
| *** mdnadeem has joined #openstack-kolla | 05:37 | |
| *** skramaja has joined #openstack-kolla | 05:47 | |
| *** jamesbenson has joined #openstack-kolla | 05:49 | |
| *** jamesbenson has quit IRC | 05:53 | |
| *** vhosakot has quit IRC | 05:55 | |
| openstackgerrit | Merged openstack/kolla-kubernetes master: Update deployment guide to use 1.6.4. https://review.openstack.org/481218 | 05:58 |
| *** junbo has quit IRC | 06:01 | |
| *** jamesbenson has joined #openstack-kolla | 06:10 | |
| *** yangyape_ has quit IRC | 06:10 | |
| *** yangyapeng has joined #openstack-kolla | 06:11 | |
| *** yangyape_ has joined #openstack-kolla | 06:13 | |
| *** jamesbenson has quit IRC | 06:14 | |
| *** yangyapeng has quit IRC | 06:16 | |
| *** aolwas28 has quit IRC | 06:24 | |
| *** lpetrut has joined #openstack-kolla | 06:25 | |
| *** ajafo has quit IRC | 06:26 | |
| *** ajafo has joined #openstack-kolla | 06:26 | |
| *** yangyape_ has quit IRC | 06:30 | |
| *** MarginHu has quit IRC | 06:30 | |
| *** yangyapeng has joined #openstack-kolla | 06:30 | |
| *** jamesbenson has joined #openstack-kolla | 06:30 | |
| *** yangyapeng has quit IRC | 06:32 | |
| *** yangyapeng has joined #openstack-kolla | 06:33 | |
| *** yangyapeng has quit IRC | 06:33 | |
| *** yangyapeng has joined #openstack-kolla | 06:34 | |
| *** jamesbenson has quit IRC | 06:35 | |
| *** caowei has quit IRC | 06:36 | |
| *** caowei has joined #openstack-kolla | 06:36 | |
| *** mcnanci has joined #openstack-kolla | 06:36 | |
| *** thomasem_ has quit IRC | 06:36 | |
| *** sayantani01 has quit IRC | 06:38 | |
| *** cinerama has quit IRC | 06:38 | |
| *** hawi has quit IRC | 06:38 | |
| *** thomasem_ has joined #openstack-kolla | 06:38 | |
| openstackgerrit | Taeha Kim proposed openstack/kolla master: Fixed the invalid link in the kolla/image/build.py https://review.openstack.org/481457 | 06:38 |
| *** aolwas28 has joined #openstack-kolla | 06:39 | |
| *** hawi has joined #openstack-kolla | 06:39 | |
| *** cinerama has joined #openstack-kolla | 06:40 | |
| *** sayantani01 has joined #openstack-kolla | 06:42 | |
| *** jbadiapa has quit IRC | 06:43 | |
| *** jamesbenson has joined #openstack-kolla | 06:51 | |
| duonghq | anybody try to deploy multinode cluster and poweroff and start one of the nodes? | 06:55 |
| *** jamesbenson has quit IRC | 06:56 | |
| *** jbadiapa has joined #openstack-kolla | 06:58 | |
| *** shardy has joined #openstack-kolla | 07:06 | |
| *** magicboiz has joined #openstack-kolla | 07:09 | |
| *** athomas has joined #openstack-kolla | 07:10 | |
| *** jamesbenson has joined #openstack-kolla | 07:12 | |
| *** lpetrut has quit IRC | 07:13 | |
| *** itlinux has quit IRC | 07:16 | |
| *** jamesbenson has quit IRC | 07:16 | |
| *** unicell1 has joined #openstack-kolla | 07:24 | |
| *** Seyong has joined #openstack-kolla | 07:25 | |
| *** unicell has quit IRC | 07:26 | |
| *** jamesbenson has joined #openstack-kolla | 07:33 | |
| *** reidrac_ has joined #openstack-kolla | 07:34 | |
| *** reidrac_ has quit IRC | 07:34 | |
| *** reidrac has joined #openstack-kolla | 07:35 | |
| *** manheim has joined #openstack-kolla | 07:37 | |
| *** jamesbenson has quit IRC | 07:37 | |
| *** egonzalez has joined #openstack-kolla | 07:38 | |
| duonghq | egonzalez, hi | 07:39 |
| *** manheim has quit IRC | 07:41 | |
| *** gfidente has joined #openstack-kolla | 07:41 | |
| *** manheim has joined #openstack-kolla | 07:41 | |
| *** manheim has joined #openstack-kolla | 07:41 | |
| *** serlex has joined #openstack-kolla | 07:48 | |
| *** manheim_ has joined #openstack-kolla | 07:50 | |
| *** jamesbenson has joined #openstack-kolla | 07:53 | |
| *** manheim has quit IRC | 07:54 | |
| *** jamesbenson has quit IRC | 07:58 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Implement blazar Docker images https://review.openstack.org/454745 | 08:02 |
| *** jascott1 has quit IRC | 08:06 | |
| *** jascott1 has joined #openstack-kolla | 08:08 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Finish/fix neutron sfc service https://review.openstack.org/461745 | 08:14 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Fix wrong extra name for glance_store https://review.openstack.org/481493 | 08:14 |
| *** jamesbenson has joined #openstack-kolla | 08:14 | |
| *** mgoddard_ has joined #openstack-kolla | 08:16 | |
| *** athomas has quit IRC | 08:16 | |
| openstackgerrit | Yotaro Konishi proposed openstack/kolla-ansible master: Prevent unexpected abort when initializing fernet key authentication https://review.openstack.org/477818 | 08:17 |
| *** athomas has joined #openstack-kolla | 08:17 | |
| *** jamesbenson has quit IRC | 08:18 | |
| *** david-lyle has quit IRC | 08:19 | |
| *** david-lyle has joined #openstack-kolla | 08:19 | |
| *** yangyap__ has joined #openstack-kolla | 08:25 | |
| *** yangyapeng has quit IRC | 08:26 | |
| *** YuYangWang has joined #openstack-kolla | 08:27 | |
| *** jascott1 has quit IRC | 08:31 | |
| *** jascott1 has joined #openstack-kolla | 08:32 | |
| *** jamesbenson has joined #openstack-kolla | 08:35 | |
| *** jascott1 has quit IRC | 08:36 | |
| openstackgerrit | zhouya proposed openstack/kolla-ansible master: Support assigning HA traffic to dedicated interface https://review.openstack.org/481503 | 08:37 |
| *** jamesbenson has quit IRC | 08:39 | |
| openstackgerrit | zhouya proposed openstack/kolla-ansible master: Support assigning HA traffic to dedicated interface https://review.openstack.org/481503 | 08:40 |
| *** dixiaoli has quit IRC | 08:45 | |
| *** dixiaoli has joined #openstack-kolla | 08:45 | |
| *** lyang__ has quit IRC | 08:45 | |
| *** jamesbenson has joined #openstack-kolla | 08:56 | |
| *** mandre_away is now known as mandre | 08:56 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Support cinder backend for glance https://review.openstack.org/481517 | 08:56 |
| *** tvignaud has quit IRC | 08:57 | |
| *** jamesbenson has quit IRC | 09:00 | |
| *** lyang__ has joined #openstack-kolla | 09:01 | |
| *** tvignaud has joined #openstack-kolla | 09:11 | |
| *** lpetrut has joined #openstack-kolla | 09:14 | |
| *** jamesbenson has joined #openstack-kolla | 09:16 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Fix the incompatible of docker-py 2.0.0 https://review.openstack.org/481537 | 09:18 |
| *** yangyapeng has joined #openstack-kolla | 09:19 | |
| *** jamesbenson has quit IRC | 09:21 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Standardize Keystone domain variables https://review.openstack.org/479287 | 09:21 |
| *** yangyap__ has quit IRC | 09:22 | |
| openstackgerrit | zhouya proposed openstack/kolla-ansible master: Support assigning HA traffic to dedicated interface https://review.openstack.org/481503 | 09:24 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Use latest docker engine in gate https://review.openstack.org/480212 | 09:25 |
| *** yeyon has quit IRC | 09:27 | |
| *** YuYangWang has quit IRC | 09:27 | |
| *** jamesbenson has joined #openstack-kolla | 09:37 | |
| *** jamesbenson has quit IRC | 09:42 | |
| *** dixiaoli_ has joined #openstack-kolla | 09:42 | |
| *** dixiaoli_ has quit IRC | 09:42 | |
| *** YuYangWang has joined #openstack-kolla | 09:42 | |
| *** YuYangWang has quit IRC | 09:43 | |
| *** YuYangWang has joined #openstack-kolla | 09:44 | |
| *** dixiaoli has quit IRC | 09:46 | |
| *** openstackgerrit has quit IRC | 09:48 | |
| *** caoyuan has quit IRC | 10:05 | |
| *** sambetts|afk is now known as sambetts | 10:06 | |
| *** egonzalez has quit IRC | 10:07 | |
| *** tovin07_ has quit IRC | 10:07 | |
| *** egonzalez has joined #openstack-kolla | 10:09 | |
| *** caowei has quit IRC | 10:11 | |
| *** duonghq has quit IRC | 10:14 | |
| *** mgoddard_ has quit IRC | 10:19 | |
| *** iniazi_ has joined #openstack-kolla | 10:43 | |
| *** iniazi has quit IRC | 10:46 | |
| *** zhurong has quit IRC | 10:46 | |
| *** mgoddard_ has joined #openstack-kolla | 10:51 | |
| *** pbourke has quit IRC | 10:52 | |
| *** pbourke has joined #openstack-kolla | 10:54 | |
| manheim_ | hi, I have an ansible question | 11:03 |
| manheim_ | http://paste.ubuntu.com/25038692/ | 11:03 |
| manheim_ | can I use a conditional set in the globals.yml file to install or nor an item from a with_items? | 11:04 |
| manheim_ | I don't want to create a separate task only for this | 11:04 |
| *** yangyapeng has quit IRC | 11:44 | |
| *** pcaruana has joined #openstack-kolla | 11:49 | |
| *** rwallner has joined #openstack-kolla | 11:51 | |
| *** rwallner has quit IRC | 11:51 | |
| *** rwallner has joined #openstack-kolla | 11:52 | |
| *** rhallisey has joined #openstack-kolla | 11:53 | |
| *** rwsu has quit IRC | 12:05 | |
| *** rwsu has joined #openstack-kolla | 12:12 | |
| *** shardy has quit IRC | 12:18 | |
| *** schwicht has joined #openstack-kolla | 12:20 | |
| *** emccormick has joined #openstack-kolla | 12:25 | |
| *** ansmith has joined #openstack-kolla | 12:26 | |
| *** hrw has quit IRC | 12:26 | |
| *** skramaja has quit IRC | 12:29 | |
| *** schwicht has quit IRC | 12:30 | |
| *** hrw has joined #openstack-kolla | 12:32 | |
| *** goldyfruit has joined #openstack-kolla | 12:35 | |
| *** awiddersheim has quit IRC | 12:36 | |
| *** awiddersheim has joined #openstack-kolla | 12:36 | |
| *** YuYangWang has quit IRC | 12:45 | |
| *** schwicht has joined #openstack-kolla | 12:49 | |
| *** schwicht has quit IRC | 12:51 | |
| *** goldyfruit has quit IRC | 12:56 | |
| *** goldyfruit has joined #openstack-kolla | 12:57 | |
| *** goldyfruit has quit IRC | 12:58 | |
| *** ducttape_ has joined #openstack-kolla | 12:59 | |
| *** lucasxu has joined #openstack-kolla | 12:59 | |
| *** goldyfruit has joined #openstack-kolla | 13:01 | |
| *** ducttape_ has quit IRC | 13:01 | |
| *** janki has quit IRC | 13:03 | |
| *** schwicht has joined #openstack-kolla | 13:06 | |
| *** mdnadeem has quit IRC | 13:14 | |
| *** mdnadeem has joined #openstack-kolla | 13:14 | |
| *** zhurong has joined #openstack-kolla | 13:23 | |
| *** magicboiz has quit IRC | 13:24 | |
| egonzalez | manheim_, yep you can but is somekind ugly http://paste.openstack.org/show/614770/ | 13:27 |
| manheim_ | isee | 13:28 |
| manheim_ | one other question | 13:28 |
| manheim_ | so in juju for example, nova-compute services run directly on the bare metal compute host, not in containers | 13:29 |
| manheim_ | but the nova kvm services for nova all run in containers | 13:29 |
| manheim_ | even libvirt | 13:29 |
| manheim_ | how does this work? | 13:29 |
| egonzalez | no idea about juju | 13:30 |
| sbezverk | egonzalez : qq when I try to build the image I get error :base:manifest unknown: manifest unknown | 13:31 |
| manheim_ | it's just an example, it's a deployer that puts OS services in LXDs | 13:31 |
| manheim_ | but for some services you have to put them on localhost | 13:31 |
| manheim_ | like cinder-volumes, nova-comute etc | 13:32 |
| sbezverk | egonzalez any idea what I do wrong? | 13:32 |
| egonzalez | sbezverk, never seen that error, is a new image o failing building existing images? | 13:32 |
| sbezverk | egonzalez : new image | 13:33 |
| sbezverk | egonzalez : here is my command line: ./build.py --base centos --base-tag 4.0.1 --type source --namespace kolla targetd | 13:34 |
| sbezverk | it used to work in the past | 13:35 |
| sbezverk | egonzalez just checked and even old image is failing to build with the same error. :( I guess something got broken with my kolla setup | 13:37 |
| manheim_ | i mean how can you run libvirt in a container and not on baremetal node | 13:37 |
| egonzalez | sbezverk, are you able to do a direct pull from any image in dockerhub? | 13:39 |
| egonzalez | or, is the parent image pulling from a hash instead of a tag? | 13:40 |
| sbezverk | egonzalez : yep, just pulled centos:latest | 13:40 |
| *** jtriley has joined #openstack-kolla | 13:40 | |
| egonzalez | manheim_, libvirt container shares processes with the baremetal, VMs really runs in the host | 13:41 |
| sbezverk | egonzalez after removing --base-tag it started working.. | 13:41 |
| egonzalez | sbezverk, base-tag is the centos tag to use, not kolla tag | 13:44 |
| sbezverk | egonzalez : cool, thanks for clarification | 13:45 |
| egonzalez | ./build.py --base centos --tag 4.0.1 --base-tag <CENTOS_TAG(latest,7.3) --type source --namespace kolla targetd | 13:45 |
| egonzalez | sbezverk, ^^ that would be the command to build the image, if want to use centos:7 not need to use base-tag because will use 7 by default | 13:46 |
| sbezverk | egonzalez thanks! | 13:50 |
| *** eaguilar has joined #openstack-kolla | 13:56 | |
| *** vhosakot has joined #openstack-kolla | 14:03 | |
| *** hongbin has joined #openstack-kolla | 14:08 | |
| *** jtriley has quit IRC | 14:12 | |
| *** jtriley has joined #openstack-kolla | 14:14 | |
| *** jtriley has quit IRC | 14:14 | |
| *** prateek has quit IRC | 14:14 | |
| *** itlinux has joined #openstack-kolla | 14:19 | |
| *** pradk has joined #openstack-kolla | 14:22 | |
| *** zhurong has quit IRC | 14:26 | |
| *** jtriley has joined #openstack-kolla | 14:26 | |
| *** lrensing has joined #openstack-kolla | 14:27 | |
| *** mdnadeem has quit IRC | 14:31 | |
| *** duonghq has joined #openstack-kolla | 14:31 | |
| duonghq | evening guys | 14:31 |
| *** mdnadeem has joined #openstack-kolla | 14:32 | |
| *** mdnadeem has quit IRC | 14:32 | |
| *** jamesbenson has joined #openstack-kolla | 14:34 | |
| manheim_ | hello | 14:39 |
| *** ducttape_ has joined #openstack-kolla | 14:39 | |
| vhosakot | inc0: rwellum ping | 14:42 |
| vhosakot | duonghq: I'll create an etherpad for the rolling upgrade talk and send it to you soon for feedback. | 14:42 |
| inc0 | I'm here vhosakot | 14:43 |
| inc0 | good morning | 14:43 |
| vhosakot | inc0: rwellum: I think we need to separate the DOckerHub+gate talk and custom images talk as two separate talks so we double our probability of getting accepted :) | 14:43 |
| vhosakot | inc0: morning! | 14:43 |
| inc0 | vhosakot: that doesn't work this way | 14:44 |
| vhosakot | inc0: :) | 14:44 |
| *** YuYangWang has joined #openstack-kolla | 14:44 | |
| vhosakot | inc0: rwellum does the custom images have anything to do with gates? isn't it a buildtime thing/tweak? | 14:44 |
| inc0 | I was thinking of doing talk about images in general | 14:45 |
| inc0 | when to download and when to build | 14:45 |
| inc0 | downloading - how it works and why and when | 14:45 |
| inc0 | building - how it works and why and when | 14:45 |
| vhosakot | inc0: ah I see, general talk about images (including customizing them and publishing to DocerkHUb from gate).. got it | 14:45 |
| *** iniazi has joined #openstack-kolla | 14:45 | |
| inc0 | do you agree? | 14:46 |
| rwellum | Well - sort of. There's building them, and then running them? | 14:46 |
| rwellum | deploying | 14:46 |
| vhosakot | inc0: yes, if the talk is about the "usability of kolla images in the best possible way for ops", then, yes, I agree, we can combine DockerHub+gate and custom images, and both are about building and running.... | 14:47 |
| inc0 | and I think it's going to be useful talk | 14:47 |
| vhosakot | sure, one talk (DockerHub publisher) emphasizes on a strong/intelligent gate, while the other (custom images) has nothing to do with upstream gates and involves custom gating in any company. | 14:48 |
| vhosakot | so, both topics are about building, running, gating... | 14:49 |
| *** iniazi_ has quit IRC | 14:49 | |
| vhosakot | do you agree inc0 rwellum ^^^ ? | 14:49 |
| inc0 | yeah | 14:49 |
| inc0 | we can explain how to build CI like ours, as ideas | 14:49 |
| inc0 | what is our current coverage | 14:50 |
| vhosakot | rwellum: yes, the DockerHub talk totally involves running images... in fact, it is all about the easiness/comfort of running tested/golden images without worrying about building images from scratch. | 14:50 |
| rwellum | So little confused, vhosakot you proposing one talk or two? | 14:51 |
| inc0 | one talk rwellum | 14:51 |
| vhosakot | rwellum: I first thought two, now I'm thinking one.. what do you think? | 14:51 |
| inc0 | you me and Vikram about images (dockerhub and build your own, including customizations) | 14:51 |
| rwellum | I just think there's a lot of content there for 45m talk. | 14:51 |
| inc0 | and vhosakot we could have second talk with duonghq about upgrades | 14:51 |
| rwellum | But basically yes. | 14:51 |
| vhosakot | yeah, all three of us of course... | 14:52 |
| inc0 | rwellum: we don't need to get to technical details on talk | 14:52 |
| vhosakot | yeah, to cover them in 45 mins is what I'm thinking, plus a demo? | 14:52 |
| rwellum | Thats what I asked yesterday - does this include a demo? | 14:52 |
| inc0 | I think it's better to explain ideas and point to correct places to find more information | 14:52 |
| vhosakot | I've seen speakers play pre-recorded demo as a video.. | 14:52 |
| inc0 | not sure about demo, we could build one image and show full customizations | 14:53 |
| inc0 | or do full build, record, and cut "watching grass grow" in the middle | 14:53 |
| *** itlinux has quit IRC | 14:53 | |
| vhosakot | DockerHub is about awesome gates, while custom images is about awesome buildtime-tweak, both could take like 20+ mins each may be? | 14:53 |
| vhosakot | inc0: right, we can cut the boring ("growing beard grow part") | 14:54 |
| inc0 | we'll figure it out as we go vhosakot | 14:54 |
| rwellum | So the title I proposed yesterday is probably not accurate: OpenStack Deployment using Custom/Home Grown Kolla Containers - with Ansible and Kubernetes | 14:54 |
| inc0 | right now we just need titile and abstract | 14:54 |
| vhosakot | rwellum: yes, we need a title to cover the DockerHub publisher topic too | 14:55 |
| jamesbenson | Thanks inc0 and duonghq for the +2's and merge :-) | 14:55 |
| inc0 | hehe welcome to official club of Kolla commiters jamesbenson ;) | 14:55 |
| inc0 | next step - do reviews;) | 14:55 |
| jamesbenson | the cherrypick is still on the cutting board whenever you get around to it ;-) | 14:57 |
| jamesbenson | reviews coming up. ;-) | 14:57 |
| vhosakot | rwellum: a company can use the DockerHub publisher idea to push tested/golden images to an internal registry and the devs in the company can just pull custom tested/golden images and deploy without deploying.. so, this is how we can overlap/form a segway between talks. | 14:57 |
| vhosakot | rwellum: tested/golden custome images to an internal registry, I meant | 14:57 |
| rwellum | vhosakot: yes agreed | 14:57 |
| inc0 | Images to your imagination. Advanced image usage in Kolla | 14:58 |
| vhosakot | inc0: rwellum: the key here is when and build and push, once pushed, things are self-explanatory/easy. | 14:58 |
| inc0 | (not sure about english, so native speakers feel free to correct me please) | 14:58 |
| inc0 | I think key messages is *when* you should download and when to build | 14:59 |
| inc0 | explain what our "golden images" are and what they are not | 14:59 |
| inc0 | and then we can finish up with building one image, something quick, with a lot of customizations | 15:00 |
| vhosakot | inc0: rwellum; by combining the two talks, both topics kinda complement each other... DockerHub publisher topic can touch about doing it for customized images, and the custom image topic can touch about doing it with in internal regitry so devs do not have to build custom images everyday. | 15:00 |
| inc0 | yeah I like that | 15:00 |
| rwellum | Agreed | 15:00 |
| rwellum | To both | 15:01 |
| rwellum | I get this a lot when I talk about Kolla btw - what IS a kolla image? Where do they come from? How to I get them and customize? Etc. | 15:01 |
| vhosakot | the idea is basically: "I don't have to build" (DockerHub topic) and "I can use custom/interal images" (custom image topic) | 15:01 |
| vhosakot | rwellum: I think the answer for "Where do they come from? How to I get them" shuold be found in the DockerHub topic | 15:03 |
| *** lucasxu has quit IRC | 15:04 | |
| *** egonzalez has quit IRC | 15:04 | |
| vhosakot | I bet ops want to do both - 1) not build images.. 2) use custom images.... we just need to find a nice segway and compliment the two topics and show how both topics can totally do the other using Kolla. | 15:04 |
| *** duonghq has quit IRC | 15:05 | |
| *** gfidente has quit IRC | 15:05 | |
| *** gfidente has joined #openstack-kolla | 15:05 | |
| rwellum | Yeah agreed. | 15:09 |
| *** duonghq has joined #openstack-kolla | 15:09 | |
| vhosakot | inc0: rwellum: good chat, now we need a title :) | 15:09 |
| rwellum | inc0: is kolla image tag 4.0.0 - newton? | 15:09 |
| inc0 | ocata rwellum | 15:10 |
| rwellum | so 3.x? | 15:10 |
| inc0 | 3x is newton, but it's eol already | 15:10 |
| inc0 | sorry | 15:10 |
| inc0 | I take that back | 15:10 |
| inc0 | newton is not EOL | 15:10 |
| *** sudipto has joined #openstack-kolla | 15:10 | |
| *** sudipto_ has joined #openstack-kolla | 15:10 | |
| vhosakot | yeah 4.0.0 is Ocata | 15:11 |
| rwellum | mitaka is - but 2.x? | 15:11 |
| rwellum | Is this documented? | 15:11 |
| vhosakot | yes, Liberty was 1.1.0 I know | 15:11 |
| rwellum | What I;ve found online seems wrong. | 15:11 |
| vhosakot | rwellum: https://releases.openstack.org/teams/kolla.html | 15:14 |
| vhosakot | rwellum: Mitaka (2.x) - https://docs.openstack.org/releasenotes/kolla/mitaka.html | 15:14 |
| duonghq | vhosakot, sorry, I'm late, my bad | 15:15 |
| inc0 | no you're not;) | 15:15 |
| rwellum | ty vhosakot | 15:16 |
| inc0 | duonghq: so, upgrade talk? | 15:16 |
| inc0 | didn't you have talk like that in Boston? | 15:16 |
| duonghq | not yet :) | 15:16 |
| duonghq | I have some thought: the gate: it's more like Kolla's workflow for image building and CI | 15:17 |
| duonghq | smt like that | 15:17 |
| duonghq | I mean the topic vhosakot talked above | 15:17 |
| duonghq | I mean we share our ideas on maintaining a set of Docker image | 15:17 |
| inc0 | I see, 4 presenters is a bit too much and I thought you wanted to do upgrade talk:( | 15:17 |
| inc0 | sorry | 15:18 |
| vhosakot | inc0: duonghq: upgrade is totally away/opposite of building images. I'd say upgrade is all about runtime, rolling, in-place activity with min downtime, and how Kolla can do this. | 15:18 |
| duonghq | ya, upgrade is totally different thing | 15:18 |
| inc0 | yeah, I agree vhosakot and we can do upgrade talk too | 15:18 |
| inc0 | as separate | 15:18 |
| inc0 | btw duonghq are you going to openstack days china? | 15:18 |
| rwellum | Upgrade - ansible and k8s? | 15:18 |
| vhosakot | duonghq, inc0 and I can do it... upgrade talk was duonghq's idea right? | 15:18 |
| duonghq | ya, I hope that I can join in upgrade talk in Sydney | 15:18 |
| duonghq | rwellum, ansible first | 15:19 |
| inc0 | I'd focus on ansible and I'd focus on Ocata->Pike | 15:19 |
| duonghq | inc0, no, I'm Vietnamese :) | 15:19 |
| vhosakot | rwellum: mostly Kolla-ansible... k8s upgrade is kinda indepedant/differnet than kolla-ansible's upgrade playbook/arch. | 15:19 |
| inc0 | I know, but it's closer than Australia;) | 15:19 |
| inc0 | just asking, lots of Kolla will be present there afaik | 15:20 |
| duonghq | hmm, not even know that, it's not official event from OpenStack foundation so I cannot get fund for go to that | 15:20 |
| rwellum | Are we testing Pike? :) | 15:20 |
| inc0 | we will, extensively | 15:21 |
| inc0 | and hopefully in gate before summit << looking at Vikram >> | 15:21 |
| inc0 | ;) | 15:21 |
| duonghq | so, inc0 and vhosakot will have topic about gating and CI? | 15:21 |
| inc0 | anyway, we can do full deep dive, explain all the issues with upgrading O -> P | 15:22 |
| vhosakot | rwellum: k8s' upgrade (kubectl rolling-update <image name> --image=image:<new version/tag>) is a k8s feature and we do not plan to cover that... the upgrade talk is how kolla-ansible's ansible playbook can use used to upgrade Kolla images (minor-to-minor, major-to-major release) in a rolling fashion. | 15:22 |
| duonghq | I'm not sure I've catched up with you guys :P | 15:22 |
| vhosakot | yes, I"m ready to start upgrade gate inc0 :) | 15:22 |
| *** eaguilar has quit IRC | 15:22 | |
| inc0 | duonghq: no, I was thinking we could put CI for upgrdaes into upgrade talk | 15:22 |
| inc0 | duonghq: ok, so to summarize | 15:22 |
| inc0 | we'll have 2 talks: | 15:22 |
| vhosakot | inc0: duonghq yes, testing upgrades in CI is nice. | 15:22 |
| inc0 | 1. me, vhosakot and rwellum - images, dockerhub pushing, building custom images etc | 15:23 |
| *** eaguilar has joined #openstack-kolla | 15:23 | |
| inc0 | 2. duonghq vhosakot and me - upgrades, deep dive into O->P, upgrade gate architecture etc | 15:23 |
| duonghq | awesome | 15:24 |
| inc0 | I think if we promise to talk about O->P and promise to explain all the special cases for this release, operators will join en masse | 15:24 |
| inc0 | and at the end of talk we'll make live demo of upgrade | 15:24 |
| vhosakot | I'd talk about what upgrade does - stop existing containers, back them up, pull next release, start them, migrate DB, usual OpenSTack upgrade path | 15:24 |
| rwellum | +1 | 15:24 |
| duonghq | ya, if we can make sure it will be done successfully, we need really big room | 15:25 |
| inc0 | vhosakot: that too | 15:25 |
| inc0 | we usually get big rooms:) | 15:25 |
| duonghq | vhosakot, I hope that I can make rolling upgrade asap, so we can bring up to the demo | 15:25 |
| inc0 | yeah would be cool | 15:26 |
| inc0 | ok, I think that's a good plan guys | 15:26 |
| duonghq | I've done some work on ansible strategy but some work need to be done | 15:26 |
| vhosakot | duonghq: let's do it, I can work with you. | 15:26 |
| duonghq | vhosakot, https://review.openstack.org/#/c/480081/ -> it's my proposal | 15:26 |
| rwellum | https://etherpad.openstack.org/p/kolla-sydney-custom-repo-talk - feel free to hack this up guys. | 15:26 |
| duonghq | rwellum, nice, can I join in idea drafting? | 15:27 |
| rwellum | Yes please | 15:27 |
| duonghq | vhosakot, I think the strategy is almost done, but something happen so the restart container handler is not triggered | 15:27 |
| vhosakot | duonghq: I see, I'll test it, thanks for the pointer. | 15:28 |
| inc0 | brb | 15:29 |
| duonghq | vhosakot, it'd be better if I add restarting percentage to the strategy, but I'm kept on busy on figure out why handler is not invoked | 15:29 |
| vhosakot | if not a rolling upgrade, customers must do some pre-upgrade day-0 stuff... like stop their apps in VMs, back them up, backup VMs' volumes, shutdown VMs. | 15:30 |
| duonghq | ya, cold upgrade is always an option | 15:30 |
| duonghq | btw, did you test bring an cluster up by Kolla, and restart some node(s) in the cluster? | 15:31 |
| duonghq | brb in ~10mins | 15:31 |
| duonghq | :P | 15:31 |
| *** jascott1 has joined #openstack-kolla | 15:33 | |
| *** manheim_ has quit IRC | 15:33 | |
| vhosakot | duonghq: I'll first manually run the keystone upgrade play in the review, and then post comments. | 15:33 |
| *** manheim has joined #openstack-kolla | 15:34 | |
| * duonghq back | 15:35 | |
| duonghq | vhosakot, thanks | 15:35 |
| vhosakot | the last time I ran upgrade, some nova VMs went to error state after the new nova-compute container was restarted, I'll check it | 15:37 |
| inc0 | back | 15:37 |
| *** jascott1 has quit IRC | 15:38 | |
| duonghq | I got trouble even with running cluster | 15:38 |
| duonghq | just restart one node and it's kicked out from cluster when backed | 15:38 |
| inc0 | vhosakot rwellum "Images that fit your imagination. Deep dive into container images in Kolla" | 15:38 |
| inc0 | thoughts? ^:) | 15:38 |
| *** manheim has quit IRC | 15:38 | |
| vhosakot | inc0: rwellum: that's nice... how about "Gate-love gone wild".... lol | 15:40 |
| vhosakot | I'll also think some titles | 15:40 |
| duonghq | inc0, how do you think slim down image on this cycle, or it should take lower priority than rolling upgrade in this cycle? | 15:40 |
| inc0 | duonghq: for me upgrades are almost top priority always, that's what ops care about | 15:41 |
| inc0 | and cycle ends soon | 15:41 |
| vhosakot | duonghq: I've seen you mention slim images in a meeting too.. what is a slim image basically? aren't kolla images already slim per-service with a big parent layer with common packages? | 15:41 |
| inc0 | however we can meet up at Denver and have PTG discussion about slim images | 15:41 |
| inc0 | vhosakot: slim == smaller | 15:41 |
| inc0 | I want slim images, I just don't have good idea how to do it in a way that will be significant | 15:42 |
| rwellum | "Kolla Image Wizardry: Pull, Use, Customize, - imagination is your only limit." | 15:42 |
| duonghq | inc0, ya, if I can catch up with you guys on Denver | 15:42 |
| vhosakot | inc0: I see nothing unwanted in a kolla image to be removed to make it smaller, is there? | 15:42 |
| inc0 | and doing arch change to gain insignificant optmization isn't worth it imo | 15:42 |
| duonghq | vhosakot, it's more likely image refactoring subtopic | 15:43 |
| inc0 | duonghq: PTG is way more interesting to us than summit | 15:43 |
| inc0 | us = devs | 15:43 |
| duonghq | yes, I'm waiting for TSP result | 15:43 |
| vhosakot | what'd be removed to make a kolla image slimmer? | 15:45 |
| rwellum | duonghq: https://en.wikipedia.org/wiki/Trisodium_phosphate (TSP) | 15:45 |
| inc0 | vhosakot: thing is removing (rm -rf stuff) will have opposite result | 15:45 |
| inc0 | because images have CoW layers | 15:46 |
| duonghq | rwellum, lol | 15:46 |
| inc0 | how much of it do you need to get to PTG? | 15:46 |
| vhosakot | well, I used the "removed" incorrectly... how can the size of a kolla image be reduced/slimmed down? | 15:46 |
| inc0 | there are few ideas | 15:47 |
| inc0 | 1. docker --squash | 15:47 |
| duonghq | inc0, I'm not even estimate that, ticket, flight, hotel.... | 15:47 |
| inc0 | yeah it's expensive | 15:47 |
| inc0 | as for hotel, I'm volunteering to share my room with someone | 15:48 |
| inc0 | but that's for later | 15:48 |
| inc0 | duonghq: did you try travel support program? | 15:48 |
| duonghq | inc0, I applied for it, but still waiting for result | 15:49 |
| *** itlinux has joined #openstack-kolla | 15:49 | |
| inc0 | ahh...TSP | 15:49 |
| inc0 | that's what tsp stands for;) | 15:49 |
| vhosakot | inc0: I see, remove intermediate stuff using compress/squash so registry size is less... I'm getting the idea | 15:49 |
| inc0 | but that's slow and experimental | 15:50 |
| duonghq | good news is it seem that I can use my Visa for Denver | 15:50 |
| inc0 | we can use lightweight distro instead of ubuntu/centos | 15:50 |
| vhosakot | inc0: should we unsquash/uncomress before deploying Kolla? | 15:50 |
| inc0 | no need vhosakot | 15:50 |
| vhosakot | yeah, a light base is alwasy a good option | 15:50 |
| rwellum | inc0: possibly unrelated - but you've seen the LOCI effort right? Any lessons we can learn there? | 15:50 |
| inc0 | I've seen it | 15:51 |
| vhosakot | yeah, I'm thinking of LOCI now too | 15:51 |
| duonghq | rwellum, what is LOCI? | 15:51 |
| inc0 | SamYaple made it | 15:51 |
| inc0 | and portdirect | 15:51 |
| sdake | inc0 newton eoling is happening soon | 15:51 |
| inc0 | loci uses single layer so they can rm -rf unneeded things like apt cache after image build | 15:52 |
| inc0 | but we use layers in our images | 15:52 |
| inc0 | so that's not feasable without --squash | 15:52 |
| sdake | i think the big delta reallly between loci and kolla isn't the image flatness - it is the ABI | 15:52 |
| duonghq | ah, okay | 15:53 |
| rwellum | ABI? | 15:53 |
| sdake | yes kolla contianers have an ABI for communication with them | 15:53 |
| inc0 | kolla images has some kolla-specific stuff in it | 15:53 |
| sdake | loci does not | 15:53 |
| inc0 | like set_configs or startup scripts | 15:53 |
| sdake | rwellum let me explain | 15:53 |
| inc0 | LOCI only has source code (no binary) and minimum required packages to run it | 15:54 |
| *** yangyapeng has joined #openstack-kolla | 15:54 | |
| sdake | rwellum https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/neutron/templates/neutron-server.json.j2 | 15:54 |
| vhosakot | sdake: ABI = Application Binary INterface? | 15:54 |
| sdake | vhosakot ya - i guess it may be considered an API - although its pretty binary :) | 15:54 |
| sdake | api/abi - call it what you want | 15:55 |
| sdake | its a commonized industry standardized interface to openstack containers | 15:55 |
| vhosakot | I see.. cool :) basically, an interface between two binary programs at runtime | 15:55 |
| sdake | rwellum if you look at that file, that is fed into every container | 15:55 |
| sdake | and does all the magic of securing and configuring the container | 15:56 |
| sdake | this makes kolla containers idempotent | 15:56 |
| sdake | as well as provides other benefits | 15:56 |
| rwellum | idempotent but not immutable | 15:56 |
| sdake | wrong | 15:57 |
| sdake | its also immutable | 15:57 |
| sdake | the config files are loaded once | 15:57 |
| sdake | and then not loaded again | 15:57 |
| sdake | once the container is started - its contents dont change | 15:57 |
| rwellum | Interesting... | 15:57 |
| portdirect | its not immutable i think - as i would define that at somthing whos contents never change | 15:57 |
| sdake | unless using COPY_ALWAYS - whcih is of course the operator's choice | 15:57 |
| portdirect | so you can/could run with a read only rootfs | 15:58 |
| *** mgoddard_ has quit IRC | 15:58 | |
| sdake | ya - running with a read only root fs should be on kolla's roadmap | 15:59 |
| sdake | however, the fact that kolla does not run with a read only rootfs does nto make it not immutable | 15:59 |
| portdirect | no - but its a pretty good acid test of how far towards being immutable it is :) | 16:00 |
| vhosakot | no container is read-only is it.. app will write something (logs for example) to container at some point? | 16:00 |
| sdake | vhosakot no - they write them to either docker volumes or kubernetes pvcs | 16:00 |
| sdake | vhosakot it is possible to implement read only containers | 16:00 |
| portdirect | or to sdout | 16:00 |
| vhosakot | I can wrote an app to touch a file in a container without volumes, can;t I sdake? | 16:01 |
| portdirect | sdake: is on point here | 16:01 |
| sdake | portdirect - right - more addressing the persistent storage aspect ;) | 16:01 |
| sdake | vhosakot containers may be launched with read only filesystems | 16:01 |
| vhosakot | doesn't immutability come into picture when the container _restarts_ as everything not in a volume is lost and container goes back to known golden state. | 16:02 |
| sdake | docker run -d --read-only --tmpfs /run --tmpfs /tmp IMAGE | 16:02 |
| vhosakot | sdake: read-only container with read-only fs means, read-only app? | 16:02 |
| sdake | vhosakot no - it means putting your persistent data on PVCs in the kubernetes case or docker volumes in the docker case | 16:03 |
| vhosakot | right, use volumes to write, or do read-only stuff if no volumes.. | 16:03 |
| sdake | rwellum anyway back on the original point rwellum | 16:03 |
| portdirect | and your temp files in to somting like the tmpfs vols created in sdakes example above | 16:03 |
| sdake | rwellum that config file instructs the container how to bheave | 16:03 |
| sdake | i am not sure if kubernetes has read only volume support yet or not | 16:04 |
| inc0 | images are RO by default;) containers are instantiated, therefore they're not | 16:04 |
| sdake | rather container support | 16:04 |
| sdake | a read only instantion o a container image is the ultimate security tool | 16:05 |
| inc0 | still, we can get some of LOCI stuff in once --squash becomes stable | 16:05 |
| sdake | it didn't become a feature unitl 1.10 | 16:05 |
| inc0 | it will make builds longer, but containers will be smaller | 16:06 |
| sdake | or it would have likely meandered its way into the code base | 16:06 |
| inc0 | however *all* of containers will consume more space due to lack of sharing | 16:06 |
| sdake | inc0 also large compute farms will take longer to launch | 16:07 |
| vhosakot | https://github.com/openstack/loci | 16:07 |
| inc0 | with our env not that much longer - sharing of layers is our ally in this case | 16:07 |
| sdake | inc0 i mean with squash ;) | 16:07 |
| inc0 | a yeah | 16:08 |
| sdake | squash was integrated at the most inoopportune time | 16:08 |
| sdake | 1.13 feature | 16:08 |
| sdake | 1.13 = dead to me | 16:08 |
| vhosakot | inc0: sdake so, LOCI does not have kolla_start concept and no startup/entrypoint mechanism? | 16:08 |
| inc0 | it's in experimental branch | 16:08 |
| inc0 | so as far as I'm concenrned, it's not implemented yet | 16:08 |
| sdake | vhosakot its been awhile since i've looked at the code - but as of 3-4 moonths ago that is correct | 16:08 |
| inc0 | no vhosakot, idea behind LOCI is to overlay all your custom stuff on top of LOCI images | 16:09 |
| inc0 | which are supposed to just give you binaries | 16:09 |
| *** manheim has joined #openstack-kolla | 16:09 | |
| sdake | the key thing here is - kolla doesn''t really have custom stuff | 16:09 |
| inc0 | so for us to use LOCI in kolla-ansible for example we'd still need to create image FROM loci/nova:latest -> ADD extended start | 16:09 |
| sdake | we have stuff that implements our abi | 16:10 |
| inc0 | sdake: we have, just not ansible-specific | 16:10 |
| vhosakot | right, kolla bakes extended start into the image | 16:10 |
| inc0 | and our abi is custom;) | 16:10 |
| sdake | inc0 i'd say industry standard is a better way of putting it :) | 16:10 |
| inc0 | if industry means people who use kolla images;) | 16:10 |
| inc0 | still, I think it's great value | 16:11 |
| inc0 | and you could simulate something like this on k8s by mounting configmaps with scritps in them | 16:11 |
| inc0 | but I personally think it's very ugly, even if k8s folk do it | 16:11 |
| vhosakot | using Kolla is easier than LOCI right since ops must figure out the overlay part (proper base, extended start/entrypoint)? | 16:11 |
| *** jascott1 has joined #openstack-kolla | 16:12 | |
| kfox1111 | +1. injecting python/shell scripts via configmaps is hackuy. | 16:12 |
| kfox1111 | hacky. | 16:12 |
| kfox1111 | should be used very sparingly, not as the base of your whole archetecture. | 16:13 |
| inc0 | it does put some bootstrapping logic into container, LOCI doesn't, LOCI on the other hand is significantly smaller and images don't share anything so they're independent from each othere | 16:13 |
| inc0 | tradeoffs | 16:13 |
| inc0 | we had this discussion on PTG | 16:13 |
| kfox1111 | you can still have kolla containers all b e independent of each other and still have a well defined entrypoint. | 16:13 |
| *** manheim has quit IRC | 16:14 | |
| kfox1111 | there is no tradeoff there. | 16:14 |
| inc0 | yeah | 16:14 |
| inc0 | well, size is tradeoff | 16:14 |
| inc0 | want independent - it's going to be big | 16:14 |
| vhosakot | I like Kolla images, because it comes with the startup mechanism (kolla_start, extended start) baked into the image, all I need to do is carry the right config | 16:14 |
| inc0 | want shared layers - it's not independent | 16:14 |
| vhosakot | imagine figuring out the entrypoint/extended start for neutron as ops | 16:14 |
| kfox1111 | I think they are papering over "share nothing". | 16:14 |
| kfox1111 | it still shares a common base image. | 16:14 |
| kfox1111 | so you still have dependeies in the tree. just fewer. | 16:15 |
| *** openstackgerrit has joined #openstack-kolla | 16:16 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 16:16 |
| sdake | kfox1111 see pm | 16:16 |
| *** klindgren has joined #openstack-kolla | 16:17 | |
| vhosakot | as an ops guy treating a kolla container as my "unit of deployment", I prefer kolla as it starts an OpenStack service when I start a kolla container... as ops, I don;t want to figure out the overlay part (what to start inside my container and what entrypoint should I overlay). | 16:17 |
| *** eaguilar has quit IRC | 16:25 | |
| duonghq | see you later guys | 16:26 |
| inc0 | good night duonghq | 16:27 |
| duonghq | bye inc0 | 16:27 |
| *** duonghq has quit IRC | 16:27 | |
| *** reidrac has quit IRC | 16:32 | |
| *** janki has joined #openstack-kolla | 16:38 | |
| *** jascott1 has quit IRC | 16:44 | |
| *** unicell1 has quit IRC | 16:51 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 16:59 |
| *** janki has quit IRC | 17:00 | |
| *** athomas has quit IRC | 17:01 | |
| SamYaple | vhosakot: remember, LOCI started as a POC for changing the Kolla build process. it only "forked" into its own project when the build process couldnt be done in Kolla with the existing bits | 17:08 |
| openstackgerrit | Merged openstack/kolla-kubernetes master: Make wait_for_pods.py work inside kubernetes cluster https://review.openstack.org/478941 | 17:10 |
| *** daidv has quit IRC | 17:10 | |
| *** eaguilar has joined #openstack-kolla | 17:11 | |
| *** hieulq has quit IRC | 17:12 | |
| *** unicell has joined #openstack-kolla | 17:15 | |
| openstackgerrit | Marcus Williams proposed openstack/kolla-ansible master: Add OpenDaylight role https://review.openstack.org/416367 | 17:16 |
| openstackgerrit | Marcus Williams proposed openstack/kolla-ansible master: Add OpenDaylight role https://review.openstack.org/416367 | 17:19 |
| vhosakot | SamYaple: ah I see, thanks for the info :) | 17:20 |
| sdake | SamYaple i'm not sure anyone said you did anything wrong - no need to give an explination | 17:20 |
| sdake | fwiw I dont think loci is a fork - seems like new original work to me | 17:20 |
| sdake | and as always SamYaple - your highly creative | 17:20 |
| vhosakot | ^^^ | 17:21 |
| vhosakot | :) | 17:21 |
| vhosakot | yeah, I like the concept of lightweight/slim images | 17:21 |
| *** unicell1 has joined #openstack-kolla | 17:22 | |
| SamYaple | vhosakot: yea it was made as a POC to fix the external build tool/shared layer problem with kolla that makes kolla really hard to work with in CICD. when it was clear that it wasnt going to land that methodology went its own way and we have been growing it with a focus on CICD and upstream compatible builing aproaches (like building in dockerhub) | 17:23 |
| SamYaple | vhosakot: with the new docker 17.06 FROM image stuff, kolla might be able to achieve a similiar result in size at a significantly greater cost in time, but it would retain the set_config stuff | 17:23 |
| *** dave-mccowan has joined #openstack-kolla | 17:24 | |
| SamYaple | inc0: you have seen the build-args FROM stuff, yes? | 17:25 |
| *** unicell has quit IRC | 17:25 | |
| *** daidv has joined #openstack-kolla | 17:25 | |
| *** jascott1 has joined #openstack-kolla | 17:25 | |
| jascott1 | inc0 I sent that email | 17:25 |
| *** hieulq has joined #openstack-kolla | 17:26 | |
| vhosakot | SamYaple: cool, yeah set_config is a grest feature | 17:26 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 17:27 |
| rwellum | sbezverk or kfox1111: have you tried kolla-k8s with docker 17.x? | 17:28 |
| sdake | rwellum last i tried - kubeadm does not support that version of docker | 17:29 |
| sdake | nor did it work when i tried anyway | 17:29 |
| *** ducttape_ has quit IRC | 17:31 | |
| *** sambetts is now known as sambetts|afk | 17:34 | |
| *** sudipto has quit IRC | 17:35 | |
| *** sudipto_ has quit IRC | 17:35 | |
| sbezverk | rwellum nope | 17:37 |
| sbezverk | I heard somebody tried and failed | 17:37 |
| *** gfidente has quit IRC | 17:43 | |
| *** tonanhngo has joined #openstack-kolla | 17:44 | |
| *** jascott1 has quit IRC | 17:44 | |
| *** jascott1 has joined #openstack-kolla | 17:45 | |
| sdake | rwellum kolla-k8s may work with a different installation tool | 17:48 |
| sdake | with docker 17.x | 17:48 |
| sdake | such as kargo (or kubespray?) | 17:48 |
| kfox1111 | rwellum: I've been avoiding it. | 17:50 |
| kfox1111 | I think the distro's are going to drop docker soon and switch to containerd. | 17:50 |
| kfox1111 | over the whole docker swarm -> docker fieasco. | 17:51 |
| rwellum | Yeah kfox1111, sdake, sbezverk - I just mention it because SamYaple did above. | 17:54 |
| kfox1111 | yeah. not sure how thats going to effect kolla-ansible/kolla-kubernetes. | 17:54 |
| sdake | kfox1111 that is a kubernetes centric world view | 17:54 |
| kfox1111 | the paths could potentially diverge a bit there. | 17:54 |
| sdake | red hat fwiw wants to moe to OCID i beieve | 17:55 |
| kfox1111 | sdake: its a community centric worldview. | 17:55 |
| sdake | kfox1111 what i mean is there is nthing to replace docker in bare metal at present | 17:55 |
| sdake | containerd doesn't do the full job | 17:55 |
| kfox1111 | the general community is focused around containerd, not docker inc. | 17:55 |
| kfox1111 | why do you say that? | 17:55 |
| sdake | lots of people use docker's apis | 17:55 |
| sdake | in a kubernetes world using docker's apis can be abstracted away | 17:56 |
| kfox1111 | Sure. but most of that api is been renamed to containerd. | 17:56 |
| kfox1111 | by docker inc. | 17:56 |
| sdake | ok - well i guess I'm out of the loop :) | 17:56 |
| kfox1111 | ah. | 17:56 |
| kfox1111 | yeah, docker inc did a shell game a couple months ago. | 17:56 |
| kfox1111 | "docker" now means something very different then "docker" then. | 17:57 |
| kfox1111 | "docker" now is what use to be called docker swarm for the most part. | 17:57 |
| rwellum | But I believe you need containerd + something though kfox1111 - containerd is not a 1-to-1 of the 'old' docker. | 17:57 |
| kfox1111 | and what use to be docker is now containerd. | 17:57 |
| kfox1111 | see containerd.io | 17:57 |
| kfox1111 | yeah, its not 100% yet. but they are working hard on 1.0. should be out this month I think, the way they are going. | 17:58 |
| kfox1111 | this is way the community basically stopped at 1.12/1.13. | 17:58 |
| kfox1111 | because containerd's what the community wants, not docker swarm. | 17:58 |
| rwellum | Ah it contains runc - that starts to make a little more sens.e | 17:58 |
| kfox1111 | but docker inc wants to keep the docker trademark for their product. | 17:58 |
| kfox1111 | and their product now is swarm. | 17:59 |
| vhosakot | docker does not exist anymore :) https://github.com/docker/docker goes to https://github.com/moby/moby :) all container runtime upstream is CNCF's containerd now... I heard about it too | 17:59 |
| kfox1111 | but "docker" has all the praise from users for years. | 17:59 |
| rwellum | Right - deliberate obfuscation? | 17:59 |
| kfox1111 | so they are redirecting that praise to docker swarm. | 17:59 |
| vhosakot | docker is legacy :) | 17:59 |
| kfox1111 | oh, docker exists. | 17:59 |
| *** serlex has quit IRC | 17:59 | |
| kfox1111 | docker is basically the rhel of docker swarm. | 17:59 |
| kfox1111 | moby is now the fedora of docker swarm. | 18:00 |
| rwellum | Ok what is moby :) | 18:00 |
| rwellum | Argh! | 18:00 |
| kfox1111 | and containderd is what use to be called docker. | 18:00 |
| kfox1111 | yeah. they did a huge terminology shift to claim the docker name. :) | 18:00 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 18:00 |
| vhosakot | docker exists as the non-open-source (commercial) bit | 18:00 |
| kfox1111 | right. | 18:00 |
| kfox1111 | docker is now all locked up. | 18:00 |
| rwellum | So is the k8s community working towards containerd then? | 18:01 |
| kfox1111 | containerd is what redhat/ubuntu/google/etc cared about in docker of old. | 18:01 |
| kfox1111 | yup. | 18:01 |
| kfox1111 | thats why they enabled cri by default in 1.6 even though it was beta. | 18:01 |
| kfox1111 | they were so anoyed at dockerinc that they pushed hard to put in an abstraction quick. | 18:01 |
| vhosakot | what confuses me is, why did docker fork https://github.com/docker/containerd from https://github.com/containerd/containerd... which of the two is the greatest/newest/best container runtime? | 18:01 |
| *** bit_lySLH2uSZHed has joined #openstack-kolla | 18:02 | |
| kfox1111 | it was https://github.com/docker/containerd | 18:02 |
| kfox1111 | then it was given over to the cncf, | 18:02 |
| kfox1111 | and then became https://github.com/containerd/containerd | 18:02 |
| sbezverk | kfox1111: do you know when containerd will be published in repos? | 18:02 |
| vhosakot | so, both will be developed? | 18:02 |
| *** bit_lySLH2uSZHed has left #openstack-kolla | 18:02 | |
| kfox1111 | sbezverk: as soon as its deamed stable by redhat/ubuntu. | 18:02 |
| kfox1111 | they have a vested interest in getting off of docker 1.12. so I'd think as soon as possible. | 18:03 |
| sbezverk | kfox1111: ok so in next few years then ;) | 18:03 |
| *** ducttape_ has joined #openstack-kolla | 18:03 | |
| kfox1111 | no, I think docker's much more toxic then that. | 18:03 |
| sbezverk | kfox1111 : oh ok | 18:03 |
| kfox1111 | I would be really shocked if it wasnt this year. | 18:04 |
| vhosakot | kfox1111: haha, you're not the only one that thinks docker is becoming toxic (money-minded) :) | 18:04 |
| kfox1111 | docker inc burned a huge amount of bridges with docker swarm -> docker. | 18:04 |
| kfox1111 | it means any company selling swarms competitor (k8s) now has to support swarm too or drop docker support. | 18:04 |
| kfox1111 | the community wouldn't stand for that. so its making changes as fast as possible to get docker out. | 18:05 |
| vhosakot | ha, why would a k8s' seller support docker swarm too, interesting... | 18:05 |
| *** stvnoyes has joined #openstack-kolla | 18:05 | |
| *** stvnoyes has left #openstack-kolla | 18:06 | |
| kfox1111 | yeah. having to install swarm just to install k8s seems bad. | 18:06 |
| vhosakot | I wouldn't need swarm _with_ k8s, when k8s can do everything swarm can and even more. | 18:07 |
| *** lpetrut has quit IRC | 18:08 | |
| kfox1111 | yeah. that really rubbed all the k8s supperters wrong. | 18:09 |
| kfox1111 | we saw traces of this when docker kept duplicating systemd functionality. | 18:09 |
| kfox1111 | but most distros weren't supporting microdistro's yet, so not many cared about it exept CoreOS. thats why that bridge was burned first. | 18:10 |
| kfox1111 | then docker inc relized people want to pay for COE, not for individual container launching support. | 18:10 |
| kfox1111 | and k8s was providing that, not docker. | 18:10 |
| kfox1111 | so they had to push swarm as hard as possibe. | 18:10 |
| kfox1111 | so they reclaimed the name to trick people into going with swarm instead of k8s, | 18:10 |
| kfox1111 | due to so many people not knowing docker but hearing good devs/ops saying "docker is great" | 18:11 |
| sdake | kfox1111 you use the term "pay' looszely above kfox1111 :) | 18:11 |
| kfox1111 | sdake: there are more then one way of paying. ;) | 18:11 |
| kfox1111 | $ is just one form. | 18:11 |
| rwellum | Um - I was at a presentation recently where they claimed docker swarm was 5x faster than k8s. :) | 18:11 |
| kfox1111 | devs time, kudo's, documentation, blog articles, etc are others. | 18:11 |
| kfox1111 | rwellum: at what? faster at failing to be flexible enough to deploy my apps? :) | 18:12 |
| kfox1111 | really, with the new nomenclature, they are claiming 5x faster at doing COE things I guess. | 18:13 |
| kfox1111 | http://containerd.io/images/chart-f.png shows what the community will look like when the migration is done. | 18:14 |
| rwellum | https://blog.docker.com/2016/03/swarmweek-docker-swarm-exceeds-kubernetes-scale/ - it was along these lines. | 18:15 |
| kfox1111 | "The study and article, commissioned by Docker" | 18:17 |
| kfox1111 | ah. they are doing stuff like benchmarking api calls too. | 18:18 |
| kfox1111 | yeah. older article. k8s has better scaling now. also has focused on devs with minikube. | 18:19 |
| kfox1111 | so small to huge just like swarm. | 18:19 |
| kfox1111 | I'd focus more on articles that come from independent third parties, not from docker inc. they produce a lot of stuff very slanted in their favor. (its their job to do so. I don't blame them) | 18:20 |
| rwellum | Yeah - working for a vendor - I get all sorts of customers pitching me stuff. | 18:21 |
| kfox1111 | the issue is kind of bad right now in that, a lot of people are saying, | 18:26 |
| kfox1111 | "I heard docker is great, I need docker training" | 18:26 |
| kfox1111 | so they go and hire someone to do docker training. | 18:26 |
| kfox1111 | and then they get all trained up on swarm. :/ | 18:26 |
| kfox1111 | cause they didn't know to ask. | 18:27 |
| *** itlinux has quit IRC | 18:27 | |
| sdake | kfox1111 thats sort of amusing | 18:29 |
| kfox1111 | amusing is one word for it. | 18:30 |
| kfox1111 | I've seen it first hand a copule of times now. :/ | 18:30 |
| sdake | what does this eman | 18:31 |
| sdake | la/'], u'container_name': u'neutron_vpnaas_agent', u'privileged': True}}) | 18:31 |
| sdake | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TemplateRuntimeError: {{ service_plugins|selectattr('enabled', 'equalto', true)|list }}: no test named 'equalto' | 18:31 |
| sdake | fatal: [localhost]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""} | 18:31 |
| sdake | i've seen it and fixed it a bunch of times | 18:31 |
| sdake | i dont recall waht i did to fix it | 18:31 |
| sdake | inc0 ^^ | 18:31 |
| vhosakot | rwellum: yeah, I did some POCs to research comparing the post-day-1 capacities of swarm vs k8s, yes, swarm is pretty cool too with k8s-ish feature (LB, rolling upgrades, auto scaling, storage volume, resiliency manager, etc). | 18:32 |
| sdake | the fact is swarm isn't hip | 18:32 |
| sdake | so it is already done for | 18:32 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 18:32 |
| vhosakot | sdake: I've seen that 'equalto' error. 'equalto' is in jinja 2.8. https://github.com/ansible/ansible/issues/10123#issuecomment-71949731 | 18:33 |
| kfox1111 | sdake: the problem is, with COE's still relatively new, the docker swarm-> docker rename might actually work to get them primary market share. | 18:33 |
| kfox1111 | basically taking advantage of all the good will the community put into docker before the rename. and throwing out the communities solution. | 18:33 |
| kfox1111 | rubs me the wrong way. :/ | 18:34 |
| vhosakot | I've heard people say they do k8s instead of swarm just because Google started k8s. | 18:34 |
| rwellum | sdake: point jinja2 installation to 2.8.1 | 18:34 |
| sdake | rwellum thanks | 18:34 |
| kfox1111 | I tried early swarm and early k8s. | 18:34 |
| sdake | early k8s was a pile of fgarb | 18:34 |
| kfox1111 | swarm/compose couldn't do a huge amount of things that k8s could. | 18:34 |
| sdake | early swarm was worse | 18:34 |
| kfox1111 | the whole problem I had was k8s has pods, and swarm didnt. | 18:35 |
| sdake | compose was simpel and elegant but ultimately useless | 18:35 |
| kfox1111 | pods really are critical for most of my workloads. | 18:35 |
| sdake | also I submitted a 2 line change upstream and it took 6 months to merge | 18:35 |
| kfox1111 | yeah, k8s is slow at times. :/ | 18:35 |
| sbezverk | sdake: there is a positive side to this, less garbage gets in :) | 18:36 |
| sdake | i was talking about compose (2 line patch) | 18:36 |
| kfox1111 | compose worked ok. k8s worked so much better. | 18:37 |
| kfox1111 | k8s was a bit more complicated to get into. | 18:37 |
| sdake | sbezverk also fwiw that is a very limited viewpoint - nearly every major innovation was considered "garbage" by those that came before | 18:37 |
| rwellum | Which solution is more secure? | 18:37 |
| kfox1111 | but overall, complexity is down. | 18:37 |
| kfox1111 | I think both have pretty good security these days. | 18:38 |
| sdake | kfox1111 if you think kubernetes isn't complex - well - i got some ocean front property in arizona to sell ya :) | 18:38 |
| vhosakot | rwellum: both k8s and swarm can do TLS at nerwork-level. | 18:38 |
| kfox1111 | sdake: thats not what I said. | 18:38 |
| sdake | apologies then | 18:38 |
| kfox1111 | the overall system I'm maintaining, I think complexity is down due to adding k8s. | 18:38 |
| kfox1111 | as k8s's added complexity was offset by the removal of complexity elsewhere in the system. | 18:39 |
| vhosakot | don't about container-native security features in the two tho. | 18:39 |
| kfox1111 | as k8s matures, that complexity also keeps shrinking. | 18:39 |
| kfox1111 | k8s 1.7 just gained the big security feature I was after. | 18:39 |
| rwellum | what is that? | 18:39 |
| kfox1111 | only allowing secrets to be downloaded by a kubelet if it has a pod scheduled for that node that references the secret. | 18:39 |
| rwellum | Ah... | 18:40 |
| kfox1111 | then a comped node can't access anything it shouldnt secret wise. | 18:40 |
| sbezverk | sdake: I think there is no issue with k8s complexity, the issue is with people who does not want to learn, they want to jump/hump and run away.. ;) | 18:43 |
| *** itlinux has joined #openstack-kolla | 18:47 | |
| *** lpetrut has joined #openstack-kolla | 18:54 | |
| jascott1 | jump/hump and run away... lol | 18:56 |
| rwellum | Still on the 'jump | 18:56 |
| rwellum | personally... | 18:56 |
| rwellum | kfox1111, inc0: stupid question.... With configs bloating kolla images. Why not attach a hash to each individual config, store the actual config somewhere else and just the hash in the image. When the config is needed - access it via the hash - use it dynamically? | 19:01 |
| jamesbenson | how do you build specific docker images in kolla? kolla-build -t source -b ubuntu --registry 127.0.0.1:4000 --push kolla-toolbox ? | 19:01 |
| vhosakot | nothing is complex... I don't mind getting stuck in the fail-redo-fail-redo-fail-redo-fail-redo-fail-redo-fail-redo loop.. sometimes, I like that loop :) | 19:01 |
| kfox1111 | rwellum: not sure I follow. | 19:01 |
| kfox1111 | in k8s, we use configmaps to inject the config into the container. | 19:01 |
| kfox1111 | so the config lives in k8s, not the container. | 19:02 |
| rwellum | Referring to the large image conversation above. I thought it was the intelligence that is added to the kolla image that is chewing up the space. | 19:03 |
| rwellum | jamesbenson: that looks correct to me - if you're using dockerhub images. | 19:04 |
| jamesbenson | thanks :-) | 19:04 |
| rwellum | kfox1111: so wondering if it can be stored outside the container | 19:05 |
| kfox1111 | rwellum: no, the other way around. loci isn't sharing common images, so the same packages have to be added to seperate images. | 19:05 |
| kfox1111 | it makes their images bigger when multiple are used on the same host. | 19:05 |
| kfox1111 | kolla shares some common base that all inherit from. the config stuff talked about above is shell scripts for processing the config, not the config itself. | 19:06 |
| kfox1111 | and its in the base layer, so it is only added one. | 19:06 |
| rwellum | Ok that helps - but the same question - can we store the shell scripts outside the image? | 19:06 |
| kfox1111 | philosophy question. | 19:07 |
| kfox1111 | some of us believe the image is the distribution mechanism. it should provide a well tested/stable api for launching the thing in question. | 19:08 |
| kfox1111 | others want the container to be a blob of stuff, | 19:08 |
| kfox1111 | and something injected into it at runtime makes sense of the blob and provides a standard api. | 19:08 |
| vhosakot | jamesbenson: just pass the name/regex of the name of the container to be built to kolla-build --> https://github.com/openstack/kolla/blob/master/kolla/image/build.py#L824 | 19:08 |
| kfox1111 | I think the former is cleaner and the latter is playing with fire. but thats just me. | 19:08 |
| vhosakot | jamesbenson: https://github.com/openstack/kolla/blob/master/doc/image-building.rst | 19:09 |
| rwellum | kfox1111: so if instead of the scripts in the image, but a hash to the scripts, does that break the former? To me it's the same. | 19:09 |
| kfox1111 | if it doesnt have the scripts built in, but has to fetch them, | 19:10 |
| kfox1111 | it goes against the phylosophy of the image being the tested/binary identicle image shipped everywhere. | 19:10 |
| kfox1111 | I don't think the container should change during runtime. | 19:11 |
| kfox1111 | like, "you don't run yum install xxxx" | 19:11 |
| vhosakot | kfox1111: I'm a fan of not injecting anything to a container apart from passing a good config file at runtime. I think the container built must already have the mechanism to run the thing in question and be a sensible blob. | 19:11 |
| kfox1111 | if that happens, then things can drift/break. | 19:11 |
| kfox1111 | vhosakot: yeah. | 19:11 |
| kfox1111 | makes it easy to mirror locally when you have a firewall in the way, or ensure what you tested on the test system is exactly the same as whats being deployed to the prod system. | 19:12 |
| vhosakot | kfox1111: All I want to do is start a container and pass it some config. I know want to know what/how to start things at runtime when I start the container. I start a neutron container, I expect a golden neutron-server up waiting for my config. | 19:12 |
| kfox1111 | +1 | 19:12 |
| vhosakot | I meant. I _dont_ know want to know what/how to start things at runtime when I start the container | 19:12 |
| kfox1111 | yeah. the provider of the container should provide that. | 19:12 |
| kfox1111 | as part of the container. | 19:13 |
| vhosakot | "something injected into it at runtime makes sense of the blob and provides a standard api" - wonder if this has a valid usecase kfox1111? | 19:15 |
| kfox1111 | I've had to do it in kolla-kubernetes as a workaround while we get some stuff standardized in the kolla containers. | 19:16 |
| kfox1111 | generally feel uneasy about it. | 19:17 |
| vhosakot | ah I see | 19:17 |
| kfox1111 | I think the k8s pods should be pure instruction to k8s on how to orchestrate the pod. | 19:17 |
| kfox1111 | not how to manage the container processes. | 19:17 |
| vhosakot | yeah, I agree too | 19:18 |
| kfox1111 | like, "hook to safely shut down neutron-server" | 19:18 |
| kfox1111 | that should be a standard part of the api. | 19:18 |
| kfox1111 | or haproxy, or apache. etc. | 19:18 |
| kfox1111 | so then the orchestration piece doesn't have to know if the right shutdown for the container is actually apachectl xxxx or kill -SIGUSR1 foo | 19:19 |
| kfox1111 | it just calls kolla_safe_shutdown or something. | 19:19 |
| kfox1111 | and the container writer implements the right behavior for the right implementation of the container. | 19:19 |
| kfox1111 | openstack-helm side steps it with loci and does it: | 19:20 |
| kfox1111 | make a configmap containing shell scripts that probe if neutron-server is running under python, or apache, or whatever, and what version of neutron, and try and figure out the right way to safely shut it down. | 19:20 |
| kfox1111 | so they check the box on having the logic not be in the k8s orchestration object, | 19:21 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 19:21 |
| kfox1111 | but the logic for safe shutdown of the container is not written by the folks writing the container. so there is risk of mismatch. | 19:21 |
| vhosakot | yeah, I like the hook being part of the standard API concept, helps debuggin/remote cases when troubleshooting... | 19:22 |
| *** ducttap__ has joined #openstack-kolla | 19:23 | |
| *** ducttape_ has quit IRC | 19:27 | |
| rwellum | I get what you're saying kfox1111. Thing is - hashes/keys etc are a very trusted mechanism to validate information is sane. So if you trusted the hash/key then it's as if the script themselves are in the container. It's just as tested, secure, trust-worthy etc. Without the bloating. | 19:27 |
| rwellum | I've done this before. When memory is limited, hash it down, store the hash, pull it back in when needed. | 19:28 |
| vhosakot | rwellum: how would the stuff (configs/scripts) behind the hash pulled at runtime? from a remote box over the network when the container starts? | 19:29 |
| *** ducttape_ has joined #openstack-kolla | 19:30 | |
| rwellum | vhosakot: yes - along those lines. | 19:30 |
| rwellum | Haven't thought it through - just curious. | 19:30 |
| vhosakot | or mounted from the host at runtime, if not thryu network, in which case the host must carry per-hash stuff , not so portable if container needs stuff on host.. | 19:31 |
| vhosakot | over the network is a portable idea.. yeah | 19:32 |
| *** ducttap__ has quit IRC | 19:32 | |
| *** goldyfruit has quit IRC | 19:37 | |
| sdake | hey quick q- my resolvc.conf is not being honored by kubernetes 1.7.0 | 19:38 |
| sdake | any suggestions | 19:38 |
| *** goldyfruit has joined #openstack-kolla | 19:39 | |
| jamesbenson | inc0, fyi: centos7 with binary 4.0.0 images fails as well at the same ceph bootstrap stage. | 19:39 |
| inc0 | jamesbenson: host ubuntu or host centos? | 19:45 |
| jamesbenson | centos7 with cent images | 19:45 |
| jamesbenson | I tried both host os's | 19:46 |
| inc0 | lol so it's something on the hardware lvl? oO | 19:46 |
| inc0 | mtu? | 19:46 |
| inc0 | what's your networking look like? | 19:46 |
| jamesbenson | so it seems to fail with host os Ubuntu16 or centos7 and with docker images ubuntu-source and cent-binary | 19:46 |
| kfox1111 | rwellum: its adistribution issue, not an issue of trust. | 19:47 |
| kfox1111 | putting all the runable code together ensures its shipped together with the same mechanism. | 19:47 |
| jamesbenson | have time to videochat? might be easier... | 19:47 |
| kfox1111 | you could put in a hash, and a curl in the container, and then it launches then curls the exact hashed files down. | 19:48 |
| kfox1111 | but then you have 2 distribution mechanisms. the docker one, and an additional one. | 19:48 |
| sdake | kfox1111 - resolv.conf workaround seems not to work with 1.7.0 | 19:48 |
| sdake | kfox1111 any suggestions | 19:48 |
| kfox1111 | sdake: hmm... | 19:48 |
| sdake | glance-api or e.g. is not resolvable | 19:48 |
| kfox1111 | resolve_conf_net_host_workaround=false work? | 19:49 |
| kfox1111 | if not, we may need to put in a patch set that sets the 1.6+ setting for fixing net=host and resolv... | 19:50 |
| sdake | kfox1111 where does that resolve_conf_net_host_woraround=false go | 19:52 |
| kfox1111 | its a helm argument. | 19:53 |
| kfox1111 | or values file. | 19:53 |
| kolla-slack | <portdirect> how are you finding k8s 1.7.0 with helm? | 19:53 |
| *** Teck7 has joined #openstack-kolla | 19:53 | |
| kolla-slack | Action: portdirect is still to kick the tyres properly | 19:53 |
| inc0 | haven't tried it yet | 19:54 |
| *** rfxn has quit IRC | 19:55 | |
| kfox1111 | if that doesn't work, we will need to add this attribute to all the pods with the workaround: https://github.com/kubernetes/kubernetes/issues/17406 | 19:55 |
| jamesbenson | inc0: 2 nic's, here's my ifconfig: http://paste.openstack.org/show/614809/ | 19:55 |
| jamesbenson | inc0: neutron_external_interface: "veno1"; kolla_external_vip_interface: "br0"; network_interface: "eno1" | 19:56 |
| inc0 | and it fails on osd bootstrap because timeout? | 19:57 |
| *** Teck72 has joined #openstack-kolla | 19:57 | |
| *** serlex has joined #openstack-kolla | 19:58 | |
| sdake | kfox1111 should I not run setup-resolve-conf? | 20:00 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 20:00 |
| *** serlex has left #openstack-kolla | 20:00 | |
| *** Teck7 has quit IRC | 20:01 | |
| kfox1111 | sdake: shouldn't have to if thats off. | 20:01 |
| jamesbenson | inc0, ceph_mon can't reach other nodes or itself I think | 20:01 |
| jascott1 | sdake have we installed the services at this point? maybe its not resovling cause they are not there yet? | 20:02 |
| jamesbenson | inc0: http://paste.openstack.org/show/614810/ | 20:02 |
| jamesbenson | but if I do a one node, it fails too unless I'm missing somehting when I modified ceph.conf | 20:02 |
| sdake | jascott1 no - that resolv.conf sh file replaces resolv.conf in each ocntainer | 20:02 |
| sdake | jascott1 in every container loaded | 20:03 |
| inc0 | damn, what's so special about your stuff?:/ | 20:03 |
| sdake | jascott1 that doesn't appear to be happening with the default settings | 20:03 |
| sdake | inc0 deploy in progress | 20:03 |
| sdake | if your interetedin watching | 20:03 |
| jamesbenson | inc0: old dell r410/r610/r710 junk? | 20:04 |
| inc0 | jamesbenson: my suggestion now would be - try to use ceph deploy | 20:05 |
| inc0 | it's nice on it's own | 20:05 |
| inc0 | and deploy kolla with external ceph | 20:05 |
| inc0 | http://docs.ceph.com/docs/master/rados/deployment/ | 20:05 |
| jamesbenson | yeah, I'm thinking that as well now that I've exhausted all of my options. | 20:05 |
| sdake | kfox1111 ok - well turning that off in cloud.yaml and not running resolv.conf results in the same thing | 20:06 |
| jamesbenson | just don't understand why this doesn't work... | 20:06 |
| inc0 | me neither... | 20:06 |
| inc0 | if ceph deploy fails in similar fashon | 20:06 |
| inc0 | then it's something deeper | 20:06 |
| inc0 | if it succeed, it's something with kolla | 20:06 |
| jamesbenson | okay | 20:06 |
| *** ducttape_ has quit IRC | 20:07 | |
| sdake | kfox1111 even this step which I would expect to work does not | 20:07 |
| jamesbenson | inc0: if kolla, that's going to be fun to debug... | 20:08 |
| inc0 | right | 20:08 |
| sdake | [sdake@localhost deploy]$ kubectl run -i -t $(uuidgen) --image=busybox --restart=Never | 20:08 |
| sdake | nsloIf you don't see a command prompt, try pressing enter. | 20:08 |
| sdake | Server: 10.3.3.10 | 20:08 |
| sdake | Address 1: 10.3.3.10 kube-dns.kube-system.svc.cluster.local | 20:08 |
| sdake | nslookup: can't resolve 'glance-api' | 20:08 |
| sdake | oh nm, it does seem to work as you described it | 20:09 |
| sdake | although deployment doesnt | 20:09 |
| * sdake groans | 20:09 | |
| jamesbenson | question, after bootstrap, kolla doesn't do anything with the disk images, correct? Because I do a reboot afterwards. I'm thought I remember someone mentioning something with the disk id or something? ::memory fails me:: | 20:10 |
| jamesbenson | just wondering if a reboot between bootstrap and deploy can impact anything | 20:11 |
| inc0 | well, with external ceph it's not relevant | 20:11 |
| inc0 | ntp jamesbenson? ;) | 20:11 |
| jamesbenson | lol, fixed that ;-) | 20:11 |
| jamesbenson | hey, you're welcome to dig around into these machines if you want... | 20:12 |
| inc0 | brain no worky today:( | 20:12 |
| jamesbenson | tell me about it, fighting near migraines for the past 2 days... hard to focus... | 20:14 |
| sdake | jascott1 - sadly have a meeting shortly | 20:14 |
| sdake | jascott1 if you want ot continue poking at it, you hae seen my workflow - feel free | 20:15 |
| *** ducttape_ has joined #openstack-kolla | 20:15 | |
| jascott1 | sdake im going to take a look around the tiller source | 20:15 |
| jascott1 | but yeah I will give it a go when I get an idea of wtf to do next ;) | 20:15 |
| sdake | jascott1 not sure how tha twill be hepful but up to you :) | 20:15 |
| jascott1 | im convinced out ultimate problem is tiller going out to lunch | 20:16 |
| sdake | this is failing: | 20:16 |
| sdake | Args: | 20:16 |
| sdake | /usr/bin/openstack endpoint create --region "RegionOne" "image" "admin" "http://glance-api:9292"; | 20:16 |
| sdake | State: Waiting | 20:16 |
| sdake | Reason: CrashLoopBackOff | 20:16 |
| sdake | Last State: Terminated | 20:16 |
| sdake | Reason: Error | 20:16 |
| sdake | Exit Code: 1 | 20:16 |
| jamesbenson | inc0, ceph-deploy (https://github.com/ceph/ceph-deploy) should be equivalent right? | 20:16 |
| inc0 | not sure, I think sl | 20:16 |
| inc0 | so | 20:16 |
| jamesbenson | looks like it's ceph deploy just automated | 20:17 |
| sdake | ok whoever figured out what the problem was - feel free to deploy :) | 20:23 |
| sdake | lets see if she works | 20:23 |
| *** ansmith has quit IRC | 20:25 | |
| sdake | ok deploy away :) | 20:29 |
| sdake | i gotta jet for about 1.5 hrs | 20:29 |
| sdake | remember remove /var/lib/kollavolumes/* | 20:29 |
| sdake | :) | 20:29 |
| jascott1 | thanks | 20:29 |
| *** jtriley has quit IRC | 20:32 | |
| *** lrensing has quit IRC | 20:36 | |
| Reepicheep | ansible question? If I want to override the openstack-infra.conf for the karbor-protection container should this have a "vint | 20:38 |
| Reepicheep | oops .. fingers got off | 20:38 |
| Reepicheep | should it have a "with_first_found" or something like that? | 20:39 |
| Reepicheep | https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/karbor/tasks/config.yml | 20:39 |
| jascott1 | sdake image kolla/cnetos-source-kolla-toolbox:3.0.2 not found | 20:41 |
| *** sambetts|afk has quit IRC | 20:42 | |
| *** sambetts has joined #openstack-kolla | 20:43 | |
| sdake | jascott1 looks like deploy is working to me | 20:45 |
| sdake | oh - thats because my network ided :) | 20:45 |
| jascott1 | oh | 20:45 |
| jascott1 | but the other containers failed previously | 20:46 |
| sdake | readiness probe failed on glance- registry | 20:49 |
| sdake | Readiness: http-get http://:9191/healthcheck delay=5s timeout=5s period=10s #success=1 #failure=3 | 20:49 |
| sdake | doesn't look correct | 20:50 |
| sdake | same thing on neutron-server | 20:51 |
| sdake | need to enable haproxy I think | 20:51 |
| sdake | or disable it | 20:52 |
| sdake | soeone changed teh docs | 20:52 |
| sdake | the docs dont work | 20:52 |
| sdake | pretty sure kfox1111 changed that iirc - i recall reviewing it | 20:52 |
| kfox1111 | haproxy should be turned on by default. | 20:53 |
| kfox1111 | it is on in the gates too. | 20:53 |
| kfox1111 | so well tested. | 20:53 |
| sdake | doesn't work with the docs | 20:55 |
| sdake | probably an ansible implementation gap | 20:57 |
| *** ducttape_ has quit IRC | 21:09 | |
| *** ducttape_ has joined #openstack-kolla | 21:09 | |
| *** klindgren has quit IRC | 21:11 | |
| jamesbenson | inc0 I think I found part of the problem, selinux | 21:13 |
| inc0 | on ubuntu? | 21:14 |
| inc0 | hmm | 21:14 |
| jamesbenson | centos right now | 21:14 |
| inc0 | yeah | 21:14 |
| inc0 | makes total sense | 21:14 |
| jamesbenson | but still having issues with ceph-deploy | 21:14 |
| inc0 | try kolla ceph with setenforce 0 | 21:14 |
| inc0 | on all nodes | 21:15 |
| jamesbenson | that's the command? | 21:15 |
| vhosakot | Reepicheep: template override is when a child block inherits/extends a parent template and overrides parent's blocks/sections. I think what you're looking for is to reconfigure/tweak openstack-infra.conf. You can just update the jinja file https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/karbor/templates/providers.d/openstack-infra.conf.j2 before deploying, or update it in /etc/kolla, and run "k | 21:15 |
| vhosakot | olla-ansible reconfigure". | 21:15 |
| jamesbenson | sorry, being stupid. | 21:16 |
| jamesbenson | ansible -i ./multinode -m shell -a "setenforce 0" all | 21:16 |
| jascott1 | uh where is tillers configmap? | 21:17 |
| vhosakot | inc0: I see kolla-ansible does not have stable/newton branch, but kolla repo still has stable/newton branch. is there a plan to remove stable/newton in kolla repo? | 21:17 |
| jamesbenson | I'm running deploy again now... | 21:17 |
| inc0 | vhosakot: kolla-ansible was created in ocata | 21:18 |
| vhosakot | ah, somebody asked me about it... so, kolla's newton branch should have the ansible playbooks as well then right? | 21:18 |
| vhosakot | yes I see it, thanks for the info inc0 | 21:19 |
| *** Pavo has joined #openstack-kolla | 21:20 | |
| *** ducttape_ has quit IRC | 21:24 | |
| *** itlinux has quit IRC | 21:26 | |
| *** iniazi_ has joined #openstack-kolla | 21:26 | |
| jamesbenson | inc0: still fails... | 21:29 |
| *** iniazi has quit IRC | 21:29 | |
| *** klindgren has joined #openstack-kolla | 21:32 | |
| *** lpetrut has quit IRC | 21:34 | |
| *** ansmith has joined #openstack-kolla | 21:36 | |
| Reepicheep | vhosakot: that is basicly what I did and it didn't work as expected | 21:38 |
| Reepicheep | I put my own openstack-infra.conf file in /etc/kolla/config/karbor/providers.d/openstack-infra.conf which did not get deployed when I ran a "reconfigure" | 21:39 |
| Reepicheep | idealy for karbor we would be able to add more providers, files in providers.d/ | 21:40 |
| *** dave-mccowan has quit IRC | 21:41 | |
| *** goldyfruit has quit IRC | 21:46 | |
| *** openstackgerrit has quit IRC | 21:47 | |
| *** schwicht has quit IRC | 21:49 | |
| *** jamesbenson has quit IRC | 21:49 | |
| *** tonanhngo has quit IRC | 21:51 | |
| *** JamesBenson-mobi has joined #openstack-kolla | 21:53 | |
| JamesBenson-mobi | Inc0: leaving my office now, but ceph deploy just gave me a health ok status.... | 21:54 |
| *** openstackstatus has quit IRC | 21:56 | |
| *** openstack has joined #openstack-kolla | 22:00 | |
| *** JamesBenson|phon has joined #openstack-kolla | 22:00 | |
| *** iniazi_ has quit IRC | 22:01 | |
| *** Pavo has quit IRC | 22:01 | |
| *** jamesPR has joined #openstack-kolla | 22:02 | |
| *** JamesBenson-mobi has quit IRC | 22:03 | |
| *** jamesPR has quit IRC | 22:04 | |
| *** JamesBenson-mobi has joined #openstack-kolla | 22:09 | |
| *** JamesBenson|phon has quit IRC | 22:12 | |
| *** rhallisey has quit IRC | 22:12 | |
| *** JamesBenson|phon has joined #openstack-kolla | 22:29 | |
| *** JamesBenson-mobi has quit IRC | 22:33 | |
| *** manheim has joined #openstack-kolla | 22:34 | |
| *** openstackgerrit has joined #openstack-kolla | 22:36 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 22:36 |
| *** JamesBenson|phon has quit IRC | 22:49 | |
| *** Pavo has joined #openstack-kolla | 22:50 | |
| sbezverk | kfox1111: ping | 22:55 |
| kolla-slack | <kfox1111> Hi | 22:55 |
| kolla-slack | <kfox1111> What's up? | 22:56 |
| kolla-slack | <kfox1111> [Not at a computer] | 22:57 |
| *** Pavo has quit IRC | 23:02 | |
| sbezverk | kfox1111: got it.. will ping you later then or next week | 23:03 |
| kolla-slack | <kfox1111> K | 23:03 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding alternative iscsi_helper support https://review.openstack.org/481690 | 23:12 |
| *** JamesBenson-mobi has joined #openstack-kolla | 23:14 | |
| *** schwicht has joined #openstack-kolla | 23:15 | |
| *** manheim has quit IRC | 23:38 | |
| *** ducttape_ has quit IRC | 23:38 | |
| *** bmace has quit IRC | 23:41 | |
| *** bmace has joined #openstack-kolla | 23:41 | |
| *** manheim has joined #openstack-kolla | 23:41 | |
| *** jtriley has joined #openstack-kolla | 23:44 | |
| *** ducttape_ has joined #openstack-kolla | 23:47 | |
| *** jtriley has quit IRC | 23:50 | |
| JamesBenson-mobi | Inc0 I think I know why it is failing.... | 23:52 |
| kfox1111 | sbezverk: ping | 23:52 |
| kfox1111 | back at a computer. | 23:52 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
