| *** vhosakot has quit IRC | 00:05 | |
| kklimonda | hmm, my libery deployment (1.1.2 from git) is failing at "ceph | Fetching Ceph keyrings" with ValueError: No JSON object could be decoded | 00:10 |
|---|---|---|
| kklimonda | it looks like ceph_mon containers are not being started | 00:10 |
| kklimonda | I can see this part of the playbook being skipped | 00:10 |
| kklimonda | looks like I had an existing ceph_mon_config from previous installation | 00:16 |
| *** diogogmt has quit IRC | 00:18 | |
| *** ravig has joined #openstack-kolla | 00:19 | |
| *** sacharya has joined #openstack-kolla | 00:20 | |
| *** sacharya_ has quit IRC | 00:20 | |
| *** Jeffrey4l has joined #openstack-kolla | 00:23 | |
| *** mbound has joined #openstack-kolla | 00:25 | |
| *** wmiller has quit IRC | 00:26 | |
| *** wmiller has joined #openstack-kolla | 00:27 | |
| *** yuanying has quit IRC | 00:27 | |
| *** yuanying has joined #openstack-kolla | 00:27 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Look in kolla-kubernetes directory instead of kolla https://review.openstack.org/329058 | 00:28 |
| *** mbound has quit IRC | 00:30 | |
| *** sacharya has quit IRC | 00:41 | |
| *** jtriley has joined #openstack-kolla | 00:51 | |
| *** daneyon_ has joined #openstack-kolla | 00:52 | |
| *** daneyon_ has quit IRC | 00:57 | |
| *** zhiwei has joined #openstack-kolla | 01:08 | |
| *** rhallisey has quit IRC | 01:16 | |
| *** fragatina has quit IRC | 01:23 | |
| *** tfukushima has joined #openstack-kolla | 01:44 | |
| *** jtriley has quit IRC | 01:48 | |
| *** sacharya has joined #openstack-kolla | 01:59 | |
| *** sacharya has quit IRC | 01:59 | |
| *** ravig has quit IRC | 02:16 | |
| openstackgerrit | Merged openstack/kolla: The quickstart guide still says to use Anisble 1.9.4 https://review.openstack.org/329015 | 02:21 |
| *** tfukushima has quit IRC | 02:29 | |
| *** dave-mccowan has quit IRC | 02:32 | |
| *** sacharya has joined #openstack-kolla | 02:34 | |
| *** tfukushima has joined #openstack-kolla | 02:38 | |
| *** daneyon_ has joined #openstack-kolla | 02:40 | |
| *** jtriley has joined #openstack-kolla | 02:44 | |
| *** daneyon_ has quit IRC | 02:45 | |
| *** jtriley has quit IRC | 02:49 | |
| *** sacharya_ has joined #openstack-kolla | 02:49 | |
| *** yuanying has quit IRC | 02:50 | |
| *** sacharya has quit IRC | 02:52 | |
| *** yuanying has joined #openstack-kolla | 02:52 | |
| *** ravig has joined #openstack-kolla | 02:54 | |
| *** klint has joined #openstack-kolla | 02:56 | |
| *** openstackgerrit has quit IRC | 03:11 | |
| *** openstackgerrit has joined #openstack-kolla | 03:11 | |
| *** sacharya_ has quit IRC | 03:27 | |
| *** sacharya has joined #openstack-kolla | 03:28 | |
| *** coolsvap has joined #openstack-kolla | 03:28 | |
| *** sacharya_ has joined #openstack-kolla | 03:30 | |
| *** sacharya has quit IRC | 03:32 | |
| *** yuanying has quit IRC | 03:33 | |
| *** sacharya has joined #openstack-kolla | 03:33 | |
| *** yuanying has joined #openstack-kolla | 03:34 | |
| *** sacharya_ has quit IRC | 03:36 | |
| *** jtriley has joined #openstack-kolla | 03:52 | |
| *** ayoung has quit IRC | 03:56 | |
| *** berendt has quit IRC | 03:56 | |
| *** jtriley has quit IRC | 03:59 | |
| *** yuanying has quit IRC | 04:05 | |
| *** yuanying has joined #openstack-kolla | 04:07 | |
| *** berendt has joined #openstack-kolla | 04:44 | |
| *** fragatina has joined #openstack-kolla | 04:45 | |
| *** fragatina has quit IRC | 04:50 | |
| *** ravig has quit IRC | 04:55 | |
| *** ravig has joined #openstack-kolla | 04:57 | |
| openstackgerrit | Merged openstack/kolla: Remove Heat CLI deprecated commands https://review.openstack.org/329125 | 05:11 |
| *** sacharya has quit IRC | 05:16 | |
| *** sacharya has joined #openstack-kolla | 05:16 | |
| *** salv-orl_ has quit IRC | 05:17 | |
| *** salv-orlando has joined #openstack-kolla | 05:18 | |
| *** ravig has quit IRC | 05:25 | |
| *** salv-orl_ has joined #openstack-kolla | 05:29 | |
| *** salv-orlando has quit IRC | 05:31 | |
| *** salv-orl_ has quit IRC | 05:34 | |
| *** salv-orlando has joined #openstack-kolla | 05:34 | |
| openstackgerrit | Merged openstack/kolla: Fix missing ceilometer user in the sudoers list for binary type https://review.openstack.org/328581 | 05:38 |
| *** SiRiuS__ has joined #openstack-kolla | 05:46 | |
| *** tfukushi_ has joined #openstack-kolla | 05:47 | |
| *** tfukushima has quit IRC | 05:49 | |
| *** jtriley has joined #openstack-kolla | 05:54 | |
| *** mbound has joined #openstack-kolla | 05:55 | |
| *** jtriley has quit IRC | 06:00 | |
| *** mbound has quit IRC | 06:00 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Create ansible home directory in kolla-toolbox container https://review.openstack.org/329274 | 06:07 |
| *** salv-orlando has quit IRC | 06:13 | |
| *** salv-orlando has joined #openstack-kolla | 06:14 | |
| *** daneyon_ has joined #openstack-kolla | 06:17 | |
| Mech422 | wirehead_: were they still making Mac clones in the pII era? I thought apple squashed that earlier ? | 06:17 |
| *** daneyon_ has quit IRC | 06:22 | |
| *** openstackgerrit has quit IRC | 06:48 | |
| *** openstackgerrit has joined #openstack-kolla | 06:48 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added unit test for start container https://review.openstack.org/327247 | 06:56 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added unit test for start container https://review.openstack.org/327247 | 07:03 |
| *** sacharya has quit IRC | 07:03 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update kollaglue repo on docker to kolla https://review.openstack.org/329289 | 07:04 |
| *** cloudnautique has quit IRC | 07:08 | |
| *** Serlex has joined #openstack-kolla | 07:08 | |
| *** jeh has quit IRC | 07:09 | |
| Mech422 | mag009_: Did you say something about missing ceph.conf file earlier ? I can't remember who said it, and cant find it in scrollback now | 07:09 |
| *** tfukushi_ has quit IRC | 07:20 | |
| *** tfukushima has joined #openstack-kolla | 07:21 | |
| *** pece has joined #openstack-kolla | 07:21 | |
| *** mikelk has joined #openstack-kolla | 07:22 | |
| *** tfukushima has quit IRC | 07:22 | |
| *** tfukushima has joined #openstack-kolla | 07:22 | |
| *** dmk0202 has joined #openstack-kolla | 07:32 | |
| *** pece has quit IRC | 07:32 | |
| *** athomas has joined #openstack-kolla | 07:33 | |
| *** tfukushima has quit IRC | 07:37 | |
| *** tfukushima has joined #openstack-kolla | 07:39 | |
| *** tfukushima has quit IRC | 07:40 | |
| *** tfukushima has joined #openstack-kolla | 07:41 | |
| *** salv-orlando has quit IRC | 07:44 | |
| *** salv-orlando has joined #openstack-kolla | 07:44 | |
| *** jeh has joined #openstack-kolla | 07:48 | |
| *** jeh has joined #openstack-kolla | 07:48 | |
| *** SiRiuS__ has quit IRC | 07:48 | |
| *** fragatina has joined #openstack-kolla | 07:49 | |
| *** fragatina has quit IRC | 07:55 | |
| *** jtriley has joined #openstack-kolla | 07:55 | |
| *** jtriley has quit IRC | 08:00 | |
| *** shardy has joined #openstack-kolla | 08:04 | |
| *** Mech422__ has joined #openstack-kolla | 08:17 | |
| *** Mech422 has quit IRC | 08:19 | |
| *** pece has joined #openstack-kolla | 08:28 | |
| *** athomas has quit IRC | 08:34 | |
| *** athomas has joined #openstack-kolla | 08:42 | |
| *** matrohon has joined #openstack-kolla | 08:50 | |
| *** daneyon_ has joined #openstack-kolla | 09:00 | |
| *** prithiv has joined #openstack-kolla | 09:04 | |
| *** daneyon_ has quit IRC | 09:04 | |
| openstackgerrit | Merged openstack/kolla: Added test document https://review.openstack.org/328069 | 09:06 |
| *** Mech422__ is now known as Mech422 | 09:27 | |
| mdnadeem_ | Jeffrey4l, when get time please visit https://review.openstack.org/#/c/327247/5 | 09:33 |
| patchbot | mdnadeem_: patch 327247 - kolla - Added unit test for start container | 09:33 |
| Mech422 | mdnadeem_: was it you or mag009_ that mentioned missing ceph configs ? | 09:35 |
| *** tyrola has joined #openstack-kolla | 09:44 | |
| *** jtriley has joined #openstack-kolla | 09:56 | |
| *** tfukushima has quit IRC | 10:01 | |
| *** jtriley has quit IRC | 10:01 | |
| *** sgordon` is now known as sgordon | 10:06 | |
| *** dmk0202 has quit IRC | 10:08 | |
| *** mbound has joined #openstack-kolla | 10:13 | |
| *** pbourke has quit IRC | 10:14 | |
| *** pbourke has joined #openstack-kolla | 10:15 | |
| Mech422 | soo...this is odd: http://pastebin.com/F6AqYiNk | 10:17 |
| Mech422 | cinder-api is consistently unable to find the default volume type - but it only hangs about half the time ?? | 10:18 |
| *** jmccarthy has quit IRC | 10:18 | |
| *** jmccarthy has joined #openstack-kolla | 10:19 | |
| mdnadeem_ | Mech422, Its not me | 10:27 |
| Mech422 | mdnadeem_: I was sure I saw someone mention it today...but I can't find it in my scrollback | 10:28 |
| Mech422 | mdnadeem_: but now I'm trying to figure out why its working 50% of the time :-P | 10:28 |
| mdnadeem_ | Mech422, I too cant find it in my scroll | 10:33 |
| Mech422 | mdnadeem_: oh! thanks for checking :-) | 10:33 |
| Mech422 | mdnadeem_: I'm probably just going crazy | 10:34 |
| mdnadeem_ | Mech422, :) | 10:34 |
| Mech422 | mdnadeem_: could you check something for me - its quick... | 10:37 |
| Mech422 | could you grep -i raise exception.VolumeTypeNotFoundByName cinder-api.log ? | 10:37 |
| *** salv-orlando has quit IRC | 10:38 | |
| *** salv-orlando has joined #openstack-kolla | 10:39 | |
| mdnadeem_ | Mech422, I have not cinder enable env | 10:39 |
| Mech422 | mdnadeem_: ahh darn...thanks anyway :-) | 10:39 |
| *** daneyon_ has joined #openstack-kolla | 10:48 | |
| *** SiRiuS__ has joined #openstack-kolla | 10:50 | |
| *** daneyon_ has quit IRC | 10:52 | |
| *** salv-orlando has quit IRC | 10:58 | |
| *** salv-orlando has joined #openstack-kolla | 10:58 | |
| *** klint has quit IRC | 11:05 | |
| *** klint has joined #openstack-kolla | 11:11 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Remove the duplication of the inventory https://review.openstack.org/328891 | 11:21 |
| *** mliima has joined #openstack-kolla | 11:22 | |
| *** rhallisey has joined #openstack-kolla | 11:28 | |
| *** salv-orl_ has joined #openstack-kolla | 11:28 | |
| *** salv-orlando has quit IRC | 11:32 | |
| *** jtriley has joined #openstack-kolla | 11:47 | |
| berendt | is somebody currently working on neutron-vpnaas container and ansible integration? | 11:49 |
| *** liyi has joined #openstack-kolla | 11:50 | |
| *** coolsvap has quit IRC | 11:51 | |
| openstackgerrit | Merged openstack/kolla: Create ansible home directory in kolla-toolbox container https://review.openstack.org/329274 | 11:52 |
| *** jtriley has quit IRC | 11:52 | |
| *** ravig has joined #openstack-kolla | 12:05 | |
| *** fragatina has joined #openstack-kolla | 12:11 | |
| *** matrohon has quit IRC | 12:15 | |
| *** fragatina has quit IRC | 12:16 | |
| DuncanT | Has anybody done a recent kolla how-to? If not I'll document anything I hit while trying, but if somebody's written something then it saves me time | 12:20 |
| rhallisey | DuncanT, what do you mean by a how-to? | 12:20 |
| DuncanT | rhallisey: Last time I tried kolla (early days) there were a bunch of gotchas (globally installed things, docker config changes, etc) that weren't clearly documented... | 12:21 |
| DuncanT | rhallisey: There were acouple of blog posts that walked through them all | 12:21 |
| DuncanT | rhallisey: Wondering if something similar is available (or even needed) now | 12:22 |
| rhallisey | DuncanT, gotcha. The quickstart guide has seen a lot of improvement since then | 12:22 |
| rhallisey | DuncanT, I would recommend try that and let the community know if you think it has decent coverage | 12:22 |
| DuncanT | rhallisey: Great, I'll give it a go if nobody else suggests anything better before I start | 12:23 |
| DuncanT | rhallisey: Patches welcome, I assume? | 12:23 |
| rhallisey | DuncanT, always | 12:23 |
| rhallisey | they would be greatly appreciated | 12:23 |
| *** matrohon has joined #openstack-kolla | 12:25 | |
| *** berendt has quit IRC | 12:32 | |
| *** salv-orl_ has quit IRC | 12:33 | |
| DuncanT | Is anybody publishing periodic (or milestone/release) kolla containers anywhere? | 12:33 |
| *** salv-orlando has joined #openstack-kolla | 12:34 | |
| DuncanT | Ah, never mind, I found them | 12:34 |
| *** daneyon_ has joined #openstack-kolla | 12:36 | |
| *** daneyon_ has quit IRC | 12:41 | |
| DuncanT | Looking through the quickstart, I'm still required to globally install pip requirements, which I find causes me issues - has anybody used kolla purely in a venv, or should I run it in a VM? | 12:47 |
| rhallisey | VM should be fine | 12:48 |
| *** jtriley has joined #openstack-kolla | 12:49 | |
| *** ravig has quit IRC | 12:58 | |
| *** klint has quit IRC | 12:59 | |
| *** stvnoyes has quit IRC | 12:59 | |
| *** salv-orlando has quit IRC | 13:00 | |
| *** salv-orlando has joined #openstack-kolla | 13:00 | |
| *** stvnoyes has joined #openstack-kolla | 13:00 | |
| *** chopmann_ has joined #openstack-kolla | 13:04 | |
| chopmann_ | hello everyone how do I label the disks as journals? I have found this https://bugs.launchpad.net/kolla/+bug/1560049, and looked at the commit, but i dont see something like KOLLA_CEPH_JOURNAL_BOOTSTRAP | 13:08 |
| openstack | Launchpad bug 1560049 in kolla " Allow external ceph journals and fix bootstrap" [Wishlist,Confirmed] - Assigned to Sam Yaple (s8m) | 13:08 |
| *** diogogmt has joined #openstack-kolla | 13:13 | |
| *** ppowell has quit IRC | 13:18 | |
| *** ppowell has joined #openstack-kolla | 13:18 | |
| *** dwalsh has joined #openstack-kolla | 13:19 | |
| *** dave-mccowan has joined #openstack-kolla | 13:20 | |
| sean-k-mooney | chopmann_: ya i was reading that code yesterday and taught the same. was not sure if it would allow external journal fo rthe cache drivers or not | 13:23 |
| chopmann_ | sean-k-mooney: https://github.com/openstack/kolla/blob/master/docker/kolla-toolbox/find_disks.py#L110 if i understand it right--> We need to add '_J' to KOLLA_CEPH_OSD_BOOTSTRAP | 13:25 |
| sean-k-mooney | i think the existing code should work though | 13:25 |
| sean-k-mooney | hum let me have a look | 13:25 |
| liyi | Hi for those who want to deploy kolla with kubernetes, you might interested with this: https://github.com/liyimeng/kubernetes/tree/k8s-kolla | 13:28 |
| *** diogogmt has quit IRC | 13:28 | |
| liyi | In that particular k8s branch, I have cherry picked the missing features in k8s in order to satisfy kolla deployment need | 13:29 |
| rhallisey | liyi, how about a doc explaining it | 13:30 |
| sean-k-mooney | chopmann_: on one hand looking at https://github.com/openstack/kolla/blob/master/ansible/roles/ceph/tasks/bootstrap_osds.yml#L57-L87 and https://github.com/openstack/kolla/blob/master/docker/ceph/ceph-osd/extend_start.sh | 13:31 |
| sean-k-mooney | chopmann_: i think the code should work as is but i have not tested it | 13:31 |
| sean-k-mooney | chopmann_: have you to use an external journal for the cache drive? | 13:32 |
| Mech422 | sean-k-mooney: add _J to the journal part name | 13:32 |
| rhallisey | liyi, since kube 1.3 is closing up soon, it would be nice to have a doc until kube supports the feature | 13:32 |
| liyi | sure Ryan: if you deploy kolla with k8s now, you will hit two major issues: k8s dose not allow undescore as key in configmap, which require you rename many config filename in kolla | 13:32 |
| Mech422 | sean-k-mooney, chopmann_ : you also want to watch for stale partition names... | 13:32 |
| chopmann_ | so, label SSD with KOLLA_CEPH_OSD_BOOTSTRAP_J and HDD with KOLLA_CEPH_OSD_BOOTSTRAP --> magic&unicorns --> Ceph with extenal Journals? :D | 13:32 |
| *** d_code has quit IRC | 13:33 | |
| liyi | another issues is that k8s is not allow you to mount a volume with propagation=shared. | 13:33 |
| Mech422 | chopmann_: should do - but I've only used partitions - not full disks | 13:33 |
| Mech422 | chopmann_: also - watch out for stale partition names .. | 13:33 |
| liyi | Both iusses have folks working on it in k8s community, | 13:34 |
| chopmann_ | aye aye | 13:34 |
| rhallisey | liyi, awesome. We can carry a doc on how to rebuild hypercube until it's fixed | 13:34 |
| *** ravig has joined #openstack-kolla | 13:34 | |
| Mech422 | chopmann_: gah - can't find my patch - too many open windows | 13:34 |
| rhallisey | since it likely won't be until 1.4 | 13:34 |
| *** d_code has joined #openstack-kolla | 13:35 | |
| Mech422 | chopmann_: I patched find_disks.py to call sgdisk directly - it could also be patched to call 'udevadm trigger' after updating partition names | 13:35 |
| liyi | but not merged at the moment, I cherry pick them in a single branch, which allow you build a hyperkube for kolla need | 13:35 |
| sean-k-mooney | Mech422: right looking at sams blog you just add KOLLA_CEPH_OSD_CACHE_BOOTSTRAP_1_J | 13:35 |
| sean-k-mooney | as the partition lable | 13:35 |
| Mech422 | sean-k-mooney: for partitions yes | 13:35 |
| Mech422 | sean-k-mooney: for full disk - I think you just leave off the '_1' - not sure how you get external with full disk though | 13:36 |
| liyi | that is right rhallisey! since it involve api change, the reviewing progress is pretty slow. | 13:36 |
| rhallisey | wirehead_, when you're around ^ | 13:36 |
| chopmann_ | sean-k-mooney: can you post the link, please? | 13:37 |
| Daviey | Mech422: Am i right in saying that only the first disk label is added to ceph, and not all disks with that label? | 13:37 |
| sean-k-mooney | Mech422: i dont think it ever makes sense to dedicate a full disk to one journal though right? | 13:37 |
| sean-k-mooney | sure i got it form Mech422 yesterday https://yaple.net/2016/03/20/deploying-openstack-mitaka-with-kolla-docker-and-ansible/ | 13:37 |
| Daviey | sean-k-mooney: a full disk of SSD? Why not? | 13:37 |
| chopmann_ | i though the script would take the whole disk and partition it to have multiple journals | 13:37 |
| Mech422 | Daviey: you add a unique tag to the partition name | 13:37 |
| sean-k-mooney | Daviey: a full disk just for a journal | 13:37 |
| Mech422 | Daviey: _1 + _1_J , _2 + _2_J etc | 13:38 |
| Mech422 | sean-k-mooney: yeah - but what about a full disk with an external (SSD) partition journal ? | 13:38 |
| Mech422 | Daviey: and again - watch out for stale partition table data | 13:39 |
| sean-k-mooney | Mech422: that is what i would expect | 13:39 |
| Mech422 | sean-k-mooney: I mean I don't know the incantation for that combo :-) | 13:39 |
| sean-k-mooney | whole spinning disk for osd and journal partition on ssd | 13:39 |
| Mech422 | sean-k-mooney: which is probably what most people want ? | 13:39 |
| sean-k-mooney | having said that you can just create a singel partition that fills the whole disk to get the same effect and alwasy use partitions | 13:40 |
| Mech422 | Oh! here's my patches for stale-partition stuff : https://review.openstack.org/#/c/326609 | 13:41 |
| Mech422 | sean-k-mooney: ohh! you had your coffee already, didn't you :-P | 13:41 |
| Mech422 | sean-k-mooney: let me get a cup so I don't look so stupid :-) | 13:41 |
| Daviey | Mech422: and in the case of a disk replacement, re-adding the _X that was from the failed drive.. re-run the playbook and it DTRT? | 13:42 |
| sean-k-mooney | Mech422: did you update those to use the udev trigger thing. haha lookally im gmt time so i have the timezone advantage on my side :) | 13:42 |
| *** d_code_ has joined #openstack-kolla | 13:42 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the novnc configure in the nova.conf https://review.openstack.org/328882 | 13:42 |
| Mech422 | Daviey: ohh - umm...no... | 13:43 |
| Mech422 | Daviey: It would create a new OSD I think... | 13:43 |
| sean-k-mooney | ya it would create a new OSD | 13:44 |
| Mech422 | Daviey: to replace a drive...you ... umm... just replace the drive and let ceph sync it ? | 13:44 |
| sean-k-mooney | though with ceph if a disk fails do you care that the replacement has the old osd id | 13:44 |
| *** d_code has quit IRC | 13:45 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the novnc configure in the nova.conf https://review.openstack.org/328882 | 13:45 |
| *** d_code_ is now known as d_code | 13:45 | |
| Daviey | Well yes... but I'd need to add the new disk to 'kolla'.. note, it might have different enumeration | 13:45 |
| Mech422 | sean-k-mooney: Didn't update the patch for udevadm trigger... I mentioned it yesterday and no-one seemed to have a strong feeling about it | 13:45 |
| Daviey | I mean, should _X always increment.. or should numbers be re-used? | 13:45 |
| Mech422 | sean-k-mooney: so either it gets merged as is, or I'll hit it up next time I submit a patch ? | 13:46 |
| *** berendt has joined #openstack-kolla | 13:46 | |
| Mech422 | Daviey: 'X' is meaningless after you start ceph | 13:46 |
| Mech422 | Daviey: its just a 'tag' so ceph knows which partition goes with which journal | 13:46 |
| Daviey | I get that... but without doing something.. the disk won't be added through ansible to anything? No? | 13:46 |
| sean-k-mooney | Mech422: i had no strong feeling either way on udevadm trigger vs not using /sys | 13:47 |
| Mech422 | Daviey: I don't think kolla has a 'failure' play ? | 13:47 |
| Daviey | hmm | 13:47 |
| Mech422 | Daviey: I would probably replace the drive by hand at this point | 13:47 |
| Mech422 | Daviey: or run on RAID-1, then if a disk fails, you can use plain old raid recovery? | 13:48 |
| Daviey | Mech422: Ceph on RAID? | 13:48 |
| Mech422 | Daviey: I said it was an idea...never said good idea :-) | 13:49 |
| Mech422 | Daviey: but it would make recovery much simpler - and double your read speeds | 13:49 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin to the Ubuntu 16.04 https://review.openstack.org/329454 | 13:50 |
| Mech422 | sean-k-mooney: I was thinking - since Kolla already has the notion of a 'deployment node'... | 13:50 |
| Mech422 | sean-k-mooney: it might make sense to try and figure out how to use the 'normal' ceph-deploy | 13:51 |
| Daviey | At the moment, I'm trying to decide between LVM'ing a JBOD... Adding an OSD to each disk in each server.. or Using Ceph on RAID... OSD on each disk seems cleaner, but i'm worried about the maintenance of replacing disks with kolla | 13:51 |
| Mech422 | sean-k-mooney: then you could do all the add/remove disk/osd/whatever stuff easily | 13:51 |
| *** jmccarthy has quit IRC | 13:52 | |
| wirehead_ | liyi: I think I have a fix for the underscore issue that doesn't involve a custom k8s build. | 13:52 |
| *** jmccarthy has joined #openstack-kolla | 13:52 | |
| Mech422 | sean-k-mooney: of course, ceph-deploy doesn't speak 'container' so it would be interesting... | 13:53 |
| wirehead_ | liyi: https://blueprints.launchpad.net/kolla-kubernetes/+spec/fix-underscores-in-paths | 13:53 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Remove verbose parameter from lbaas_agent.ini file https://review.openstack.org/329457 | 13:54 |
| *** coolsvap has joined #openstack-kolla | 13:54 | |
| *** dwalsh has quit IRC | 13:56 | |
| *** ravig has quit IRC | 13:59 | |
| *** jtriley has quit IRC | 14:00 | |
| *** jtriley has joined #openstack-kolla | 14:04 | |
| liyi | Hi wirehead_, that is nice. But we still need to k8s customized built to have shared mount propagation | 14:06 |
| liyi | Hi wirehead_, that is nice! But we still need a k8s customized built to have shared mount propagation | 14:06 |
| *** dwalsh has joined #openstack-kolla | 14:11 | |
| *** zhiwei has quit IRC | 14:12 | |
| wirehead_ | liyi: what is broken with shared mount proagation? | 14:14 |
| *** esharao_ has joined #openstack-kolla | 14:15 | |
| mag009_ | morning | 14:15 |
| Mech422 | Morning mag009_ | 14:15 |
| mag009_ | anyone deployed ironic with kolla ? | 14:15 |
| mliima | mag009_, https://bugs.launchpad.net/kolla/+bug/1565936 | 14:16 |
| openstack | Launchpad bug 1565936 in kolla "Ironic inspector not installed for Ubuntu" [Medium,Triaged] | 14:16 |
| mag009_ | mliima: anyone working on this bug ? | 14:17 |
| mag009_ | guess I can if no one is | 14:17 |
| mliima | do it mag009_ :) | 14:17 |
| mag009_ | i just enabled it for fun and it failed because it detect my compute aren't using openvswitch | 14:18 |
| mag009_ | why does it require all compute to install ironic on it | 14:19 |
| mag009_ | all i need is the server/controller to server ironic | 14:19 |
| Mech422 | anyone with a recently deployed cinder that could 'grep raise exception.VolumeTypeNotFoundByName cinder-api.log' for me ? | 14:20 |
| mliima | I think that sbezverk_, can help you Mech422 | 14:21 |
| liyi | wirehead_ it is not support at all with current version of k8s, neither in next release 1.3. Hopefully it could make the way through in 1.4 | 14:21 |
| *** mliima is now known as mliima_away | 14:22 | |
| Mech422 | mliima_away: oh - thanks | 14:22 |
| wirehead_ | liyi: Yeah, I'm getting pessimistic about that. what's broken on the OpenStack side without shared mount propagation? | 14:22 |
| Mech422 | I just wanna see if it happens with all installs or just ceph or just mine (cuz I'm special :-P ) | 14:22 |
| liyi | neutron won't work | 14:23 |
| liyi | then we have to go with nova-network instead. Not so much excited then ;) | 14:23 |
| *** daneyon_ has joined #openstack-kolla | 14:24 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin to the Ubuntu 16.04 https://review.openstack.org/329454 | 14:26 |
| wirehead_ | liyi: k. Neutron's going to be the most annoying part of this all. | 14:27 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the novnc configure in the nova.conf https://review.openstack.org/328882 | 14:28 |
| *** daneyon_ has quit IRC | 14:29 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Implement Rally Dockerfile https://review.openstack.org/326414 | 14:30 |
| liyi | wirehead_ not much worse that nova. And we love chanllence :D | 14:30 |
| *** vhosakot has joined #openstack-kolla | 14:30 | |
| *** jtriley_ has joined #openstack-kolla | 14:32 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/329480 | 14:33 |
| mag009_ | is there any plan of deploying ironic in standalone mode ? | 14:35 |
| *** jtriley has quit IRC | 14:35 | |
| *** ayoung has joined #openstack-kolla | 14:36 | |
| *** jrist has quit IRC | 14:38 | |
| *** ravig has joined #openstack-kolla | 14:38 | |
| *** jtriley_ has quit IRC | 14:38 | |
| esharao_ | hi good morning.. quick question.. i'm using vagrant with Virtual Box on a windows machine and I keep on getting an error with Kolla_start not found when i run kolla-ansible deploy.. Anyone else faced similar issue? | 14:39 |
| d_code | morning. working with neutron and trying to customize configs for a particular host in ml2_conf.ini… is this a typo? https://github.com/openstack/kolla/blob/ba30579b3b3e69536f5bd59e57d39ed4b63a2dd6/ansible/roles/neutron/tasks/config.yml#L72 | 14:43 |
| d_code | or are the neutron configs interchangable? | 14:44 |
| *** inc0 has joined #openstack-kolla | 14:46 | |
| inc0 | good morning Kolla! | 14:46 |
| d_code | I guess since it’s an ini file you could just merge them all together? just seems inconsistent with other merge_configs that I’ve done (which is mostly nova.conf) | 14:46 |
| d_code | morning inc0 | 14:46 |
| inc0 | << you should read it as in "good morning Vietnam" >> | 14:46 |
| inc0 | d_code, which one? | 14:47 |
| inc0 | which conf you're reffering to? | 14:47 |
| d_code | ml2_conf.ini | 14:47 |
| d_code | inc0: that _is_ how I read it =) | 14:47 |
| inc0 | :D | 14:47 |
| d_code | the link…line 72…. it reads in various ml2_conf.ini files, but for inventory_hostname specific, it reads in nova.conf | 14:48 |
| d_code | oh…guess you didn’t see the link | 14:48 |
| d_code | :p | 14:48 |
| d_code | https://github.com/openstack/kolla/blob/ba30579b3b3e69536f5bd59e57d39ed4b63a2dd6/ansible/roles/neutron/tasks/config.yml#L72 | 14:48 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Change base container of neutron-server container to neutron-lbaas-agent https://review.openstack.org/329487 | 14:48 |
| inc0 | d_code, bug. | 14:48 |
| inc0 | should be fixed | 14:49 |
| berendt | can somebody please have a look on this review, I think the solution is a bit hacky, but I think it is the only way to add the neutron-lbaas-agent sources into the neutron-server container | 14:49 |
| d_code | it’s been like that since at least Liberty | 14:49 |
| inc0 | a bug nonetheless | 14:49 |
| d_code | which is why I thought I was crazy | 14:49 |
| d_code | cool | 14:49 |
| berendt | or is it possible to add two source archives defined in kolla-build.conf into one container? | 14:49 |
| d_code | I’ll create the bug | 14:49 |
| Mech422 | inc0: Morning | 14:49 |
| *** jrist has joined #openstack-kolla | 14:50 | |
| Mech422 | could you 'grep raise exception.VolumeTypeNotFoundByName cinder-api.log' for me ? | 14:50 |
| inc0 | berendt, hmm...lbaas doesn't depend on server? | 14:51 |
| *** jtriley has joined #openstack-kolla | 14:51 | |
| inc0 | Mech422, as soon as I redeploy stuff | 14:51 |
| berendt | inc0, when enabling lbaas agent the start of neutron-server failed with the following issue: ImportError: Plugin 'neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2' not found | 14:52 |
| Mech422 | inc0: thanks! I opened a bug last night about it - but it appears its not fatal | 14:52 |
| Mech422 | inc0: also, I got some sort of wierd behavior going on, where VMs only start after about 50% of deploys - its almost like a race or something | 14:53 |
| inc0 | ehh all software is broken | 14:54 |
| inc0 | did you get the idea what's wrong? | 14:54 |
| Mech422 | inc0: not for the race - the bug is potentially a 1 liner... | 14:54 |
| Mech422 | inc0: https://bugs.launchpad.net/kolla/+bug/1592356 | 14:55 |
| openstack | Launchpad bug 1592356 in kolla "cinder-api exception due to default_volume_type not being created" [Undecided,New] | 14:55 |
| inc0 | most of bugs are | 14:55 |
| Mech422 | inc0: I think the 'race' might be the nova/rabbitmq thing someone bugged earlier | 14:55 |
| Mech422 | inc0: redeploying now to see if I can re-create it | 14:56 |
| *** mliima_ has joined #openstack-kolla | 14:57 | |
| inc0 | if you manage to repeat this state, keep it that way so we can get to the bottom of this | 14:57 |
| Mech422 | inc0: sure - I'm good at doing nothing :-) | 14:58 |
| *** sacharya has joined #openstack-kolla | 14:58 | |
| Mech422 | its one of my finer skills! | 14:58 |
| inc0 | mag009_, ping | 14:59 |
| *** mliima_away has quit IRC | 14:59 | |
| mag009_ | inc0: pong | 14:59 |
| mag009_ | fast response eh! | 14:59 |
| inc0 | http://paste.fedoraproject.org/379025/59164001/ got sth like that while deploying etcd | 15:00 |
| inc0 | and it's restarting | 15:00 |
| *** sacharya_ has joined #openstack-kolla | 15:01 | |
| rhallisey | new spec: https://review.openstack.org/#/c/304182/ | 15:01 |
| patchbot | rhallisey: patch 304182 - kolla - Spec: Deploy Kolla images using Kubernetes | 15:01 |
| inc0 | seems like non-idempotency on etcd side | 15:01 |
| inc0 | you might want to add check if there is existing cluster | 15:01 |
| mag009_ | is it possible you've deployed it already and the cleanup didn't flush the volume ? | 15:01 |
| mag009_ | so it keep the old data but I've been redeploying it over and over here without problem | 15:02 |
| inc0 | yeah, possible, but now it seems to flush it correctly | 15:02 |
| inc0 | however, even if that'd be true | 15:02 |
| mag009_ | etcdctl member list ? | 15:02 |
| inc0 | you might want to try to bootstrap it only if there is no volume | 15:02 |
| mag009_ | yes i will add the bootstraping | 15:03 |
| openstackgerrit | Derek Ditch proposed openstack/kolla: Makes ml2_conf.ini configuration consistent https://review.openstack.org/329506 | 15:03 |
| *** sacharya has quit IRC | 15:04 | |
| inc0 | mag009_, use this approach: https://github.com/openstack/kolla/blob/master/ansible/roles/ceph/tasks/bootstrap_mons.yml#L25 | 15:04 |
| mag009_ | ok i will | 15:06 |
| *** Jeffrey4l has quit IRC | 15:07 | |
| *** dwalsh has quit IRC | 15:08 | |
| wirehead_ | rhallisey: I'll chime in a bit later today. | 15:08 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Make cleanup remove restarting/exited containers https://review.openstack.org/329511 | 15:13 |
| rhallisey | wirehead_, thanks | 15:14 |
| *** pece has quit IRC | 15:15 | |
| *** dwalsh has joined #openstack-kolla | 15:19 | |
| *** prithiv has quit IRC | 15:21 | |
| mag009_ | :P | 15:23 |
| *** salv-orlando has quit IRC | 15:25 | |
| *** salv-orlando has joined #openstack-kolla | 15:26 | |
| *** Serlex has quit IRC | 15:28 | |
| *** salv-orlando has quit IRC | 15:30 | |
| *** salv-orlando has joined #openstack-kolla | 15:30 | |
| inc0 | mag009_, soo...neutron_l3 agent is deploying but incorrectly | 15:31 |
| inc0 | you need l3 agent with calico at all? | 15:32 |
| Mech422 | inc0: blah - fired up 3 vms no problem | 15:33 |
| Mech422 | inc0: redeploying again - gimme 30 minutes | 15:33 |
| inc0 | lol | 15:34 |
| Mech422 | 65% of the time, it works every time :-P | 15:34 |
| Mech422 | what? no everquest 2 fans around ? | 15:35 |
| inc0 | here is term for that | 15:35 |
| inc0 | Heisenbug | 15:35 |
| inc0 | bug that is affected by observation | 15:35 |
| Mech422 | Heh | 15:35 |
| inc0 | it doesn't appear when you observe | 15:35 |
| Mech422 | but is it dead or alive ? | 15:36 |
| inc0 | as for everquest 2..I kinda wanted to play it but never got to do it;) | 15:36 |
| inc0 | Thats Shroedingerbug | 15:36 |
| Mech422 | inc0: its still a load of fun... I go back to it every year or two | 15:36 |
| inc0 | I have issues in mmo | 15:37 |
| inc0 | I manage to build comunity, go to guild and such | 15:37 |
| inc0 | and when I get bored by game, I leave people behind and feel bad | 15:37 |
| inc0 | so I stopped playing mmos | 15:37 |
| Mech422 | inc0: yeah - I feel ya - we go thru the same... | 15:37 |
| Mech422 | inc0: the girlfriend and I have been in 1 guild like 5-10 years... | 15:38 |
| Mech422 | inc0: we drop in for a month or two, then feel bad when we don't go back for 6-1year | 15:38 |
| inc0 | yeah it happened to me in number of games | 15:39 |
| inc0 | FF14, EVE online, WoW....I played a lot of these | 15:39 |
| Mech422 | we did FFIV:AAR, and just about everything except Eve, WoW and warhammer | 15:40 |
| Mech422 | we even played aion for a bit - it turned into a decent PVE mmo... | 15:40 |
| inc0 | I did these 3;) | 15:40 |
| Mech422 | looks a bit plasticey though | 15:40 |
| inc0 | warhammer could be great mmo, just not enough people | 15:40 |
| inc0 | wh had one good thing, you could lvlup by pvp:D | 15:41 |
| Mech422 | the girlfriend requires cosmetics, housing/decorating, and mounts/pets... | 15:41 |
| Mech422 | so we don't really play anything that doesn't have those | 15:41 |
| inc0 | ultima online;) | 15:41 |
| inc0 | go ahead, build a town | 15:41 |
| Mech422 | oh! guild wars 2 was like that - whole seperate pvp campaign | 15:41 |
| *** SiRiuS__ has quit IRC | 15:41 | |
| Mech422 | inc0: have you seen lord british's new one ? | 15:41 |
| inc0 | I wouldn't call it a campaign really, but it was cool | 15:42 |
| Mech422 | shroud of astaria ? | 15:42 |
| *** SiRiuS__ has joined #openstack-kolla | 15:42 | |
| inc0 | nope | 15:42 |
| Mech422 | It might have potential - but its REALLY freaking pricey :-P | 15:42 |
| Mech422 | people are paying $10K USD for 'guild towns' | 15:43 |
| inc0 | lol | 15:43 |
| Mech422 | its almost as bad as entropia universe | 15:43 |
| Mech422 | eq2 has pretty much everything - but its starting to look a bit dated. LoTR has the BEST story quests - and the graphics hold up a bit better, but housing sucks | 15:44 |
| inc0 | I'm wondering if they add multiplayer to no man sky | 15:45 |
| Mech422 | inc0: oh! I thought it was multiplayer ? | 15:45 |
| inc0 | well, hard to tell | 15:45 |
| Mech422 | inc0: we want to duo in it...gf loves to explore - so procedural generation is awesome for her | 15:45 |
| Mech422 | inc0: Planet explorers was shaping up well too | 15:46 |
| inc0 | yeah, you should try Elite: Dangerous | 15:46 |
| inc0 | if no man sky give me 2 things, I'll be super happy | 15:46 |
| *** d_code_ has joined #openstack-kolla | 15:47 | |
| inc0 | VR and mmo | 15:47 |
| Mech422 | inc0: I played the original Elite on C-64 ... I could never get out of the dam station - lousy driver :-P | 15:47 |
| apuimedo | Ibm announced they would collaborate on sword art online mmo iirc | 15:47 |
| inc0 | lol | 15:47 |
| inc0 | ibm into gaming? | 15:47 |
| openstackgerrit | Merged openstack/kolla: Remove verbose parameter from lbaas_agent.ini file https://review.openstack.org/329457 | 15:47 |
| Mech422 | apuimedo: oh? | 15:47 |
| apuimedo | Apparently it's supposed to be very high tech | 15:48 |
| Mech422 | inc0: no mans sky would be awesome on vive... | 15:48 |
| Mech422 | apuimedo: never heard of that one...I'll have to check it out :-) | 15:48 |
| *** d_code has quit IRC | 15:48 | |
| *** d_code_ is now known as d_code | 15:48 | |
| apuimedo | https://www.google.es/url?q=https://developer.ibm.com/dwblog/sword-art-online-vrmmo/&sa=U&ved=0ahUKEwiVp-3_7qfNAhVCMBoKHVucAesQFggLMAA&usg=AFQjCNElxtbc4IvT-dRw5eXhkl_G3uhUWg | 15:48 |
| Mech422 | ArcheAge was neat - but it was like a second job :-P | 15:48 |
| Mech422 | apuimedo: Danke! | 15:49 |
| apuimedo | It's based on a very famous manga/anime | 15:49 |
| apuimedo | I save it for my retirement :) | 15:50 |
| apuimedo | I'm going to be playing vmmorpg all the time | 15:50 |
| Mech422 | apuimedo: wow..body scan -> avatar? that um...well, actually - I'm ugly so maybe not... | 15:51 |
| Mech422 | apuimedo: yeah - its always nice when a game has a lot of lore to draw from... | 15:51 |
| apuimedo | I think having similar aspect or at least height may help with the immersion | 15:51 |
| Mech422 | apuimedo: makes for the best worlds | 15:52 |
| *** d_code_ has joined #openstack-kolla | 15:52 | |
| *** d_code has quit IRC | 15:53 | |
| *** d_code_ is now known as d_code | 15:53 | |
| inc0 | btw sword art online.... | 15:54 |
| inc0 | it's anime inspired by mmo | 15:54 |
| inc0 | so ibm is making mmo based on anime based on mmo | 15:55 |
| Mech422 | inc0: a recursive mmo ? | 15:55 |
| apuimedo | Yes. Going full circle | 15:55 |
| Mech422 | or would that be mmo^2 | 15:55 |
| apuimedo | The first arc of the anime is good enough to be watched | 15:56 |
| *** chopmann_ has quit IRC | 16:03 | |
| Mech422 | inc0: crud - 10 VMs no problem... | 16:03 |
| *** matrohon has quit IRC | 16:04 | |
| Mech422 | inc0: I might have to rebuild from scratch and not just re-deploy | 16:04 |
| inc0 | yeah, try that | 16:04 |
| inc0 | btw, deploy takes 30mins? | 16:04 |
| inc0 | how big many nodes? | 16:04 |
| *** ravig has quit IRC | 16:04 | |
| *** jtriley has quit IRC | 16:05 | |
| Mech422 | inc0: pull and deploy from a local docker repo takes 20 minutes... | 16:05 |
| *** mummer has joined #openstack-kolla | 16:05 | |
| *** mikelk has quit IRC | 16:06 | |
| Mech422 | inc0: I actually timed it last night after talking to sdake - turns out its closer to an hour for bare-metal to cluster not 2 hours | 16:06 |
| inc0 | kk | 16:06 |
| inc0 | still slowish | 16:06 |
| Mech422 | inc0: build & push - 18 minutes | 16:06 |
| *** ravig has joined #openstack-kolla | 16:06 | |
| inc0 | deploy from local registry on 10vms takes me 6mins circa | 16:06 |
| inc0 | ceph | 16:06 |
| inc0 | and everything | 16:06 |
| inc0 | btw they enabled support for ascii arts in git commit messages | 16:07 |
| inc0 | in 2.9 | 16:07 |
| Mech422 | inc0: 10GigE ? | 16:07 |
| inc0 | imagine the possibilities | 16:07 |
| inc0 | Mech422, 1gig, but that shouldn't matter that much | 16:07 |
| Mech422 | inc0: (oh - I'm on 7 hardware nodes - 1Gig) | 16:08 |
| inc0 | ssd or spindle | 16:08 |
| inc0 | ? | 16:08 |
| Mech422 | inc0: huh - I'll look at mine later.. | 16:08 |
| Mech422 | inc0: SSD, 1GigE, Dell managed switches | 16:08 |
| inc0 | so it should be quicker | 16:08 |
| Mech422 | inc0: could be because I use ovs for base network ? Didn't think it was that much slower though | 16:09 |
| inc0 | shouldn't matter | 16:09 |
| inc0 | try to add pipelining to your ansible.conf | 16:09 |
| inc0 | and bring up number of threads to some big number, like 100 | 16:09 |
| Mech422 | inc0: dunno then - its just a bog standard trusty install...Could try that, mine builds in 2 'batches' | 16:10 |
| Mech422 | 5 nodes, then the last 2 come in | 16:10 |
| *** zhiwei has joined #openstack-kolla | 16:10 | |
| Mech422 | inc0: btw - Am I suposed to only get 1 keepalived container ? | 16:11 |
| *** daneyon_ has joined #openstack-kolla | 16:13 | |
| *** jtriley has joined #openstack-kolla | 16:13 | |
| *** zhiwei has quit IRC | 16:15 | |
| *** daneyon_ has quit IRC | 16:18 | |
| *** ravig has quit IRC | 16:22 | |
| *** prithiv has joined #openstack-kolla | 16:22 | |
| *** prithiv has quit IRC | 16:31 | |
| *** fragatina has joined #openstack-kolla | 16:36 | |
| *** fragatin_ has joined #openstack-kolla | 16:36 | |
| *** fragatina has quit IRC | 16:40 | |
| inc0 | keepalived+haproxy sits on network nodes | 16:41 |
| inc0 | if you have more than one, you should have more than one keepalived | 16:41 |
| Mech422 | inc0: I figured as much - I don't need network node soo... | 16:42 |
| inc0 | you can put it on controllers | 16:42 |
| inc0 | just add same nodes as controllers to network group | 16:42 |
| Mech422 | inc0: yeah - I stuffed a 'network node' on the controller hosts | 16:42 |
| inc0 | there is no reason why groups in inv needs to be exclusive | 16:42 |
| Mech422 | yeah - I just didn't feel like mucking in the inventory file too much yet...wanted to get used to how its 'supposed' to work first | 16:43 |
| inc0 | well inv file is supposed to be mucked in | 16:44 |
| *** ravig has joined #openstack-kolla | 16:44 | |
| Mech422 | inc0: yeah - I just did't want too much 'magical setup' to start out with.. too easy to get wierd errors....I'm pushing my luck with overriding OVS as it is | 16:45 |
| inc0 | so network and control on same nodes is how I deploy it | 16:45 |
| inc0 | and storage and compute on same nodes | 16:45 |
| Mech422 | yeah - I do storage and compute on sames nodes | 16:45 |
| openstackgerrit | Merged openstack/kolla: Implement Rally Dockerfile https://review.openstack.org/326414 | 16:45 |
| Mech422 | and I'll run network and control since I don't really need network for much | 16:46 |
| Mech422 | but one step at a time | 16:46 |
| Mech422 | lots of knobs and dials I haven't explored yet :-) | 16:46 |
| *** coolsvap has quit IRC | 16:46 | |
| Mech422 | I actually generate kolla's inventory from ansible on another machine | 16:47 |
| *** salv-orlando has quit IRC | 16:49 | |
| *** salv-orlando has joined #openstack-kolla | 16:50 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Make cleanup remove restarting/exited containers https://review.openstack.org/329511 | 16:58 |
| inc0 | ehh I tried to edit stuff in gerrit | 16:59 |
| inc0 | should've known better | 16:59 |
| Mech422 | inc0: heh | 16:59 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Make cleanup remove restarting/exited containers https://review.openstack.org/329511 | 17:00 |
| Mech422 | inc0: speaking of too much magic...this: allow_world_readable_tmpfiles = True is why my ceph deploy didn't bomb out | 17:01 |
| Mech422 | inc0: it let me hijack the .ansible dir | 17:01 |
| inc0 | ahh | 17:01 |
| inc0 | well | 17:01 |
| inc0 | fix was merged | 17:01 |
| inc0 | brb lunch | 17:01 |
| Mech422 | oh nice - that was fast.... | 17:02 |
| *** athomas has quit IRC | 17:02 | |
| *** huikang has joined #openstack-kolla | 17:08 | |
| *** d_code has quit IRC | 17:15 | |
| *** d_code has joined #openstack-kolla | 17:17 | |
| *** mdnadeem_home has joined #openstack-kolla | 17:19 | |
| *** d_code_ has joined #openstack-kolla | 17:20 | |
| *** d_code has quit IRC | 17:21 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added unit test for start container https://review.openstack.org/327247 | 17:22 |
| *** mdnadeem_home has quit IRC | 17:23 | |
| *** jtriley has quit IRC | 17:24 | |
| *** d_code_ has quit IRC | 17:25 | |
| *** ravig has quit IRC | 17:25 | |
| *** salv-orlando has quit IRC | 17:26 | |
| *** d_code has joined #openstack-kolla | 17:26 | |
| *** salv-orlando has joined #openstack-kolla | 17:26 | |
| *** salv-orl_ has joined #openstack-kolla | 17:29 | |
| openstackgerrit | Ken Wronkiewicz proposed openstack/kolla: Keystone interface address and memcached override https://review.openstack.org/328569 | 17:31 |
| *** The_Ball has quit IRC | 17:31 | |
| *** salv-orlando has quit IRC | 17:32 | |
| *** jtriley has joined #openstack-kolla | 17:35 | |
| *** ravig has joined #openstack-kolla | 17:36 | |
| *** JoseMello has joined #openstack-kolla | 17:40 | |
| *** ravig has quit IRC | 17:43 | |
| *** The_Ball has joined #openstack-kolla | 17:44 | |
| *** ravig has joined #openstack-kolla | 17:46 | |
| *** ravig has quit IRC | 17:48 | |
| *** Lyncos has quit IRC | 17:54 | |
| Mech422 | inc0: blah - I'm just gonna try starting like 50 VMs at once and see what happens | 18:00 |
| *** huikang has quit IRC | 18:01 | |
| *** tyrola has quit IRC | 18:02 | |
| inc0 | back | 18:03 |
| *** kimk0316 has joined #openstack-kolla | 18:03 | |
| Mech422 | inc0: heh - spawning 30 vms at once blew it up nice | 18:03 |
| Mech422 | inc0: lemme look at logs | 18:03 |
| kimk0316 | Anyone familiar with this error? The error was: IOError: [Errno 13] Permission denied: '/usr/share/ansible/kolla_keystone_service.py' | 18:04 |
| Mech422 | inc0: blah - just ran out of disk quota | 18:04 |
| inc0 | kimk0316, I've seen it in difeerent place | 18:04 |
| inc0 | let me see if it's same issue | 18:05 |
| kimk0316 | I think it's error on kolla_toolbox container. | 18:05 |
| inc0 | kimk0316, this was kolla_toolbox container | 18:05 |
| inc0 | ? | 18:05 |
| inc0 | yeah | 18:05 |
| kimk0316 | right. I try to figure out what's wrong with kolla_toolbox | 18:06 |
| *** huikang has joined #openstack-kolla | 18:06 | |
| inc0 | kimk0316, did you submit bug for that? | 18:06 |
| kimk0316 | I didn't. I wanted to make sure this isn't something that I did wrong. | 18:07 |
| inc0 | it's not, would you kindly do that? and let me know which bug id it got | 18:07 |
| inc0 | I see problem already, fixing it as we speak | 18:08 |
| *** ravig has joined #openstack-kolla | 18:08 | |
| *** dmk0202 has joined #openstack-kolla | 18:09 | |
| *** huikang_ has joined #openstack-kolla | 18:09 | |
| kimk0316 | Is this issue with permission for /usr/share/ansible/kolla_keystone_service.py file? | 18:10 |
| *** salv-orl_ has quit IRC | 18:10 | |
| *** salv-orlando has joined #openstack-kolla | 18:11 | |
| inc0 | it's perm for whole /usr/share | 18:11 |
| inc0 | /usr/share/ansible | 18:11 |
| *** salv-orlando has quit IRC | 18:11 | |
| *** ravig has quit IRC | 18:12 | |
| *** salv-orlando has joined #openstack-kolla | 18:12 | |
| *** huikang has quit IRC | 18:13 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Change permissions of usr/share/ansible and /etc https://review.openstack.org/329610 | 18:14 |
| inc0 | kimk0316, try with this patch | 18:14 |
| kimk0316 | Sure, Thank you inc0 | 18:14 |
| inc0 | mag009_, soo...with calico my neutron_l3_agent is in restarting loop | 18:15 |
| kimk0316 | Do I need to rebuild kolla-tollbox image or just container? | 18:18 |
| *** ravig has joined #openstack-kolla | 18:18 | |
| inc0 | kimk0316, you can | 18:18 |
| inc0 | cant rebuild container;) | 18:18 |
| inc0 | rebuild image and run redeploy | 18:18 |
| kimk0316 | okay. thank you. | 18:18 |
| *** huikang_ has quit IRC | 18:19 | |
| openstackgerrit | Merged openstack/kolla: Make cleanup remove restarting/exited containers https://review.openstack.org/329511 | 18:20 |
| *** ravig has quit IRC | 18:20 | |
| inc0 | sdake, ping | 18:22 |
| *** huikang has joined #openstack-kolla | 18:26 | |
| wirehead_ | Can I get some core reviewer eyes on https://review.openstack.org/#/c/327925/ and https://review.openstack.org/#/c/320744/ ? That’s holding up a lot of kubernetes patches. | 18:26 |
| patchbot | wirehead_: patch 327925 - kolla - Enable kolla k8s to override bind api bind address... | 18:26 |
| patchbot | wirehead_: patch 320744 - kolla - Add kolla-kubernetes bootstrap capability to mariadb | 18:26 |
| *** huikang has quit IRC | 18:27 | |
| *** dmk0202 has quit IRC | 18:30 | |
| *** harlowja has quit IRC | 18:30 | |
| *** huikang has joined #openstack-kolla | 18:31 | |
| *** harlowja has joined #openstack-kolla | 18:32 | |
| *** ravig has joined #openstack-kolla | 18:41 | |
| *** salv-orlando has quit IRC | 18:45 | |
| *** salv-orlando has joined #openstack-kolla | 18:46 | |
| openstackgerrit | Merged openstack/kolla: Run more kolla-ansible task in the gate https://review.openstack.org/326281 | 18:46 |
| *** berendt has quit IRC | 18:57 | |
| *** berendt has joined #openstack-kolla | 18:59 | |
| *** ravig has quit IRC | 19:01 | |
| *** ravig has joined #openstack-kolla | 19:01 | |
| openstackgerrit | Merged openstack/kolla: Change the search path to look locally https://review.openstack.org/329044 | 19:02 |
| *** dmk0202 has joined #openstack-kolla | 19:05 | |
| *** sdake has quit IRC | 19:06 | |
| openstackgerrit | Marc-Andre Gatien proposed openstack/kolla: merge calico integration to kolla https://review.openstack.org/327759 | 19:07 |
| *** sdake has joined #openstack-kolla | 19:07 | |
| inc0 | mag009_, so rebase your patch plz | 19:09 |
| openstackgerrit | Marc-Andre Gatien proposed openstack/kolla: merge calico integration to kolla https://review.openstack.org/327759 | 19:10 |
| mag009_ | i did rebase | 19:10 |
| mag009_ | git rebase -i master | 19:11 |
| *** huikang has quit IRC | 19:11 | |
| *** fragatin_ has quit IRC | 19:12 | |
| *** huikang has joined #openstack-kolla | 19:12 | |
| mag009_ | what am i doing wrong | 19:12 |
| *** zhiwei has joined #openstack-kolla | 19:12 | |
| *** huikang has quit IRC | 19:16 | |
| *** ravig has quit IRC | 19:16 | |
| *** zhiwei has quit IRC | 19:17 | |
| inc0 | mag009_, did you call git fetch origin? | 19:17 |
| inc0 | so you need to do git fetch origin && git rebase -i origin/master | 19:18 |
| *** huikang has joined #openstack-kolla | 19:18 | |
| inc0 | although I think I see issue;) | 19:18 |
| inc0 | look at my comment | 19:19 |
| sdake | inc0 sup | 19:19 |
| *** ravig has joined #openstack-kolla | 19:19 | |
| inc0 | sdake, https://review.openstack.org/#/c/326734/ | 19:19 |
| patchbot | inc0: patch 326734 - kolla - Macro install packages | 19:19 |
| mag009_ | cause I copied an older file | 19:19 |
| inc0 | we can separate heat stuff to other patch? | 19:19 |
| mag009_ | shit | 19:19 |
| sdake | inc0 that wfm | 19:20 |
| sdake | macros shoudl be one patch each service one patch | 19:20 |
| inc0 | I'll decouple heat stuff then | 19:20 |
| inc0 | yeah | 19:20 |
| sdake | eacch container set i mean | 19:20 |
| sdake | i was going to take it over though becausse you asked me to do so | 19:20 |
| inc0 | so I'll decouple that | 19:20 |
| openstackgerrit | Marc-Andre Gatien proposed openstack/kolla: merge calico integration to kolla https://review.openstack.org/327759 | 19:20 |
| inc0 | well, I wanted you to drive contianer sets;) | 19:21 |
| sdake | cool | 19:21 |
| inc0 | let's get this one merged | 19:21 |
| sdake | i can definately make that happen | 19:21 |
| inc0 | and following one | 19:21 |
| wirehead_ | I can hardly contain my excitement. | 19:21 |
| sdake | would be nice to have all the macros implemented tho | 19:21 |
| inc0 | https://review.openstack.org/#/c/327271/2 this is required too | 19:21 |
| patchbot | inc0: patch 327271 - kolla - Customizations continued | 19:21 |
| inc0 | wirehead_, did you get lifetime supply of free beer? | 19:22 |
| inc0 | or what else could be that exciting? | 19:22 |
| wirehead_ | :P | 19:22 |
| inc0 | sdake, how about we merge just this both patchsets as-is and fill up heat in following patch? | 19:23 |
| inc0 | so we'll have a template and example how to do this | 19:23 |
| inc0 | or hold on | 19:23 |
| inc0 | I have better idea | 19:23 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Customizations continued https://review.openstack.org/327271 | 19:25 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Macro install packages https://review.openstack.org/326734 | 19:25 |
| openstackgerrit | Marc-Andre Gatien proposed openstack/kolla: merge calico integration to kolla https://review.openstack.org/327759 | 19:28 |
| openstackgerrit | Ken Wronkiewicz proposed openstack/kolla: Keystone interface address and memcached override https://review.openstack.org/328569 | 19:28 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Customizations continued https://review.openstack.org/327271 | 19:28 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Macro install packages https://review.openstack.org/326734 | 19:28 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Customizations for heat https://review.openstack.org/329651 | 19:31 |
| inc0 | sdake, https://review.openstack.org/#/c/329651/1 feel free to take this one over | 19:32 |
| patchbot | inc0: patch 329651 - kolla - Customizations for heat | 19:32 |
| inc0 | and to start distributing work | 19:32 |
| inc0 | once all of requirements gets merged | 19:32 |
| *** ravig has quit IRC | 19:32 | |
| inc0 | I'll in the meantime will keep tweaking mechanism itself | 19:32 |
| inc0 | I have few more ideas to make it better | 19:33 |
| inc0 | but the dockerfile structure shouldn't change so we can go ahead with work | 19:33 |
| *** ayoung_ has joined #openstack-kolla | 19:34 | |
| *** ravig has joined #openstack-kolla | 19:36 | |
| Mech422 | inc0: looks like xpoint isn't till end of year: http://www.computerworld.com/article/3082658/data-storage/intel-lets-slip-roadmap-for-optane-ssds-with-1000x-performance.html | 19:37 |
| *** tyrola has joined #openstack-kolla | 19:40 | |
| sdake | inc0 we need more then rpm replacement | 19:40 |
| inc0 | I'll need to comandeer one of these and test out ceph on it | 19:40 |
| inc0 | sdake, yeah, we need all | 19:41 |
| inc0 | but mechanism supports whatever | 19:41 |
| sdake | i think we get macros and heat in a position where it has all things then we merge them | 19:41 |
| sdake | that will be alot easier to distribute work | 19:41 |
| sdake | then go look at 6 patches and figure out what you need to do | 19:41 |
| inc0 | well, we need macros for rest, but we can start with packages | 19:42 |
| sdake | it takes alomst as much time to touch packages as it does the whole dockerfile in one swoop | 19:42 |
| inc0 | I don't think we should use macros for keys | 19:42 |
| *** ravig has quit IRC | 19:42 | |
| sdake | use blocks then? | 19:43 |
| inc0 | no | 19:43 |
| inc0 | https://review.openstack.org/#/c/326688/5 | 19:43 |
| patchbot | inc0: patch 326688 - kolla - Add customizations for apt keys and packages | 19:43 |
| *** salv-orlando has quit IRC | 19:43 | |
| inc0 | filter | customizable will work with list | 19:44 |
| *** salv-orlando has joined #openstack-kolla | 19:44 | |
| sdake | the downside is we would have to copy 232 to 235 to every dockerfile | 19:44 |
| inc0 | https://review.openstack.org/#/c/326688/5/docker/base/Dockerfile.j2 line 232 | 19:44 |
| patchbot | inc0: patch 326688 - kolla - Add customizations for apt keys and packages | 19:44 |
| sdake | macro is 1 line not 3 | 19:44 |
| inc0 | change {% for key in base_apt_keys %} to {% for key in base_apt_keys | customizable %} | 19:45 |
| inc0 | and it works | 19:45 |
| inc0 | we do keys in one dockerfile... | 19:45 |
| sdake | keys go with repos, we need per container repos, hence per container keys | 19:45 |
| *** harlowja_ has joined #openstack-kolla | 19:45 | |
| inc0 | but we do not set keys/repos in every dockerfile | 19:45 |
| inc0 | if you want to do that, use blocks | 19:45 |
| sdake | yes but that is what i nee | 19:46 |
| inc0 | use blocks | 19:46 |
| inc0 | or rebuild base | 19:46 |
| inc0 | or build new base for those changed ones | 19:46 |
| sdake | rebuiiding base each time is pokey | 19:46 |
| inc0 | you'll end up doing that anyway | 19:46 |
| sdake | it would take 3-4 hours to build that way | 19:46 |
| inc0 | but you are aware that if you modify this var in one container, base stays base | 19:47 |
| inc0 | it won't change | 19:47 |
| inc0 | only way to achieve what you want to do is: | 19:47 |
| sdake | unlesss base is overriden to contain nothing | 19:47 |
| inc0 | 1. different base for changed containers | 19:47 |
| inc0 | 2. use normal base, then in block uninstlal old keys and install new keys, new repos, new containers (not thin) | 19:48 |
| sdake | i dont see any point in uninstalling when we can just not install | 19:48 |
| inc0 | 3. build normal base, build all the normal containers, build new base, build changed containers | 19:48 |
| sdake | 3 would work today without changes | 19:48 |
| sdake | but it takes forever | 19:48 |
| inc0 | not really | 19:48 |
| inc0 | what you need really is 2 separate bases | 19:49 |
| inc0 | changed and not changed | 19:49 |
| inc0 | otherwise you lose all layering | 19:49 |
| *** harlowja has quit IRC | 19:49 | |
| inc0 | and this would be horrible antipattern | 19:49 |
| sdake | i understand the tradeoff | 19:49 |
| *** daneyon_ has joined #openstack-kolla | 19:49 | |
| inc0 | just build 2 bases, customized and not customized | 19:50 |
| sdake | i've have to build 40 bases | 19:50 |
| inc0 | yeah, same amount of time as changing it per container | 19:50 |
| sdake | because each container can have diffferent repos | 19:50 |
| inc0 | only if there happen to be 2 containers with same base | 19:50 |
| inc0 | you can actually make use of that | 19:50 |
| sdake | i dont care about optimizations for this use case | 19:51 |
| inc0 | that is some horribly strange and probably poorly engineered requirement | 19:51 |
| openstackgerrit | Merged openstack/kolla: Added unit test for start container https://review.openstack.org/327247 | 19:51 |
| inc0 | but well, not my problem | 19:51 |
| inc0 | anyway | 19:51 |
| inc0 | you can do that | 19:51 |
| inc0 | could | 19:51 |
| inc0 | by building 40 bases | 19:51 |
| *** tyrola has quit IRC | 19:52 | |
| inc0 | or build empty base | 19:52 |
| sdake | why on earth would yo udo that when yo ucould just specify keys=nil repos=nil in teh bsae docker file override | 19:52 |
| inc0 | and inject code to block in every contianer | 19:52 |
| sdake | right | 19:52 |
| inc0 | block | 19:52 |
| inc0 | and macros for keys install | 19:52 |
| inc0 | all it takes | 19:52 |
| Mech422 | inc0: I don't get it - why would you need new bases? wouldnt the customizations just be a layer? (I thought that was the point of aufs?) | 19:52 |
| inc0 | Mech422, what sdake is talking about is to have different sources.list per container | 19:53 |
| inc0 | from the beggining | 19:53 |
| inc0 | sdake, just do it with blocks plz | 19:53 |
| Mech422 | oh - I thought it was just modifing repos in the container | 19:54 |
| inc0 | don't put it into main kolla tree | 19:54 |
| inc0 | Mech422, if you modify repos after installing some stuff in base | 19:54 |
| inc0 | then you are in dirty state | 19:54 |
| inc0 | when your packages were installed from different repo than you have | 19:54 |
| *** daneyon_ has quit IRC | 19:54 | |
| inc0 | so all it takes is one version mismatch to make stuff bad | 19:55 |
| inc0 | if you want to build repos per contianer, you essentially say bye bye to docker image inheritance mechanism | 19:55 |
| sdake | i understand the loss of layering if someone uses this approach | 19:56 |
| sdake | but it wont be default and it will be on them to take the performance hit on layering | 19:56 |
| inc0 | so sdake one thing we could do for you is to customize FROM | 19:56 |
| inc0 | then you can put all your specialized containers at FROM ubuntu:1404 instad of base | 19:57 |
| Mech422 | inc0: I figured base just had the 'base' repos (like the ubuntu/rh repos) and the 'customizations' were like the OpenStack repos...so it would be sorta 'additive' rather then replacment | 19:57 |
| inc0 | and do all that stuff on your own | 19:57 |
| inc0 | Mech422, still I don't really want to have this in kolla tree, I'll make it possible, not super-easy | 19:57 |
| Mech422 | inc0: not arguing - just trying ot understand... | 19:58 |
| inc0 | if it's additive, then it's ok | 19:58 |
| inc0 | but if it's not, things become complicated | 19:58 |
| Mech422 | inc0: yeah - that part I see...it just caught me off guard, since I thought the point of aufs was inheritance... | 19:59 |
| inc0 | sdake, I'll provide you with hooks where you can inject your stuff | 19:59 |
| Mech422 | inc0: I see now its not a limitation of aufs, its just to keep things neat and consisstent | 19:59 |
| sdake | when i get home tomorrow i'll test build the world one container set a ta time | 19:59 |
| sdake | if its not 4 hours then maybe ewe can proceed in tht way | 19:59 |
| inc0 | sdake, it will be | 20:00 |
| inc0 | as you need to do apt-get update with new set of repos EVERY SINGLE TIME | 20:00 |
| sdake | i cant have a 4 hour build time | 20:00 |
| inc0 | no inheritance | 20:00 |
| inc0 | so you need to reeenineer your crap | 20:00 |
| sdake | apt-get update only happens once in our model | 20:00 |
| inc0 | what? | 20:01 |
| *** esharao__ has joined #openstack-kolla | 20:01 | |
| sdake | are you talking about installing openstack-base? | 20:01 |
| inc0 | you want different set of repos per container | 20:01 |
| inc0 | hence EVERY container has to have apt-cache up-to-date and in sync with its own repos | 20:01 |
| sdake | inc0 let me get back with you | 20:01 |
| inc0 | if you change repo addresses, it doesn't automatically change cache mind you... | 20:02 |
| inc0 | that's what apt-get update is for... | 20:02 |
| *** tyrola has joined #openstack-kolla | 20:02 | |
| Mech422 | inc0: you know... you could reduce build time if you used the kolla nodes as well as the 'deployment' node to build... | 20:04 |
| Mech422 | inc0: I was thinking that last night..as I rebuilt.... again... | 20:04 |
| inc0 | Mech422, well, technically | 20:04 |
| inc0 | but lot's of complexity to get little benefit imho | 20:05 |
| Mech422 | inc0: ok | 20:05 |
| inc0 | also not really | 20:05 |
| *** esharao_ has quit IRC | 20:05 | |
| inc0 | as you'd need base image in every case | 20:05 |
| inc0 | and building base+openstack base is most of the work | 20:05 |
| *** ravig has joined #openstack-kolla | 20:05 | |
| *** huikang has quit IRC | 20:05 | |
| inc0 | so technically you'd build base and openstack base in one node, distribute it and build rest of container in distributed manned | 20:06 |
| *** huikang has joined #openstack-kolla | 20:06 | |
| inc0 | but that would give you few mins of performance at expanse of LOTS of compexity | 20:06 |
| Mech422 | inc0: really? that blows :-/ | 20:06 |
| Mech422 | I just need some of those xpoint ssds :-) | 20:07 |
| *** tyrola has quit IRC | 20:09 | |
| *** huikang_ has joined #openstack-kolla | 20:09 | |
| *** huikang has quit IRC | 20:10 | |
| *** prithiv has joined #openstack-kolla | 20:20 | |
| inc0 | Mech422, yeah, something's funky in your env;) | 20:22 |
| *** ravig has quit IRC | 20:22 | |
| Mech422 | inc0: 'something' more like everything :-P | 20:25 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Convert Keystone to HostNetwork=False, Fix bootstrap. https://review.openstack.org/328567 | 20:25 |
| Mech422 | inc0: I'm pretty sure I need to redo DNS - reverses don't work at least...god knows what I'm doing for forwards | 20:25 |
| Mech422 | inc0: but its a test bed - it gets torn down and rebuilt all the time | 20:25 |
| *** dmk0202 has quit IRC | 20:26 | |
| *** dmk0202 has joined #openstack-kolla | 20:26 | |
| *** diogogmt has joined #openstack-kolla | 20:27 | |
| inc0 | harlowja_, so I spoke with John Garbutt about cells | 20:27 |
| inc0 | he suggest we move straight to v2 with newton | 20:27 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Added documentation for building Kolla containers. https://review.openstack.org/324116 | 20:28 |
| *** prithiv has quit IRC | 20:28 | |
| harlowja_ | inc0 lol | 20:29 |
| harlowja_ | nice | 20:29 |
| harlowja_ | sucks for v1 folks, ha | 20:29 |
| harlowja_ | i'm sure v3 will be even better :-P | 20:29 |
| inc0 | v1 gets removed once v2 lands according to him | 20:29 |
| inc0 | so yeah...:P | 20:30 |
| inc0 | sux:P | 20:30 |
| inc0 | anyway, maybe you guys could help with adding cells to kolla? | 20:31 |
| inc0 | shouldn't be too hard as mitaka is already a cell | 20:31 |
| *** mliima_ has quit IRC | 20:31 | |
| *** dmk0202 has quit IRC | 20:33 | |
| *** mliima_ has joined #openstack-kolla | 20:33 | |
| *** jtriley has quit IRC | 20:37 | |
| *** prithiv has joined #openstack-kolla | 20:38 | |
| *** diogogmt has quit IRC | 20:43 | |
| wirehead_ | rhallisey: So, Kolla-kubernetes and SkyDNS. Some stuff doesn’t quite work unless it’s running and our current setup instructions don’t include setting that up. | 20:44 |
| kimk0316 | I got this error on kolla_toolbox container: Host file not found: /etc/ansible/hosts | 20:46 |
| rhallisey | wirehead_, did it already merge? | 20:46 |
| wirehead_ | rhallisey: there’s a SkyDNS patch from sbezverk_ that’s been sitting there. | 20:47 |
| rhallisey | wirehead_, what specifically doesn't work | 20:47 |
| *** d_code_ has joined #openstack-kolla | 20:47 | |
| rhallisey | I haven't tried it yet | 20:47 |
| wirehead_ | So, my thinking is that I really don’t want kolla-kubernetes to act like an installer for Kubernetes. | 20:47 |
| wirehead_ | There’s already plenty o’ those. | 20:48 |
| *** diogogmt has joined #openstack-kolla | 20:48 | |
| *** ravig has joined #openstack-kolla | 20:48 | |
| rhallisey | right | 20:48 |
| wirehead_ | So I kinda wanted to have an answer for Hyperkube + SkyDNS so that we can actually do a plausable set of gating tests. | 20:49 |
| wirehead_ | But not one that would lead users to use that in production. :) | 20:49 |
| kimk0316 | inc0, can you check if permission for /etc/ansible/hosts file on kolla_toolbox container is correct? | 20:49 |
| *** d_code has quit IRC | 20:50 | |
| *** d_code_ is now known as d_code | 20:50 | |
| *** shardy has quit IRC | 20:53 | |
| *** mbound has quit IRC | 20:54 | |
| sbezverk_ | wirehead_ I think adding skydns as a service to kolla-k8s makes perfect sense | 20:56 |
| sbezverk_ | we have done it already in kolla with tool box | 20:56 |
| *** ayoung_ has quit IRC | 20:57 | |
| wirehead_ | hm | 20:57 |
| *** d_code has quit IRC | 20:57 | |
| *** ayoung has quit IRC | 20:57 | |
| sbezverk_ | wirehead_ bascially we do not rely on on what cluster has we bring services we need to function with kolla-k8s | 20:57 |
| *** mbound has joined #openstack-kolla | 20:58 | |
| sbezverk_ | wirehead_ I think it will be more difficult to check existing k8s cluster what it has or has not | 20:58 |
| rhallisey | sbezverk_, ya but we can get the kubernetes service ip from the kubenetes api endpoint | 20:58 |
| sbezverk_ | than just to bring what we need in a non very intrusive way ;-) | 20:58 |
| rhallisey | well | 20:58 |
| rhallisey | so kube exposes all service at env vars for each pod | 20:59 |
| wirehead_ | Most of the installer options other than hyperkube are going to come with DNS turned on. | 20:59 |
| sbezverk_ | wirehead_ I tried a couple and I guess I was not lucky non of them brought up dns | 21:00 |
| sbezverk_ | wirehead_ my concern that different flavors might have different hickups and gotchas | 21:01 |
| sbezverk_ | if we really need dns service (based on rhallisey comment we might not) then it is safer to offer the one we tested and know it is working than hope for the best ;-) | 21:01 |
| *** dwalsh has quit IRC | 21:03 | |
| wirehead_ | The Kube docs specifically call out SkyDNS as an “Addon” | 21:03 |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Moving all path resolution and searching for program files into one place https://review.openstack.org/329676 | 21:03 |
| *** d_code has joined #openstack-kolla | 21:03 | |
| wirehead_ | Which puts it at the same level as cluster-level logging and container resource monitoring. | 21:04 |
| *** fragatina has joined #openstack-kolla | 21:05 | |
| rhallisey | sbezverk_, if you mariadb, checkout out what variables are in the environemnt | 21:05 |
| rhallisey | it turns out kube exposes the service ip to the pods | 21:06 |
| rhallisey | as an ev var | 21:06 |
| rhallisey | pretty neat | 21:06 |
| rhallisey | so we may not need it | 21:06 |
| wirehead_ | Yeah, I’d wanted to use more of that, but substituting env variables in the config files is going to be an intrusive config patch. | 21:06 |
| *** inc0 has quit IRC | 21:06 | |
| sbezverk_ | rhallisey right but what is keystone pod starts up before mariadb, I am not sure if kube will add new pod viariables to already running pods | 21:06 |
| wirehead_ | Also that. | 21:07 |
| *** mliima_ has quit IRC | 21:07 | |
| sbezverk_ | also I am not sure if it updates them if pod moves from one compute node to another | 21:07 |
| rhallisey | true, but then the pods will fail | 21:07 |
| wirehead_ | Also correct. :) | 21:07 |
| rhallisey | and restart | 21:07 |
| wirehead_ | Gut feel says that, over time, it’s going to be ‘normal’ to expect a k8s cluster to have DNS setup. | 21:08 |
| *** timss has quit IRC | 21:08 | |
| rhallisey | sbezverk_, re your second point I'm not sure either | 21:08 |
| sbezverk_ | rhallisey I guess it is a matter of approaches possibly both will work, it is just I feel as a operator safer to rely on DNS ;-) | 21:08 |
| wirehead_ | Oh, right, the VIP doesn’t change. | 21:09 |
| rhallisey | although let's say it doesn't an the pod fails | 21:09 |
| rhallisey | wirehead_, nvm, it won't change | 21:09 |
| rhallisey | you're right | 21:09 |
| wirehead_ | Yah, so if you have DNS set up and look up ‘mariadb’, it’s going to get the mariadb VIP | 21:09 |
| rhallisey | the method is the same then | 21:10 |
| wirehead_ | It’s still a bit of an intrusive patch to Kolla and we’ve already got a bunch of patches in Kolla waiting for +2’s. | 21:10 |
| wirehead_ | Also, headless services break the model. | 21:10 |
| wirehead_ | We’re going to need headless services to make Neutron work. | 21:10 |
| rhallisey | wirehead_, it's not intrusive :) | 21:10 |
| *** inc0 has joined #openstack-kolla | 21:10 | |
| *** d_code has quit IRC | 21:11 | |
| wirehead_ | Do all of the config files we’re going to be touching do env var replacement? | 21:11 |
| wirehead_ | e.g. keystone.conf | 21:12 |
| *** huikang_ has quit IRC | 21:13 | |
| rhallisey | what would you be replacing? | 21:13 |
| rhallisey | so I understand the context | 21:13 |
| *** timss has joined #openstack-kolla | 21:13 | |
| *** huikang has joined #openstack-kolla | 21:13 | |
| wirehead_ | So, in keystone.conf, you’d have to put the memcache_servers to whatever the variable of the memcached service env var. | 21:14 |
| wirehead_ | also, for the database, the env variable for the mariadb service env var | 21:14 |
| dcwangmit01 | rhallisey: The lack of DNS had me banging my head on how you and wirehead_ got keystone up. Eventually I figured out i needed to create the services first, then go back and edit globals to set the IPs of mariadb and memcached. It's a manual step in the kolla-kubernetes workflow if no DNS. | 21:14 |
| rhallisey | we could have a readiness check do this | 21:15 |
| rhallisey | I think | 21:15 |
| dcwangmit01 | I don't think a rediness check would help. You have to modify /etc/kolla/globals.yaml with the IPs of teh services, after the services have been created with "kolla-kubernetes bootstrap mariadb; kolla-kbuernets bootstrap memcached" | 21:16 |
| dcwangmit01 | modify, then genconfig, then bootstrap keystone | 21:16 |
| rhallisey | readiness check could be a bash script | 21:16 |
| dcwangmit01 | It requires editing the configs. | 21:17 |
| wirehead_ | Well, you could add a script that runs at startup for the container and rewrites the config file. | 21:17 |
| rhallisey | readiness check could check and see if the service ips are avilable | 21:17 |
| rhallisey | wirehead_, yes | 21:17 |
| dcwangmit01 | Ah. We should probably get some sort of DNS working instead | 21:17 |
| wirehead_ | The service IP changes every time you create the service. | 21:17 |
| dcwangmit01 | Every kube env, GKE, ones created from kube-up, seems to support DNS | 21:17 |
| dcwangmit01 | wirehead_: That's true, but that's why you don't blow away services | 21:18 |
| *** berendt has quit IRC | 21:18 | |
| wirehead_ | Overall, I think that providing a way for us to install SkyDNS in environments where it’s not present is going to be simpler. | 21:18 |
| dcwangmit01 | I want to add a "purge" command to kolla-kubernetes, do do things like that. Clear configmaps, services, etc. Things that aren't removed with kolla-kuberenets kill | 21:19 |
| dcwangmit01 | Right now I have to do that manually by pasting in commands | 21:19 |
| rhallisey | dcwangmit01, so when we delete configmaps it's not clearing them out? | 21:19 |
| rhallisey | wirehead_, I think dns would be the cleanest | 21:20 |
| rhallisey | https://review.openstack.org/#/c/328814/ | 21:21 |
| patchbot | rhallisey: patch 328814 - openstack-infra/project-config - Add gates for deploying kolla-kubernetes (MERGED) | 21:21 |
| rhallisey | gates are coming | 21:21 |
| rhallisey | hold the door! | 21:21 |
| wirehead_ | Barbarians at the gate! | 21:22 |
| dcwangmit01 | rhallisey: I misspoke. Configmaps are indeed cleared | 21:22 |
| rhallisey | dcwangmit01, gotcha. If anything isn't cleaned by kill, we should make sure it is | 21:22 |
| rhallisey | anything kube owned that is | 21:22 |
| wirehead_ | If I watched Game of Thrones, I’d make a reference now. | 21:22 |
| rhallisey | hodor? | 21:23 |
| wirehead_ | Or something. | 21:23 |
| rhallisey | hodor hodor | 21:23 |
| rhallisey | lol.. | 21:23 |
| rhallisey | I g2g guys | 21:23 |
| dcwangmit01 | Bye bye, east coaster | 21:23 |
| rhallisey | let's continue the discussion if there is anymore tmr | 21:23 |
| dcwangmit01 | rhallisey: We need to talk about how to get some of our kolla patches through | 21:24 |
| rhallisey | dcwangmit01, I'll bring it up at the meeting tmr | 21:24 |
| dcwangmit01 | My test branches maintain at least 10 patches | 21:24 |
| rhallisey | ya same :/ | 21:24 |
| wirehead_ | We made the strategic mis-step of involving core Kolla members in our patches. | 21:24 |
| wirehead_ | :D | 21:25 |
| wirehead_ | SO, rhallisey we probably want to exclude skydns from “all" | 21:25 |
| wirehead_ | That’s my only intrusive thought | 21:25 |
| rhallisey | we just need firendly reminders | 21:25 |
| rhallisey | ya we'll see | 21:26 |
| *** ravig has quit IRC | 21:26 | |
| rhallisey | anyway.. see ya! | 21:26 |
| wirehead_ | See ya tomorrow, rhallisey | 21:26 |
| *** rhallisey has quit IRC | 21:26 | |
| wirehead_ | The problem is that we can’t make dcwangmit01’s cute little dog IRC-enabled. Otherwise, he could threaten you with the consequences of making a cute little fluffy puppy-dog sad by not +2ing. | 21:27 |
| *** berendt has joined #openstack-kolla | 21:27 | |
| *** ravig has joined #openstack-kolla | 21:28 | |
| *** zhiwei has joined #openstack-kolla | 21:29 | |
| *** d_code has joined #openstack-kolla | 21:31 | |
| *** ravig has quit IRC | 21:31 | |
| *** zhiwei has quit IRC | 21:33 | |
| *** inc0 has quit IRC | 21:34 | |
| *** jtriley has joined #openstack-kolla | 21:35 | |
| *** daneyon_ has joined #openstack-kolla | 21:41 | |
| *** ravig has joined #openstack-kolla | 21:45 | |
| *** d_code has quit IRC | 21:47 | |
| *** jtriley has quit IRC | 21:48 | |
| *** ravig has quit IRC | 21:48 | |
| *** huikang has quit IRC | 21:49 | |
| *** huikang has joined #openstack-kolla | 21:50 | |
| *** daneyon_ has quit IRC | 21:50 | |
| *** huikang_ has joined #openstack-kolla | 21:51 | |
| *** huikang has quit IRC | 21:54 | |
| *** huikang_ has quit IRC | 21:55 | |
| *** d_code has joined #openstack-kolla | 21:56 | |
| *** huikang has joined #openstack-kolla | 22:01 | |
| *** huikang has quit IRC | 22:06 | |
| *** openstackstatus has quit IRC | 22:25 | |
| *** openstack has joined #openstack-kolla | 22:25 | |
| *** openstackstatus has joined #openstack-kolla | 22:26 | |
| *** ChanServ sets mode: +v openstackstatus | 22:26 | |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Moving all path resolution and searching for program files into one place https://review.openstack.org/329676 | 22:28 |
| esharao__ | hi.. running into the following issue - failed: [localhost] (item=haproxy) => {"failed": true, "item": "haproxy", "msg": "AnsibleUndefinedVariable: 'dict object' has no attribute u'ansible_eth1'"} | 22:29 |
| esharao__ | is there a way to fix this | 22:29 |
| *** huikang has joined #openstack-kolla | 22:31 | |
| *** cloudna__ has joined #openstack-kolla | 22:34 | |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Consolidating path resolution and searching for program files https://review.openstack.org/329676 | 22:35 |
| *** daneyon_ has joined #openstack-kolla | 22:36 | |
| *** mummer has quit IRC | 22:38 | |
| *** mummer has joined #openstack-kolla | 22:40 | |
| *** esharao__ has quit IRC | 22:41 | |
| *** daneyon_ has quit IRC | 22:41 | |
| *** cloudna__ has quit IRC | 22:44 | |
| *** berendt has quit IRC | 22:49 | |
| *** MarMat has joined #openstack-kolla | 22:50 | |
| *** vhosakot has quit IRC | 22:52 | |
| *** JoseMello has quit IRC | 22:59 | |
| *** salv-orlando has quit IRC | 23:14 | |
| *** salv-orlando has joined #openstack-kolla | 23:14 | |
| *** ravig has joined #openstack-kolla | 23:14 | |
| *** ayoung has joined #openstack-kolla | 23:17 | |
| *** huikang has quit IRC | 23:17 | |
| *** huikang has joined #openstack-kolla | 23:18 | |
| *** huikang has quit IRC | 23:22 | |
| openstackgerrit | Ken Wronkiewicz proposed openstack/kolla-kubernetes: Add documentation for setting up a local SkyDNS + Kube2Sky https://review.openstack.org/329717 | 23:24 |
| *** Jeffrey4l has joined #openstack-kolla | 23:29 | |
| *** huikang has joined #openstack-kolla | 23:29 | |
| *** salv-orl_ has joined #openstack-kolla | 23:29 | |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Consolidating path resolution and searching for program files https://review.openstack.org/329676 | 23:31 |
| *** mummer has quit IRC | 23:31 | |
| *** salv-orlando has quit IRC | 23:32 | |
| *** sdake has quit IRC | 23:32 | |
| *** sdake has joined #openstack-kolla | 23:33 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the rabbitmq crash on CentOS binary and source deploy https://review.openstack.org/328960 | 23:38 |
| openstackgerrit | Ken Wronkiewicz proposed openstack/kolla-kubernetes: Add documentation for setting up a local SkyDNS + Kube2Sky https://review.openstack.org/329717 | 23:40 |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Consolidating path resolution and searching for program files https://review.openstack.org/329676 | 23:44 |
| *** prithiv1 has joined #openstack-kolla | 23:45 | |
| *** prithiv has quit IRC | 23:45 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin to the Ubuntu 16.04 https://review.openstack.org/329454 | 23:46 |
| *** SiRiuS__ has quit IRC | 23:55 | |
| *** zhiwei has joined #openstack-kolla | 23:56 | |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: Consolidating path resolution and searching for program files https://review.openstack.org/329676 | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!