| *** vinkman has quit IRC | 00:00 | |
| *** sdake_ has joined #openstack-kolla | 00:02 | |
| *** jrist has quit IRC | 00:16 | |
| *** zhiwei has joined #openstack-kolla | 00:18 | |
| *** ayoung has joined #openstack-kolla | 00:20 | |
| *** rahuls has quit IRC | 00:20 | |
| *** jrist has joined #openstack-kolla | 00:21 | |
| *** zhiwei has quit IRC | 00:23 | |
| *** david-lyle has quit IRC | 00:25 | |
| *** rhallisey has joined #openstack-kolla | 00:26 | |
| *** david-lyle has joined #openstack-kolla | 00:26 | |
| *** david-lyle has quit IRC | 00:30 | |
| wirehead_ | rhallisey: I’ve got an updated https://review.openstack.org/#/c/320744/ | 00:36 |
|---|---|---|
| patchbot | wirehead_: patch 320744 - kolla - Add kolla-kubernetes bootstrap capability to mariadb | 00:36 |
| rhallisey | wirehead_, go ahead an post over my patch | 00:36 |
| wirehead_ | rhallisey: I’m not sure how to do that correctly. :D | 00:37 |
| rhallisey | were you editing on top of my patch? | 00:37 |
| wirehead_ | Yeah. | 00:37 |
| rhallisey | so add the changes. git commit --amend | 00:38 |
| rhallisey | and git review that | 00:38 |
| rhallisey | also add yourself as a co-author | 00:38 |
| wirehead_ | Err, I lied. I’m editing on top of master. | 00:38 |
| wirehead_ | Which of the options under “Download” is the right one? | 00:39 |
| rhallisey | wirehead_, is there massive change? Would it be painful to merge the commits? | 00:39 |
| wirehead_ | Naw. It’s 4 lines. | 00:39 |
| rhallisey | ya cherry pick under download | 00:39 |
| wirehead_ | Okay, so I’m in detatched head. So I just create a new branch on my end? | 00:40 |
| rhallisey | wait did you use the checkout link? or cherrypick? | 00:41 |
| rhallisey | here do this: git stash. cherry-pick. git stash pop | 00:41 |
| rhallisey | and fix any conflicts | 00:41 |
| wirehead_ | ok | 00:43 |
| *** daneyon_ has joined #openstack-kolla | 00:43 | |
| wirehead_ | then I just ammend the change? | 00:43 |
| wirehead_ | Too much time with Github has made my git-fu weak. :P | 00:43 |
| rhallisey | yes amend it | 00:44 |
| rhallisey | no worries | 00:45 |
| ccesario | hey folks.... | 00:46 |
| ccesario | does soeone have any idea about this error ? http://paste.openstack.org/show/507417/ | 00:46 |
| ccesario | I 'm trying deploy using Ubuntu 16.04 as host..... | 00:47 |
| *** daneyon_ has quit IRC | 00:47 | |
| *** phuongnh has joined #openstack-kolla | 00:48 | |
| openstackgerrit | Ken Wronkiewicz proposed openstack/kolla: Add kolla-kubernetes bootstrap capability to mariadb https://review.openstack.org/320744 | 00:49 |
| rhallisey | wirehead_, nice | 00:49 |
| wirehead_ | This means that the containers don’t need to know about etcd anymore. :) | 00:50 |
| wirehead_ | And you can do `kubectl describe po mariadb` and get the status of the bootstrap. | 00:50 |
| wirehead_ | (you can use the --v=8 flag with any kubectl command to see what API calls it’s doing) | 00:51 |
| rhallisey | sweet | 00:52 |
| rhallisey | I'll try it out tmr | 00:56 |
| *** rhallisey has quit IRC | 00:56 | |
| *** ssurana has quit IRC | 01:06 | |
| *** weiyu has joined #openstack-kolla | 01:09 | |
| *** asalkeld has quit IRC | 01:13 | |
| *** eaguilar has joined #openstack-kolla | 01:15 | |
| sdake_ | my note5 died so I bought a samsung galaxy s7 for 5$ a month :) | 01:23 |
| *** zhiwei has joined #openstack-kolla | 01:30 | |
| *** vinkman1 has left #openstack-kolla | 01:35 | |
| openstackgerrit | Joshua Harlow proposed openstack/kolla: Make kolla-ansible a top level python command https://review.openstack.org/324982 | 02:04 |
| *** mummer has quit IRC | 02:17 | |
| *** eaguilar has quit IRC | 02:17 | |
| *** eaguilar has joined #openstack-kolla | 02:17 | |
| sdake_ | harlowja quick question | 02:22 |
| sdake_ | i just saw your python kolla-ansible command | 02:22 |
| sdake_ | looksgood | 02:22 |
| sdake_ | one issue is we have -vvv to get maximum output | 02:22 |
| sdake_ | and -vv | 02:22 |
| sdake_ | and -vvvv | 02:22 |
| sdake_ | ansible has different levels by the number of vs iiuc | 02:23 |
| *** eaguilar has quit IRC | 02:25 | |
| *** eaguilar has joined #openstack-kolla | 02:25 | |
| *** eaguilar has quit IRC | 02:25 | |
| *** unicell has quit IRC | 02:31 | |
| openstackgerrit | Merged openstack/kolla: Install missing package in spiceproxy container https://review.openstack.org/323650 | 02:32 |
| *** inc0 has joined #openstack-kolla | 02:35 | |
| inc0 | good evening | 02:36 |
| coolsvap | good morning folks | 02:36 |
| inc0 | hey coolsvap rested well? | 02:36 |
| coolsvap | not much but i have a good (or bad) habit of waking up early whatever time i sleep | 02:36 |
| openstackgerrit | Merged openstack/kolla: Add .vimrc to .gitignore https://review.openstack.org/324628 | 02:37 |
| openstackgerrit | Merged openstack/kolla: Fix the cinder backup permission issue when enable ceph https://review.openstack.org/323330 | 02:37 |
| coolsvap | so sometimes i even get up in couple of hrs but thats fine with me | 02:37 |
| coolsvap | i take cumulative sleep over weekends | 02:37 |
| *** sacharya has joined #openstack-kolla | 02:37 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-mesos: Fix typos in readme.rst https://review.openstack.org/311792 | 02:39 |
| sdake_ | coolsvap can you approve that please so I can retire kolla-mesos | 02:39 |
| sdake_ | inc0 cn you approve that please | 02:39 |
| sdake_ | coolsvap lol | 02:39 |
| sdake_ | jpeeler or Jeffrey4l_ can you approve 311792 | 02:40 |
| inc0 | done | 02:40 |
| sdake_ | thanks | 02:40 |
| sdake_ | coolsvap how about a workfklow on https://review.openstack.org/#/c/311792/ | 02:41 |
| patchbot | sdake_: patch 311792 - kolla-mesos - Fix typos in readme.rst | 02:41 |
| sdake_ | 2 pages of outstanding reviews | 02:42 |
| * sdake_ groans | 02:42 | |
| inc0 | yeah I just started reviews | 02:43 |
| openstackgerrit | Merged openstack/kolla-mesos: Fix typos in readme.rst https://review.openstack.org/311792 | 02:43 |
| Jeffrey4l_ | +workflow | 02:43 |
| inc0 | tomorrow I'll do full review and bugfix day | 02:43 |
| inc0 | since we have this DSL thing sorted out | 02:43 |
| sdake_ | thanks :) | 02:44 |
| inc0 | sdake_, check out harlowja patch series, you wanted build.py optimizations | 02:46 |
| sdake_ | inc0 i have looked at some of them already | 02:47 |
| sdake_ | i dont care about optimizaations | 02:47 |
| sdake_ | i want to simplify what we have rather then make it more complex ;) | 02:47 |
| sdake_ | the barrier change is good | 02:47 |
| sdake_ | i also liked the change on the items thing | 02:47 |
| sdake_ | but i haven't tested it | 02:47 |
| sdake_ | and afraid to touch build.py :) | 02:47 |
| sdake_ | so I will need to test it persoanlly | 02:48 |
| sdake_ | before I +2 it | 02:48 |
| inc0 | I'm testing full series now | 02:48 |
| inc0 | seems cool and faster than usual build | 02:48 |
| inc0 | also did you try customization thing? | 02:49 |
| coolsvap | inc0, sdake_ https://review.openstack.org/317240/ https://review.openstack.org/317256/ | 02:49 |
| inc0 | on my review list coolsvap ;) | 02:49 |
| *** yuanying has quit IRC | 02:50 | |
| * coolsvap just wanted highlight, pending from along time, lot of rebases | 02:50 | |
| sdake_ | inc0 was that email representative of what we agreed to and discussed | 02:56 |
| sdake_ | inc0 i've been trying to clean up my backlog today | 02:56 |
| sdake_ | but friday I am going to spend all day working on that | 02:56 |
| inc0 | sdake_, mail looks about right | 02:59 |
| inc0 | not sure about 8 places per dockerfile but I don't knwo what you mean | 02:59 |
| inc0 | so just make first set and we'll go from thee | 02:59 |
| *** unicell has joined #openstack-kolla | 03:01 | |
| sdake_ | dmsimard quick q about ara | 03:02 |
| dmsimard | sdake_: wazza. | 03:02 |
| sdake_ | dmsimard i am totallly slammed with work - i saw your announcement and wanted to try it out | 03:02 |
| sdake_ | but I just dont have time and probably wont for atleast 2-3 weeks | 03:02 |
| dmsimard | sure. | 03:03 |
| sdake_ | I'm wondering what the output looks like with kolla AIO if you ahve tried it | 03:03 |
| sdake_ | or what it does preciesely :) | 03:03 |
| dmsimard | sdake_: I haven't, is kolla ansible >2 yet ? | 03:03 |
| dmsimard | if it is I can run it for you | 03:03 |
| sdake_ | yup all the ansible bugs are fixed ;) | 03:03 |
| inc0 | dmsimard, indeed it is | 03:03 |
| sdake_ | dmsimard that would be great - I'd like to see the text | 03:03 |
| sdake_ | or html | 03:03 |
| sdake_ | or whatever | 03:03 |
| sdake_ | i should just do it myself | 03:04 |
| dmsimard | sdake_: I can put it on a hosted demo | 03:04 |
| sdake_ | but seriously 2 pages of outstanding reviews | 03:04 |
| sdake_ | ahlf of them are in merge conflict | 03:04 |
| dmsimard | sdake_: I haven't touched kolla in a couple months, link to aio docs ? | 03:04 |
| sdake_ | http://docs.openstack.org/developer/kolla/ | 03:05 |
| sdake_ | quickstart is sort of AIO :) | 03:05 |
| sdake_ | the docs have been undergoing alot of changing | 03:05 |
| dmsimard | yeah that quickstart docs says <2.0 | 03:07 |
| dmsimard | I guess I should use master branch ? | 03:07 |
| inc0 | dmsimard, master should be ansible 2 | 03:08 |
| sdake_ | master is 2.0 or 2.1 | 03:11 |
| sdake_ | stable branches are on 1.9.4 | 03:11 |
| *** weiyu has quit IRC | 03:13 | |
| dmsimard | 2.1 is fine ? | 03:14 |
| sdake_ | yup | 03:14 |
| dmsimard | k | 03:14 |
| inc0 | there was some issues with 2.1, not sure tho what's it's about | 03:16 |
| sdake_ | pretty sure 2.1 is fixxed | 03:17 |
| sdake_ | according to ml | 03:17 |
| inc0 | will test tomorrow | 03:27 |
| inc0 | good night guys | 03:27 |
| sdake_ | later dude | 03:27 |
| *** inc0 has quit IRC | 03:27 | |
| dmsimard | kolla-build is running | 03:29 |
| dmsimard | you made me bump into a req mismatch issue between ara and openstack global-reqs :) | 03:29 |
| dmsimard | https://github.com/dmsimard/ara/issues/97 | 03:29 |
| dmsimard | yay for finding bugs | 03:30 |
| *** timonwong has joined #openstack-kolla | 03:30 | |
| sdake_ | coolsvap did we make a decision on :: on the line where the : should go, or on a separate line in the docs | 03:38 |
| dmsimard | I'm gonna fall asleep if those images don't finish building soon :p | 03:43 |
| *** yuanying has joined #openstack-kolla | 03:49 | |
| *** mdnadeem has joined #openstack-kolla | 03:52 | |
| coolsvap | sdake_, yes | 03:53 |
| coolsvap | if there is no trailing : | 03:53 |
| coolsvap | it should be on the same line | 03:53 |
| sdake_ | so example | 03:54 |
| sdake_ | Here is hte information: | 03:54 |
| sdake_ | should be | 03:54 |
| sdake_ | Here is the information:: | 03:54 |
| coolsvap | no | 03:54 |
| coolsvap | in this case the :: should be no next line | 03:54 |
| *** v1k0d3n has quit IRC | 03:54 | |
| *** mdnadeem_ has quit IRC | 03:55 | |
| coolsvap | i think i wrote it wrong | 03:55 |
| coolsvap | no i did write it correctly | 03:55 |
| coolsvap | w.r.t. what you said, if there is trailing : | 03:56 |
| coolsvap | :: should not be on the same line, it should be on next line | 03:56 |
| coolsvap | sdake_, ^^ | 03:56 |
| *** dwalsh has quit IRC | 03:59 | |
| *** ssurana has joined #openstack-kolla | 04:04 | |
| *** tfukushima has joined #openstack-kolla | 04:05 | |
| *** diogogmt has quit IRC | 04:07 | |
| *** ssurana has quit IRC | 04:09 | |
| *** ayoung has quit IRC | 04:20 | |
| sdake_ | coolsvap so the docs should be | 04:20 |
| sdake_ | "This is an example: | 04:20 |
| sdake_ | <cr> | 04:20 |
| sdake_ | :: | 04:20 |
| sdake_ | four spaces then text? | 04:21 |
| coolsvap | yes | 04:21 |
| coolsvap | no | 04:21 |
| sdake_ | ok well one of your patches is busted | 04:21 |
| coolsvap | which one? | 04:21 |
| sdake_ | coolsvap read review 3 https://review.openstack.org/#/c/317084 | 04:22 |
| sdake_ | Mauricio Lima | 04:23 |
| sdake_ | put it on a single line.May 20 11:12 AM | 04:23 |
| sdake_ | Steven Dake | 04:23 |
| sdake_ | 9:22 PM | 04:23 |
| sdake_ | We made a documentation decision at some point in the last couple of weeks based upon reading the RST documentation that the above syntax is indeed correct. | 04:23 |
| sdake_ | For example, this patch merged. | 04:23 |
| sdake_ | https://review.openstack.org/#/c/320933/ | 04:23 |
| patchbot | sdake_: patch 320933 - kolla - Update the image-building documentation (MERGED) | 04:23 |
| *** v1k0d3n has joined #openstack-kolla | 04:24 | |
| sdake_ | I guess that was mandre's patch | 04:24 |
| coolsvap | let me write explanation here | 04:24 |
| sdake_ | here where | 04:24 |
| sdake_ | respond on my unmerged patch please | 04:24 |
| sdake_ | so i know what to fix :) | 04:24 |
| coolsvap | on the unmerged patch | 04:24 |
| coolsvap | yes | 04:24 |
| sdake_ | http://docs.openstack.org/developer/kolla/image-building.html | 04:25 |
| sdake_ | this renders correctly | 04:25 |
| coolsvap | yes | 04:26 |
| coolsvap | your syntax is correct | 04:26 |
| *** kbyrne has quit IRC | 04:32 | |
| *** cloudnautique has quit IRC | 04:33 | |
| *** kbyrne has joined #openstack-kolla | 04:35 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Document upgrading for operators https://review.openstack.org/317084 | 04:35 |
| *** tfukushima has quit IRC | 04:40 | |
| *** tfukushima has joined #openstack-kolla | 04:41 | |
| *** tfukushima has quit IRC | 04:46 | |
| *** dave-mccowan has quit IRC | 04:47 | |
| dmsimard | sdake_: seeing this error in the precheck http://paste.openstack.org/show/507488/ | 04:55 |
| dmsimard | 2.0.1.0 or 2.1.0.0, same error | 04:55 |
| sdake_ | dmsimard not sure what would cause that - are ou using the same globals.yml from before or a new one? if so can you paste it | 04:56 |
| dmsimard | sdake_: I'm using whatever came from master, did not edit it | 04:56 |
| dmsimard | built centos binary and just did kolla-genpwd | 04:56 |
| sdake_ | oh you have to modify globals.yml slightly | 04:56 |
| sdake_ | to specify an internal vip address | 04:57 |
| sdake_ | and your docker registry | 04:57 |
| sdake_ | it should be in the documentation | 04:57 |
| dmsimard | it looks like it used the defaults | 04:57 |
| dmsimard | shrug, kolla-ansible deploy is going now | 04:57 |
| dmsimard | I don't plan on troubleshooting this tonight, just want to send you the results | 04:58 |
| sdake_ | sounds good | 04:58 |
| sdake_ | if its too hard I understand - I'll get to it eventuallly - i saw your demo - which was helpful | 04:58 |
| *** timonwong has quit IRC | 05:01 | |
| dmsimard | sdake_: nope, got it | 05:05 |
| dmsimard | sdake_: have a gift for you :) | 05:05 |
| dmsimard | sdake_: https://dmsimard.com/ara.tar.gz | 05:06 |
| dmsimard | generated static version of what I ran with kolla precheck and kolla deploy | 05:06 |
| dmsimard | if you have any feedback, email or https://github.com/dmsimard/ara/issues is fine, sleep for me now before I pass out :) | 05:07 |
| *** tfukushima has joined #openstack-kolla | 05:17 | |
| *** yuanying has quit IRC | 05:17 | |
| openstackgerrit | Merged openstack/kolla: Use a task model and dumb workers https://review.openstack.org/321317 | 05:26 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Build openvswitch image using source type https://review.openstack.org/309726 | 05:51 |
| *** Mr_Broken has joined #openstack-kolla | 05:54 | |
| *** Mr_Broken has quit IRC | 06:00 | |
| *** yuanying has joined #openstack-kolla | 06:05 | |
| *** Mr_Broken has joined #openstack-kolla | 06:06 | |
| openstackgerrit | Merged openstack/kolla: add curlrc file to base image https://review.openstack.org/301911 | 06:12 |
| *** openstackgerrit has quit IRC | 06:17 | |
| *** openstackgerrit has joined #openstack-kolla | 06:18 | |
| *** phuongnh has quit IRC | 06:21 | |
| *** phuongnh has joined #openstack-kolla | 06:22 | |
| *** yuanying has quit IRC | 06:28 | |
| *** yuanying has joined #openstack-kolla | 06:29 | |
| *** lmiccini has quit IRC | 06:29 | |
| *** mkoderer has quit IRC | 06:30 | |
| *** yuanying has quit IRC | 06:31 | |
| mdnadeem | Hello, is there anyone who face the issue "IOError: [Errno 2] No such file or directory: '/var/lib/kolla/config_files/config.json'" while deploying mariadb service ? | 06:31 |
| *** mkoderer has joined #openstack-kolla | 06:31 | |
| *** yuanying has joined #openstack-kolla | 06:31 | |
| mdnadeem | >> https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py#L150 | 06:32 |
| *** lmiccini has joined #openstack-kolla | 06:32 | |
| *** yuanying has quit IRC | 06:33 | |
| *** yuanying has joined #openstack-kolla | 06:34 | |
| *** harlowja has quit IRC | 06:35 | |
| mdnadeem | Jeffrey4l_, could you please let me know how to generate /var/lib/kolla/config_files/config.json file or what value is need to set in env variable "KOLLA_CONFIG" ? | 06:47 |
| *** unicell has quit IRC | 07:00 | |
| *** unicell1 has joined #openstack-kolla | 07:00 | |
| *** tyrola has joined #openstack-kolla | 07:02 | |
| *** openstackgerrit has quit IRC | 07:03 | |
| *** openstackgerrit has joined #openstack-kolla | 07:03 | |
| *** tyrola has quit IRC | 07:04 | |
| *** SiRiuS__ has joined #openstack-kolla | 07:06 | |
| *** tyrola has joined #openstack-kolla | 07:07 | |
| *** sacharya has quit IRC | 07:07 | |
| *** phuongnh has quit IRC | 07:11 | |
| *** phuongnh has joined #openstack-kolla | 07:12 | |
| *** mikelk has joined #openstack-kolla | 07:15 | |
| *** tyrola has quit IRC | 07:25 | |
| openstackgerrit | Joshua Harlow proposed openstack/kolla: Make kolla-ansible a top level python command https://review.openstack.org/324982 | 07:25 |
| *** matrohon has joined #openstack-kolla | 07:30 | |
| *** reddydodda has quit IRC | 07:41 | |
| *** yuanying has quit IRC | 07:50 | |
| *** reddydodda has joined #openstack-kolla | 07:53 | |
| *** tyrola has joined #openstack-kolla | 07:54 | |
| *** daneyon_ has joined #openstack-kolla | 07:55 | |
| openstackgerrit | Merged openstack/kolla: Document upgrading for operators https://review.openstack.org/317084 | 07:56 |
| *** elemoine_ has quit IRC | 07:58 | |
| *** tpeoples has quit IRC | 07:58 | |
| *** daneyon_ has quit IRC | 08:00 | |
| *** rmart04 has joined #openstack-kolla | 08:02 | |
| *** sdake_ has quit IRC | 08:03 | |
| *** mgoddard has joined #openstack-kolla | 08:04 | |
| *** sacharya has joined #openstack-kolla | 08:08 | |
| *** sacharya has quit IRC | 08:13 | |
| *** cu5 has joined #openstack-kolla | 08:27 | |
| *** jmccarthy has quit IRC | 08:32 | |
| *** bradjones has quit IRC | 08:39 | |
| *** cu5 has quit IRC | 08:39 | |
| *** tonyb has quit IRC | 08:39 | |
| *** dasm has quit IRC | 08:39 | |
| *** tonyb has joined #openstack-kolla | 08:39 | |
| *** huikang has joined #openstack-kolla | 08:39 | |
| *** yuanying has joined #openstack-kolla | 08:39 | |
| *** bradjones has joined #openstack-kolla | 08:39 | |
| *** bradjones has quit IRC | 08:39 | |
| *** bradjones has joined #openstack-kolla | 08:39 | |
| *** dasm has joined #openstack-kolla | 08:40 | |
| *** phuongnh has quit IRC | 08:48 | |
| *** phuongnh has joined #openstack-kolla | 08:48 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add vars to the merge_config task in config-neutron-fake.yml https://review.openstack.org/325139 | 09:04 |
| *** huikang has quit IRC | 09:08 | |
| *** tpeoples has joined #openstack-kolla | 09:08 | |
| *** jmccarthy has joined #openstack-kolla | 09:09 | |
| *** elemoine_ has joined #openstack-kolla | 09:15 | |
| *** dwalsh has joined #openstack-kolla | 09:21 | |
| *** SiRiuS__ has quit IRC | 09:33 | |
| -openstackstatus- NOTICE: CI is experiencing issues with test logs, all jobs are currently UNSTABLE as a result. No need to recheck until this is fixed! Thanks for your patience. | 09:37 | |
| *** logan_ has joined #openstack-kolla | 09:57 | |
| *** logan_ is now known as Guest62971 | 09:58 | |
| *** logan- has quit IRC | 09:59 | |
| *** Guest62971 is now known as logan- | 10:01 | |
| *** elemoine_ has quit IRC | 10:02 | |
| *** tpeoples has quit IRC | 10:02 | |
| -openstackstatus- NOTICE: CI is experiencing issues with test logs, all jobs are currently UNSTABLE as a result. No need to recheck until this is fixed! Thanks for your patience. | 10:08 | |
| *** ChanServ changes topic to "CI is experiencing issues with test logs, all jobs are currently UNSTABLE as a result. No need to recheck until this is fixed! Thanks for your patience." | 10:08 | |
| *** pbourke_ has quit IRC | 10:10 | |
| *** pbourke_ has joined #openstack-kolla | 10:11 | |
| *** tfukushima has quit IRC | 10:11 | |
| *** rhallisey has joined #openstack-kolla | 10:14 | |
| *** mkoderer has quit IRC | 10:16 | |
| *** slagle has quit IRC | 10:16 | |
| Jeffrey4l_ | mdnadeem, never saw this. is other container ok? | 10:30 |
| *** tpeoples has joined #openstack-kolla | 10:32 | |
| mdnadeem | Jeffrey4l_, Actually i am trying to deploy mariadb using kolla-kubernetes : https://github.com/openstack/kolla-kubernetes/blob/master/doc/source/quickstart.rst | 10:33 |
| Jeffrey4l_ | this file should be exist on `/etc/kolla/mariadb/config.json`, and then map to the container by using `-v` | 10:34 |
| Jeffrey4l_ | the file in `/etc/kolla/mariadb/config.json` is generate by the ansible( in kolla ). it come from the templates folder in the mariadb role. | 10:35 |
| Jeffrey4l_ | mdnadeem, ^ | 10:35 |
| mdnadeem | Jeffrey4l_, yes, this file is present at my host, however i dont know why set-config.py looks for /var/lib/kolla/config_files/conf.json ? | 10:37 |
| *** rmart04 has quit IRC | 10:38 | |
| *** daneyon_ has joined #openstack-kolla | 10:38 | |
| Jeffrey4l_ | set_config will copy the config file to the right position in the container. the source and the dest info is wrote in the config.js | 10:38 |
| Jeffrey4l_ | config.json | 10:38 |
| *** rmart04 has joined #openstack-kolla | 10:38 | |
| mdnadeem | >> https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py#L150 | 10:39 |
| *** elemoine_ has joined #openstack-kolla | 10:40 | |
| mdnadeem | Jeffrey4l_, source directory is /var/lib/kolla/config_files/galera.cnf, however on my host there is no /var/lib/kolla directory | 10:41 |
| rhallisey | mdnadeem, https://github.com/openstack/kolla-kubernetes/blob/master/kolla_kubernetes/common/file_utils.py#L40 | 10:41 |
| Jeffrey4l_ | mdnadeem, this file is in the containers. | 10:42 |
| Jeffrey4l_ | mdnadeem, | 10:42 |
| rhallisey | mdnadeem, are you using tools/kolla-kubernetes.py ? | 10:42 |
| rhallisey | curious how you're running it | 10:42 |
| mdnadeem | rhallisey, i run it directly kolla-kubernetes run mariadb | 10:43 |
| *** rmart04 has quit IRC | 10:43 | |
| mdnadeem | as mentioned in guide : https://github.com/openstack/kolla-kubernetes/blob/master/doc/source/quickstart.rst | 10:43 |
| rhallisey | run it from tools. I'm wondering if that bit is broken | 10:43 |
| *** daneyon_ has quit IRC | 10:43 | |
| mdnadeem | ok, let me try it | 10:44 |
| *** phuongnh has quit IRC | 10:44 | |
| *** rmart04 has joined #openstack-kolla | 10:45 | |
| *** mkoderer has joined #openstack-kolla | 10:49 | |
| *** slagle has joined #openstack-kolla | 10:49 | |
| *** orwell.freenode.net changes topic to "Kolla IRC meetngs on Wednesdays @ 16:30 UTC even weeks, 23:00 UTC odd weeks - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA)" | 10:49 | |
| mdnadeem | rhallisey, try as you say, the pods still show in pending state from last 3-4 minutes | 10:55 |
| mdnadeem | rhallisey, http://paste.openstack.org/show/507644/ | 10:58 |
| rhallisey | mdnadeem, can you run with -d and paste the output | 11:00 |
| mdnadeem | rhallisey, >> http://paste.openstack.org/show/507645/ | 11:03 |
| mdnadeem | rhallisey, Error from server: configmaps "mariadb-configmap" already exists | 11:03 |
| rhallisey | mdnadeem, ./tools/kolla-kubernetes kill mariadb | 11:07 |
| rhallisey | then run it again | 11:07 |
| rhallisey | mdnadeem, bootstrap then run | 11:07 |
| rhallisey | mdnadeem, if you look at line 49, it's pick up the config files | 11:07 |
| rhallisey | but I think you maraidb configmap is left over | 11:08 |
| mdnadeem | rhallisey, I have done the same thing, i had run kill mariadb | 11:08 |
| rhallisey | mdnadeem, kubectl describe configmap mariadb-configmap | 11:10 |
| mdnadeem | >> http://paste.openstack.org/show/507646/ | 11:11 |
| rhallisey | so it looks like it has config | 11:12 |
| mdnadeem | rhallisey, what you mean by bootstrap then run, is it mean first i run ./tools/kolla_kubernetes.py -d run mariadb-bootstrap and then again run ./tools/kolla_kubernetes.py -d run mariadb ? | 11:13 |
| rhallisey | ./tools/kolla_kubernetes.py -d bootstrap mariadb | 11:13 |
| rhallisey | yes | 11:13 |
| rhallisey | be sure to remove /var/lib/mysql | 11:14 |
| rhallisey | or bootstrap will error | 11:14 |
| mdnadeem | ok | 11:14 |
| *** coolsvap_ has joined #openstack-kolla | 11:16 | |
| rhallisey | mdnadeem, one more thing :) | 11:18 |
| rhallisey | https://review.openstack.org/#/c/320744/ | 11:18 |
| patchbot | rhallisey: patch 320744 - kolla - Add kolla-kubernetes bootstrap capability to mariadb | 11:18 |
| rhallisey | you need to build the mariadb container with that patch | 11:18 |
| mdnadeem | rhallisey, ok, i will try by this patch, Thanks | 11:19 |
| rhallisey | no probelm | 11:19 |
| rhallisey | mdnadeem, I'll brb in 30 min | 11:19 |
| rhallisey | let me know if it works for you | 11:19 |
| mdnadeem | sure | 11:19 |
| *** rhallisey has quit IRC | 11:19 | |
| ccesario | morning guys!! | 11:22 |
| *** coolsvap_ has quit IRC | 11:27 | |
| *** mliima has joined #openstack-kolla | 11:31 | |
| mliima | morning guys | 11:32 |
| *** coolsvap_ has joined #openstack-kolla | 11:39 | |
| -openstackstatus- NOTICE: CI is experiencing issues with test logs, all jobs are currently UNSTABLE as a result. No need to recheck until this is fixed! Thanks for your patience. | 11:41 | |
| *** ChanServ changes topic to "CI is experiencing issues with test logs, all jobs are currently UNSTABLE as a result. No need to recheck until this is fixed! Thanks for your patience." | 11:41 | |
| *** rhallisey has joined #openstack-kolla | 11:54 | |
| *** coolsvap_ has quit IRC | 12:05 | |
| *** SiRiuS has joined #openstack-kolla | 12:07 | |
| *** sacharya has joined #openstack-kolla | 12:10 | |
| *** sacharya has quit IRC | 12:15 | |
| *** tyrola has quit IRC | 12:23 | |
| *** Mr_Broken has quit IRC | 12:26 | |
| *** tyrola has joined #openstack-kolla | 12:28 | |
| *** Mr_Broke_ has joined #openstack-kolla | 12:31 | |
| *** dave-mccowan has joined #openstack-kolla | 12:44 | |
| *** tyrola has quit IRC | 12:44 | |
| *** cu5 has joined #openstack-kolla | 12:46 | |
| *** sdake has joined #openstack-kolla | 12:49 | |
| sdake | hey cats | 12:50 |
| sdake | sweet my upgrade doc merged | 12:50 |
| Mech422 | sdake: Morning :-) | 12:52 |
| Daviey | sdake: Have you thought about how to backup the control plane data, and restore/rollback in the event of upgrade failure? | 12:53 |
| sdake | harmw around? | 13:01 |
| sdake | restore/rollback | 13:01 |
| sdake | its 6am | 13:01 |
| sdake | give me 15 mins to get rolling pls | 13:01 |
| sdake | harmw ignore, I was pingin gsomeone else | 13:01 |
| Daviey | sdake: i'll give you 20 mins. :) | 13:02 |
| *** stvnoyes has quit IRC | 13:05 | |
| *** stvnoyes has joined #openstack-kolla | 13:05 | |
| *** berendt has joined #openstack-kolla | 13:12 | |
| *** mark-casey has joined #openstack-kolla | 13:12 | |
| dmsimard | sdake: did you retrieve that link I sent you last night ? | 13:17 |
| sdake | dmsimard not yet but its on my list for before 8am: ) | 13:17 |
| dmsimard | sdake: ok, let me know when you got it, I'll delete it from my server :p | 13:18 |
| *** daneyon_ has joined #openstack-kolla | 13:20 | |
| Mech422 | sdake: btw - the VLAN stuff works great with VETH devices. Found an issue with OpenVSwitch that irc0 had a nice fix for | 13:23 |
| sdake | Mech422 fantastic | 13:23 |
| sdake | dmsimard i did download it | 13:24 |
| sdake | dmsimard lasst night | 13:24 |
| sdake | I just haven't opened it up yet | 13:24 |
| *** daneyon_ has quit IRC | 13:25 | |
| *** ppowell has joined #openstack-kolla | 13:27 | |
| mark-casey | Mech422: is your VLAN mention for VLAN configured in Neutron by chance? | 13:30 |
| mdnadeem | rhallisey, Please have a look at this : http://paste.openstack.org/show/507667/ | 13:33 |
| *** ruhe has left #openstack-kolla | 13:34 | |
| *** inc0 has joined #openstack-kolla | 13:34 | |
| inc0 | good morning | 13:34 |
| ccesario | morning inc0 :) | 13:35 |
| inc0 | pbourke_, mandre DSL thing is concluded I think | 13:35 |
| *** huikang has joined #openstack-kolla | 13:36 | |
| rhallisey | mdnadeem, clear our any old python remnants of kolla-kubernetes | 13:36 |
| pbourke_ | inc0: yeah I saw the email | 13:36 |
| rhallisey | and reinstall it | 13:36 |
| rhallisey | it's not picking up the bootstrap commnd | 13:36 |
| mandre | inc0: without even a drop of blood | 13:36 |
| Mech422 | mark-casey: no - umm - I'm setting up Kolla host with OVS and vlans for the various nets (storage,internal,external) - then using VETH pairs to pass that to kolla | 13:36 |
| inc0 | true mandre, I'm almost disapointed;) | 13:37 |
| Mech422 | mark-casey: working really nicely - only thing is, Kolla starts new OpenVSwitch daemons in containers - so you need to comment out 3 tasks to prevent that | 13:37 |
| inc0 | Mech422, ovs-db* 3 tasks you commented are about db;) | 13:38 |
| ccesario | inc0, have tested kolla deploy using ubuntu 16.04 ? I did not have success.. | 13:38 |
| inc0 | ccesario, hmm, not yet | 13:38 |
| ccesario | http://paste.openstack.org/show/507653/ http://paste.openstack.org/show/507654/ | 13:38 |
| inc0 | Lyncos tested it out tho afaik | 13:38 |
| Mech422 | inc0: oh yeah - openvswitch-vswitchd is still running - my bad | 13:38 |
| inc0 | you might want to talk to him when he appears | 13:39 |
| inc0 | but he tested stable mitaka | 13:39 |
| mandre | inc0, pbourke_, sdake: what do you think of http://paste.openstack.org/show/507325/ as a way to simplify the configuration of the build? | 13:39 |
| mdnadeem | rhallisey, ok | 13:39 |
| mandre | all you have to pass to the build script in the end is a profile | 13:39 |
| inc0 | mandre, looks good, we need to have long critical look at our build | 13:40 |
| ccesario | inc0, http://paste.openstack.org/show/507668/ | 13:40 |
| inc0 | and refactor it greatly | 13:41 |
| ccesario | inc0, rigth.... I will check with him | 13:41 |
| pbourke_ | what is is simplifying | 13:41 |
| inc0 | ccesario, check haproxy logs | 13:41 |
| pbourke_ | *is it | 13:41 |
| mdnadeem | rhallisey, i should manually delete >> rm -rf /usr/local/share/kolla-kubernetes/ and again re-install it . right ? | 13:42 |
| inc0 | pbourke_, we don't really have a way to add another dockerfile to our set | 13:42 |
| inc0 | custom one I mean | 13:42 |
| mark-casey | Mech422: what does that allow? sounds very convenient but I'm trying to think why the Kolla host has to participate in those networks | 13:42 |
| rhallisey | mdnadeem, I think you just need to get rid of the python bits then reinstall | 13:42 |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: WIP: Updates Magnum for Mitaka https://review.openstack.org/314805 | 13:42 |
| pbourke_ | we do you just drop it in | 13:42 |
| pbourke_ | under docker/ | 13:42 |
| pbourke_ | I feel the kolla-build.conf is quite straight forward to use | 13:43 |
| *** mbound has joined #openstack-kolla | 13:43 | |
| Mech422 | mark-casey: it allows me to use 1 pair of bonded 10GigE interfaces for all of Kolla/openstack | 13:43 |
| mdnadeem | rhallisey, can you please provide command to do that, actually i am not getting it | 13:43 |
| Mech422 | mark-casey: so I get HA on the nics | 13:43 |
| rhallisey | mdnadeem, sure. rm -f /usr/lib/python2.7/site-packages/kolla-kubernetes* | 13:44 |
| rhallisey | s/-f/-rf/ | 13:44 |
| Mech422 | mark-casey: (I suppose it also protects against port failures on the swithc too) | 13:44 |
| mdnadeem | rhallisey, ok, then un install kolla-kubernetes by executing command sudo pip uninstall . | 13:45 |
| *** tyrola has joined #openstack-kolla | 13:45 | |
| *** mgoddard_ has joined #openstack-kolla | 13:45 | |
| ccesario | inc0, http://sprunge.us/JDjQ | 13:46 |
| inc0 | uhh...dunno | 13:46 |
| inc0 | will test it later on | 13:46 |
| Mech422 | inc0: btw - since I'm manually configuring br-ex, I can use OVS patch connections instead of VETH for a bit more performance | 13:46 |
| sdake | guys i've got a on fire email i need to deal with i will read scrollbakc in about 30 minutes whwen i' mdone and respond | 13:47 |
| inc0 | well now technically you can do whatever | 13:47 |
| rhallisey | mdnadeem, yes | 13:47 |
| sdake | Daviey i'll bbiaf | 13:47 |
| *** mgoddard has quit IRC | 13:49 | |
| *** huikang has quit IRC | 13:49 | |
| mark-casey | Mech422: cool. (was reading chat log and...) I would also love to see that in the advanced config doc. I've got two gige and two 10gig ports on my motherboard(s) | 13:49 |
| *** huikang has joined #openstack-kolla | 13:50 | |
| Mech422 | mark-casey: http://pastebin.com/wfMEwTVC | 13:50 |
| Mech422 | mark-casey: I haven't figured out the required hoops to jump to submit that for 'official' use | 13:50 |
| Mech422 | mark-casey: I think my boss needs to sign some agreement | 13:51 |
| Mech422 | mark-casey: also - sam yapple has some really good info on setting Ceph up to use partitions rather then whole devices on his blog - that should prolly be in the advanced docs too | 13:52 |
| *** huikang has quit IRC | 13:52 | |
| *** huikang has joined #openstack-kolla | 13:52 | |
| *** tyrola has quit IRC | 13:52 | |
| Mech422 | mark-casey: basically - just tag whatever gpt partitions you want kolla/ceph to use with the name "KOLLA_CEPH_BOOTSTRAP_x" and "KOLLA_CEPH_BOOTSTRAP_x_J" to create a data/journal pair for ceph | 13:53 |
| mdnadeem | rhallisey, http://paste.openstack.org/raw/507669/ | 13:53 |
| mark-casey | Mech422: someone may correct me (and I'm not a lawyer), but stackalytics is based on Affiliations you specify when signing the individual contributor agreement. you may or may not have to get the company version signed | 13:53 |
| Mech422 | mark-casey: ('x' is just a incrementing number, and is unique per HOST - not cluster) | 13:53 |
| Mech422 | mark-casey: dunno - it was boring legal crap - more fun to hack on the cluster :-P | 13:54 |
| mdnadeem | rhallisey, this time mariadb container name change to "mariadb-db6l5" | 13:54 |
| rhallisey | mdnadeem, ya it deploying a replication controller | 13:54 |
| rhallisey | mdnadeem, did you include the patch that split the cli? | 13:55 |
| mark-casey | Mech422: and yea. I think the gpt flags were used in the demo in Tokyo. really cool | 13:55 |
| rhallisey | mdnadeem, sorry, not everything has merged yet. There's bits and pieces all over the place | 13:55 |
| Mech422 | mark-casey: yeah - its a really great feature. you can tag whole drives, or just the partitions you want :-) | 13:55 |
| Mech422 | mark-casey: very handy when you don't want to have a custom provisioning setup just for OSD nodes | 13:56 |
| mdnadeem | rhallisey, please provide me the link, i will try with the patch | 13:56 |
| rhallisey | https://review.openstack.org/#/c/324113/ | 13:56 |
| patchbot | rhallisey: patch 324113 - kolla-kubernetes - Add bootstrap method to the quickstart doc | 13:56 |
| mark-casey | rhallisey: latest maria bootstrap I saw looks slick. what is it talking to on localhost:8080? | 13:56 |
| Mech422 | mark-casey: (Our provisioning sets up partition 2 as '/', which is why I had to go looking to begin with...) | 13:56 |
| rhallisey | mdnadeem, you need to pip uninstall then apply that | 13:56 |
| rhallisey | mark-casey, I need to ask wirehead_ about that | 13:57 |
| mark-casey | cool | 13:57 |
| mdnadeem | rhallisey, but this patch have changes only in doc | 13:57 |
| rhallisey | mdnadeem, oops sorry wrong patch | 13:58 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Implements elemental.yml DSL for nova https://review.openstack.org/325313 | 13:58 |
| rhallisey | https://review.openstack.org/#/c/320668/ | 13:58 |
| patchbot | rhallisey: patch 320668 - kolla-kubernetes - Break deployment procedure into two steps | 13:58 |
| rhallisey | mdnadeem, that one ^ | 13:58 |
| mdnadeem | rhallisey, ok, | 13:59 |
| *** mgoddard_ has quit IRC | 13:59 | |
| *** ChanServ changes topic to "Kolla IRC meetngs on Wednesdays @ 16:30 UTC even weeks, 23:00 UTC odd weeks - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA)" | 14:00 | |
| -openstackstatus- NOTICE: Cleanup from earlier block storage disruption on static.openstack.org has been repaired, and any jobs which reported an "UNSTABLE" result or linked to missing logs between 08:00-14:00 UTC can be retriggered by leaving a "recheck" comment. | 14:00 | |
| *** mgoddard has joined #openstack-kolla | 14:00 | |
| *** ayoung has joined #openstack-kolla | 14:00 | |
| mark-casey | rhallisey: with 'api/v1/namespaces/default/replicationcontrollers' in the path I'm pretty sure it's posting annotations for the RC to k8s itself (some part of it). | 14:01 |
| mark-casey | quite cool | 14:02 |
| *** huikang has quit IRC | 14:03 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Implements elemental.yml DSL for nova https://review.openstack.org/325313 | 14:03 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Install keepalived in neutron_l3_agent container https://review.openstack.org/325316 | 14:03 |
| rhallisey | mark-casey, ya I'm trying it now | 14:03 |
| rhallisey | mark-casey, ok I get how it works | 14:04 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Document the use of external Ceph journal drives https://review.openstack.org/324539 | 14:04 |
| *** salv-orlando has joined #openstack-kolla | 14:05 | |
| *** b_bezak has joined #openstack-kolla | 14:09 | |
| *** Mr_Broke_ has quit IRC | 14:10 | |
| mdnadeem | rhallisey, http://paste.openstack.org/show/507670/ | 14:14 |
| *** huikang has joined #openstack-kolla | 14:15 | |
| mdnadeem | rhallisey, containers is in pending state, any idea how much time generally it will take to come to running state ? | 14:15 |
| rhallisey | mdnadeem, I think the latest addition to the mariadb container is breaking it. It's looking to append something to a replication controller that doesn't exist | 14:15 |
| mdnadeem | rhallisey, ohk, can i try on some other service like keystone or any other, please suggest ? | 14:17 |
| rhallisey | mdnadeem, https://review.openstack.org/#/c/320744/3 was working for me. The newest patch not yet | 14:17 |
| patchbot | rhallisey: patch 320744 - kolla - Add kolla-kubernetes bootstrap capability to mariadb | 14:17 |
| rhallisey | I think wirehead_ was running it without the split CLI | 14:17 |
| rhallisey | since in that scenario it could signal the RC | 14:17 |
| *** mbound has quit IRC | 14:18 | |
| *** Mr_Broken has joined #openstack-kolla | 14:19 | |
| *** egonzalez has joined #openstack-kolla | 14:19 | |
| *** jtriley has joined #openstack-kolla | 14:19 | |
| mdnadeem | rhallisey, ohk, Thanks , its enogh for today, time to go home, will see u soon :) | 14:20 |
| rhallisey | mdnadeem, no problem. It will become more stable soon enough :) | 14:20 |
| mdnadeem | rhallisey, yah , very exiting to use it :) | 14:21 |
| jmccarthy | Hiya, wondering about is there some work going on at the moment to use ansible 2.1 to do deploys ? | 14:33 |
| inc0 | well, not sure if it's "going" | 14:34 |
| *** reddydodda has quit IRC | 14:34 | |
| inc0 | but it's surely a priority | 14:34 |
| *** matrohon has quit IRC | 14:35 | |
| jmccarthy | Ok, what version is expected to be used at the minute, 2.0 is it ? | 14:36 |
| pbourke_ | isn't it just ceph that doesn't work with 2.1? | 14:37 |
| *** pbourke_ is now known as pbourke | 14:38 | |
| sdake | pbourke can we backup in kolla to swift? | 14:38 |
| pbourke | sdake: dont know. it may be possible to configure with augment files. I've never tried it | 14:39 |
| *** egonzalez has quit IRC | 14:43 | |
| *** huikang has quit IRC | 14:44 | |
| jmccarthy | inc0: Is there a bp or something tracking the ansible 2.1 work ? I'm wondering about how to know if or how it's going | 14:45 |
| wirehead_ | rhallisey, mdnadeem : Yeah, the afforementioned mysterious call is the Kubernetes API. | 14:48 |
| rhallisey | wirehead_, ya I've been playing around with it | 14:48 |
| rhallisey | the issue with calling the RC though is with the CLI split the RC doesn't exists | 14:49 |
| wirehead_ | rhallisey, mdnadeem : It's bad practice to talk directly to etcd. | 14:49 |
| wirehead_ | Yeah, I saw that. Hrm. | 14:49 |
| rhallisey | wirehead_, is there a way to change the amount of replicas? | 14:49 |
| wirehead_ | Yeah, you just update the replica key. | 14:49 |
| rhallisey | could have bootstrap also be an rc | 14:49 |
| wirehead_ | I was going to say that. :D | 14:49 |
| wirehead_ | Bootstrap creates the RC, with replica count 0. | 14:50 |
| wirehead_ | start sets replica count to 1. | 14:50 |
| rhallisey | ya that's what I'm trying now | 14:50 |
| rhallisey | I'm just not getting it's key :) | 14:50 |
| *** rmart04 has quit IRC | 14:51 | |
| rhallisey | wirehead_, there's also something to consider. If the key lives in the bootstrap data, will it be lost when the bootstrap container gets deleted? | 14:52 |
| rhallisey | or does it even live with the bootstrap | 14:52 |
| rhallisey | the lock could also live with the master | 14:53 |
| wirehead_ | So, the reason why I put it in the RC in the first place is so that it's lifetime will match the lifetime of MariaDB. | 14:57 |
| wirehead_ | If you do a `kolla-kubernetes kill mariadb` then any information about the bootstrap goes away. | 14:57 |
| wirehead_ | (which is what motivated me to get that patched in the first place. Got tired of reprovisioning my Vagrant box because mariadb only provisioned right on a fresh config) | 14:58 |
| inc0 | jmccarthy, well, so I don't think we did much more besides get it running | 14:59 |
| inc0 | for 2.0 | 14:59 |
| inc0 | if it's not running for 2.1 then that's a bug | 14:59 |
| *** mgoddard_ has joined #openstack-kolla | 15:01 | |
| rhallisey | I think we want it to last the lifetime of mariadb. But for lights out recovery that won't work | 15:01 |
| wirehead_ | Depends on what you mean by "lights out recovery". | 15:02 |
| rhallisey | node gets unplugged | 15:02 |
| *** zhiwei has quit IRC | 15:02 | |
| inc0 | wirehead_, lights out recovery is recovery from restart of all of galera cluster nodes | 15:03 |
| inc0 | and we need to set them back up without losing data | 15:03 |
| inc0 | that is non-trivial | 15:03 |
| *** mgoddard has quit IRC | 15:04 | |
| rhallisey | hmm maybe we would recover | 15:04 |
| rhallisey | since the bootstrap wouldn | 15:04 |
| rhallisey | 'be run again | 15:04 |
| *** mliima has quit IRC | 15:05 | |
| inc0 | rhallisey, talking about k8s? | 15:05 |
| rhallisey | yes | 15:05 |
| inc0 | you won't have this issue at all | 15:05 |
| inc0 | as you don't run galera cluster, only single mariadb | 15:05 |
| inc0 | issue appears when you have cluster out of it | 15:05 |
| wirehead_ | So, my Kubernetes demo is aother piece of software we've been working on. And we delete all of the nodes live and onstage. | 15:06 |
| wirehead_ | As long as the contents of the etcd are preserved, the next time that k8s comes up, it'll try to re-schedule all of the RCs until it has nodes to schedule them on. | 15:06 |
| rhallisey | I'm just worried about how we handle the locking so recovery works | 15:06 |
| wirehead_ | Thus, outside of zapping things back to bare metal, the annotation will be preserved. | 15:06 |
| wirehead_ | It's little more than a 'safe' proxy to etcd. | 15:07 |
| sdake | pbourke if I dind't already thank yu thanks for the resposne :) | 15:07 |
| * sdake deep in fire email | 15:07 | |
| rhallisey | ok so storing the lock with the rc should be fine | 15:08 |
| mark-casey | inc0: rhallisey: with the patch to make haproxy consider donor and/or resyncing galera nodes, I think it's now less troublesome to run a cluster in k8s. Cluster now has more theoretical reliability than a single mariadb that gets restarted on fail | 15:09 |
| wirehead_ | So, right now, if you zap a node, it's going to try to re-schedule the MariaDB RC to another node. | 15:09 |
| inc0 | mark-casey, so in k8s it works differently | 15:09 |
| inc0 | whole model | 15:09 |
| inc0 | basically you run only single db node, mount storage over network | 15:09 |
| wirehead_ | ^ what inc0 said. | 15:10 |
| inc0 | if node dies, k8s is responsible of rescheduling it somewhere else and reconnecting network | 15:10 |
| inc0 | so you never really get net partition | 15:10 |
| inc0 | as you never run cluster | 15:10 |
| wirehead_ | It's sick as an angry greased armadillo in practice. | 15:10 |
| inc0 | only caveat to that is situation when etcd loses node, but actual container works | 15:11 |
| *** mgoddard has joined #openstack-kolla | 15:11 | |
| inc0 | (for example etcd uses mgmt network and storage uses stor network. interface of etcd dies and maria keeps running) | 15:11 |
| inc0 | therefore before restarting k8s has to ensure that storage is fenced and supposedly down node will not write to it | 15:12 |
| *** dwalsh has quit IRC | 15:12 | |
| inc0 | doable. | 15:12 |
| wirehead_ | So, what I've heard from larger-scale ops is that k8s handles a control plane outage OK. | 15:12 |
| inc0 | then it's really safe and your data is a lot safer than with galera | 15:12 |
| inc0 | yeah, so I heard | 15:12 |
| *** mgoddard_ has quit IRC | 15:12 | |
| inc0 | jmccarthy, so I'm deploying with ansible 2.1 now | 15:13 |
| wirehead_ | Also, you can set up a resilient etcd cluster and there you get better guarantees on distributed database linearization than the usual yolo distributed database. | 15:13 |
| inc0 | no issues so far | 15:13 |
| mark-casey | mounting storage over network adds overhead to new users. I don't run ceph with Kolla now and am not thrilled about being forced into it to use k8s. A restarting galera node wipes the datadir and replicates it from healthy node. This removes the need for any anchoring of db pods. As long as the k8s health check knows what a resyncing/donor node looks like you don't lose queries on fail/recover | 15:14 |
| inc0 | mark-casey, so with k8s you think about ha a bit differently | 15:14 |
| inc0 | mark-casey, you need to do this with k8s anyway | 15:14 |
| inc0 | otherwise k8s is just a glorious restarter of containers | 15:14 |
| mark-casey | emptydir volumes are persistent to pod deletion | 15:15 |
| inc0 | you need to store data somewhere... | 15:15 |
| jmccarthy | inc0: Oh ! Ok cool I was just thinking to setup a vm or something and try a deploy as well, do you have ceph enabled ? | 15:16 |
| rhallisey | ya shared storage here is a must | 15:16 |
| inc0 | well, it failer;) | 15:16 |
| inc0 | on cinder | 15:16 |
| mark-casey | right, as long as the pod isn't deleted from all k8s workers at the same time, your data is inside the container. which is exactly how kolla-ansible does it - without shared storage | 15:16 |
| inc0 | and yeah, ceph enabled | 15:16 |
| inc0 | 10 node + ceph | 15:17 |
| wirehead_ | You can probably use a NAS or NAS-like thing | 15:17 |
| inc0 | mark-casey, not exactly | 15:17 |
| wirehead_ | I think that's kinda a NAS-ty solution by comparison. :D | 15:17 |
| inc0 | right now data is in named volume | 15:17 |
| inc0 | and I mean k8s becomes a restarter | 15:17 |
| rhallisey | mark-casey, it's on the host now | 15:17 |
| inc0 | let me put it that way | 15:17 |
| mark-casey | I may be missing a piece. I thought if you delete all db containers you have a bad day | 15:17 |
| inc0 | mark-casey, no | 15:18 |
| inc0 | it would be equivalent to service mariadb stop on nodes | 15:18 |
| inc0 | if you redeploy containers, it'll work just fine | 15:18 |
| rhallisey | mark-casey, all of mariadb's data exists in /var/lib/docker | 15:18 |
| inc0 | (if you follow recovery procedure) | 15:18 |
| inc0 | no persistent data is ever stored in container | 15:18 |
| inc0 | we used to do that in liberty and this huge backport was because of this | 15:19 |
| mark-casey | ok. but in that case the data in /var/lib/docker is not shared between hosts | 15:19 |
| inc0 | it never is | 15:19 |
| inc0 | so you have persistent data locally | 15:19 |
| inc0 | if node dies, well, shame | 15:20 |
| mark-casey | right. I think it would be better to do the same in k8s | 15:20 |
| inc0 | I don't agree | 15:20 |
| rhallisey | mark-casey, then the db can't float | 15:20 |
| inc0 | as 1. k8s manages volumes really well | 15:20 |
| rhallisey | it would be tied to nodes | 15:20 |
| inc0 | you don't need ceph for that, you have lots of options | 15:20 |
| mark-casey | isn't it tied to nodes now outside of k8s | 15:20 |
| mark-casey | *? | 15:20 |
| inc0 | and as Ryan say, you want k8s to manage and restart containers across cluster | 15:20 |
| rhallisey | mark-casey, yes, ansible will deploy to the nodes you specifcy | 15:21 |
| rhallisey | the db data is stored on those nodes | 15:21 |
| inc0 | mark-casey, yeah, but you get software that manages storage really well to manage storage instead of having every single thing managing it's own | 15:21 |
| inc0 | some good, some not so good | 15:21 |
| inc0 | as for new users...k8s is not really for new users imho | 15:22 |
| rhallisey | kube will abstract maraidb out so that it guarantees it be active somewhere | 15:22 |
| inc0 | if you want k8s, you should be able to tell why you want it | 15:22 |
| *** mliima has joined #openstack-kolla | 15:22 | |
| rhallisey | inc0, lol add that to the readme | 15:22 |
| inc0 | :) | 15:22 |
| inc0 | well, that's true to most of software | 15:23 |
| inc0 | and a lot of times not met:) | 15:23 |
| mark-casey | rhallisey: you'd said the db can't float. why was that? | 15:23 |
| rhallisey | mark-casey, no it can float | 15:23 |
| inc0 | mark-casey, becasue storage will be local | 15:23 |
| rhallisey | in kubne | 15:24 |
| rhallisey | usuing ceph | 15:24 |
| inc0 | it requires net mounted volumes to float | 15:24 |
| rhallisey | without ceph, so using hostMount, it can't float | 15:24 |
| mark-casey | sure it can | 15:24 |
| rhallisey | how would it, hostMount is relative to a node | 15:25 |
| Mech422 | pbourke: on https://review.openstack.org/325316 - my understanding is its NOT a bug, but you need a 'tag' after KOLLA_CEPH_OSD_BOOTSTRAP ( eg KOLLA_CEPH_OSD_BOOTSTRAP_1, KOLLA_CEPH_OSD_BOOTSTRAP_1_J) | 15:25 |
| mark-casey | the mysql datadir is evaluated on galera start and if the node is too far behind or has no data the datadir is explicitly rm -rf and resynced from a healthy node | 15:25 |
| Mech422 | pbourke: basically, it uses the 'tag' to pair the data partition with a journal partition | 15:27 |
| rhallisey | are you talking about clustering here? I'm talking about one mariadb service | 15:27 |
| *** huikang has joined #openstack-kolla | 15:28 | |
| inc0 | mark-casey, again, no galera in k8s | 15:29 |
| inc0 | you don't want galera in k8s | 15:29 |
| rhallisey | mark-casey, plus in your example, it would only be able to float between those 2 nodes | 15:29 |
| rhallisey | we want it to float everywhere | 15:29 |
| sdake | ok scrollback | 15:29 |
| inc0 | cuz it will float and if it floats away it will cause partition | 15:29 |
| inc0 | a lot of partitions | 15:29 |
| inc0 | a lot of replications | 15:29 |
| sdake | Daviey still around? | 15:30 |
| sdake | mandre I withhold judgement on any changes to build.py until we finish the refactor we seem to be in agreement about relating to customization | 15:30 |
| sdake | mandre then th efloodgates open to fix kolla-build | 15:31 |
| rhallisey | wirehead_, have you figured out what the key is? | 15:32 |
| *** vhosakot has joined #openstack-kolla | 15:32 | |
| rhallisey | to change replicas | 15:32 |
| *** nilk has quit IRC | 15:33 | |
| sdake | mark-casey what kind of gear has 10gig ethernet on motherboard? | 15:33 |
| mark-casey | http://www.newegg.com/Product/Product.aspx?Item=2NS-000A-000B9http://www.newegg.com/Product/Product.aspx?Item=2NS-000A-000B9 | 15:33 |
| mark-casey | sdake: | 15:33 |
| mark-casey | oops double paste | 15:33 |
| mark-casey | http://www.newegg.com/Product/Product.aspx?Item=2NS-000A-000B9 | 15:33 |
| rhallisey | wow | 15:34 |
| rhallisey | worth more than my laptop | 15:34 |
| sdake | mark-casey everyone has to sign the individual license agreement, even if a company has signed it - to contribute to openstack | 15:34 |
| mark-casey | sdake: right. I was trying to allude to the company not signing | 15:34 |
| sdake | Mech422 did sam make a parition patch availalbe, or does it work with kolla as is? | 15:35 |
| sdake | Mech422 sam and I had lots of conversations about partition support in ceph and he said it was difficult - i assume that means it requires a change | 15:35 |
| Mech422 | sdake: it appears to be in the code... | 15:36 |
| sdake | mliima I think we are on to the plan we discussed on the google hangout last night, which means you dont need to do nova or neutron at this time | 15:36 |
| Mech422 | sdake: though I'm still working on pulling up all the various services | 15:36 |
| mark-casey | rhallisey: nvme slot, soldered-on octa Xeon-D that can be passively cooled, kvm over IPMI. I'm in love. we got three of them | 15:36 |
| Mech422 | sdake: he has a blog post about how to use it - let me dig it up... | 15:36 |
| rhallisey | mark-casey, lol nice! | 15:36 |
| sdake | jmccarthy we are working on ansible 2.1, 2.0 works, and I think 2.1 works as well but not cetain | 15:37 |
| sdake | but only in master | 15:37 |
| inc0 | mark-casey, sounds like something I'd like under my desk | 15:37 |
| *** dwalsh has joined #openstack-kolla | 15:37 | |
| inc0 | passively cooled including power supply? so it won't be noisy as fuck? | 15:37 |
| mark-casey | :D | 15:37 |
| mark-casey | PSU has a fan | 15:37 |
| mliima | i did it yesterday sdake :/ | 15:38 |
| Mech422 | sdake: http://yaple.net/2016/03/20/deploying-openstack-mitaka-with-kolla-docker-and-ansible/ (the explanation is right above the first code snippet) | 15:38 |
| mark-casey | and there is a version of the whole board with a fan | 15:38 |
| sdake | wirehead_ lights out recovery means that the datacenter has had a lights out scenario (power loss) and mariadb doesn't recover from that without manual (or in our case automated) intervention | 15:38 |
| inc0 | so I'm going back to shitload of nucs idea | 15:38 |
| sdake | rhallisey please don't use any locking - it doesns't scale | 15:38 |
| sdake | inc0 I agree with single db node approach - please use that ;) | 15:39 |
| inc0 | sdake, for k8s | 15:39 |
| rhallisey | sdake, the locking has to do with determining if we're in bootstrap mode | 15:39 |
| inc0 | http://www.amazon.com/Single-Computers-Intel-BoardM-2-pieces/dp/B0131V6F0S/ref=sr_1_11?ie=UTF8&qid=1464968352&sr=8-11&keywords=nuc+board&refinements=p_89%3AIntel | 15:39 |
| Mech422 | inc0: I have 2xDell c6100's and 1 x HP 6500 8-node chassis | 15:40 |
| rhallisey | kube doesn't have dependancies | 15:40 |
| Mech422 | inc0: noisy as all get out | 15:40 |
| *** jtriley has quit IRC | 15:40 | |
| sdake | inc0 lol glorious restarter of ontainers | 15:40 |
| Mech422 | inc0: but at $250/node with 24+G RAM, 2xquadcore, IPMI, etc - its dirt ass cheap | 15:40 |
| rhallisey | inc0, that's what compute will be :) | 15:40 |
| inc0 | so my plan is to buy several nucs, all passively cooled | 15:41 |
| mark-casey | on k8s out of curiosity, thus far, what else requires shared storage? | 15:41 |
| inc0 | or put 2 silent fans once I stack them together | 15:41 |
| inc0 | mark-casey, rabbitmq | 15:41 |
| Mech422 | inc0: the nucs are nice - but don't they 'kit out' at like $400/node ? | 15:42 |
| inc0 | and that's about it | 15:42 |
| inc0 | well, you can | 15:42 |
| inc0 | depends on a nuc | 15:42 |
| mark-casey | inc0: same situation with single node for rabbit or is it an IPC type issue? | 15:42 |
| rhallisey | wirehead_, so another idea | 15:42 |
| inc0 | single node rabbit, same thing | 15:42 |
| rhallisey | wirehead_, we could probe the bootstrap pod/rc to see that it exited | 15:43 |
| sdake | mark-casey cool thanks - super cheap price for that | 15:43 |
| rhallisey | or doesn't exist | 15:43 |
| sdake | my 10gig nics in my minidells were 300$ a crack | 15:43 |
| rhallisey | that would be a signal to run | 15:43 |
| sdake | mliima all good :) | 15:43 |
| sdake | mliima sorry for wasting yoru time :( | 15:43 |
| Mech422 | sdake: I've been seeing 10g adapters on EBay for like $25... | 15:44 |
| Mech422 | sdake: problem is the dam switches are still way expensive | 15:44 |
| Mech422 | sdake: especially compared to how fast the infi-band switch prices dropped | 15:44 |
| mark-casey | sdake: for sure. the only nitpick I have so far is that the pcie channel the 10g nics are on (internally) isn't quite high enough bandwidth. though I suppose that may be an issue for addin cards too. | 15:44 |
| sdake | rhallisey no kube has a shitload of deps, they just get statitcally built - some form github master branches... | 15:44 |
| mliima | don't worry sdake :) | 15:44 |
| rhallisey | sdake, no I mean dependencies on other tasks | 15:45 |
| sdake | Mech422 my switch was 850 bucks | 15:45 |
| Mech422 | sdake: heh - yeah, too rich for my home setup - I get 4 full nodes populated for $1k | 15:46 |
| *** absubram has joined #openstack-kolla | 15:46 | |
| sdake | good price | 15:46 |
| sdake | I have 6 nodes total, 3 with 32gb ram core i7s ssd storage and 4tb of disk | 15:46 |
| sdake | 1 with 1.2tb nvme ssd, 128gb ram | 15:46 |
| Mech422 | sdake: yeah - the dell c6100's are decent - but I sorta prefer the hp 6500 chassis | 15:46 |
| sdake | 2 with 12 gb ram | 15:46 |
| sdake | all with 10gig dual nic cards | 15:46 |
| *** jtriley has joined #openstack-kolla | 15:46 | |
| sdake | I have a netgear prosafe 8 port 10gig switch | 15:47 |
| sdake | seems to work | 15:47 |
| sdake | little noisy | 15:47 |
| sdake | little/lot | 15:47 |
| Mech422 | sdake: mine are 24/48G RAM, 2xquard core, 120G SSD per node | 15:47 |
| sdake | my office sounds like a jet factory | 15:47 |
| sdake | ok all caught up on scrollback for the morning :) | 15:47 |
| Mech422 | sdake: and 8 of my nodes have a mellanox 10G fiber card | 15:47 |
| mark-casey | rhallisey: inc0: hey guys are you expecting the shared storage ahead of time or are you planning to deploy it to k8s first as part of deploy? | 15:47 |
| sdake | Daviey still around? | 15:47 |
| inc0 | mark-casey, shared storage will be requirement prior to deploy k8s | 15:48 |
| inc0 | k8s doesn't really handle storage | 15:48 |
| rhallisey | mark-casey, good question. Likely before | 15:48 |
| inc0 | it needs to have some external storage | 15:48 |
| rhallisey | if we leave that to be handled externally, it will make our lives easier | 15:48 |
| mark-casey | inc0: I do get most of the persistence issues, but I still disagree :) | 15:48 |
| inc0 | one idea I had is to use our current ansible to deploy ceph and k8s;) | 15:48 |
| mark-casey | I like that | 15:49 |
| rhallisey | agreed | 15:49 |
| Mech422 | sdace: my real 'find' was an IBM 24U half rack for $300.... took me 6 months to find one for sale locally | 15:49 |
| *** sacharya has joined #openstack-kolla | 15:49 | |
| sdake | wirehead_ you ahve some work on ceph don't you? | 15:52 |
| sdake | Mech422 nice | 15:54 |
| *** mikelk has quit IRC | 15:58 | |
| wirehead_ | sdake: David is doing the ceph work. | 16:01 |
| wirehead_ | So, I am trying really really really really hard to not have a jet factory in my geekroom. | 16:01 |
| *** dwalsh has quit IRC | 16:02 | |
| wirehead_ | It’s disruptive to my other obsessions in life besides code. Such as music. | 16:02 |
| wirehead_ | So, I’m actually going into the office today instead of wfh. I haven’t gotten to looking at the patches yet. | 16:02 |
| dmsimard | fyi spotted an ansible 2.1 regression https://github.com/ansible/ansible/issues/16125 | 16:03 |
| Mech422 | wirehead_: yeah - I turned my son's old bedroom into a 'server room' - can still hear the servers thru the shared wall | 16:03 |
| wirehead_ | dcwangmit01_ and I have been going back and forth about Ceph. We’ve got our own cluster set up via Ansible. | 16:03 |
| *** daneyon_ has joined #openstack-kolla | 16:03 | |
| sdake | wirehead i hear you with the music | 16:03 |
| wirehead_ | I like my NUC linux box. | 16:04 |
| wirehead_ | It’s silent. | 16:04 |
| sdake | wirehead_ https://drive.google.com/open?id=0B8q6xDPETSkHcGNpdDVHbEpmcmc | 16:05 |
| sdake | Mech422 i did that to my daughter when we remodeled | 16:05 |
| sdake | she almost killed me i'm pretty sure :) | 16:05 |
| openstackgerrit | Merged openstack/kolla: Enable HAProxy consider MariaDB wsrep_local_state https://review.openstack.org/322200 | 16:06 |
| sdake | wirehead_ dr feickert blackbird turntable, line magnetic 219ia SET tube amp, Audio Research phono se2 | 16:06 |
| mark-casey | score! @ https://review.openstack.org/#/c/322200/ | 16:06 |
| patchbot | mark-casey: patch 322200 - kolla - Enable HAProxy consider MariaDB wsrep_local_state (MERGED) | 16:06 |
| sdake | i am buying an axiom when vestmast hits | 16:06 |
| mark-casey | so excited about that one | 16:07 |
| Mech422 | wirehead_: yeah - quiet is definately nice.. but it would have double the cost - and I have 16 nodes, so it adds up | 16:07 |
| Mech422 | sdake: LOL - mine moved cross country so he hasn't found out yet :-P | 16:07 |
| sdake | wirehead_ this is what the axiom looks like https://www.google.com/search?q=axiom+tonearm&espv=2&tbm=isch&imgil=b5VL3-1PM1H7VM%253A%253B-T_OdTOI4r3nfM%253Bhttp%25253A%25252F%25252Fwww.myhifilife.com%25252F2014%25252F06%25252F09%25252Fdiscovering-germanys-acoustical-systems%25252F&source=iu&pf=m&fir=b5VL3-1PM1H7VM%253A%252C-T_OdTOI4r3nfM%252C_&usg=__RZW8lYO8EgfAr7lbZZz1zohfASI%3D&biw=960&bih=475&ved=0ahUKEwjut_ndnozNAhVJ7mMKHRG0DigQyjcIKQ&ei=R6tRV674CM | 16:08 |
| sdake | ncjwOR6LrAAg#imgrc=b5VL3-1PM1H7VM%3A | 16:08 |
| sdake | woops | 16:08 |
| sdake | http://www.myhifilife.com/wp-content/uploads/2014/06/axiom.jpg | 16:08 |
| sdake | super excited about that | 16:08 |
| *** daneyon_ has quit IRC | 16:08 | |
| sdake | wirehead_ the gear on the second shelf is for digital musicand power distribution | 16:10 |
| wirehead_ | So I’m very much a home recording nerd instead of an audiophile. | 16:10 |
| wirehead_ | So I’ve got some Yamaha HS50M studio monitors: https://www.flickr.com/photos/cyberspace/8216074388/ | 16:10 |
| sdake | wirehead_ oh i see :) | 16:10 |
| wirehead_ | As well as some crappy Logitech speakers for testing on a ‘real’ setup. | 16:11 |
| sdake | well for recording you need absolute silence | 16:11 |
| wirehead_ | Which reminds me, I need to stop being lazy about soundproofing foam. | 16:11 |
| *** tyrola has joined #openstack-kolla | 16:11 | |
| sdake | have you found a vendor for any? | 16:11 |
| sdake | i'd like to get some on my walls to absorb sound | 16:11 |
| sdake | since i have tile now | 16:11 |
| wirehead_ | http://www.foambymail.com/acoustical-foam-products.html | 16:12 |
| sdake | and my audio gear sunds worse | 16:12 |
| sdake | bummer only comes in charcoal | 16:12 |
| wirehead_ | Naw, it comes in colors too | 16:12 |
| sdake | the one i want only has charcoal | 16:12 |
| sdake | or would want | 16:12 |
| sdake | I saw a really cool sound imrpovement board at my dealer | 16:13 |
| sdake | but he said the company is out of business | 16:13 |
| sdake | its basically like a picture with nothign on it :) | 16:13 |
| wirehead_ | One of the Rackspace SF offices had noise problems, so we got a bunch of bright colored foam from there. Eventually they got rid of it. So I’ve got a lot of bright blue foam squares to reduce reflectivity. | 16:14 |
| wirehead_ | Also bookshelves. | 16:15 |
| sdake | i may bite the bullet and get a rug | 16:15 |
| wirehead_ | Bookshelves loaded with books are actually decent ersatz sound reflectivity dampers. | 16:16 |
| sdake | something cheap and thick | 16:16 |
| *** ravig has joined #openstack-kolla | 16:16 | |
| sdake | yes bookshelves rock | 16:16 |
| sdake | but i don't have pace in my listening room | 16:16 |
| wirehead_ | That’s what real recording studios use. Vibe + soundproofing. | 16:16 |
| sdake | what is vibe | 16:16 |
| sdake | i want some refelctions | 16:16 |
| wirehead_ | Psychological vibe. | 16:16 |
| sdake | but i have too many now | 16:16 |
| sdake | the soundstage is muddled slightly from where it was | 16:16 |
| sdake | although it is still fantastic | 16:16 |
| openstackgerrit | Merged openstack/kolla: Use an image object, recorder object and status constants https://review.openstack.org/321844 | 16:17 |
| *** jmccarthy has left #openstack-kolla | 16:18 | |
| wirehead_ | Hm. So I can’t find right now the geekroom arrangement panorama, from chair view. | 16:21 |
| wirehead_ | Oh, there’s also http://www.auralex.com/ sdake | 16:23 |
| sdake | wirehead_ sweet thanks for the link | 16:24 |
| sdake | i've book maked thatone :) | 16:24 |
| wirehead_ | They’ve got some interesting-looking things under the Diffusion lineup | 16:25 |
| *** jmccarthy has joined #openstack-kolla | 16:25 | |
| sdake | agree | 16:25 |
| wirehead_ | Well, if you come to the SF office, sdake, Andi Parker is a huge audiophile nerd, of your tribe. | 16:26 |
| sdake | nice | 16:26 |
| inc0 | so in topic of sound | 16:26 |
| inc0 | I found really good speakers for the buck | 16:26 |
| wirehead_ | There’s a combination coffee shop and audiophile store in SF. | 16:26 |
| inc0 | http://www.amazon.com/Micca-MB42X-Bookshelf-Speakers-Tweeter/dp/B00E7H8GG2?ie=UTF8&psc=1&redirect=true&ref_=oh_aui_detailpage_o02_s00 I was surprised by sound quality for that kind of money | 16:27 |
| *** ssurana has joined #openstack-kolla | 16:27 | |
| wirehead_ | Well, given today’s technology, it takes dedicated research *COUGH*bose*COUGH* to sound truly bad. | 16:28 |
| inc0 | well, bose doesn't sound super bad, but its so overpriced that for that kind of money you can get superb stuff | 16:29 |
| inc0 | I actually was surprised that in Europe this stuff is cheaper | 16:29 |
| inc0 | but maybe because I'm used to european brands and looked mostly at these | 16:30 |
| inc0 | even with this it seems cheaper to get good quality stuff | 16:30 |
| ccesario | inc0, Lyncos told me that he used 14.04 not 16.04 | 16:30 |
| inc0 | at host? | 16:31 |
| inc0 | ok then I misunderstood him | 16:31 |
| inc0 | anyway I'm more headphones guy than speakers | 16:31 |
| *** esharao has joined #openstack-kolla | 16:33 | |
| wirehead_ | rhallisey: I mean, we could just create a file on the drive to signal that bootstrapping is complete. | 16:33 |
| esharao | hi guys.. quick question - I'm getting the following error.. 404 Client Error: Not Found (\"Container command \\'kolla_start\\' not found or does not exist.\")\\n'"} | 16:33 |
| inc0 | wirehead_, negative | 16:33 |
| ccesario | inc0, eheheh well, I 'm confirming it with him (by email,he still not answer me) | 16:34 |
| rhallisey | wirehead_, no that would be host specific | 16:34 |
| inc0 | this has to be available across the cluster | 16:34 |
| *** Lyncos has joined #openstack-kolla | 16:34 | |
| wirehead_ | Yeah, that would only work for the mysql, mostly. | 16:34 |
| rhallisey | wirehead_, I'll have work on a patch after I finish lunch | 16:35 |
| inc0 | even for mysql you want it in etcd | 16:35 |
| rhallisey | I think it's better this way | 16:35 |
| rhallisey | don't have to deal with passing around lock | 16:35 |
| wirehead_ | So, there’s a thirdpartyresource that you could create to track the lifespan of the mysql cluster, but that’s beta. | 16:35 |
| rhallisey | I also saw there are events | 16:35 |
| *** inc0_ has joined #openstack-kolla | 16:35 | |
| wirehead_ | The other thing on my mental obsessiveness list was splitting the bootstrap into discrete pieces. | 16:36 |
| Lyncos | Hey guuys | 16:39 |
| wirehead_ | rhallisey: OK, so, you are going to generate the next patch for mariadb bootstrap? I had some stuff I wanted to do to Keystone and bootstrap. | 16:39 |
| *** inc0 has quit IRC | 16:40 | |
| Lyncos | nova-api-metadata is not implemented | 16:40 |
| Lyncos | ? | 16:40 |
| rhallisey | wirehead_, ya let's work on the concept a little | 16:40 |
| *** SiRiuS has quit IRC | 16:40 | |
| Lyncos | this was a question | 16:40 |
| sdake | wirehead_ re that other cat working on ceph | 16:45 |
| sdake | wirehead_ is he on irc? | 16:45 |
| sdake | wirehead_ if not, might as well get him here :) | 16:45 |
| wirehead_ | sdake: dcwangmit01_ | 16:45 |
| inc0_ | Lyncos, isn't metadata served as normal nova api call? | 16:45 |
| *** ravig has quit IRC | 16:46 | |
| sdake | dcwangmit01_ mind sharing what your thinking is with ceph? Kolla has a ceph implementation already built for bare metal but its not sacrosanct | 16:46 |
| *** tyrola has quit IRC | 16:46 | |
| sbezverk | rhallisey have you thought how to deal with keystone wsgi listen ip addresses? | 16:46 |
| sdake | and by bare metal I mean it runs in containers on bare metal | 16:46 |
| rhallisey | sbezverk, remind me of the issue with it | 16:47 |
| wirehead_ | I was thinking of setting mariadb and keystone to not use a host port but to be exposed merely as a kubernetes service and seeing what would happen. | 16:47 |
| wirehead_ | Obviously this will break for some services. :) | 16:48 |
| wirehead_ | But at least mariadb and keystone ought to be more towards things that will be OK with kube networking. | 16:48 |
| mag009_ | hey guys | 16:50 |
| inc0_ | so folks important news | 16:50 |
| inc0_ | just got email | 16:50 |
| *** esimone has joined #openstack-kolla | 16:50 | |
| mag009_ | i believe you didn't test ansible 2.0.X with ceph | 16:50 |
| inc0_ | we'll have 130 nodes from OSIC cluster between 7/22 and 8/14 | 16:50 |
| wirehead_ | But… there’s a clear use case for the control plane being on k8s. | 16:50 |
| inc0_ | sdake, | 16:50 |
| wirehead_ | How you set up the storage and compute nodes depends on the install. | 16:51 |
| inc0_ | mag009_, I just failed on cinder conf so kinda yeah | 16:51 |
| mag009_ | cinder I believe I fixed it | 16:51 |
| mag009_ | the main problem is the ceph | 16:51 |
| wirehead_ | Some folks are going to want to have the base layer be k8s, at which point ceph inside of k8s starts to make sense, albeit with a lot of potential circular dependencies. | 16:51 |
| mag009_ | at the delegate part | 16:51 |
| mag009_ | it try to ssh with no user | 16:52 |
| mag009_ | I'm trying to figure this one out :) | 16:52 |
| inc0_ | mag009_, ceph deployed no issues | 16:52 |
| mag009_ | hm | 16:52 |
| inc0_ | HEALTH_OK | 16:52 |
| sdake | inc0_ midcycle is 7-12/7-13 | 16:52 |
| sbezverk | rhallisey right now keystone wsgi file has two lines with ip address and ports where httpd will listen for authentications requests | 16:52 |
| inc0_ | ansible 2.1 | 16:52 |
| sdake | so that sounds great | 16:52 |
| sdake | inc0 we can spend face to face time figuring out what to do with the nodes | 16:52 |
| inc0_ | cool so at midcycle we can brainstorm all the tests | 16:53 |
| sbezverk | rhallisey current config generation put VIP there, which is not acceptable for k8s | 16:53 |
| inc0_ | great | 16:53 |
| sdake | inc0_ i'd like people to be prepared with ideas | 16:53 |
| inc0_ | I'll send email on list | 16:53 |
| sdake | inc0_ you read my mind :) | 16:53 |
| sdake | mention the midcycle, mention we are ogin to brainstorm there, mention come prepared with ideas | 16:53 |
| wirehead_ | Other folks are going to be OK with kicking off ansible to install ceph, k8s, and compute nodes. | 16:53 |
| sbezverk | rhallisey we need either to change it to 0.0.0.0:35357 or during the bootstrap process replace it with ip address assigned to keystone pod | 16:53 |
| sdake | mention the date of the midcycle as well pls | 16:53 |
| sdake | wirehead_ can't we use k8s to run compute nodes? | 16:54 |
| wirehead_ | sdake: you can. | 16:54 |
| sdake | there are already projects that install k8s | 16:54 |
| sdake | i'd prefer to just use them if they work and have a compatible license | 16:54 |
| sdake | (instal k8s using ansible) | 16:55 |
| *** harlowja has joined #openstack-kolla | 16:55 | |
| wirehead_ | k8s using Ansible is going to be the standard installer in the future. | 16:55 |
| sdake | right thatis the one i am tlaking about | 16:55 |
| sdake | its called "saltstack 2.0" I guess ;) | 16:55 |
| wirehead_ | Right now, it’s some messy salt stack where the best thing I can say about it is that it’s better than DevStack. | 16:55 |
| sdake | deploying kubernetes is straightforward | 16:56 |
| sdake | a heat tempalte is already available for it | 16:56 |
| sdake | that can be manually converted to ansible as a stopgap | 16:56 |
| wirehead_ | sbezverk: see, that’s why I think we want to do 0.0.0.0 and a set port number, and then use kubernetes to map the port number of the container to the exposed port. | 16:56 |
| *** huikang has quit IRC | 16:56 | |
| sdake | i just dont want to be in the business of maintaining a k8s installer unless wee have to do so | 16:56 |
| wirehead_ | Hell no. | 16:57 |
| wirehead_ | That sounds like the worst of all possible futures. :D | 16:57 |
| Lyncos | Heat template for k8s https://github.com/metral/corekube/blob/master/corekube-openstack.yaml | 16:57 |
| Lyncos | :-) | 16:57 |
| sdake | the reason we don't bind to 0.0.0.0 is its a security risk | 16:57 |
| sdake | Lyncos i was talking about the one in magnum | 16:57 |
| sdake | i guess there is more then one :) | 16:57 |
| sdake | like all good thigns in life | 16:57 |
| wirehead_ | Does Keystone need to be HostNetwork? | 16:57 |
| sdake | docker proxy is slower then a 28.8k modem | 16:58 |
| inc0_ | mail sent | 16:58 |
| sdake | https://www.youtube.com/watch?v=PivpCKEiQOQ | 16:58 |
| Lyncos | ahhh | 16:58 |
| inc0_ | wirehead_, technically no | 16:58 |
| mag009_ | inc0_: it worked if I remove delegate_to in distribute_keyrings.yml | 16:59 |
| sbezverk | wirehead_ if I remember right all pods are as per agreement | 16:59 |
| inc0_ | so delegate to is there, because you might have external deploy host | 16:59 |
| inc0_ | I'm not sure if it works this way | 16:59 |
| sbezverk | wirehead_ and specs | 17:00 |
| wirehead_ | Yeah, I guess I’m going to wander in and play with the third rail, then. :D | 17:00 |
| mag009_ | hm let me try something :) | 17:00 |
| wirehead_ | For connecting to MariaDB and Keystone, my gut feel is that everything is going to go much more smoothly if we don’t make those HostNetwork. | 17:01 |
| sbezverk | sdake since openstack services are internal to the cluster, maybe it is not that much of a risk? | 17:01 |
| wirehead_ | For things like Nova and Swift, maybe other way around. :) | 17:01 |
| sbezverk | wirehead_ neutron as well | 17:02 |
| wirehead_ | yah | 17:02 |
| inc0_ | mag009_, one thing, I'm running ansible 2.1 | 17:03 |
| inc0_ | maybe they fixed something | 17:03 |
| mag009_ | same | 17:03 |
| sdake | host networking is used for two reasons: performance and security | 17:03 |
| sdake | many of the services could use a port bind | 17:03 |
| sdake | and that would preserve security | 17:03 |
| sdake | but performance would still be slower then the 28.8k modem linked above ;) | 17:03 |
| sdake | but do whatever you think is right | 17:04 |
| sdake | we can refactor later | 17:04 |
| wirehead_ | They improved the proxy performance, after realizing that it was slower than a 28.8k modem. | 17:04 |
| sdake | as long as we dont become religious about going to a hostnetworkingmode only later | 17:04 |
| sdake | wirehead_ it still blows | 17:04 |
| sdake | wirehead_ for a datapoint, mirantis uses ports in their docker implementation | 17:05 |
| sdake | and all their customers complain about openstack performance | 17:05 |
| inc0_ | so as for OSIC cluster one test I'd like to do is: | 17:06 |
| sdake | (not fuel performance) | 17:06 |
| inc0_ | deploy kolla on 130 hardware | 17:06 |
| mag009_ | I've found the problem | 17:06 |
| inc0_ | run 1000 vms | 17:06 |
| *** huikang has joined #openstack-kolla | 17:06 | |
| inc0_ | deploy kolla on 1k vms;) | 17:06 |
| inc0_ | mag009_, do tell | 17:06 |
| sdake | inc0 we should deploy at max capacity of the cluster | 17:06 |
| sdake | but ya thats definately a test | 17:06 |
| sean-k-mooney | inc0_: that would be cool to see. | 17:06 |
| wirehead_ | docker networking != kubernetes networking. | 17:06 |
| sdake | we could also use the gear to implement multi-region | 17:06 |
| inc0_ | and cells | 17:06 |
| mag009_ | i'm using a jenkins that create a docker with ansible on it as my deploy host | 17:06 |
| sdake | wirehead_ yes I know, I wrote the initial code for magnum | 17:07 |
| sdake | i am intimately familiar with kuberntes up to about 0.97 or so | 17:07 |
| mag009_ | it fail that way | 17:07 |
| mag009_ | but if I use the jenkins host as deploy host it work | 17:07 |
| inc0_ | ahh shame, that was one thing I wanted to do too | 17:07 |
| sean-k-mooney | inc0_: we are doning some testing with the fake driver currently. trying to emulate 250 fake nodes on a singel host and then run 2500 fake vms on top | 17:07 |
| mag009_ | i don't know why it would fail tho... use to work with version 1.9.X | 17:07 |
| mag009_ | seem they've changed the behaviour like you said | 17:08 |
| inc0_ | sean-k-mooney, my real point is to run 1k node kolla cluster | 17:08 |
| sdake | inc0 yu mean kolla on kolla? | 17:08 |
| inc0_ | yeah | 17:08 |
| sdake | we may have trobule getting ip addressing space | 17:08 |
| sean-k-mooney | inc0_: yep that is why we are trying to get 250 fake compute nodes running so when we go to 4 physical nodes we can have 1K | 17:08 |
| inc0_ | nah | 17:08 |
| sdake | but that sounds good | 17:08 |
| inc0_ | ip addresses will be all local | 17:09 |
| sean-k-mooney | inc0_: when we had that running we want to then try it on the osic cluster | 17:09 |
| inc0_ | sean-k-mooney, sounds like you want to participate in tests?;) | 17:09 |
| inc0_ | (you are more than welcome to) | 17:09 |
| mag009_ | inc0 i'll try to find a way to do it via a docker because it allow me to test different version of ansible etc.. in a docker | 17:10 |
| sean-k-mooney | inc0_: perhaps. pritiv currently looking at this work | 17:10 |
| *** sacharya_ has joined #openstack-kolla | 17:11 | |
| mag009_ | also i had some problem with the cleanup sometime the cleanup does not delete the ceph_mon_config volume | 17:11 |
| *** salv-orlando has quit IRC | 17:11 | |
| sean-k-mooney | inc0_: we wanted to simulate the effect of controling nodes over a high latency link so after the kolla fake nodes are spawn we are in increaseing the latency using tc the using rally to spawn the fake vms | 17:11 |
| *** salv-orlando has joined #openstack-kolla | 17:12 | |
| inc0_ | interesting, any good findings? | 17:13 |
| *** sacharya has quit IRC | 17:13 | |
| sean-k-mooney | not yet but we will be happy to share when we do | 17:13 |
| inc0_ | I'd expect rabbitmq to act up | 17:14 |
| sean-k-mooney | am well we were haveing some good results initally | 17:14 |
| inc0_ | mag009_, ad cinder.conf fail, did you publish any bugfix for that? | 17:14 |
| sean-k-mooney | we have started hitting issue with running out of ram on the host | 17:14 |
| *** esharao has quit IRC | 17:14 | |
| inc0_ | yeah, I'd expect that to be the case | 17:14 |
| mag009_ | inc0: i'm testing it as we speak | 17:14 |
| mag009_ | I think I have a fix | 17:15 |
| mag009_ | not sure yet | 17:15 |
| mag009_ | :D | 17:15 |
| inc0_ | kk | 17:15 |
| mark-casey | OSIC tests running kolla-ansible or k8s? | 17:16 |
| inc0_ | I'd put priority on ansible | 17:16 |
| inc0_ | but we might squeeze k8s in | 17:16 |
| inc0_ | time permitting | 17:16 |
| sean-k-mooney | inc0_: we got upto 250 fake compute nodes spawned on a singel host and were able to get 1 | 17:16 |
| rhallisey | inc0_, if we're ready | 17:16 |
| rhallisey | who knows | 17:17 |
| inc0_ | yeha, that too | 17:17 |
| sean-k-mooney | inc0_: we got upto 250 fake compute nodes spawned on a singel host and were able to get 1K fake vms spawned before we added latency | 17:17 |
| sean-k-mooney | inc0_: but sorry did not want to derail the osic cluster topic | 17:21 |
| inc0_ | sean-k-mooney, you didnt, anyway, we need to think how to make best use of this cluster | 17:22 |
| sean-k-mooney | inc0_: out of interest did you make the request to use the cluster or was that done by someone else? | 17:23 |
| inc0_ | I did | 17:24 |
| sean-k-mooney | cool i might ping you at some point about how to do that | 17:24 |
| sdake | for access to gear time, k8s seems fine, I think the way we should approach it is people submit their ideas for tests with a time window, and we provide it - using the cluster around the clock since our team is highly distributed | 17:25 |
| *** huikang has quit IRC | 17:25 | |
| sean-k-mooney | the osic cluster is part of the 2 500 node dataceneter intel and rackspace set up right? or is it a seperate cluster? | 17:26 |
| *** dwalsh has joined #openstack-kolla | 17:27 | |
| sdake | sean-k-mooney same thing | 17:30 |
| sean-k-mooney | sdake: cool thanks. i remeber hearing/reading about the anouchment but never really looked into it | 17:32 |
| *** ravig has joined #openstack-kolla | 17:35 | |
| *** ravig has quit IRC | 17:38 | |
| wirehead_ | I watched the grand opening. | 17:43 |
| *** d_code has joined #openstack-kolla | 17:43 | |
| *** v1k0d3n has quit IRC | 17:44 | |
| wirehead_ | sdake: The network improvements didn’t hit till 1.1 | 17:45 |
| inc0_ | sean-k-mooney, yup, this one | 17:46 |
| d_code | is Mauricio Lima or Paul Bourke or coolvsap here? | 17:46 |
| d_code | had question on my patch | 17:46 |
| inc0_ | mliima, pbourke ^ | 17:47 |
| d_code | thanks | 17:47 |
| d_code | mliima or pbourke …regarding changing the SR-IOV patch to branch off master…. do I just rebase? | 17:48 |
| d_code | nevermind…I’ll just blow away my dir…then apply the patches to master and push back up | 17:50 |
| d_code | I think | 17:50 |
| sdake | d_code do htis | 17:52 |
| sdake | git format-patch -1 | 17:52 |
| sdake | git checkout master | 17:52 |
| sdake | git am 0001* | 17:52 |
| sdake | git review -s | 17:52 |
| sdake | git review | 17:52 |
| sdake | you may need to rebase the other patch off of the stable branch | 17:52 |
| sdake | to do that, git rebase -i | 17:52 |
| sdake | delete the patch line | 17:52 |
| sdake | type in "noop" | 17:52 |
| sdake | save vi | 17:52 |
| dcwangmit01_ | sdake: Are your referring to the ceph comment I made in https://review.openstack.org/#/c/304182/15 ? | 17:53 |
| patchbot | dcwangmit01_: patch 304182 - kolla - Spec: Deploy Kolla images using Kubernetes | 17:53 |
| sdake | dcwangmit01_ no I am talking to d_code | 17:54 |
| sdake | dcwangmit01_ oh you mean your ceph work | 17:54 |
| sdake | dcwangmit01_ wirehead_ said you were working on ceph in some indepenent fashion | 17:54 |
| sdake | or maybe its part of kolla, I dont know | 17:54 |
| *** eaguilar has joined #openstack-kolla | 17:55 | |
| mliima | d_code, i'm here | 17:56 |
| dcwangmit01_ | sdake: Yes. I'm no expert on Ceph. My work using it was to get ceph working for persistence, for my bare-metal kubernetes cluster. Documented here: https://www.davidwang.com/2016/05/24/ceph-rbd-bare-metal-install-for-kubernetes-on-packet-net/ . The existing ceph-ansible scripts work really well. | 17:56 |
| d_code | mliima: sdake gave me a direction….but I screwed up trying to do `git rebase -i master`….and….well….it’s a mess =) | 17:57 |
| d_code | but…I”m working through it | 17:57 |
| mliima | d_code, ok :) | 17:57 |
| mliima | d_code, i'm sorry for the late | 17:58 |
| dcwangmit01_ | sdake: The kolla-kubernetes spec mentioned Ceph, but it was unclear if it was a Ceph cluster on top of Kubernetes, or a distinct and separate cluster from Kubernetes | 17:58 |
| openstackgerrit | Merged openstack/kolla: Use debian repos for debian base docker image https://review.openstack.org/288936 | 17:59 |
| *** zhiwei has joined #openstack-kolla | 18:00 | |
| d_code | so…trying to apply that patch… looks like nova-libvirt and nova-compute now include mounting /dev completely.... | 18:02 |
| d_code | for SR-IOV we only need /dev/vfio | 18:03 |
| d_code | soo….is it possible to backport that mount to mitaka? | 18:03 |
| d_code | should I skip that and just merge my docs for SR-IOV? | 18:03 |
| *** zhiwei has quit IRC | 18:04 | |
| Lyncos | Guys.. how metadata is handled in kolla ? | 18:04 |
| Lyncos | I don't see any metadata container | 18:05 |
| d_code | Lyncos: wrt to instance / flavor metadata? | 18:05 |
| Lyncos | I'm talking about cloud-init stuff | 18:05 |
| Lyncos | I think it need a nova-api-metadata or something .. because sending the metadata requests to nova-api dosen't seems to work | 18:06 |
| d_code | yeah…I believe all that goes into the mysql database | 18:06 |
| d_code | mariadb | 18:06 |
| d_code | I know flavor metadata is stored in the database | 18:06 |
| Lyncos | Anyone can do a test for me ? | 18:06 |
| mark-casey | sdake: inc0_: if you get time for k8s on OSIC I'd very highly recommend beating up the database a bit. I suppose I can get on board with the single node db plan :D, but keep in mind galera reads scale linearly to cluster size. Compared to a kolla-ansible cluster of the same size you're already down to a fraction of the query processing you had, and then running it over a distributed disk with a double-write penalty | 18:06 |
| openstackgerrit | Derek Ditch proposed openstack/kolla: Adds documentation of how to perform SR-IOV https://review.openstack.org/325440 | 18:07 |
| Lyncos | I would like someone to test deploying Coreos with a specific cloud-init file to see if it works.... our deployment is a bit different as we are using calico I would like to know where it is failing | 18:07 |
| Lyncos | Please deploy Coreos instance with following cloud-init http://paste.openstack.org/show/507811/ and paste me /run/systemd/system/etcd2.service/20-.... | 18:08 |
| d_code | mliima: kk..I pushed the docs to review off master | 18:08 |
| d_code | https://review.openstack.org/325440 | 18:08 |
| d_code | not sure….are the other patches just ignored? | 18:09 |
| mliima | d_code, we don't backport doc changes to stable branches in kolla | 18:10 |
| d_code | so, it’s not the doc that needs backported…it is the same | 18:10 |
| d_code | the issue is the mount | 18:10 |
| mliima | this ps isn't needed https://review.openstack.org/#/c/322334/ | 18:10 |
| patchbot | mliima: patch 322334 - kolla (stable/mitaka) - Adds documentation of how to perform SR-IOV | 18:10 |
| d_code | mitaka doesn’t have /dev/vfio mounted in nova-libvirt | 18:11 |
| sean-k-mooney | mark-casey: in the k8s setup will there be a single db per serivce or one total? similar question for rabbit | 18:11 |
| mliima | ok, i know d_code | 18:11 |
| mark-casey | I dunno. I only kinda jumped in on another discussion. kinda been lurking on k8s | 18:12 |
| mark-casey | sean-k-mooney: ^^ | 18:12 |
| d_code | k…I abandoned that change, mliima | 18:12 |
| mliima | but you don't need do a backport of documentation | 18:12 |
| mliima | :) | 18:12 |
| sean-k-mooney | mark-casey: :) no worries same | 18:12 |
| mark-casey | sean-k-mooney: I think one total | 18:12 |
| d_code | mliima: agreed. it was a failure of my understanding of the process. That’s been fixed from the PS | 18:13 |
| d_code | s/fixed/removed/ | 18:13 |
| sean-k-mooney | inc0_: not sure if you are still about but in the summit discusstion you were suggesting using one rabit instance per servce for k8s. would we do the same for the db? | 18:13 |
| d_code | I thought | 18:13 |
| d_code | dang | 18:14 |
| mliima | can you abbandon this ps https://review.openstack.org/#/c/322334/ ? | 18:14 |
| patchbot | mliima: patch 322334 - kolla (stable/mitaka) - Adds documentation of how to perform SR-IOV | 18:14 |
| mliima | This ps https://review.openstack.org/#/c/325440/1 It will be the "official" | 18:15 |
| patchbot | mliima: patch 325440 - kolla - Adds documentation of how to perform SR-IOV | 18:15 |
| d_code | k.. fixed | 18:16 |
| d_code | I’m a github guy…I like the organization Gerrit provides…but it’s a bit counter-intuitive for me at the moment | 18:16 |
| kklimonda | what would be the correct way to add support for nfs backend for nova instances? the only nfs related work I see so far is https://blueprints.launchpad.net/kolla/+spec/nfs-support-in-cinder | 18:17 |
| *** daneyon_ has joined #openstack-kolla | 18:19 | |
| *** MarMat has joined #openstack-kolla | 18:20 | |
| *** eaguilar has quit IRC | 18:20 | |
| mliima | d_code, don't worry :) | 18:21 |
| inc0_ | sean-k-mooney, here | 18:22 |
| inc0_ | well, yeah | 18:22 |
| inc0_ | that's the model we want to pursue with k8s | 18:22 |
| *** huikang has joined #openstack-kolla | 18:22 | |
| inc0_ | for both db and rabbit | 18:22 |
| *** d_code has left #openstack-kolla | 18:23 | |
| *** d_code has joined #openstack-kolla | 18:23 | |
| sean-k-mooney | kklimonda: are you refering to adding support for nfs root or just having nova using an nfs share for its ephemeral instance storage? | 18:23 |
| *** thumpba has joined #openstack-kolla | 18:23 | |
| kklimonda | sean-k-mooney: just nfs share for ephemeral instance storage | 18:24 |
| inc0_ | kklimonda, well, you can do this by putting /var/lib/docker/volumes/nova_data on nfs | 18:24 |
| inc0_ | on host level | 18:24 |
| inc0_ | I don't think we'll support it in kolla anytime soon | 18:25 |
| sean-k-mooney | inc0_: yep that was what i was thinking too | 18:25 |
| inc0_ | still possible now with a bit of host tinkering | 18:25 |
| inc0_ | nova_compute is the name of volume | 18:26 |
| *** daneyon_ has quit IRC | 18:26 | |
| inc0_ | but that should work | 18:26 |
| sean-k-mooney | inc0_: well it depends on the storage diriver you are using though correct | 18:26 |
| inc0_ | I don't think so | 18:26 |
| inc0_ | volumes are storage-agnostic I think | 18:26 |
| sean-k-mooney | inc0_: if you are using aufs or overlay fs it should work | 18:26 |
| inc0_ | it's just a dir | 18:26 |
| kklimonda | ok, that's what I was thinking of doing in the first place - is there anything I should be aware of when doing that, or no one has done it yet and lived to tell the tale? ;) | 18:27 |
| inc0_ | kklimonda, noone I know did that;) | 18:27 |
| *** mgoddard has quit IRC | 18:27 | |
| inc0_ | but you're free to try;) | 18:27 |
| mag009_ | inc0_: I've fixed the problem with cinder but i'm just trying to fix another problem before I submit the patch | 18:27 |
| mag009_ | to retest it | 18:27 |
| inc0_ | I'd say use ceph if on prod, but well... | 18:27 |
| inc0_ | kk, thanks mag009_ | 18:28 |
| inc0_ | one thing I need to check out is if nova compute puts libvirt xmls in volume too | 18:28 |
| kklimonda | I don't really have hardware for ceph - it's a weird deployment where compute nodes have only 1 drive, and space for only one more, and yet there is no shared storage to speak of.. | 18:29 |
| *** cu5 has quit IRC | 18:29 | |
| inc0_ | kklimonda, so you can still play around it | 18:29 |
| sean-k-mooney | inc0_: kklimonda one ting to keep in mind is that placing /var/lib/docker on nfs probably wont work with the btrfs storage driver just incase you plan to use it | 18:29 |
| inc0_ | not with vanilla kolla, but possible | 18:29 |
| inc0_ | sean-k-mooney, might work still, just you'd need to have separate partition for /var/lib/docker/volumes | 18:30 |
| kklimonda | sean-k-mooney: I'll be using aufs probably - seems to be working better than devicemapper, and I don't really have experience with btrfs to use it. | 18:30 |
| inc0_ | put it in ln -s or sth | 18:30 |
| inc0_ | not sure | 18:31 |
| inc0_ | kklimonda, aufs is *way* better than devicemapper | 18:31 |
| inc0_ | btrfs is a little better than aufs | 18:31 |
| inc0_ | but aufs works well | 18:31 |
| inc0_ | well enough | 18:31 |
| sean-k-mooney | kklimonda: ya aufs should work fine | 18:31 |
| sean-k-mooney | the btrfs comment relates to the fact that the btrfs driver requires the file system of /var/lib/docker to be btrfs | 18:32 |
| inc0_ | yeah, but agian you could mount /var/lib/docker/volumes on nfs I think | 18:32 |
| inc0_ | it's hacky but works | 18:32 |
| inc0_ | and kklimonda ceph can be configured to use file as OSD | 18:33 |
| sean-k-mooney | inc0_: ya that might work just never tried it | 18:33 |
| inc0_ | not right now with kolla, but should be doable with a bit of tinkering | 18:33 |
| inc0_ | or a partition | 18:33 |
| kklimonda | inc0_: but it's still going to be 42 7200rpm drives, I'm not sure how much performance I can get out of it | 18:33 |
| inc0_ | well, sure | 18:34 |
| inc0_ | but you get data persistence;) | 18:34 |
| inc0_ | well, decision you guys need to make, just saying it's possible and I'd at least cosider it | 18:34 |
| inc0_ | other than that, try nfs hack I described, should work just fine | 18:35 |
| sean-k-mooney | kklimonda: you would be serprised how well ceph works if you add an ssd cache teir even a small one in front of slow disks | 18:35 |
| kklimonda | sean-k-mooney: I'd be surpsised, because all testing I've done so far wasn't that promising ;) | 18:35 |
| inc0_ | that too, if you'd fill remaining disk slot with even small ssd it might go rocket | 18:35 |
| inc0_ | kklimonda, issue is, you'd want journal on ssd | 18:36 |
| kklimonda | are we talking about ceph cache tier, or a journal on ssd? | 18:36 |
| inc0_ | you can do both | 18:36 |
| inc0_ | partition ssd for journal and cache | 18:36 |
| kklimonda | mhm, that could work | 18:36 |
| inc0_ | and use both over spindle | 18:36 |
| sean-k-mooney | kklimonda: i was refering to a ssd cache tier | 18:37 |
| inc0_ | sean-k-mooney, you want fast journal regardless | 18:37 |
| sean-k-mooney | but moving the journal to an ssd also helps | 18:37 |
| inc0_ | so issue with kolla currently is it only understands block devices | 18:37 |
| inc0_ | because we use udev for that | 18:37 |
| inc0_ | but I have it in my backlog to enable other options for OSD too | 18:38 |
| kklimonda | how many ceph nodes should I have to really test it out? | 18:38 |
| inc0_ | well, you need at very least as much as replication factor | 18:38 |
| inc0_ | so I believe 3 osd | 18:38 |
| mag009_ | inc0_: did you had problem with neutron sysctl ? | 18:38 |
| sean-k-mooney | kklimonda: you can test with one but its really depends on what you want to test. | 18:38 |
| inc0_ | mag009_, not yet, I'm waiting for cinder fix;) | 18:38 |
| inc0_ | one will not produce relevant data | 18:39 |
| inc0_ | you can get it to stand up with 1 | 18:39 |
| inc0_ | it will even work | 18:39 |
| inc0_ | but replication changes things | 18:39 |
| kklimonda | sean-k-mooney: I'd really like to see how well will performance scale as I keep adding nodes | 18:39 |
| *** ppowell has quit IRC | 18:39 | |
| inc0_ | yeah, sure, ceph handles that well | 18:39 |
| *** ppowell has joined #openstack-kolla | 18:40 | |
| inc0_ | so my suggestion, get an ssd for your nodes and deploy ceph | 18:40 |
| sean-k-mooney | inc0_: you can change the replication policy to the osd level to have replication with a singel node but it sounds like kklimonda would be better with multinode | 18:40 |
| inc0_ | you might want to put cgroup limit on cpu on OSD if you want same node to be shared with compute services | 18:40 |
| inc0_ | OSD can be cpu hungry | 18:40 |
| inc0_ | sean-k-mooney, as I said, you can make it work, just not a relevant data would come out of it;) | 18:41 |
| inc0_ | I suggest playing with ceph more, I became a fan;) | 18:41 |
| sean-k-mooney | inc0_: ceph experts would hate my home setup hehe but it works for my usecase :) | 18:42 |
| inc0_ | it's a home setup;) | 18:43 |
| inc0_ | what I'd like to see is ceph with NVMe cache | 18:43 |
| inc0_ | sdake, you have NVMe right? | 18:43 |
| sean-k-mooney | inc0_: what i want to see is ceph running on xpoint when that launches | 18:44 |
| kklimonda | heh, I have two nvme drives just lying around - but I can't get them into this cluster ;) | 18:44 |
| inc0_ | yeah, that would go skyrocket | 18:44 |
| inc0_ | << storage porn >> | 18:45 |
| sean-k-mooney | :) | 18:45 |
| *** stvnoyes has left #openstack-kolla | 18:45 | |
| openstackgerrit | Marc-Andre Gatien proposed openstack/kolla: adding else statement with empty string, starting ansible 2.X it require a else statement. https://review.openstack.org/325483 | 18:48 |
| Mech422 | sean-k-mooney: thats why I only put a single small SSD on my nodes - waiting for xpoint... | 18:49 |
| mag009_ | inc0_: i just submitted it | 18:49 |
| sdake | inc0_ of coure - best tech i've bought in years | 18:49 |
| sdake | i have a 4tb disk in each machine and 400gb PCIE intel 750ssd | 18:50 |
| inc0_ | sdake, we can do some crazy fast ceph with that | 18:50 |
| *** eaguilar has joined #openstack-kolla | 18:50 | |
| sdake | in my main workstation i have a 1.2tb pcie nvmessd | 18:50 |
| sdake | inc0_ do you need to test? | 18:50 |
| sdake | inc0_ if so I can givey ou an ssh login | 18:51 |
| * Mech422 has storage envy... | 18:51 | |
| inc0_ | don't have time, but someday sure | 18:51 |
| sdake | inc0_ cool ya I dont have time to do much - my review queue outstanding is down to 1 page ;) | 18:51 |
| sdake | so I'd like to make progress on that | 18:51 |
| inc0_ | dayjob takes its toll, but yeah someday I'd love to try out exactly how fast ceph can be | 18:52 |
| sdake | inc0_ my gear also has intel 10gb nics | 18:52 |
| sdake | only one port is connected though as I have only 1 8 port switch | 18:52 |
| inc0_ | yeah, vxoffloading <3 | 18:52 |
| sdake | inc0_ ceph can be run single node | 18:52 |
| inc0_ | sdake, can be run != should be run' | 18:53 |
| inc0_ | it's as much value as aio k8s;) | 18:54 |
| inc0_ | and I said today what's aio k8s;) | 18:54 |
| sdake | inc0_ yees but it is testable as aio :) | 18:54 |
| sean-k-mooney | inc0_: you can still have data redundency with singel node ceph | 18:54 |
| sdake | i have 3 nodes i can make immediately available if someone wntas to test | 18:54 |
| sdake | the problem is my home partition is on the ssd | 18:54 |
| sdake | sorry the boot partition | 18:54 |
| sean-k-mooney | inc0_: but if that one node goes down then its all offlien | 18:55 |
| sdake | and last I heard, ceph doesnt work with partition tables | 18:55 |
| inc0_ | sean-k-mooney, well...I know you can do that | 18:55 |
| sdake | although someone today linked a blog about making it work | 18:55 |
| sdake | but sam said it was impossible | 18:55 |
| sdake | and its sam's blog | 18:55 |
| sdake | so who knows :) | 18:55 |
| sean-k-mooney | sdake: ceph can work with partitions but the kolla code does not supprot that currently | 18:56 |
| sdake | yes that is what i mean | 18:56 |
| sdake | I think sam's blog post has some info about making it work | 18:57 |
| sdake | but I didn't read it | 18:57 |
| mag009_ | mliima: i need to open a bug in the launchpad? | 18:57 |
| sdake | because frankly, I am tired of reading | 18:57 |
| *** ppowell has quit IRC | 18:58 | |
| inc0_ | sean-k-mooney, it can even be run with file on disk | 18:59 |
| sean-k-mooney | ah the joys of waithing for devstack to finish. its almost as fun as watching code compile | 18:59 |
| inc0_ | don't use devstack:( | 18:59 |
| mliima | mag009_, yes, but if it is not necessary to make a backport this ps, add only one TrivialFix. | 18:59 |
| sean-k-mooney | inc0_: im a maintainer of a devstack plugin so i have to ocationally | 18:59 |
| inc0_ | mliima, it is not as mitaka is ansible 1.9 | 19:00 |
| inc0_ | and this is 2.0 issue | 19:00 |
| mliima | oh | 19:00 |
| sean-k-mooney | inc0_: if its any concilation its running in a vm hosted on kolla | 19:00 |
| inc0_ | :D | 19:00 |
| mag009_ | i just filled a bug report in the launchpad now -_- | 19:00 |
| mag009_ | I was about to ammend my comment with it | 19:00 |
| mliima | so, a TrivialFix is enough inc0_ mag009_ | 19:00 |
| inc0_ | mag009_, it's ok, it's always better to do it with bug | 19:00 |
| mliima | don't worry mag009_ | 19:01 |
| inc0_ | TrivialFix is our kolla specific way to avoid creating bugs and such for every-single-thing | 19:01 |
| inc0_ | however as mliima said, if you need to backport it, you need bug | 19:01 |
| mag009_ | how do I tag it as trivialfix ? | 19:01 |
| mliima | did you report a bug? | 19:02 |
| inc0_ | mag009_, just add TrivialFix to commit message | 19:02 |
| mag009_ | yes ah ok | 19:02 |
| mag009_ | for this one I'll report it as a bug | 19:02 |
| inc0_ | but if yuou added bug, make it Closes-bug: #numberofabug | 19:02 |
| inc0_ | you'll get launchpad elo points for reporting and closing a bug;) | 19:02 |
| openstackgerrit | Marc-Andre Gatien proposed openstack/kolla: adding else statement with empty string starting ansible 2.X it require a else statement. https://review.openstack.org/325483 | 19:03 |
| mag009_ | lol | 19:03 |
| mliima | mag009_, follow inc0, he knows things | 19:03 |
| inc0_ | hehe | 19:03 |
| mag009_ | there you go | 19:03 |
| mag009_ | I'll get it... the more I do | 19:03 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Add kolla-kubernetes bootstrap capability to mariadb https://review.openstack.org/320744 | 19:04 |
| inc0_ | sure | 19:04 |
| mag009_ | still a new problem now with neutron sysctl | 19:05 |
| inc0_ | harlowja, ping | 19:06 |
| inc0_ | mliima, I've put +w on mag009_ patch | 19:08 |
| inc0_ | if gates fail, it won't merge anyway | 19:09 |
| mliima | I did not know that inc0_ | 19:09 |
| mliima | cool | 19:09 |
| inc0_ | well, I take that back, we don't have voting gates | 19:09 |
| inc0_ | but if we'd have it still goes through gates before merge | 19:10 |
| *** mark-casey has quit IRC | 19:10 | |
| wirehead_ | Complaining about the OpenStack gate is pretty much how we start conversations. Like complaining about the weather in London. | 19:10 |
| inc0_ | wirehead_, I've been to London, now I'm in Texas | 19:10 |
| inc0_ | trust me, latter is worse | 19:10 |
| inc0_ | and I'm talking about rai | 19:10 |
| inc0_ | n | 19:10 |
| wirehead_ | Well, it’s always been beautiful when I’ve visited London. | 19:11 |
| wirehead_ | By comparison, I was in San Antonio last year and drove through a torrential downpour that was the most rain I’ve ever traversed. | 19:11 |
| wirehead_ | Given that it’s been nice ever time I’ve visited Seattle, I suspect the tourism boards are investing in weather control. | 19:12 |
| inc0_ | mag009_, with your patch full deploy succeeded | 19:12 |
| inc0_ | ubuntu 15.10 on host and ansible2.1 | 19:13 |
| mag009_ | hm i'm still having issues.. | 19:13 |
| inc0_ | wirehead_, I live in San Antonio. We have storm every day at least | 19:13 |
| inc0_ | yesterday we had 3 storms | 19:13 |
| inc0_ | aaaand I see next one comming | 19:14 |
| mliima | 3? | 19:14 |
| inc0_ | yeah | 19:14 |
| inc0_ | 3 storms in same day | 19:14 |
| inc0_ | full package, downpour, thunders, wind | 19:14 |
| sean-k-mooney | we got pretty lucky for the summit so | 19:14 |
| inc0_ | yeah you did | 19:14 |
| inc0_ | like month before summit we got hailstorp | 19:14 |
| wirehead_ | So, the entire time I worked at Rackspace and therefore had to visit SAT frequently, it was drought and thus no storms. | 19:14 |
| inc0_ | size of tennis ball | 19:14 |
| mliima | 3 in same day | 19:15 |
| mliima | man | 19:15 |
| mliima | run man, run! | 19:15 |
| inc0_ | couple kms from San Antonio there was hail of size of grapefruit | 19:15 |
| *** salv-orl_ has joined #openstack-kolla | 19:15 | |
| inc0_ | FUCKING GRAPERFUIT-SIZE CHUNKS OF ICE FALLING FROM THE SKY | 19:16 |
| inc0_ | pardon my language | 19:16 |
| wirehead_ | Hee hee hee. | 19:16 |
| wirehead_ | Hail and freezing rain are mother nature giving you the middle finger. | 19:16 |
| sean-k-mooney | being in ireland we dont really ever get really bad weather but we also dont really get really great weather. its been in the mid 20's for the last week though so cant complain too much | 19:17 |
| *** zhiwei has joined #openstack-kolla | 19:17 | |
| inc0_ | mid 20's is my favorite | 19:17 |
| inc0_ | here it's already 30+ | 19:17 |
| wirehead_ | Rain, fine. Snow, fine. Hail hurts and freezing rain is really quite dangerous. | 19:17 |
| inc0_ | gets up to 45 | 19:17 |
| inc0_ | in summer | 19:17 |
| *** salv-orlando has quit IRC | 19:18 | |
| inc0_ | wirehead_, if hail makes holes in cars windshields it also might hurt | 19:18 |
| inc0_ | https://www.google.com/search?q=san+antonio+hail&num=50&tbm=isch&tbo=u&source=univ&sa=X&ved=0ahUKEwjm85G4yYzNAhVS2GMKHZrZCDcQsAQIUA&biw=1920&bih=971 | 19:18 |
| wirehead_ | Yeah, I’ve had itty-bitty hail land on me and that was painful. | 19:19 |
| wirehead_ | I do NOT want to get hit by grapefruit hail. | 19:19 |
| wirehead_ | Also, tornadoes are a particularly American form of extreme weather. | 19:19 |
| mliima | pardon inc0_ https://www.google.com.br/search?q=jericoacoara+beach&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjpyKHWyYzNAhXFKB4KHXfZB10Q_AUIBygB&biw=1920&bih=955 | 19:19 |
| mliima | :P | 19:19 |
| inc0_ | mliima, you live there?:P | 19:20 |
| mliima | same state | 19:20 |
| inc0_ | mhm...soooo...I think we have place for O midcycle | 19:21 |
| mliima | hahaha | 19:21 |
| *** daneyon_ has joined #openstack-kolla | 19:21 | |
| sean-k-mooney | inc0_: +1 lol | 19:21 |
| inc0_ | I don't even need venue, we'll buy a whiteboard and set it up on a beach | 19:22 |
| *** zhiwei has quit IRC | 19:22 | |
| wirehead_ | I’m envisoning sunburned nerds. | 19:22 |
| mliima | inc0_, +2 | 19:22 |
| mag009_ | +1 for the beach | 19:22 |
| sean-k-mooney | inc0_: on a beach we just need a stick to draw in the sand | 19:22 |
| wirehead_ | Also, season reversal. | 19:23 |
| *** thumpba_ has joined #openstack-kolla | 19:26 | |
| *** daneyon_ has quit IRC | 19:26 | |
| wirehead_ | rhallisey: how’s that for rapid review turnaround? :) | 19:27 |
| *** thumpba has quit IRC | 19:28 | |
| rhallisey | wirehead_, I was going to ask | 19:28 |
| rhallisey | since whatever I do , the hash gets added | 19:28 |
| mag009_ | inc0_: do you pass vars in your inventory ? | 19:30 |
| *** Jeffrey4l_ has quit IRC | 19:30 | |
| *** godleon has joined #openstack-kolla | 19:31 | |
| wirehead_ | rhallisey: Like, I’m wondering if we’re going to end up with a lightweight kube client, either the official kubectl, or somehing smaller and python, embedded in the Kolla images. | 19:33 |
| rhallisey | wirehead_, what would we being doing with the client? Using the endpoint is fine so far at least | 19:34 |
| *** Jeffrey4l_ has joined #openstack-kolla | 19:35 | |
| inc0_ | mag009_, I do not, but technically you could | 19:36 |
| inc0_ | I think;) | 19:36 |
| mag009_ | i know but I'm trying to find why it work on your end | 19:37 |
| wirehead_ | rhallisey: If we reach the point where it isn’t. | 19:38 |
| mag009_ | I'm stuck at neutron it fail at the when: condition of the start | 19:38 |
| wirehead_ | rhallisey: At this stage of abstraction, we seem to be fine | 19:38 |
| inc0_ | mag009_, did you rebuild containers lately? | 19:38 |
| mag009_ | nop | 19:39 |
| inc0_ | I have code from today and containers freshly | 19:39 |
| inc0_ | built | 19:39 |
| inc0_ | try that | 19:39 |
| inc0_ | get lates greatest master and do full rebuild | 19:39 |
| mag009_ | i'll try that on monday.. I did some patch to support calico | 19:40 |
| mag009_ | i need to rebase stuff | 19:41 |
| inc0_ | yeah | 19:41 |
| inc0_ | it might be disconnect between container versions and code | 19:42 |
| inc0_ | first thing you do when you hit somethign liek that is to make sure all containers are rebuilt and fresh | 19:42 |
| inc0_ | also use --no-cache flag | 19:42 |
| mag009_ | ok | 19:42 |
| rhallisey | wirehead_, ok | 19:46 |
| rhallisey | wirehead_, what do I need to do exactly get have the job get a name | 19:46 |
| rhallisey | I've added mariadb-bootstrap everywhere I coudl | 19:47 |
| wirehead_ | rhallisey: should be available as mariadb-bootstrap. | 19:47 |
| wirehead_ | rhallisey: but remember, the job will delete itself after it completes. | 19:48 |
| rhallisey | it sadly adds a hash | 19:48 |
| rhallisey | idk if it's possible to have a job without a hash | 19:48 |
| wirehead_ | hmmm | 19:48 |
| openstackgerrit | Merged openstack/kolla: Add vars to the merge_config task in config-neutron-fake.yml https://review.openstack.org/325139 | 19:48 |
| rhallisey | I mean the search I have works fine | 19:48 |
| rhallisey | but ideally we would apply a known name | 19:49 |
| rhallisey | wirehead_, also re your comment, the point is to allow someone to come along later and run the bootstrap again | 19:50 |
| rhallisey | that's how the upgrade will happen | 19:50 |
| rhallisey | I mean I guess we could have some logic saying 'you're upgrading now so it's ok to bootstrap' | 19:50 |
| rhallisey | but I think operators will want more control than that | 19:50 |
| openstackgerrit | Merged openstack/kolla: Updated from global requirements https://review.openstack.org/325454 | 19:51 |
| *** rahuls has joined #openstack-kolla | 19:51 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Add kolla-kubernetes bootstrap capability to mariadb https://review.openstack.org/320744 | 19:51 |
| wirehead_ | rhallisey: http://localhost:8080/apis/extensions/v1beta1/namespaces/default/jobs/mariadb-bootstrap | 19:52 |
| *** rahuls has quit IRC | 19:52 | |
| *** rahuls has joined #openstack-kolla | 19:53 | |
| rhallisey | wirehead_, perfect thank you | 19:54 |
| *** mliima has quit IRC | 19:54 | |
| openstackgerrit | Merged openstack/kolla: Install keepalived in neutron_l3_agent container https://review.openstack.org/325316 | 19:54 |
| mag009_ | when its a new feature do I need to use TrivialFix? | 19:56 |
| rhallisey | wirehead_, I see, pod applies a hash. But if you search more specifically you get the name | 19:56 |
| wirehead_ | rhallisey: Yeah, the hash changes as the pod gets rescheduled. | 19:56 |
| mag009_ | inc0_: I was able to pass my problem by commenting out the when: in neutron | 19:57 |
| mag009_ | everything when fine after | 19:57 |
| mag009_ | went* | 19:57 |
| wirehead_ | rhallisey: So when we kill nodes on our kube cluster, the rescheduled instances of our software get a different hash because it’s really a new instance | 19:57 |
| mag009_ | I'll try on monday like you said | 19:57 |
| wirehead_ | rhallisey: (emptyDir volumes cleared, for example) | 19:58 |
| rhallisey | wirehead_, there an issue though, this doesn't have state | 19:58 |
| wirehead_ | rhallisey: So, this depends on how deep we want to go. | 19:59 |
| rhallisey | we could add it.. | 19:59 |
| -openstackstatus- NOTICE: The infrastructure team is taking Gerrit offline for maintenance this afternoon, beginning shortly after 20:00 UTC. We aim to have it back online around 00:00 UTC. | 19:59 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Use the Kube endpoint to dictate state instead of etcd https://review.openstack.org/325503 | 20:00 |
| *** ravig has joined #openstack-kolla | 20:01 | |
| rhallisey | wirehead_, ya we need some form of state, otherwise we have to manage it ourselves | 20:01 |
| wirehead_ | rhallisey: We can change the state after each operation and thus return to the macro-block where we were. | 20:01 |
| *** absubram has quit IRC | 20:01 | |
| rhallisey | ya :/ | 20:01 |
| wirehead_ | rhallisey: except that you’d better make the macro-block safe and idempotent. | 20:01 |
| rhallisey | the hash bit could break is you had multiple bootstrap jobs running | 20:02 |
| rhallisey | that might not be possible though because of naming | 20:02 |
| rhallisey | I'm not sure what's better to go with here | 20:02 |
| wirehead_ | rhallisey: Well, I think that you get some protections with the job. | 20:02 |
| wirehead_ | rhallisey: Yeah, just verified. | 20:03 |
| wirehead_ | If you try to create two jobs with the same name, the second one will fail. | 20:03 |
| rhallisey | you also get some protections from the pod. Kubernetes manages the state vs the contianer | 20:03 |
| wirehead_ | rhallisey: Yeah, we need to protect the user (potentially) form a hazardous opperation that is trying to bootstrap a running system. And we need to make sure that the job completes fully. | 20:04 |
| rhallisey | wirehead_, well I think it's ok if the user bootstraps a running system | 20:04 |
| rhallisey | I mean that would be an upgrade step | 20:04 |
| rhallisey | I think we have to trust the operator on that one | 20:05 |
| rhallisey | but also give them the freedom to handle a strange case that needs a bootstrap suddenly | 20:05 |
| wirehead_ | rhallisey: Yeah | 20:06 |
| wirehead_ | rhallisey: So, the main purpose of the patch we’re talking about is less about mariadb bootstrapping and more about only running the steps we need to if the bootstrap halts mid-process. | 20:07 |
| openstackgerrit | Merged openstack/kolla: Use --subproject instead of deprecated --service option https://review.openstack.org/322862 | 20:07 |
| *** eaguilar has quit IRC | 20:09 | |
| -openstackstatus- NOTICE: Gerrit is offline for maintenance until 00:00 UTC | 20:09 | |
| *** ChanServ changes topic to "Gerrit is offline for maintenance until 00:00 UTC" | 20:09 | |
| inc0_ | ok, going off then;) | 20:10 |
| inc0_ | have a good weekend folks | 20:10 |
| wirehead_ | rhallisey: GAH | 20:11 |
| mag009_ | see you monday | 20:11 |
| wirehead_ | rhallisey: OK, got it in just as they announced it. | 20:11 |
| *** unicell1 has quit IRC | 20:12 | |
| *** unicell has joined #openstack-kolla | 20:12 | |
| sean-k-mooney | inc0_: enjoy the weekend | 20:13 |
| wirehead_ | rhallisey: but, yeah, Kubernetes is careful about injecting it’s own config into the environment so we don’t have to. | 20:14 |
| *** inc0_ has quit IRC | 20:14 | |
| mag009_ | question why the neutron-base include dnsmasq??? | 20:20 |
| mag009_ | shouldn't it be on the dhcp-agent image instead ? | 20:20 |
| dcwangmit01_ | rhallisey: Gerrit is down, and I've got a few uncommitted comments to the kolla-kubernetes spec. Hoping to get them into the current patch-set. | 20:31 |
| *** ravig has quit IRC | 20:31 | |
| *** eaguilar has joined #openstack-kolla | 20:33 | |
| *** ravig has joined #openstack-kolla | 20:36 | |
| rhallisey | dcwangmit01_, ok thx | 20:37 |
| rhallisey | wirehead_, I got to run. I'll catch up later | 20:37 |
| rhallisey | I'll be around tmr | 20:38 |
| *** rhallisey has quit IRC | 20:38 | |
| *** jtriley has quit IRC | 20:39 | |
| *** ravig has quit IRC | 20:39 | |
| *** unicell has quit IRC | 20:41 | |
| *** unicell has joined #openstack-kolla | 20:42 | |
| *** ravig has joined #openstack-kolla | 20:44 | |
| *** ayoung has quit IRC | 20:44 | |
| *** ravig has quit IRC | 20:45 | |
| *** ravig has joined #openstack-kolla | 20:46 | |
| *** d_code has quit IRC | 20:46 | |
| *** ravig has quit IRC | 20:55 | |
| *** mbound has joined #openstack-kolla | 21:03 | |
| *** huikang has quit IRC | 21:04 | |
| *** dwalsh has quit IRC | 21:06 | |
| *** salv-orl_ has quit IRC | 21:06 | |
| *** salv-orlando has joined #openstack-kolla | 21:07 | |
| *** mbound has quit IRC | 21:11 | |
| *** eaguilar has quit IRC | 21:11 | |
| sdake | mag009_ i think you are right but it woud require testing | 21:14 |
| *** ravig has joined #openstack-kolla | 21:16 | |
| sdake | well i guess everyone is gone | 21:16 |
| dmsimard | I'm still here | 21:17 |
| *** ravig has quit IRC | 21:17 | |
| sdake | i was going to work on the customization this afternoon | 21:17 |
| dmsimard | :P | 21:17 |
| sdake | but gerrit is bust | 21:17 |
| sdake | hey dmsimard | 21:17 |
| sdake | oh to look at ara | 21:17 |
| sdake | moment let me pull up that tarball | 21:17 |
| sdake | dmsimard why are some tsks missing names in the output? | 21:18 |
| sdake | like .../share/kolla/ansible/site.hyml | 21:18 |
| sdake | a a whole buncho f tasknames are missing but it has the action | 21:18 |
| sdake | the first one is setup for eg | 21:18 |
| *** fragatin_ has joined #openstack-kolla | 21:19 | |
| dmsimard | ansible doesn't enforce naming tasks | 21:19 |
| dmsimard | you can have - debug: msg: foo or - name: blah debug: msg: foo | 21:19 |
| dmsimard | that looks weird inline, hope you understood that | 21:19 |
| dmsimard | so there's really nothing else to show in the name column if the task doesn't have a name | 21:20 |
| *** harlowja has quit IRC | 21:20 | |
| sdake | so your syaing some of our tasks dont have names? | 21:20 |
| * dmsimard nods | 21:21 | |
| sdake | common patterns appear around include and setup | 21:21 |
| sdake | should those types be named? | 21:21 |
| dmsimard | setup can be called from - name: blah setup: and this will be named | 21:22 |
| *** fragatina has quit IRC | 21:22 | |
| dmsimard | but setup is also implicitely called from gather_facts: true | 21:22 |
| dmsimard | and that one can't be named | 21:22 |
| dmsimard | includes can be named but are not commonly named | 21:22 |
| dmsimard | or at least I think they can | 21:22 |
| sdake | recommended featuer - filter tasks | 21:22 |
| sdake | only display those with a name | 21:23 |
| sdake | (or specific actions) | 21:23 |
| sdake | just brianstorming here :) | 21:23 |
| dmsimard | yeah sure, I'm all for feature ideas | 21:23 |
| sdake | i cnat help with implementation but it looks like an interesting tool | 21:23 |
| dmsimard | we actually had something similar in the alpha | 21:23 |
| dmsimard | but it was lost in refactoring stuff | 21:23 |
| dmsimard | you could click on a particular task and you could see all the results for that task | 21:24 |
| dmsimard | across playbook runs | 21:24 |
| *** ravig has joined #openstack-kolla | 21:24 | |
| sdake | so this is sort of related to ara | 21:25 |
| sdake | we at some point want to build a cli that produces some kind of useful output | 21:25 |
| sdake | the current ansible run stuff produces absolute shit | 21:25 |
| sdake | I mean i guess its thorough | 21:25 |
| dmsimard | what kind of useful output are you after ? | 21:26 |
| dmsimard | ara has a CLI client as well | 21:26 |
| sdake | i am not exactly sure | 21:26 |
| sdake | something more concise | 21:26 |
| dmsimard | sec, I can show you. | 21:26 |
| sdake | when I run a playbook I don't wnat ot see 312 tasks | 21:26 |
| sdake | I ust want to see "mariadb deployed - then nova deployed or whtever" | 21:27 |
| sdake | so I'm wondering how you implemented ara | 21:27 |
| sdake | this probably wouldn't be helpful for you | 21:28 |
| sdake | but would be helpful for me | 21:28 |
| sdake | Instead of seeing each speciic task, i just want to see each role | 21:29 |
| *** ravig has quit IRC | 21:30 | |
| dmsimard | sdake: here: https://asciinema.org/a/4a3zmdr8oyttjwzt7dowr7jor | 21:31 |
| dmsimard | a quick view | 21:31 |
| dmsimard | and it's built on the same framework as openstackclient, so you have things like "-c <column>" to filter by column and stuff like "-f <json>" to output in json instead out of the box | 21:32 |
| *** ravig has joined #openstack-kolla | 21:32 | |
| *** zhiwei has joined #openstack-kolla | 21:33 | |
| dmsimard | I wonder if there's a way for us to detect the roles that are used | 21:33 |
| dmsimard | That would be helpful indeed. | 21:34 |
| sdake | how I want to use it is to live output | 21:35 |
| sdake | or use the tech you have | 21:35 |
| dmsimard | not sure I follow | 21:37 |
| dmsimard | you want to ditch the current ansible "streaming" output and replace it by something else ? | 21:37 |
| sdake | as kolla runs, I want to do soething like [Deploying Openstack - Major area [20% done]] | 21:37 |
| sdake | roger | 21:37 |
| sdake | i want oth | 21:37 |
| sdake | both | 21:37 |
| dmsimard | the way to do that would be through a callback of sorts | 21:38 |
| *** zhiwei has quit IRC | 21:38 | |
| dmsimard | for example we use this: https://github.com/rdo-infra/weirdo/blob/master/playbooks/library/human_log.py | 21:38 |
| dmsimard | this takes the output that ansible prints and prettyprints it so it's not inline json nonsense | 21:38 |
| dmsimard | you could have a callback that knows how many tasks there are in the playbook (somehow) and calculate progress | 21:39 |
| dmsimard | and print that progress. | 21:39 |
| sdake | dmsimard instead of a logging tool | 21:42 |
| sdake | could this be turned into a gui to display progress? | 21:42 |
| sdake | or could that be a feature? | 21:42 |
| dmsimard | you mean if ara could somehow print the progress of the playbook ? | 21:43 |
| sdake | yes | 21:43 |
| dmsimard | It'd be a nice feature, I think, though out of scope | 21:43 |
| sdake | a one pager | 21:43 |
| sdake | ya - like I said you may not be interested :) | 21:43 |
| dmsimard | No, don't get me wrong, I'm interested in something like that | 21:43 |
| dmsimard | But not in ARA proper | 21:44 |
| sdake | where would it go? | 21:44 |
| *** huikang has joined #openstack-kolla | 21:44 | |
| dmsimard | An ansible callback plugin by itself | 21:45 |
| dmsimard | however, I don't think it's possible for Ansible to know how many tasks it will have to run before going through the entire playbook | 21:45 |
| dmsimard | i.e, dynamic includes based on conditions that have not yet been evaluated | 21:45 |
| dmsimard | So, perhaps there would be a use case for sneaking that into ara because once a playbook has been run once, we can reasonably estimate how long it will take and how many tasks there will be | 21:46 |
| sdake | dnsso the plguin tells the web gui how? | 21:46 |
| dmsimard | but ara wouldn't be able to tell on the first run | 21:47 |
| sdake | well i'm good with a hardcode if we do something on our own | 21:47 |
| sdake | i am just trying to figure out how this thing works :) | 21:47 |
| dmsimard | Callbacks are hooks that Ansible calls on every event, there are hooks called like "task_on_ok", "task_on_failed" and so on. ara uses these to record events in a database but you can do anything you want with them, really | 21:48 |
| dmsimard | So, when a task completes, you could print("%s/%s tasks complete" % current_task_number, total_task_number) | 21:48 |
| dmsimard | and it'd print that on top of the output ansible prints on every task output | 21:49 |
| *** huikang has quit IRC | 21:49 | |
| sdake | so you write it to a database directly from ansible? | 21:49 |
| *** huikang has joined #openstack-kolla | 21:49 | |
| *** ravig has quit IRC | 21:50 | |
| dmsimard | that's what ara does, yeah: https://github.com/dmsimard/ara/blob/master/ara/callback/log_ara.py | 21:50 |
| dmsimard | We have sqlalchemy models and the callback adds and commits stuff | 21:50 |
| sdake | plguin idea is alot cleaner then I had | 21:51 |
| sdake | which was to hardcode a number on every task | 21:51 |
| sdake | and use tha to figure out wher ewe were in the deploy | 21:51 |
| dmsimard | the hard part is knowing how many tasks you're going to run | 21:51 |
| dmsimard | which is not really possible | 21:51 |
| sdake | you mean because some are skipped | 21:52 |
| dmsimard | some are skipped, some are dynamic (only included when certain parameters are set) | 21:52 |
| dmsimard | ansible doesn't know ahead of time how many tasks it's going to run | 21:52 |
| *** ravig has joined #openstack-kolla | 21:52 | |
| sdake | i'd be satiified with an estimate - i.e. a hardcode | 21:52 |
| dmsimard | that's why I said ara has a rough amount of knowledge after a playbook after it's been run at least once so it could provide some feedback *maybe* | 21:53 |
| sdake | i dont know how I feel about writin to a database though on the deploy host | 21:53 |
| dmsimard | in the context of kolla it could be sent to the logging container | 21:53 |
| sdake | specifically for a GUI | 21:53 |
| dmsimard | but you don't *need* to write to a database to do stuff like this | 21:53 |
| sdake | yes i just jumped rom cli to gui | 21:54 |
| *** dave-mccowan has quit IRC | 21:54 | |
| sdake | dmsimard is this the ara plulgin https://github.com/dmsimard/ara/blob/master/ara/callback/log_ara.py | 21:56 |
| sdake | or is ther emore to it | 21:56 |
| dmsimard | that's the only ansible plugin we use to "interact" with ansible, yeah | 21:56 |
| dmsimard | the rest of the ara source revolves around the flask application, cli, html, tests, etc. | 21:57 |
| *** alyson_ has quit IRC | 21:57 | |
| sdake | dmsimard these are the plugins? https://github.com/dmsimard/ara/blob/master/ara/callback/log_ara.py#L174 | 21:59 |
| *** vinkman has joined #openstack-kolla | 21:59 | |
| dmsimard | those are the hooks, yeah | 21:59 |
| sdake | ya hooks | 22:00 |
| dmsimard | the actual "plugin" is actually this whole part: https://github.com/dmsimard/ara/blob/master/ara/callback/log_ara.py#L59-L257 | 22:00 |
| dmsimard | there's some simple callbacks upstream that might be easier to look at: https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback | 22:00 |
| sdake | dmsimard you write nice ansible | 22:00 |
| dmsimard | that's python, sir :p | 22:01 |
| sdake | ya thats whta i meant | 22:02 |
| sdake | python | 22:02 |
| sdake | long day | 22:02 |
| wirehead_ | Okat. TIL, here’s how to pronunce MariaDB: https://www.tibetangeeks.com/technologies/database/mariadb/mariadb-info_and_links.txt | 22:02 |
| sdake | dmsimard the plugin directly writes the database (calls mysql api) on the local host? | 22:03 |
| wirehead_ | Also, here’s how to pronounce Kubernetes: https://twitter.com/francesc/status/487412202932936704 | 22:04 |
| dmsimard | sdake: the default config has ara configure and use a sqlite database at ~/.ara/ansible.sqlite | 22:04 |
| sdake | ok sqlite | 22:05 |
| dmsimard | sdake: but you could also configure it to use a mysql database without much effort | 22:05 |
| sdake | but ansible calls sqlite directly? | 22:05 |
| dmsimard | http://ara.readthedocs.io/en/latest/configuration.html#ara-database | 22:05 |
| sdake | i would prefer to communicate with a rest api | 22:05 |
| dmsimard | there's no rest api | 22:05 |
| dmsimard | there are sqlalchemy models | 22:05 |
| sdake | i know in ara | 22:05 |
| dmsimard | and we use those | 22:05 |
| sdake | I am thinking in my own world here ;) | 22:05 |
| dmsimard | if you want to control ansible over an api there are things like tower and semaphore | 22:05 |
| sdake | I am thinking about how to come up with a gui for kolla | 22:05 |
| sdake | semaphore? | 22:06 |
| dmsimard | https://github.com/ansible-semaphore/semaphore | 22:06 |
| sdake | i can't recommend proprietary products for use with kolla | 22:06 |
| dmsimard | I can't vouch for either tower or semaphore or other stuff, I haven't used them | 22:06 |
| dmsimard | I didn't know about semaphore until I was asked about ara vs semaphore :) | 22:07 |
| wirehead_ | I have a halfassed notion, at least for kolla-k8s, of not having the user install ansible at all, but instead running that inside of the cluster in a container. | 22:07 |
| sdake | me either | 22:07 |
| wirehead_ | I haven’t thought of it enough | 22:07 |
| sdake | wirehead_ ya that is already done in kolla-toolbox | 22:07 |
| sdake | wirehead_ just add stuff to toolbox you need | 22:07 |
| sdake | dmsimard what does semaphore look like | 22:08 |
| dmsimard | I have absolutely no clue | 22:08 |
| sdake | dmsimard here is what I got out of ara, just for the sake of sharing | 22:08 |
| sdake | oh righty ou couldn't vouch for i t | 22:08 |
| sdake | ara seems like a nice tool but what Kolla operators (atleast CIOs and sales droids) really want is a nice pretty status reporting GUI with actions attached | 22:09 |
| *** fragatin_ has quit IRC | 22:09 | |
| sdake | so think of a gui with all of kolla's actions | 22:09 |
| dmsimard | ara might have that pretty status reporting gui | 22:09 |
| dmsimard | if actual frontend-competent people would contribute | 22:10 |
| dmsimard | I'm not a UI/UX guy at all :) | 22:10 |
| sdake | whats a frontend competent | 22:10 |
| sdake | you mean gui? | 22:10 |
| *** unicell has quit IRC | 22:10 | |
| *** fragatina has joined #openstack-kolla | 22:10 | |
| sdake | I dont know anything about UI dev | 22:10 |
| dmsimard | yeah, stuff like javascript, ajax, nodejs, react, you know all those frontend buzzwords | 22:10 |
| sdake | the lasst time i wrote html was in 1996 in vi | 22:10 |
| dmsimard | the UI/UX of the web interface needs a ton of work from people that can do proper frontend | 22:11 |
| dmsimard | to make it pretty, functional, easy to browse, stuff like that | 22:11 |
| wirehead_ | I do front-end for fun. The advent of node.js and react actually makes it mostly fun. | 22:13 |
| dmsimard | css, html and javascript makes me sad | 22:14 |
| wirehead_ | And by “fun”, I mostly mean “with enough adequate abstractions layered atop each other such that it’s mostly smooth sailing, kind of like how if you put enough layers of paint on, everything’s smooth" | 22:14 |
| dmsimard | I have the utmost respect for people that can write nice frontend stuff, but I'd rather stay as far away from that kind of development as I can :) | 22:15 |
| dmsimard | Horizon is pretty cool, you know, internal popup dialogs, tabs, automatically refreshes, there's a lot of work behind the scenes and I'm sure it's not easy | 22:16 |
| wirehead_ | My idea of fun is… unusual. | 22:16 |
| *** berendt has quit IRC | 22:20 | |
| *** thumpba_ has quit IRC | 22:27 | |
| *** b_bezak has quit IRC | 22:29 | |
| *** mbound has joined #openstack-kolla | 22:30 | |
| *** b_bezak has joined #openstack-kolla | 22:30 | |
| *** b_bezak_ has joined #openstack-kolla | 22:33 | |
| *** b_bezak has quit IRC | 22:34 | |
| Daviey | sdake: sorry, i went afk | 22:40 |
| *** vinkman has left #openstack-kolla | 22:40 | |
| sdake | Daviey no problem, i forgot your question | 22:42 |
| sdake | if you remember, feel free to ask again | 22:43 |
| *** b_bezak_ has quit IRC | 22:46 | |
| *** b_bezak has joined #openstack-kolla | 22:47 | |
| *** huikang has quit IRC | 22:47 | |
| *** b_bezak has quit IRC | 22:51 | |
| *** harlowja has joined #openstack-kolla | 22:51 | |
| *** daneyon_ has joined #openstack-kolla | 22:58 | |
| *** ravig has quit IRC | 22:58 | |
| *** jrist has quit IRC | 23:01 | |
| *** jrist has joined #openstack-kolla | 23:02 | |
| *** daneyon_ has quit IRC | 23:03 | |
| *** godleon has quit IRC | 23:09 | |
| *** harlowja has quit IRC | 23:10 | |
| *** harlowja has joined #openstack-kolla | 23:10 | |
| *** vhosakot has quit IRC | 23:21 | |
| *** mummer has joined #openstack-kolla | 23:25 | |
| *** mummer has quit IRC | 23:26 | |
| *** thumpba has joined #openstack-kolla | 23:28 | |
| *** b_bezak has joined #openstack-kolla | 23:29 | |
| *** b_bezak has quit IRC | 23:31 | |
| *** b_bezak has joined #openstack-kolla | 23:31 | |
| *** thumpba has quit IRC | 23:33 | |
| *** b_bezak has quit IRC | 23:36 | |
| *** mbound has quit IRC | 23:46 | |
| *** zhiwei has joined #openstack-kolla | 23:49 | |
| sdake | harlowja around? | 23:53 |
| harlowja | sup | 23:53 |
| harlowja | i didn't do it | 23:53 |
| harlowja | lol | 23:53 |
| harlowja | it was that other guy | 23:53 |
| *** rahuls has quit IRC | 23:54 | |
| sdake | harlowja so re godaddy deploying kolla | 23:54 |
| harlowja | soooo | 23:54 |
| sdake | harlowja i ont know if thats a thing or your just playing around | 23:54 |
| *** elemoine_ has quit IRC | 23:54 | |
| sdake | but i'm sure ou hae requirements that are unmet | 23:54 |
| harlowja | soooo i'm working on it being a thing :-P | 23:54 |
| sdake | that would rock :) | 23:54 |
| *** zhiwei has quit IRC | 23:54 | |
| harlowja | i'm working on a internal-spec that is gonna explain how the team will do it | 23:54 |
| sdake | will do which | 23:55 |
| harlowja | but ya, some things that we have that are weird that i can think of (active work in progress that i need to dump) | 23:55 |
| sdake | you mean have a ci/cd pipeline with kolla? | 23:55 |
| harlowja | thats part of it | 23:55 |
| sdake | ok | 23:55 |
| sdake | well you dont have to tell me all your secrets :) | 23:55 |
| harlowja | nah, not secret | 23:55 |
| sdake | but ou hae alot of nodes | 23:55 |
| sdake | i assume | 23:55 |
| harlowja | ya | 23:55 |
| sdake | kolla has been tested at 64 | 23:55 |
| sdake | do you do multi-region or cells? | 23:55 |
| sdake | (for scalability) | 23:56 |
| harlowja | ya, we do (both), just not sure yet how that will work, but its ok | 23:56 |
| harlowja | cells and multi-region :-P | 23:56 |
| sdake | well how I recommend it working is we implement that code upstream | 23:57 |
| harlowja | agreed | 23:57 |
| sdake | and you can carry patches against an internal stable branch or some other option | 23:57 |
| harlowja | some other random thinkings | 23:57 |
| harlowja | we have patches :-/ | 23:57 |
| sdake | patches for openstack or patches for kolla | 23:57 |
| harlowja | so do people typically say point kolla at internal git repos? | 23:57 |
| harlowja | for openstack | 23:57 |
| *** elemoine_ has joined #openstack-kolla | 23:58 | |
| harlowja | and/or some mixed mode of not patched repos (which are == public ones) | 23:58 |
| sdake | in your cae i'd recommend having your ci/cd pipeline build images using a custom list of git repos | 23:58 |
| harlowja | right | 23:58 |
| sdake | that works if your bulding rom source | 23:58 |
| harlowja | we also have these other weird things (that yes, i'll eventually get the folks to get rid of) | 23:58 |
| harlowja | that are called waffles | 23:58 |
| harlowja | i think its a rackspace originated concept | 23:58 |
| sdake | would you like fries wit hthat | 23:58 |
| harlowja | supersize it thx | 23:58 |
| sdake | ;-) | 23:59 |
| harlowja | but they are basically middleware components (wsgi middleware) | 23:59 |
| harlowja | that do things with the request before it say gets to nova | 23:59 |
| harlowja | one example | 23:59 |
| wirehead_ | Oh those things. | 23:59 |
| sdake | is it like a rate limiter? | 23:59 |
| harlowja | not really | 23:59 |
| harlowja | mostly used for validation before a request gets to nova/other | 23:59 |
| sdake | so it gets a requet and doews what with it? | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
