| *** eaguilar has joined #openstack-kolla | 00:01 | |
| *** yuanying_ has joined #openstack-kolla | 00:05 | |
| *** Pavo has joined #openstack-kolla | 00:06 | |
| *** yuanying has quit IRC | 00:06 | |
| *** dixiaoli has joined #openstack-kolla | 00:09 | |
| *** dixiaoli has quit IRC | 00:14 | |
| Seyong | rwellum sbezverk: Hello, with k8s 1.7.0, kolla-kubernetes deploying works as well? I followed deployment-guide.rst but failed. Just want to know anybody succeed or not | 00:16 |
|---|---|---|
| *** yangyape_ has quit IRC | 00:17 | |
| *** yangyapeng has joined #openstack-kolla | 00:17 | |
| sbezverk | Seyong : 1.7.0 is actively tested at the gate, I check and most of gate jobs were ok | 00:17 |
| Seyong | just curious, the document is guiding that k8s 1.6.3 has a bug and giving workround fix... is the doc. uptodated? | 00:22 |
| *** yangyapeng has quit IRC | 00:22 | |
| *** manheim has quit IRC | 00:26 | |
| sbezverk | Seyong : most likely no, that bug was fixed in 1.6.4 | 00:29 |
| *** yuanying has joined #openstack-kolla | 00:31 | |
| *** yuanying_ has quit IRC | 00:31 | |
| duonghq | inc0, ping | 00:34 |
| duonghq | vhosakot, still there? | 00:36 |
| *** zhurong has joined #openstack-kolla | 00:41 | |
| *** unicell has quit IRC | 00:48 | |
| *** yuanying has quit IRC | 00:49 | |
| kolla-slack | <inc0> What's up? | 00:53 |
| *** xinliang has quit IRC | 00:53 | |
| *** emccormick has quit IRC | 00:53 | |
| kolla-slack | <inc0> Duonghq | 00:53 |
| duonghq | inc0, would to have a talk about slim Kolla image beside of gate session? | 00:56 |
| kolla-slack | <inc0> Summit talk? I'd love to see it, but I'm going to pass on presenting | 01:01 |
| kolla-slack | <inc0> I'll also have kolla project update and onboarding | 01:01 |
| *** yuanying has joined #openstack-kolla | 01:02 | |
| *** xinliang has joined #openstack-kolla | 01:06 | |
| *** xinliang has joined #openstack-kolla | 01:06 | |
| *** dixiaoli has joined #openstack-kolla | 01:09 | |
| *** gfidente has quit IRC | 01:18 | |
| *** yangyapeng has joined #openstack-kolla | 01:19 | |
| *** huzhengchuan has joined #openstack-kolla | 01:21 | |
| *** iniazi_ has joined #openstack-kolla | 01:22 | |
| *** Pavo has quit IRC | 01:23 | |
| *** yangyapeng has quit IRC | 01:24 | |
| *** iniazi has quit IRC | 01:25 | |
| *** zhubingbing has joined #openstack-kolla | 01:26 | |
| *** Pavo has joined #openstack-kolla | 01:26 | |
| *** goldyfruit has joined #openstack-kolla | 01:27 | |
| duonghq | inc0, yes, Sydney summit, I'm thinking about this topic, it'd be better if you can be co-speaker | 01:27 |
| *** MarginHu has joined #openstack-kolla | 01:28 | |
| *** yangyapeng has joined #openstack-kolla | 01:28 | |
| zhubingbing | hi guys | 01:29 |
| sdake | sup zhubingbing | 01:29 |
| zhubingbing | Sup sdake | 01:29 |
| sdake | planet on fire | 01:29 |
| duonghq | zhubingbing, sdake do you have plan go to next PTG? | 01:30 |
| sdake | yes | 01:30 |
| zhubingbing | I want to go | 01:30 |
| duonghq | nice | 01:30 |
| zhubingbing | but | 01:30 |
| zhubingbing | nit | 01:30 |
| zhubingbing | but | 01:30 |
| zhubingbing | ;( | 01:30 |
| zhubingbing | visa does not necessarily come down | 01:31 |
| duonghq | I applied for TSP again, wait for result | 01:31 |
| duonghq | ah, visa... | 01:31 |
| *** ducttape_ has joined #openstack-kolla | 01:32 | |
| *** yangyapeng has quit IRC | 01:33 | |
| *** caowei has joined #openstack-kolla | 01:34 | |
| *** yangyapeng has joined #openstack-kolla | 01:34 | |
| *** ducttape_ has quit IRC | 01:37 | |
| *** yangyape_ has joined #openstack-kolla | 01:39 | |
| *** yangyapeng has quit IRC | 01:39 | |
| *** ducttape_ has joined #openstack-kolla | 01:42 | |
| *** caoyuan has joined #openstack-kolla | 01:42 | |
| *** ducttape_ has quit IRC | 01:46 | |
| *** schwicht has quit IRC | 01:49 | |
| *** ducttape_ has joined #openstack-kolla | 01:49 | |
| *** cuongnv has joined #openstack-kolla | 01:49 | |
| *** ducttape_ has quit IRC | 01:54 | |
| *** yuanying has quit IRC | 01:57 | |
| *** yuanying has joined #openstack-kolla | 02:00 | |
| Pavo | anyone here? have a question on custom repos | 02:00 |
| jascott1 | Pavo i am here but no idea if I can help :| | 02:01 |
| *** mattmceuen has joined #openstack-kolla | 02:01 | |
| jascott1 | whats up? | 02:02 |
| Pavo | well my question is when using custom repos for building images, does it have to be a url in a normal repo format? | 02:02 |
| Pavo | or just a url likecentos/7/os/x86_64/Packages/ | 02:02 |
| *** caoyuan has quit IRC | 02:04 | |
| Pavo | also getting python errors with kolla-build | 02:04 |
| duonghq | Pavo, I'm here, | 02:05 |
| duonghq | I use custom repos on day basis | 02:06 |
| Pavo | sweet | 02:06 |
| Pavo | duonghq whats the url format need to be like? | 02:06 |
| Pavo | this is what I have | 02:06 |
| Pavo | rpm_setup_config = http://192.168.1.36:8080/centos/7/os/x86_64/Packages/ | 02:06 |
| Pavo | would that work? | 02:07 |
| vhosakot | duonghq: yes, inc0 and I discussed today that upgrades in gate would be a nice talk. we ( inc0, you and I ) can submit one if you're up for it :) | 02:07 |
| Pavo | also docs https://docs.openstack.org/kolla/latest/image-building.html very beginning isn't working | 02:08 |
| vhosakot | Pavo, rpm_setup_config per https://github.com/openstack/kolla/blob/master/doc/image-building.rst#custom-repos is for custom binary images... if you want to build custom source images, rwellum has some work and documented the steps in https://etherpad.openstack.org/p/kolla-kubernetes_with_your_own_source_images. | 02:08 |
| Pavo | says to Generating kolla-build.conf do pip install tox and then tox -e genconfig | 02:08 |
| Pavo | doesn't work | 02:08 |
| duonghq | vhosakot, I think Pavo is talking about CentOS custom repo? | 02:09 |
| Pavo | error ERROR: toxini file 'tox.ini' not found | 02:09 |
| Pavo | vhosakot yeah I always do binary | 02:09 |
| Pavo | not source | 02:09 |
| Pavo | can not trust source | 02:09 |
| vhosakot | Pavo: ah, cool | 02:09 |
| duonghq | vhosakot, upgrade gate is very nice, and it's quite mandatory when I finish rolling upgrade ps (trying to test it), and I think rolling upgrade is a good use-case for the gate | 02:10 |
| Pavo | duonghq any idea on the Generating kolla-build.conf error? | 02:10 |
| *** manheim has joined #openstack-kolla | 02:10 | |
| duonghq | Pavo, no, I haven't gotten this error :( | 02:10 |
| Pavo | hmmm | 02:10 |
| duonghq | Pavo, about the repo: you can use create_repo package | 02:11 |
| duonghq | it'll build the repo metadata | 02:11 |
| kolla-slack | <inc0> Pavo worst case scenario you can just write ini file with options | 02:11 |
| kolla-slack | <inc0> No need to generate conf really | 02:11 |
| kolla-slack | <inc0> Only to make commented defaults | 02:11 |
| Pavo | inc0 ini file? | 02:11 |
| kolla-slack | <inc0> Well build.conf | 02:11 |
| Pavo | the toc.ini file? | 02:12 |
| kolla-slack | <inc0> No, actual build.conf | 02:12 |
| *** caoyuan has joined #openstack-kolla | 02:12 | |
| kolla-slack | <inc0> It's not mandatory | 02:12 |
| Pavo | see I tried making the kolla-build.conf located in /etc/kolla/ | 02:12 |
| Pavo | and kolla-build gives python error | 02:12 |
| kolla-slack | <inc0> What error is that? | 02:13 |
| Pavo | inc0 http://paste.openstack.org/show/614520/ | 02:13 |
| kolla-slack | <inc0> Can you paste your conf? | 02:14 |
| Pavo | inc0 sure and this is the directory tree http://paste.openstack.org/show/614521/ | 02:14 |
| *** manheim has quit IRC | 02:14 | |
| Pavo | inc0 only thing I have in the kolla-build.conf is rpm_setup_config = http://192.168.1.36:8080/centos/7/os/x86_64/Packages/ | 02:14 |
| vhosakot | Pavo: does /etc/kolla/kolla-build.conf have the [DEFAULT] section? | 02:14 |
| Pavo | no | 02:15 |
| Pavo | let me add it | 02:15 |
| vhosakot | Pavo: I think it must be under the [DEFAULT] section :) | 02:15 |
| Pavo | ok | 02:15 |
| Pavo | ok guess that worked | 02:15 |
| vhosakot | inc0: duonghq: should I create an enterpad for the upgrade talk for Sydney? | 02:15 |
| vhosakot | Pavo: cool | 02:15 |
| Pavo | now duonghq getting rpm setup issue with the url I used | 02:16 |
| duonghq | vhosakot, please | 02:16 |
| Pavo | duonghq RPM setup must be provided as .rpm or .repo files. Attempted configuration was http://192.168.1.36:8080/centos/7/os/x86_64/Packages/ | 02:16 |
| vhosakot | duonghq: cool.. will do | 02:16 |
| Pavo | duonghq got an example kolla-build.conf with rpm url for custom repo? | 02:17 |
| duonghq | vhosakot, it's depend on you and inc0 if both of you'd like to keep it as private or feel free to public before we submit CFP | 02:17 |
| duonghq | Pavo, hmm, I used template override | 02:18 |
| kolla-slack | <inc0> I don't care if it's public | 02:18 |
| Pavo | duonghq hmmm | 02:18 |
| vhosakot | duonghq: inc0? what do you mean public vs private? all talks will be be public after voting starts anyway :) | 02:18 |
| kolla-slack | <inc0> Duong if you want to co present with us, I'd be happy to have you there | 02:18 |
| duonghq | vhosakot, I mean the etherpad | 02:19 |
| duonghq | inc0, sure, I want to go with you | 02:19 |
| kolla-slack | <inc0> Nothing secret out there. Presentation will be about dockerhub publisher and gates | 02:19 |
| vhosakot | duonghq: ah, the etherpad is usually worked on by whoever is speaking, we can ask the public for feedback sure, but I think the foundation limits only 4 speakers per talk or something unless it is a panel :) | 02:20 |
| kolla-slack | <inc0> 3 of us would be good team;) | 02:20 |
| duonghq | inc0, vhosakot nice, thanks | 02:21 |
| vhosakot | inc0: cool.. shouldn;t the "dockerhub publisher and gates" talk be different than the "rolling upgrades in gate" talk? are, they're the same? | 02:21 |
| duonghq | Pavo, mins, let me gather some information for you | 02:21 |
| Pavo | duonghq thank, would help because listing every single .rpm file would be insane | 02:21 |
| kolla-slack | <inc0> So cool Duong you're in:) let's talk tomorrow about it and make abstract | 02:21 |
| kolla-slack | <inc0> We'll think of something | 02:22 |
| duonghq | vhosakot, I think rolling upgrade can be one use-case for "upgrade gate", not to put in title | 02:22 |
| *** jtriley has joined #openstack-kolla | 02:22 | |
| duonghq | inc0, sure | 02:23 |
| duonghq | Pavo, https://wiki.centos.org/HowTos/CreateLocalRepos | 02:24 |
| Pavo | duonghq I have a local repo built | 02:24 |
| vhosakot | duonghq: right right.. I was thinking should we combine "dockerhub publisher and gates" and "upgrades in gate" as a single talk? or, two separate talks? | 02:24 |
| duonghq | and you can configure an http server point to it | 02:24 |
| Pavo | duonghq thats not the issue | 02:24 |
| vhosakot | inc0: what do you think? should we combine "dockerhub publisher and gates" and "upgrades in gate" as a single talk? or, two separate talks? | 02:25 |
| Pavo | its the way kolla-build.conf reads the address | 02:25 |
| duonghq | vhosakot, it'd better if 2 topis is "dockerhub publisher" and "upgrade gate"? | 02:25 |
| Pavo | the docs say The repos are accepted as a list of comma separated values and can be in the form of .repo, .rpm, or a url. but there is no example of url | 02:25 |
| vhosakot | duonghq: that is what I think too as the dockerhub publisher+gates talk does not cover upgrades.. cool | 02:25 |
| *** mattmceuen has quit IRC | 02:26 | |
| duonghq | Pava, vhosakot. sorry, I need talk a short walk around for refresh my brain | 02:27 |
| duonghq | *take a short walk | 02:27 |
| duonghq | brb | 02:27 |
| duonghq | 5mins | 02:27 |
| vhosakot | duonghq: or a beer? ;) | 02:27 |
| duonghq | no, it's my working time here | 02:27 |
| duonghq | :P | 02:27 |
| duonghq | beer or whisky is a good option in other time | 02:28 |
| vhosakot | duonghq: inc0: I'd see a talk about upgrades as a talk that shows how kolla can be used to upgrade OpenStack (both minor release-to-minor release, and major release-to-major release) in an automated, rolling way with minimum or no service interruption, and the talk can also show how our kolla gate can be used as a sample to implement rolling upgrade in any CI/CD gate internally in any company. | 02:29 |
| * duonghq back | 02:36 | |
| duonghq | vhosakot, it's another topic? | 02:36 |
| duonghq | Pavo, still here? | 02:37 |
| vhosakot | duonghq: that is what I'm thkinking. the "upgrade talk" must be separate from "dockerhub publisher+gate" talk.. | 02:37 |
| vhosakot | inc0: what do you think? should we combine "dockerhub publisher and gates" and "upgrades in gate" as a single talk? or, two separate talks? | 02:38 |
| Pavo | duonghq yeap | 02:38 |
| duonghq | vhosakot, hmm, it'll be very interesting topic | 02:38 |
| *** caoyuan has quit IRC | 02:38 | |
| duonghq | Pavo, after run createrepo, notice that it'll generate repodata dir | 02:38 |
| duonghq | the URL should be point to the dir contains the repodata | 02:39 |
| Pavo | ok | 02:39 |
| duonghq | i.e. the repodata's path is /long/long/long/path/repodata -> point URL to /long/long/long/path | 02:39 |
| Pavo | duonghq will try that but I know my local repo works because I use it for normal repo on my centos machine | 02:40 |
| Pavo | duonghq so.... http://192.168.1.36:8080/centos/7/os/x86_64/repodata/ | 02:40 |
| Pavo | I tried only pointing to http://192.168.1.36:8080/centos/7/os/x86_64 | 02:41 |
| Pavo | that doesn't work either | 02:41 |
| Pavo | duonghq same error RPM setup must be provided as .rpm or .repo files. Attempted configuration was http://192.168.1.36:8080/centos/7/os/x86_64 | 02:41 |
| *** MarginHu has quit IRC | 02:43 | |
| duonghq | what is the error? | 02:43 |
| duonghq | you can try to configure repo manual inside an container to test the repo | 02:43 |
| *** MarginHu has joined #openstack-kolla | 02:43 | |
| *** cuongnv has left #openstack-kolla | 02:44 | |
| Pavo | duonghq that is the error | 02:45 |
| Pavo | duonghq http://paste.openstack.org/show/614523/ | 02:46 |
| duonghq | RPM setup must be provided as .rpm or .repo files. | 02:47 |
| duonghq | seem that it doesn't accept url directly | 02:47 |
| Pavo | well another thing in the docs that need to be changed | 02:48 |
| Pavo | duonghq docs say "The build method allows the operator to build containers from custom repos. The repos are accepted as a list of comma separated values and can be in the form of .repo, .rpm, or a url. See examples below." | 02:48 |
| Pavo | duonghq but the example only shows .repo example | 02:49 |
| duonghq | yeah, you can put it in repo file | 02:49 |
| duonghq | I use my repo file | 02:49 |
| *** ducttape_ has joined #openstack-kolla | 02:49 | |
| Pavo | yeah I could but that would also require to put that .repo file in kolla/docker/base directory correct? | 02:52 |
| *** caoyuan has joined #openstack-kolla | 02:53 | |
| *** dave-mccowan has quit IRC | 02:53 | |
| *** ducttape_ has quit IRC | 02:54 | |
| Pavo | ok that worked | 02:55 |
| Pavo | building now | 02:55 |
| duonghq | Pavo, yeah, quite frustrate | 02:59 |
| Pavo | wow there is a ton of 3 party repos | 03:00 |
| Pavo | that aren't part of normal centos repo | 03:00 |
| duonghq | yup, many 3rd party repos already here | 03:00 |
| duonghq | I must create mirror for all of this | 03:00 |
| Pavo | yeah /etc/yum.repos.d/grafana.repo | 03:00 |
| Pavo | and others | 03:00 |
| Pavo | duonghq you have a rysnc script you use? | 03:01 |
| Pavo | would love to get a list of them all | 03:01 |
| Pavo | so I can just make a rsync script | 03:01 |
| duonghq | Pavo, I use reposync | 03:02 |
| Pavo | but you still have a list of all the repos right? | 03:02 |
| duonghq | I grad trouble repo and get its url | 03:02 |
| duonghq | You mean you want URL list? | 03:02 |
| duonghq | minutes | 03:02 |
| Pavo | yes please | 03:02 |
| *** caowei_ has joined #openstack-kolla | 03:08 | |
| *** mmehan has quit IRC | 03:10 | |
| *** caowei has quit IRC | 03:12 | |
| *** caowei_ is now known as caowei | 03:12 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible master: [WIP] Implement Keystone zero-downtime upgrade https://review.openstack.org/480081 | 03:13 |
| Pavo | duonghq you getting that list of URLs? | 03:13 |
| duonghq | Pavo, mins, I quite busy | 03:13 |
| Pavo | duonghq oh ok, mybad | 03:13 |
| Pavo | duonghq can I PM you? | 03:14 |
| duonghq | Pavo, http://paste.openstack.org/show/614525/ | 03:15 |
| duonghq | yes | 03:15 |
| duonghq | Pavo, http://paste.openstack.org/show/614526/ _> I cat * my repo dir | 03:16 |
| Pavo | oh nvm was gonna PM you with my email address so you could just sent it to me later when you had time | 03:16 |
| Pavo | but that works | 03:16 |
| Pavo | thanks | 03:16 |
| duonghq | hope that it fine, with one exception | 03:17 |
| duonghq | the delorean repo | 03:17 |
| Pavo | no need for that | 03:17 |
| Pavo | so... most of it looks like its just centos repo | 03:18 |
| Pavo | with a few other repos | 03:18 |
| duonghq | ya | 03:18 |
| Pavo | ceph, docker, mariadb, elastic and kubernetes | 03:19 |
| duonghq | mainly delorean, epel, treasuredata (for fluentd) | 03:19 |
| duonghq | docker is for my own purpose | 03:19 |
| duonghq | not related to kolla | 03:19 |
| Pavo | and this allows you to completely build all kolla images that are centos based? | 03:20 |
| duonghq | mariadb is most painful repo when create mirror | 03:20 |
| Pavo | without having any internet | 03:20 |
| duonghq | iirc, it should be enough | 03:20 |
| Pavo | ok | 03:20 |
| duonghq | not sure, I still use Internet as backup plan when I forget synchronize my mirror | 03:20 |
| Pavo | yeah guess I'll findout what doesn't build | 03:21 |
| duonghq | and please let me know if you found one, | 03:21 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add designate dashboard to horizon https://review.openstack.org/478158 | 03:22 |
| *** MarginHu has quit IRC | 03:23 | |
| *** MarginHu has joined #openstack-kolla | 03:24 | |
| *** goldyfruit has quit IRC | 03:24 | |
| *** goldyfruit has joined #openstack-kolla | 03:25 | |
| *** vhosakot has quit IRC | 03:26 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix keystone-paste.ini custom file management https://review.openstack.org/480608 | 03:27 |
| *** vhosakot has joined #openstack-kolla | 03:27 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Use proper variable in horizon copy policy task https://review.openstack.org/480649 | 03:27 |
| *** vhosakot has quit IRC | 03:27 | |
| spsurya | morning | 03:27 |
| *** vhosakot has joined #openstack-kolla | 03:27 | |
| *** vhosakot has quit IRC | 03:27 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Move mDNS to network nodes https://review.openstack.org/468534 | 03:34 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix skydive analyzer "etcd" config https://review.openstack.org/473723 | 03:40 |
| *** eaguilar has quit IRC | 03:40 | |
| *** Pavo has quit IRC | 03:46 | |
| *** mdnadeem has joined #openstack-kolla | 03:47 | |
| *** itlinux has joined #openstack-kolla | 03:56 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Fix service_type of Octavia https://review.openstack.org/474174 | 04:04 |
| duonghq | morning spsurya | 04:09 |
| openstackgerrit | Merged openstack/kolla-ansible master: Modify the configuration file of telegraf https://review.openstack.org/465446 | 04:11 |
| openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Fix Grafana config file template to use variables https://review.openstack.org/462161 | 04:12 |
| *** ipsecguy_ has joined #openstack-kolla | 04:17 | |
| *** janki has joined #openstack-kolla | 04:18 | |
| *** ipsecguy has quit IRC | 04:18 | |
| *** jtriley has quit IRC | 04:25 | |
| *** goldyfruit has quit IRC | 04:45 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla master: change fernet-key repository permissions to 750 https://review.openstack.org/480830 | 04:47 |
| *** caowei_ has joined #openstack-kolla | 04:58 | |
| *** dixiaoli has quit IRC | 04:58 | |
| *** caowei has quit IRC | 05:02 | |
| *** caowei_ is now known as caowei | 05:02 | |
| *** yeyon has joined #openstack-kolla | 05:05 | |
| spsurya | sup duonghq | 05:07 |
| *** dixiaoli has joined #openstack-kolla | 05:11 | |
| *** cargonza_ has joined #openstack-kolla | 05:13 | |
| *** unicell has joined #openstack-kolla | 05:17 | |
| *** dosaboy_ has joined #openstack-kolla | 05:20 | |
| *** skramaja has joined #openstack-kolla | 05:21 | |
| *** harbie_ has joined #openstack-kolla | 05:21 | |
| *** harlowja has quit IRC | 05:22 | |
| *** cargonza has quit IRC | 05:22 | |
| *** harbie has quit IRC | 05:22 | |
| *** devananda has quit IRC | 05:22 | |
| *** dosaboy has quit IRC | 05:22 | |
| *** harbie_ is now known as harbie | 05:22 | |
| *** harbie has quit IRC | 05:22 | |
| *** harbie has joined #openstack-kolla | 05:22 | |
| *** cargonza_ is now known as cargonza | 05:22 | |
| *** unicell1 has joined #openstack-kolla | 05:23 | |
| *** unicell has quit IRC | 05:23 | |
| *** prateek has joined #openstack-kolla | 05:24 | |
| *** mnaser has quit IRC | 05:24 | |
| *** pc_m has quit IRC | 05:25 | |
| *** calbers has quit IRC | 05:25 | |
| yeyon | systemctl docker start docker | 05:27 |
| yeyon | error message:(code =exited,status=1/FAILURE) | 05:27 |
| *** pc_m has joined #openstack-kolla | 05:27 | |
| *** calbers has joined #openstack-kolla | 05:27 | |
| yeyon | why?? | 05:27 |
| *** prateek has quit IRC | 05:28 | |
| *** prateek_ has joined #openstack-kolla | 05:30 | |
| *** prateek_ is now known as 7GHABESXC | 05:30 | |
| *** prateek_ has joined #openstack-kolla | 05:30 | |
| *** prateek_ has quit IRC | 05:30 | |
| *** mnaser has joined #openstack-kolla | 05:34 | |
| *** sgordon has quit IRC | 05:41 | |
| *** devananda has joined #openstack-kolla | 05:44 | |
| *** dgonzalez has quit IRC | 05:44 | |
| *** dgonzalez has joined #openstack-kolla | 05:46 | |
| *** sgordon has joined #openstack-kolla | 05:50 | |
| *** itlinux has quit IRC | 05:58 | |
| *** mcnanci has quit IRC | 06:01 | |
| *** calbers has quit IRC | 06:01 | |
| *** calbers has joined #openstack-kolla | 06:04 | |
| *** tovin07_ has joined #openstack-kolla | 06:17 | |
| *** zhubingbing_ has joined #openstack-kolla | 06:18 | |
| *** jascott1_ has joined #openstack-kolla | 06:19 | |
| *** ddyer2 has joined #openstack-kolla | 06:20 | |
| *** zhubingbing has quit IRC | 06:21 | |
| *** jascott1 has quit IRC | 06:21 | |
| *** ddyer has quit IRC | 06:21 | |
| *** manheim has joined #openstack-kolla | 06:37 | |
| *** manheim has quit IRC | 06:43 | |
| *** gfidente has joined #openstack-kolla | 06:44 | |
| *** gfidente has quit IRC | 06:44 | |
| *** gfidente has joined #openstack-kolla | 06:44 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla-ansible master: delete delimiter from init fernet condition check https://review.openstack.org/480853 | 06:52 |
| *** shardy has joined #openstack-kolla | 07:06 | |
| *** rwsu has quit IRC | 07:06 | |
| *** janki has quit IRC | 07:12 | |
| *** janki has joined #openstack-kolla | 07:12 | |
| *** mgoddard_ has joined #openstack-kolla | 07:14 | |
| *** mandre has quit IRC | 07:18 | |
| *** mandre has joined #openstack-kolla | 07:30 | |
| *** reidrac has joined #openstack-kolla | 07:35 | |
| *** manheim has joined #openstack-kolla | 07:43 | |
| *** mgoddard_ has quit IRC | 07:48 | |
| openstackgerrit | zhubingbing proposed openstack/kolla-ansible stable/ocata: Add option ha_vrrp_health_check_interval for l3 https://review.openstack.org/480888 | 07:49 |
| *** egonzalez has joined #openstack-kolla | 07:50 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla master: Fix searchlight_api container starting issue https://review.openstack.org/480743 | 07:54 |
| openstackgerrit | Merged openstack/kolla master: Migrate cinder-api under apache https://review.openstack.org/463535 | 07:55 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add option ha_vrrp_health_check_interval for l3 https://review.openstack.org/475614 | 07:56 |
| zhubingbing_ | sup egonzalez | 07:57 |
| egonzalez | zhubingbing_, sup | 07:57 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Migrate cinder-api under apache https://review.openstack.org/471753 | 07:59 |
| iniazi_ | is there kolla documents reorg going on? kolla-kubernetes and links from kolla docs are not working (or docs moving out of openstack.org?) | 08:00 |
| iniazi_ | i'm new to kolla so it may have been old news | 08:01 |
| egonzalez | iniazi_, which links? all openstack docs are being worked atm | 08:03 |
| openstackgerrit | Merged openstack/kolla-ansible master: Remove elasticsearch_protocol unused variable https://review.openstack.org/480181 | 08:03 |
| openstackgerrit | jangseon ryu proposed openstack/kolla-ansible master: deploy failed when didn't defined node_config_directory https://review.openstack.org/480891 | 08:07 |
| *** jascott1_ has quit IRC | 08:10 | |
| *** jascott1 has joined #openstack-kolla | 08:10 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible master: [WIP] Implement Keystone zero-downtime upgrade https://review.openstack.org/480081 | 08:10 |
| *** jascott1 has quit IRC | 08:10 | |
| duonghq | egonzalez, ping | 08:11 |
| *** jascott1 has joined #openstack-kolla | 08:11 | |
| egonzalez | duonghq, pong | 08:12 |
| duonghq | egonzalez, https://review.openstack.org/#/c/480081/ <<< I draft one strategy in the ps | 08:13 |
| duonghq | for zero-downtime upgrade | 08:13 |
| duonghq | but in my testbeds, the restart keystone* handler isn't triggered | 08:13 |
| duonghq | do you have any idea? | 08:13 |
| *** blallau has joined #openstack-kolla | 08:17 | |
| iniazi_ | egonzalez, oh docs.openstack.org/developer/kolla-kubernetes/ and https://docs.openstack.org/developer/kolla/mitaka/ but yeah, if its actively being worked on, than nevermind. was just starting to look into kubernetes install today, but got 404s | 08:18 |
| iniazi_ | well k8s was 404 | 08:19 |
| iniazi_ | actually didn't read the 404 message too carefully. so they are doing a reorg... i guess i'll have to wait a few days | 08:20 |
| iniazi_ | actually that may not even be relevant to me... | 08:20 |
| iniazi_ | well i don't know... anyway | 08:20 |
| *** mgoddard_ has joined #openstack-kolla | 08:21 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible master: [WIP] Implement Keystone zero-downtime upgrade https://review.openstack.org/480081 | 08:23 |
| *** yangyapeng has joined #openstack-kolla | 08:26 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/477363 | 08:31 |
| *** zhurong has quit IRC | 08:32 | |
| reidrac | iniazi_: is not the same, but GH renders the RST files: https://github.com/openstack/kolla-kubernetes/blob/master/doc/source/deployment-guide.rst | 08:37 |
| egonzalez | duonghq, i'm not sure whats happening there | 08:37 |
| reidrac | far from perfect, but readable | 08:37 |
| egonzalez | I dont fully understand the strategy plugins yet | 08:37 |
| duonghq | egonzalez, even without the strategy plugin, in my testbed, the handler isn't invoked | 08:38 |
| duonghq | I'm trying to revert and run upgrade again | 08:38 |
| *** zhurong has joined #openstack-kolla | 08:39 | |
| *** yangyapeng has quit IRC | 08:50 | |
| mandre | egonzalez, duonghq: hi! can I get some eyes on https://review.openstack.org/480200? | 08:56 |
| *** tvignaud has quit IRC | 08:57 | |
| iniazi_ | reidrac, thanks, i'll look at that. | 09:01 |
| mandre | thx egonzalez! | 09:05 |
| *** caoyuan has quit IRC | 09:08 | |
| *** tvignaud has joined #openstack-kolla | 09:11 | |
| *** blallau has quit IRC | 09:14 | |
| *** blallau has joined #openstack-kolla | 09:15 | |
| *** magicboiz has quit IRC | 09:16 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Migrate cinder-api under apache https://review.openstack.org/471753 | 09:16 |
| openstackgerrit | Martin André proposed openstack/kolla master: Make set_configs.py work with symlinks https://review.openstack.org/480200 | 09:20 |
| *** bmotz has joined #openstack-kolla | 09:22 | |
| *** lyang__ has quit IRC | 09:28 | |
| openstackgerrit | Pavel Gluschak (scsnow) proposed openstack/kolla-ansible master: Make cinder-backup service optional https://review.openstack.org/464620 | 09:29 |
| *** lyang__ has joined #openstack-kolla | 09:30 | |
| *** dciabrin has quit IRC | 09:35 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Migrate cinder-api under apache https://review.openstack.org/471753 | 09:42 |
| *** shardy is now known as working_shardy | 09:44 | |
| *** igordcard has quit IRC | 09:48 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix field data is too large https://review.openstack.org/476367 | 09:48 |
| *** igordcard has joined #openstack-kolla | 09:48 | |
| *** dciabrin has joined #openstack-kolla | 09:49 | |
| *** dixiaoli has quit IRC | 09:51 | |
| *** duonghq has quit IRC | 09:53 | |
| *** duonghq has joined #openstack-kolla | 09:55 | |
| openstackgerrit | Merged openstack/kolla stable/ocata: Fix apache configuration for ceilometer_api https://review.openstack.org/480088 | 09:57 |
| *** zhubingbing__ has joined #openstack-kolla | 09:59 | |
| *** tovin07_ has quit IRC | 10:05 | |
| *** caowei has quit IRC | 10:08 | |
| *** caowei has joined #openstack-kolla | 10:09 | |
| *** zhubingbing__ has quit IRC | 10:10 | |
| openstackgerrit | Merged openstack/kolla master: Fix searchlight_api container starting issue https://review.openstack.org/480743 | 10:10 |
| *** MarginHu has quit IRC | 10:11 | |
| *** jascott1 has quit IRC | 10:12 | |
| *** sambetts|afk is now known as sambetts | 10:12 | |
| *** jascott1 has joined #openstack-kolla | 10:12 | |
| *** jascott1 has quit IRC | 10:17 | |
| *** working_shardy is now known as shardy | 10:21 | |
| openstackgerrit | Pavel Gluschak (scsnow) proposed openstack/kolla-ansible master: Don't enable ceilometer-api when database type is gnocchi https://review.openstack.org/480960 | 10:24 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Fix zun log dir permissions https://review.openstack.org/480961 | 10:24 |
| *** japestinho has joined #openstack-kolla | 10:26 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix zun-api logging https://review.openstack.org/480963 | 10:27 |
| *** manheim_ has joined #openstack-kolla | 10:30 | |
| *** manheim has quit IRC | 10:33 | |
| *** yangyape_ has quit IRC | 10:35 | |
| *** rwallner has joined #openstack-kolla | 10:49 | |
| *** caowei has quit IRC | 10:49 | |
| *** pbourke has quit IRC | 10:50 | |
| *** lpetrut has joined #openstack-kolla | 10:50 | |
| *** pbourke has joined #openstack-kolla | 10:51 | |
| *** retr0h has quit IRC | 10:57 | |
| *** retr0h has joined #openstack-kolla | 10:57 | |
| openstackgerrit | Taeha Kim proposed openstack/kolla-ansible master: Correct a syntax error in the tools/cleanup-images https://review.openstack.org/480977 | 10:57 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Remove pep8 exceptions https://review.openstack.org/480978 | 11:06 |
| sdake | morning peeps | 11:10 |
| sdake | iniazi_ the docs need a 1 patch march to make them uptodate on docs.oo as a result of recent changes to infrastructure | 11:12 |
| sdake | iniazi_ as of yet nobody has published any update to trigger the new docs job bulid | 11:12 |
| *** rwsu has joined #openstack-kolla | 11:33 | |
| manheim_ | hi | 11:34 |
| manheim_ | is it possible not to give any interface for neutron_external_interface? | 11:34 |
| manheim_ | coz I'll want to do a NAT from the management network | 11:35 |
| manheim_ | i'm NIC limited | 11:35 |
| *** jtriley has joined #openstack-kolla | 11:38 | |
| *** schwicht has joined #openstack-kolla | 11:39 | |
| *** ducttape_ has joined #openstack-kolla | 11:41 | |
| *** ansmith has quit IRC | 11:44 | |
| *** ducttape_ has quit IRC | 11:46 | |
| *** jtriley has quit IRC | 11:51 | |
| *** mattmceuen has joined #openstack-kolla | 11:55 | |
| *** tumble has joined #openstack-kolla | 11:57 | |
| *** yangyapeng has joined #openstack-kolla | 12:02 | |
| rwellum | Seyong: Any more luck with kolla-k8s? I know the deployment guide needs a couple of updates. | 12:06 |
| rwellum | manheim_: my understanding is 'no'. Neutron needs a NIC with either an unused ip address or no ip address. Can you use the second scenario? | 12:09 |
| *** emccormick has joined #openstack-kolla | 12:12 | |
| openstackgerrit | Bradley Jones proposed openstack/kolla-kubernetes master: (WIP) Deploy playbook testing with nova PV https://review.openstack.org/479826 | 12:14 |
| *** oanson has quit IRC | 12:15 | |
| *** oanson has joined #openstack-kolla | 12:20 | |
| *** goldyfruit has joined #openstack-kolla | 12:21 | |
| openstackgerrit | Dennis Mueller proposed openstack/kolla-ansible master: Added neutron metering-agent support https://review.openstack.org/445858 | 12:22 |
| *** jbadiapa has quit IRC | 12:25 | |
| *** rhallisey has joined #openstack-kolla | 12:28 | |
| *** hrw has quit IRC | 12:30 | |
| *** hrw has joined #openstack-kolla | 12:32 | |
| *** ansmith has joined #openstack-kolla | 12:35 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix bandit tests directory https://review.openstack.org/481016 | 12:42 |
| *** mdnadeem has quit IRC | 12:42 | |
| spsurya | manheim_: you can create one more NIC | 12:48 |
| spsurya | for neutron | 12:48 |
| *** mattmceuen has quit IRC | 12:48 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix bandit tests directory https://review.openstack.org/481016 | 12:51 |
| manheim_ | i've added one more NIC, yes | 12:58 |
| *** yeyon has quit IRC | 13:03 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Fixed vmware typos https://review.openstack.org/480297 | 13:08 |
| *** skramaja has quit IRC | 13:09 | |
| *** 7GHABESXC has quit IRC | 13:10 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla-ansible master: delete delimiter from init fernet condition check https://review.openstack.org/480853 | 13:11 |
| *** rwsu has quit IRC | 13:14 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/kolla-ansible master: delete delimiter from init fernet condition check https://review.openstack.org/480853 | 13:15 |
| *** janki has quit IRC | 13:16 | |
| *** janki has joined #openstack-kolla | 13:16 | |
| *** iniazi has joined #openstack-kolla | 13:16 | |
| *** lucasxu has joined #openstack-kolla | 13:19 | |
| *** iniazi_ has quit IRC | 13:20 | |
| *** zhurong has quit IRC | 13:22 | |
| *** ducttape_ has joined #openstack-kolla | 13:23 | |
| *** ducttape_ has quit IRC | 13:27 | |
| *** igordcard has quit IRC | 13:28 | |
| *** igordcard has joined #openstack-kolla | 13:29 | |
| openstackgerrit | Andy Smith proposed openstack/kolla-ansible master: Add support for hybrid messaging backends https://review.openstack.org/468966 | 13:31 |
| *** huzhengchuan has quit IRC | 13:34 | |
| egonzalez | Jeffrey4l, around? mind if change this dir to /etc/kolla/version_check/releases or any other dir? https://github.com/openstack/kolla/blob/master/tools/version-check.py#L41 | 13:36 |
| egonzalez | i'm working on adding bandit tests to all code and complains about being in /tmp | 13:36 |
| *** ducttape_ has joined #openstack-kolla | 13:37 | |
| Jeffrey4l | egonzalez, no. but which folder is better? | 13:37 |
| *** ducttape_ has quit IRC | 13:37 | |
| egonzalez | hrm, not sure, I can add just a # nosec to ignore the warning | 13:38 |
| Jeffrey4l | np, i mean* :) | 13:38 |
| *** ducttape_ has joined #openstack-kolla | 13:38 | |
| egonzalez | because i dont think a git repo belongs to /etc/kolla | 13:38 |
| Jeffrey4l | hrm, another solution is use tox, then we can put the file into .tox folder ;) | 13:42 |
| *** krtaylor has quit IRC | 13:42 | |
| *** bmotz_ has joined #openstack-kolla | 13:45 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix bandit tests directory https://review.openstack.org/481016 | 13:47 |
| egonzalez | but when running version_check without tox will fail | 13:48 |
| Jeffrey4l | yes. | 13:49 |
| *** bmotz has quit IRC | 13:49 | |
| manheim_ | olla nova waiting for nova-compute to come up | 13:50 |
| manheim_ | sorry | 13:50 |
| manheim_ | https://bugs.launchpad.net/kolla/+bug/1668489 | 13:50 |
| openstack | Launchpad bug 1668489 in kolla "failed to deploy because haproxy Waiting for virtual IP to appear" [Medium,Confirmed] - Assigned to Eduardo Gonzalez (egonzalez90) | 13:50 |
| manheim_ | was this fixed? | 13:50 |
| manheim_ | i have this issue | 13:50 |
| manheim_ | the interface is up and has an IP | 13:50 |
| *** blallau has quit IRC | 13:50 | |
| Jeffrey4l | egonzalez, how about use # nosec before we have a better solution? | 13:51 |
| Jeffrey4l | or just use ".release" folder and add .release into gitignore file? | 13:52 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Add bandit check to all python code https://review.openstack.org/481087 | 13:52 |
| *** bmotz_ has quit IRC | 13:55 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Add bandit check to all python code https://review.openstack.org/481087 | 13:55 |
| egonzalez | Jeffrey4l, I like that one | 13:55 |
| egonzalez | better .releases | 13:55 |
| *** krtaylor has joined #openstack-kolla | 13:56 | |
| Jeffrey4l | egonzalez, ok. | 13:56 |
| egonzalez | thanks! | 13:56 |
| egonzalez | manheim_, which release? is rabbitmq running, are there compute nodes actives? | 13:58 |
| manheim_ | it's an AIO | 13:59 |
| manheim_ | ubuntu 16.04 host w latest upgrades, ubuntu source images, stable/ocata repo with 4.0.0 images | 14:00 |
| *** hongbin has joined #openstack-kolla | 14:01 | |
| *** bmotz_ has joined #openstack-kolla | 14:01 | |
| egonzalez | manheim_, ensure nova service-list returns nova-compute binary active, if not check if is actually running or connected to rabbitmq | 14:03 |
| manheim_ | i took out the haproxy for the moment | 14:03 |
| manheim_ | I have another issue now | 14:03 |
| manheim_ | libvirt is not starting | 14:03 |
| *** ducttape_ has quit IRC | 14:11 | |
| *** ducttape_ has joined #openstack-kolla | 14:11 | |
| *** jbadiapa has joined #openstack-kolla | 14:18 | |
| *** jtriley has joined #openstack-kolla | 14:22 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Standardize Keystone domain variables https://review.openstack.org/479287 | 14:34 |
| *** eaguilar has joined #openstack-kolla | 14:36 | |
| jamesbenson | manheim_: regarding the eth's. I use veth here and that seems to have worked... check out my interfaces file here for an idea: https://github.com/JamesOBenson/Kolla-misc/blob/master/tasks/Generate_interfaces.yml | 14:42 |
| manheim_ | ok, ty | 14:43 |
| jamesbenson | we use only 2 ports... one thing I did notice is that eno1:1 for example would fail because ansible has issues with it... but this way worked.... now I just have ceph issues to work out... | 14:44 |
| *** ducttape_ has quit IRC | 14:46 | |
| *** zhubingbing_ has quit IRC | 14:53 | |
| *** eaguilar has quit IRC | 15:01 | |
| *** ducttape_ has joined #openstack-kolla | 15:08 | |
| openstackgerrit | Merged openstack/kolla master: Make set_configs.py work with symlinks https://review.openstack.org/480200 | 15:09 |
| *** ducttape_ has quit IRC | 15:10 | |
| *** ducttape_ has joined #openstack-kolla | 15:10 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Implement blazar Docker images https://review.openstack.org/454745 | 15:18 |
| *** unicell has joined #openstack-kolla | 15:22 | |
| *** unicell1 has quit IRC | 15:23 | |
| jamesbenson | inc0: ping | 15:25 |
| *** eaguilar has joined #openstack-kolla | 15:27 | |
| sdake | kfox1111 quick quesiton about the neutron not restartble problem | 15:30 |
| sdake | kfox1111 wouodl you mind expanding on which pods cannot be restarted - local testing shows neutron-server restarts just fine | 15:31 |
| kfox1111 | morning. | 15:31 |
| kfox1111 | anything that makes a network namespace. | 15:31 |
| kfox1111 | neutron-dhcp-agent, neutron-l3-agent, neutron-openvswitch-agent. | 15:32 |
| *** janki has quit IRC | 15:32 | |
| *** mandre is now known as mandre_away | 15:33 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Use monasca master tarballs https://review.openstack.org/481118 | 15:33 |
| inc0 | good morning | 15:36 |
| *** bmotz_ has quit IRC | 15:37 | |
| inc0 | duonghq so, summit talks | 15:37 |
| inc0 | if you want me to co-present with you, I'll be happy to do so | 15:38 |
| inc0 | only if I happen to have 2 full presentations and kolla project update + onboarding, I'll need a lot of your help guys with preparation | 15:39 |
| jamesbenson | inc0, mind helping pushing this bug? https://review.openstack.org/#/c/477605/6 | 15:41 |
| inc0 | yeah hold on | 15:41 |
| jamesbenson | thanks | 15:41 |
| inc0 | so, I'll explain how you should debug gates | 15:41 |
| inc0 | first of all, click on red gate, you'll be redirected to page like this http://logs.openstack.org/05/477605/6/check/gate-kolla-ansible-dsvm-deploy-multinode-ubuntu-source-ubuntu-xenial-2-node-nv/969ec62/ | 15:42 |
| inc0 | console is stdout of gate, that's first thing to look at | 15:42 |
| jamesbenson | k | 15:43 |
| jamesbenson | looks good there from what I can see | 15:43 |
| jamesbenson | it had some sudo: unable to resolve host ubuntu-xenial-2-node-osic-cloud1-s3700-9582718: Connection timed out | 15:44 |
| jamesbenson | issues | 15:44 |
| inc0 | I see the issue | 15:45 |
| inc0 | so go to logs/ansible | 15:45 |
| inc0 | and click through various logs | 15:45 |
| inc0 | http://logs.openstack.org/05/477605/6/check/gate-kolla-ansible-dsvm-deploy-multinode-ubuntu-source-ubuntu-xenial-2-node-nv/969ec62/logs/ansible/get-logs this one | 15:45 |
| inc0 | failed | 15:45 |
| inc0 | and it failed because one of containers is in Exited state | 15:46 |
| inc0 | http://logs.openstack.org/05/477605/6/check/gate-kolla-ansible-dsvm-deploy-multinode-ubuntu-source-ubuntu-xenial-2-node-nv/969ec62/logs/docker_logs/ this one | 15:46 |
| inc0 | http://logs.openstack.org/05/477605/6/check/gate-kolla-ansible-dsvm-deploy-multinode-ubuntu-source-ubuntu-xenial-2-node-nv/969ec62/logs/kolla/mariadb/mariadb.log and this is actual log | 15:47 |
| inc0 | connection time out | 15:47 |
| inc0 | totally not helpful | 15:47 |
| jamesbenson | yeah, story of my life :-p | 15:47 |
| jamesbenson | that's why I did a recheck the first time.... because a I think 2-3 failed for timeout issues, after recheck, half failed. | 15:48 |
| *** hieulq_ has joined #openstack-kolla | 15:48 | |
| jamesbenson | just issued another recheck this morning, hasn't gone yet. | 15:49 |
| inc0 | maybe sth with infra, not sure, I'll keep looking | 15:49 |
| inc0 | I have meeting now so I'll be back in hour or so | 15:49 |
| jamesbenson | k | 15:49 |
| sdake | kfox1111 thanks for the answer earlier - i missed it - appreciate it | 15:51 |
| kfox1111 | np | 15:53 |
| *** dixiaoli has joined #openstack-kolla | 15:53 | |
| *** itlinux has joined #openstack-kolla | 15:55 | |
| *** dixiaoli has quit IRC | 15:57 | |
| *** mgiles has joined #openstack-kolla | 15:59 | |
| *** david-lyle has joined #openstack-kolla | 16:00 | |
| openstackgerrit | Rich Wellum proposed openstack/kolla master: Add a section on building custom kolla images from source. https://review.openstack.org/481127 | 16:03 |
| *** mgiles has left #openstack-kolla | 16:05 | |
| *** egonzalez has quit IRC | 16:05 | |
| *** manheim_ has quit IRC | 16:05 | |
| *** jbadiapa has quit IRC | 16:07 | |
| *** rwsu has joined #openstack-kolla | 16:12 | |
| *** eaguilar has quit IRC | 16:15 | |
| *** reidrac has quit IRC | 16:33 | |
| sdake | rwellum can you do me a solid and find something simple to fix in the kolla-kubernetes docs to fix this prooblem: https://docs.openstack.org/kolla-kubernetes | 16:34 |
| sdake | could be a typo - could be a junk commit with a revert afterwards - anything will do :) | 16:34 |
| *** sambetts is now known as sambetts|afk | 16:35 | |
| *** jtriley has quit IRC | 16:36 | |
| rwellum | sdake: yes - actually I have a couple of fixups in the bare-metal guide - will that do? | 16:37 |
| *** jbadiapa has joined #openstack-kolla | 16:38 | |
| sdake | rwellum is that in kolla-kubernetes deliverable? | 16:39 |
| sdake | if so yes | 16:39 |
| sdake | (we have a couple bre metal guide) | 16:39 |
| sdake | s | 16:39 |
| *** jtriley has joined #openstack-kolla | 16:40 | |
| rwellum | sdake: kolla-kubernetes/doc/source/deployment-guide.rst | 16:43 |
| *** hieulq_ has quit IRC | 16:43 | |
| sdake | rwellum please subit - would like to fix asap | 16:44 |
| sdake | rwellum even a one liner | 16:44 |
| rwellum | Yes - gimme a minute and will :) | 16:44 |
| sdake | cool :) | 16:44 |
| *** unicell has quit IRC | 16:45 | |
| *** gfidente has quit IRC | 16:53 | |
| openstackgerrit | Rich Wellum proposed openstack/kolla-kubernetes master: Update the deployment guide with a few corrections. https://review.openstack.org/481150 | 16:54 |
| rwellum | sdake: ^^^ | 16:55 |
| sdake | inc0 kfox1111 sbezverk can one of you rubber stmap this docs change - all docs publishing are broken on docs.oo as a result of a change to infra on july 3rd | 16:59 |
| kfox1111 | already done. | 16:59 |
| sdake | thanks | 16:59 |
| kfox1111 | np. | 16:59 |
| sdake | rwellum note i left a note in there recommedning further followup work | 17:00 |
| *** mgoddard_ has quit IRC | 17:02 | |
| *** inc0 has quit IRC | 17:02 | |
| *** inc0 has joined #openstack-kolla | 17:03 | |
| krtaylor | does anyone know of any talk to use kolla builds and maybe kolla-ansible instead of dsvm for infra CI check/gate testing? like a zuul/nodepool container alternative? | 17:04 |
| inc0 | krtaylor: there were whispers here and there | 17:04 |
| inc0 | also to use kolla as devstack alternative (patches for that are already in) | 17:04 |
| openstackgerrit | Merged openstack/kolla-kubernetes master: Update the deployment guide with a few corrections. https://review.openstack.org/481150 | 17:05 |
| inc0 | but nothing concrete | 17:05 |
| *** shardy has quit IRC | 17:07 | |
| *** rwsu has quit IRC | 17:07 | |
| *** rwsu has joined #openstack-kolla | 17:07 | |
| krtaylor | inc0, thanks, I would be interested | 17:07 |
| krtaylor | I've searched mail and irc logs | 17:07 |
| krtaylor | I'll also post to infra and see what happens | 17:07 |
| inc0 | once we have readily available images | 17:08 |
| inc0 | dockerhub + cache | 17:08 |
| inc0 | deployment of gate would be couple times faster | 17:08 |
| sdake | krtaylor we have typically steered clear of that area to avoid the hassle of duplication of effort, parallel development, forcing people to learn ansible+docker to do development work when in many cases devstack works, and finally the shitshow that could errupt as a result :) | 17:09 |
| krtaylor | yes, I have been following that, and the discussion has included gate/check deployment | 17:09 |
| krtaylor | lol, yes! | 17:09 |
| kfox1111 | yeah, but then you get the opposite issue... | 17:09 |
| sdake | krtaylor that said, i think the timing is right to make kolla-ansible a viable development environment for openstack | 17:09 |
| kfox1111 | devs that only work against something users will never use, devstack. | 17:09 |
| krtaylor | yes, the puppet vs ansible discussion... | 17:09 |
| sdake | krtaylor go forth and conquer :) | 17:10 |
| kfox1111 | devs being in their own world causes lots of other issues. :/ | 17:10 |
| sdake | kfox1111 agreed devs view the world through a different lens then opertors | 17:10 |
| kfox1111 | yeah. | 17:10 |
| sdake | kfox1111 one of those lenses is kolla-ansible :) | 17:10 |
| krtaylor | it is really only meant to be a test environment, and kolla-ansible could be so much more flexible | 17:10 |
| inc0 | so now we have wiring in Kolla to make it work like devstack | 17:11 |
| kfox1111 | so if they don't share common deployment tooling, the devs make changes and thinks the world's all rosy, and the ops suffer. :/ | 17:11 |
| inc0 | https://docs.openstack.org/kolla-ansible/latest/kolla-for-openstack-development.html | 17:11 |
| sdake | kfox1111 the only problem with that argument is there is more then one deployment tool :) | 17:11 |
| kfox1111 | yeah. but thats not reson to add yet another deployment tool only for devs. | 17:12 |
| krtaylor | hm, kolla-puppet :) | 17:12 |
| * krtaylor ducks | 17:12 | |
| inc0 | krtaylor: if someone writes that | 17:12 |
| sdake | inc0 that looks good dude | 17:12 |
| sdake | inc0 is that implemented? | 17:12 |
| inc0 | sdake: at least some of services | 17:12 |
| inc0 | and all the wiring | 17:12 |
| sdake | wiring hardest part | 17:12 |
| inc0 | so adding other services is quite easy by now | 17:12 |
| *** harlowja has joined #openstack-kolla | 17:13 | |
| sdake | adding the 50 or so services mostly C&P? | 17:13 |
| inc0 | yeah | 17:13 |
| rwellum | sdake: thanks for the note. I will do that. Should I add the 1.6.3 DOA and remove the work-around - or stagger it? | 17:13 |
| inc0 | and you get devstack which will not ruin your machine | 17:13 |
| krtaylor | ++ | 17:14 |
| inc0 | and is easy to make HA + used on prod | 17:14 |
| inc0 | lots of benefits | 17:14 |
| inc0 | but culture of devstack is pretty entrenched and changing that would take a lot of effort | 17:14 |
| sdake | rwellum i'd add 1.6.3 = DOA an remove workaround in one go | 17:15 |
| rwellum | sdake: ack | 17:15 |
| sdake | inc0 well its not like we force it upon people | 17:15 |
| sdake | inc0 its more like "bulid it they will come" :) | 17:16 |
| *** lpetrut has quit IRC | 17:16 | |
| sdake | what position are we to say that kolla-ansible dev mode is better than devstack | 17:16 |
| inc0 | yes and no, it takes effort to show them this option | 17:16 |
| sdake | ok.. I know devstack destroys your system - but still :) | 17:16 |
| inc0 | it's prod ready | 17:16 |
| inc0 | it's easy to test on multinode scenario | 17:16 |
| inc0 | it's much faster to deploy:) | 17:17 |
| inc0 | there are lots of benefits | 17:17 |
| sdake | sounds like a great summit talk to me | 17:17 |
| krtaylor | agreed, it would be interesting to present a functional comparison...sydney? | 17:17 |
| krtaylor | hehheh | 17:17 |
| sdake | i'd present it but haven't the faintest idea how it works | 17:17 |
| inc0 | indeed it does, pbourke should be one on stage tho, it's his child | 17:17 |
| sdake | krtaylor i thiink presenting comparisons needlessly creates conflict | 17:17 |
| sdake | inc0 cool - hope he will make it to syndey | 17:18 |
| sdake | we should probably see if that is the case so we can et a talk submitted | 17:18 |
| krtaylor | well, I was thinking showing how similar it could be would help the "argument" | 17:18 |
| sdake | krtaylor you used the word right there - argument | 17:18 |
| sdake | ;) | 17:18 |
| sdake | no reason to argue over it | 17:19 |
| krtaylor | I'd implement it in our CI and show as a PoC..alas, if I had 3 more me | 17:19 |
| rwellum | git clone krtaylor | 17:20 |
| rwellum | dammit - didn't work.. | 17:20 |
| krtaylor | lol, I wish | 17:20 |
| krtaylor | well, thanks for the discussion, I'll see if I can drum up a set of folks interested in kicking in a bit of time via email | 17:20 |
| inc0 | would be awesome krtaylor | 17:21 |
| inc0 | I think changing culture of devs to use more opsy prod-like env would be beneficial for OpenStack as whole | 17:21 |
| rwellum | +1 inc0 | 17:21 |
| inc0 | if we figure out how to easily do multinode code editing (nfs mounted to /opt/stack?) that would be even better | 17:22 |
| krtaylor | yeah, thats whats really driving my interest, that and the time factor, it would really speed up our testing | 17:22 |
| inc0 | "here run this 3 commands and you'll have 3 node HA openstack for your development pleasure" | 17:22 |
| rwellum | I've been using my tool to do the same for kolla-k8s environment, and there's a couple of folks from Cinder looking at it now too. | 17:22 |
| rwellum | Containers invalidate the need for dev-stack imo. | 17:23 |
| sdake | well lets not go on an all out offensive against devstackc plz :) | 17:23 |
| sdake | they are our friends | 17:23 |
| inc0 | also as I said, lots of devs uses devstack daily and switching to something else will slow down their dev process | 17:24 |
| rwellum | TBF - containers are doing that to a lot of technologies. | 17:24 |
| inc0 | at least initially | 17:24 |
| rwellum | It's one step backwards to go two steps forward inc0 :) | 17:25 |
| *** jtriley has quit IRC | 17:25 | |
| inc0 | also, while we have it written, I'm sure when we actually be used for dev lots of issues with dev workflow will appear, it'll take us time to get it all right | 17:26 |
| inc0 | so what we need today is volunteers to try it out and develop openstack on kolla-ansible instead of devstack | 17:26 |
| inc0 | and see how successful they are | 17:26 |
| sbezverk | kfox1111: With help of Intel developer, we got multi networks POD in kubernetes running :) has 3 interfaces, 1 - management (flannel), 1 - mapped to Linux bridge, 1 - SRIOV VF interrace | 17:26 |
| kfox1111 | sbezverk: nice. :) | 17:26 |
| kfox1111 | hows ip allocation done on the secondary interfaces? | 17:27 |
| inc0 | sbezverk: who was that? | 17:27 |
| sbezverk | inc0: dude from Ireland Kural | 17:27 |
| inc0 | interesting, I don't know him | 17:27 |
| sbezverk | kfox1111: each interface has its own ipam | 17:28 |
| inc0 | Intel is big... | 17:28 |
| kfox1111 | that would allow for the elimination of yet another use case of vm's over containers. :) | 17:28 |
| kfox1111 | sbezverk: cool. | 17:28 |
| sbezverk | for sriov ipam is optional, basically it is layer 2 pipe | 17:28 |
| kfox1111 | optional would be cool. | 17:28 |
| kfox1111 | is it optional for bridge too? | 17:28 |
| inc0 | bbiaf | 17:29 |
| sbezverk | kfox1111: not sure, will need to test it | 17:29 |
| rwellum | sbezverk and sdake - in the deployment guide, should I bump the k8s version recommended to 1.6.4 or 1.7.0? | 17:31 |
| kfox1111 | has anyone pushed the gate to 1.7.0 yet? | 17:31 |
| *** hieulq has quit IRC | 17:33 | |
| *** daidv has quit IRC | 17:33 | |
| *** duonghq has quit IRC | 17:33 | |
| rwellum | sbezverk: mentioned it was working in the gate | 17:33 |
| *** unicell has joined #openstack-kolla | 17:34 | |
| sdake | rwellum 1.6.1 and 1.6.2 work | 17:34 |
| sdake | rwellum i'd just != 1.6.3 | 17:34 |
| *** ducttape_ has quit IRC | 17:34 | |
| sdake | kfox1111 been recovering from a couple falls - so bsically hands off the laptop for aout 3 weeks | 17:35 |
| sdake | responding to mail and doing the minimum :) | 17:35 |
| jamesbenson | inc0, sorry I was out for a bit, did you check the recheck? all but one | 17:35 |
| sdake | kfox1111 i dont have any idea the status of the k8s gate | 17:35 |
| *** krtaylor has quit IRC | 17:35 | |
| kfox1111 | ok. we shoudl submit a patch that bumps the version to 1.7 and see what happens. | 17:35 |
| jamesbenson | inc0, failed with a no space left on device in the gates? | 17:36 |
| sbezverk | kfox1111: I check gate logs and it uses 1.7.0 | 17:36 |
| *** vhosakot has joined #openstack-kolla | 17:37 | |
| *** jtriley has joined #openstack-kolla | 17:38 | |
| *** manheim has joined #openstack-kolla | 17:41 | |
| rwellum | sdake: I think it's better to bump the min. revision past 1.6.3 (it's currently 1.6.2) - that way we avoid mentioning 1.6.3 at all. If we are DOA/EOL 1.6.3, | 17:42 |
| rwellum | sbezverk: yes because yum install just pulls in latest | 17:42 |
| sdake | 1.6..1 and 1.6.0 also work i think | 17:42 |
| rwellum | sdake: is there any advantage to using it though? | 17:43 |
| sdake | rwellum if you want to state 1.6.4 as a minimum that wfm - bt pleaes indicate 1.6.3 is DOA | 17:43 |
| *** itlinux has quit IRC | 17:43 | |
| rwellum | ok sdake | 17:43 |
| sdake | i know it is duplicative, however, we shouold be extremely clear 1.6.3 is not to be used | 17:43 |
| sdake | inc0 i'm going to have a difficult day ahead | 17:45 |
| sdake | and i had wanted to fix docs.oo | 17:45 |
| sdake | but i dont have time | 17:45 |
| sdake | can you check out the recent comments on openstack-infra | 17:45 |
| sdake | and take it on? it looks like less then 30 mins of work | 17:45 |
| *** manheim has quit IRC | 17:45 | |
| sbezverk | rwellum: I would say 1.6.6 or 1.7.0 is good for the doc | 17:47 |
| *** daidv has joined #openstack-kolla | 17:48 | |
| *** duonghq has joined #openstack-kolla | 17:48 | |
| *** hieulq has joined #openstack-kolla | 17:48 | |
| sdake | sbezverk i think we want to set the minimums we know work | 17:49 |
| sdake | we know 1.6.4 works correct? | 17:49 |
| rwellum | So right now I have 1.6.4, I explicitly mention 1.6.3 is DOA and added a note saying current version 1.7.0 is stable. | 17:50 |
| sdake | duonghq can you check out the openstack infra logs rom about 20 mins ago (search for sdake) and fix docs.oo plz | 17:50 |
| sdake | I'd do so, however, have meetings | 17:50 |
| sdake | duonghq and our docs are not published properly | 17:50 |
| sdake | which is a big problem requiring immedite attention | 17:50 |
| sdake | duonghq hi btw :) | 17:51 |
| sdake | or nayone else out there :) | 17:52 |
| sdake | plz | 17:52 |
| *** eaguilar has joined #openstack-kolla | 17:56 | |
| *** ducttape_ has joined #openstack-kolla | 18:01 | |
| *** ducttape_ has quit IRC | 18:02 | |
| *** ducttape_ has joined #openstack-kolla | 18:03 | |
| *** dave-mccowan has joined #openstack-kolla | 18:03 | |
| *** krtaylor has joined #openstack-kolla | 18:12 | |
| *** jascott1 has joined #openstack-kolla | 18:15 | |
| jascott1 | sdake what up! | 18:15 |
| jascott1 | 114F yikes | 18:16 |
| *** lpetrut has joined #openstack-kolla | 18:20 | |
| *** lpetrut has quit IRC | 18:22 | |
| *** krtaylor has quit IRC | 18:28 | |
| *** lpetrut has joined #openstack-kolla | 18:29 | |
| *** krtaylor has joined #openstack-kolla | 18:33 | |
| *** eaguilar has quit IRC | 18:34 | |
| *** jbadiapa has quit IRC | 18:45 | |
| kfox1111 | jascott1: sorry, I havent responded to you yet. been super busy trying to fix some broken servers for the physics mc right now. | 18:45 |
| kfox1111 | hopefully will be able to get back to you later today. | 18:45 |
| jascott1 | kfox1111 not a problem, i know you are busy busy | 18:47 |
| *** tonanhngo has joined #openstack-kolla | 18:49 | |
| *** eaguilar has joined #openstack-kolla | 18:51 | |
| *** hieulq has quit IRC | 18:51 | |
| *** daidv has quit IRC | 18:51 | |
| *** duonghq has quit IRC | 18:51 | |
| *** cliles_ has quit IRC | 18:52 | |
| *** cliles has joined #openstack-kolla | 18:52 | |
| jamesbenson | manheim_: ping | 18:55 |
| *** itlinux has joined #openstack-kolla | 18:58 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements https://review.openstack.org/474652 | 19:01 |
| *** Pavo has joined #openstack-kolla | 19:03 | |
| *** daidv has joined #openstack-kolla | 19:06 | |
| *** hieulq has joined #openstack-kolla | 19:06 | |
| *** duonghq has joined #openstack-kolla | 19:07 | |
| *** pcaruana has quit IRC | 19:12 | |
| *** ducttape_ has quit IRC | 19:21 | |
| *** ducttape_ has joined #openstack-kolla | 19:22 | |
| *** ducttape_ has quit IRC | 19:24 | |
| *** ducttape_ has joined #openstack-kolla | 19:24 | |
| *** ducttape_ has quit IRC | 19:31 | |
| *** vhosakot has quit IRC | 19:32 | |
| *** Pavo has quit IRC | 19:34 | |
| *** rwallner has quit IRC | 19:35 | |
| dmsimard | sdake: what do you guys need epel for on el7 ? | 19:36 |
| sdake | dmsimard iscsi package of some sort | 19:36 |
| sdake | dmsimard there may be a few other packages | 19:36 |
| sdake | dmsimard i'd really like to remove epel | 19:36 |
| sdake | dmsimard i don't have the fortitude to get the rpm packaging done in rdo | 19:37 |
| dmsimard | sdake: fwiw RDO has successful deployments with zero dependencies on EPEL | 19:37 |
| dmsimard | sdake: and I believe openstack-ansible has done away with their EPEL dependencies as well | 19:37 |
| sdake | dmsimard i hear y - perhaps our iscsi dependency can be changed | 19:37 |
| dmsimard | our dependencies have been self contained since at most liberty | 19:37 |
| sdake | dmsimard i am not a cinder expert | 19:37 |
| sdake | i can barely get it to wor | 19:37 |
| sdake | k | 19:37 |
| sdake | unless using ceph | 19:37 |
| sdake | ceph cinder is a nobrainer | 19:37 |
| sdake | ceph + lvm = painful | 19:37 |
| sdake | rather vm+cinder = painful | 19:38 |
| sdake | its the lvm+cinder that require the scsi package iirc | 19:38 |
| sdake | dmsimard thanks for the note that openstack-ansible moved to centos | 19:38 |
| sdake | dmsimard I didn't know that :) | 19:38 |
| dmsimard | sdake: they didn't move, they just support centos as well as ubuntu | 19:38 |
| *** rwallner has joined #openstack-kolla | 19:38 | |
| sdake | sorry i mispooke thats what i meant | 19:39 |
| dmsimard | They still deploy from source as usual, and they use RDO for dependencies | 19:39 |
| sdake | a few years ago i asked them to do that | 19:39 |
| sdake | the response cn't be repeated :) | 19:39 |
| dmsimard | Someone showed up to do the work :p | 19:39 |
| sdake | always the key factor | 19:39 |
| dmsimard | I think they're happy about it | 19:39 |
| sdake | someone has to show up to do the work | 19:39 |
| sdake | which begs the question, are you showing up to do the epel removal work :) | 19:40 |
| sdake | the actual removal of epel is trivial - 10 mins of work - the dependencies are a different matter | 19:40 |
| dmsimard | sdake: I don't even have the bandwidth to do the things I need to do :P | 19:41 |
| *** ducttape_ has joined #openstack-kolla | 19:41 | |
| dmsimard | sdake: but I do have the time to point it out | 19:42 |
| sdake | dmsimard so what can I hep you with | 19:42 |
| dmsimard | sdake: there actually is something now that I think of it | 19:42 |
| sdake | the other thing i want to do is move to mariadb inside rdo | 19:42 |
| dmsimard | sdake: it already is | 19:42 |
| sdake | not in kolla | 19:42 |
| dmsimard | sdake: I think you mean xtradb or percona | 19:42 |
| sdake | we are using mariadb.org | 19:42 |
| sdake | xtradb is integrated in rdo already | 19:42 |
| dmsimard | sdake: mariadb is already in RDO mirrors, not sure what else you need | 19:43 |
| *** rwallner has quit IRC | 19:43 | |
| dmsimard | http://mirror.centos.org/centos/7/cloud/x86_64/openstack-ocata/common/mariadb-server-galera-10.1.20-1.el7.x86_64.rpm | 19:44 |
| sdake | dmsimard i was just telling you the two changes i want to make to the structure of the kolla containers | 19:48 |
| sdake | remove epel - integrate rdo's version of mariadb | 19:48 |
| dmsimard | sdake: oh | 19:48 |
| sdake | i cannot remove epel without either a) help on cinder implementation without iscsi or b) addition of deps to rdo | 19:49 |
| sdake | mariadb different matter - that is https://review.openstack.org/#/c/468632/ | 19:50 |
| larsks | sdake: which iscsi tools are in epel that aren't in centos? | 20:00 |
| sdake | larsks there is one way to find out | 20:01 |
| sdake | larsks you will have to give me a bit to build the images | 20:01 |
| dmsimard | larsks: scsi-target-utils ? | 20:01 |
| dmsimard | from looking at the cinder-volume dockerfile.. | 20:01 |
| dmsimard | https://github.com/openstack/kolla/blob/master/docker/cinder/cinder-volume/Dockerfile.j2 | 20:02 |
| larsks | dmsimard: ah, thanks, I was looking for iscsi-related packages and wasn't spotting any. | 20:02 |
| larsks | Tricky package naming :) | 20:02 |
| sdake | i'm not sure if that is the only missing dependency we have | 20:02 |
| sdake | and i'm not sure if that dependency is actually necessary for correct operation | 20:02 |
| sdake | last i looked at it was june | 20:03 |
| larsks | sdake: scsi-target-utils is what provides "tgtd", so that looks pretty necessary. | 20:03 |
| sdake | tgtd required for cinder? | 20:04 |
| larsks | sdake: I'm looking at the cinder-volume Dockerfile, which appears to be configuring tgtd. | 20:04 |
| sbezverk | for iscsi-lvm to work we need tgtd, iscsid, targetcli | 20:04 |
| sbezverk | and iscsiadm | 20:04 |
| *** manheim has joined #openstack-kolla | 20:10 | |
| dmsimard | sdake: btw would love to land https://review.openstack.org/#/c/479881/ - it'd be super useful for us. | 20:11 |
| openstackgerrit | Rich Wellum proposed openstack/kolla-kubernetes master: Update deployment guide to use 1.6.4. https://review.openstack.org/481218 | 20:11 |
| dmsimard | larsks: my knowledge of cinder w/ iscsi+lvm is not exactly awesome but we don't require epel in puppet integration tests where it's setup | 20:12 |
| sdake | larsks i am now building all of kolla (with source) to see what fails to build | 20:13 |
| sdake | note source shuld build entirely | 20:13 |
| dmsimard | larsks: https://github.com/openstack/puppet-openstack-integration/blob/master/manifests/cinder.pp#L17 | 20:13 |
| sdake | although i took out epel in the proess | 20:13 |
| sdake | so we can see what deps are missing | 20:14 |
| sdake | dmsimard enjoy | 20:16 |
| dmsimard | \o/ | 20:16 |
| *** ducttape_ has quit IRC | 20:19 | |
| sdake | dmsimard either that manfiest doesn't work with rdo or kolla is using the wrong or outdated dependencies | 20:19 |
| sdake | dmsimard i had a question related to the skip thing | 20:20 |
| sdake | dmsimard you said it would be useful - I am struggling to understand the use case | 20:20 |
| sdake | dmsimard mind sharing | 20:20 |
| *** daidv has quit IRC | 20:21 | |
| *** hieulq has quit IRC | 20:21 | |
| *** duonghq has quit IRC | 20:21 | |
| larsks | dmsimard: it looks like the puppet integration tests are using 'targetcli', which is in base centos. | 20:21 |
| larsks | So that's probably the solution for configuring the cinder iscsi backend without requiring epel. | 20:22 |
| *** iniazi_ has joined #openstack-kolla | 20:23 | |
| sdake | @lalarINFO:kolla.image.build.base:No package scsi-target-utils available. | 20:23 |
| larsks | sdake: as expected... | 20:23 |
| jamesbenson | for cherry picking, you just mention 4.0.2 or stable/ocata? | 20:24 |
| sdake | here is the mitaka install guide: https://docs.openstack.org/mitaka/install-guide-rdo/cinder-storage-install.html | 20:24 |
| openstackgerrit | James Benson proposed openstack/kolla-ansible stable/ocata: Adds NTP sync process to account for ceph time sync requirements. Removes precheck portion of NTP. Corrects for redhat Fixes typo https://review.openstack.org/481224 | 20:26 |
| jamesbenson | stable/ocata, got it | 20:26 |
| sdake | rdo instal guide: https://docs.openstack.org/ocata/install-guide-rdo/cinder-storage-install.html#install-and-configure-components | 20:26 |
| *** iniazi has quit IRC | 20:27 | |
| *** ducttape_ has joined #openstack-kolla | 20:27 | |
| sdake | larsks i just asked in #openstack-cinder what the correct solution from their POV is | 20:28 |
| larsks | I am pretty sure, based on https://github.com/openstack/puppet-cinder/blob/master/manifests/backend/iscsi.pp#L105, that the answer is "use targetcli". | 20:28 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Allow tmp path to download registry https://review.openstack.org/475940 | 20:31 |
| sdake | larsks sbezverk https://review.openstack.org/#/c/474999/ | 20:31 |
| *** mandre_away is now known as mandre | 20:32 | |
| sdake | larsks the next question is how is targetcli configured | 20:35 |
| sdake | atm we hve a tgtd.conf file which works (or worked) correctly | 20:35 |
| sdake | larsks keep in mind i dont parse puppet | 20:36 |
| *** daidv has joined #openstack-kolla | 20:36 | |
| sbezverk | sdake: if this ps committed I think it will break kolla, as some binaries will be missing.. | 20:36 |
| larsks | sdake: there's no time like the present to learn :) | 20:36 |
| *** duonghq has joined #openstack-kolla | 20:36 | |
| sdake | larsks i've got about 10000 things to leran | 20:37 |
| *** hieulq has joined #openstack-kolla | 20:37 | |
| sdake | larsks puppet is about 9999 on that list | 20:37 |
| larsks | True dat. | 20:37 |
| sbezverk | here the test is only to build container, but no tests confirming sucessful deployment | 20:37 |
| sdake | the gates deploy - however - they dont test lvm | 20:38 |
| sbezverk | sdake: Above I listed binaries which must be present for iscsi-lvm to work | 20:38 |
| sdake | sbezverk let me read scrollback | 20:38 |
| sdake | larsks does tripleo deploy iscsi-lvm? | 20:40 |
| sdake | sbezverk iscsi-lvm is used as a dep for cinder | 20:40 |
| sdake | ? | 20:40 |
| larsks | sdake: I don't know for certain, but the default is probably ceph. | 20:40 |
| sdake | larsks is there any way to look at that puppet wizardlly and find out? | 20:41 |
| sdake | defualt is ceph in kolla as well | 20:41 |
| sdake | we lack kolla-ansible gates for it | 20:41 |
| larsks | I mean, tripleo probably consumes that puppet-cinder module, which would deploy iscsi using targetcli if passed the right options. | 20:42 |
| larsks | I'll see if I can figure that out over the next few days. | 20:42 |
| sdake | larsks the question is - does that deployment work :) | 20:42 |
| *** ansmith has quit IRC | 20:42 | |
| sdake | sbezverk is iscsi-lvm gated in kola-kuberntes | 20:43 |
| sbezverk | sdake: if scsi-target-utils is removed then we will not have tgtd and tgtdadm binaries and it will break lvm | 20:44 |
| *** vhosakot has joined #openstack-kolla | 20:44 | |
| *** jbadiapa has joined #openstack-kolla | 20:44 | |
| sbezverk | sdake: yes, we test lvm, but we use 3.0.0 and 4.0.0 images | 20:44 |
| *** eharney has joined #openstack-kolla | 20:46 | |
| eharney | sdake: hi | 20:46 |
| *** jtriley has quit IRC | 20:46 | |
| sdake | sbezverk eharney can you fellas discuss iscsi + lvm + cinder or whatever problem we are trying to solve here:) | 20:46 |
| sdake | eharney just to catch you up - end goal is to remove epel | 20:47 |
| sdake | iscsi-target-utils is not in rdo (nor rhel7) | 20:47 |
| eharney | great | 20:47 |
| sbezverk | sdake: we do not have problem, I see iscsi-lvm is working perfectly fine in kolla-k8s | 20:47 |
| sdake | sbezverk has indicated binaries from these packages are required in order to operate lvm | 20:47 |
| eharney | so always use lio when on el7 and then scsi-target-utils is not needed | 20:47 |
| sdake | sbezverk if you take out epel - the containers do not build | 20:47 |
| sbezverk | sdake: then do not take it out | 20:48 |
| sdake | sbezverk itt really must be done - epel is sloppy and error prone | 20:48 |
| sdake | sbezverk the alternative is to add iscsi-target-utils to epel | 20:48 |
| sdake | sbezverk what id really like to understand is if there is a htird solution | 20:48 |
| sbezverk | sdake: the goal is to have functional software and not some compliancy to some rules ;) | 20:48 |
| sdake | that involves using lio exclusviely whatever that is | 20:48 |
| sdake | sbezverk this is about cross-community collaboration | 20:49 |
| sbezverk | sdake: we cannot use 'lio' | 20:49 |
| sdake | sbezverk cool - expand plz? | 20:49 |
| sbezverk | out code does not handle this helper at all | 20:49 |
| sbezverk | sdake: at the time when I put together iscsi support for kolla, the only option avaialbe for centos was tgtdadm as helper | 20:50 |
| sdake | pardon ignorance, could it be made to work with lio | 20:50 |
| sbezverk | lio was purely rh thing | 20:50 |
| sdake | lio is kernel module right? | 20:50 |
| eharney | yes | 20:50 |
| sdake | so not a rht thing? | 20:50 |
| sbezverk | sdake: possibly, but somebody need to implement it before removing tgtdadm | 20:50 |
| sdake | sbezverk agree on order of operations | 20:51 |
| eharney | it works on Ubuntu -- the Cinder LIO check job runs it on Ubuntu | 20:51 |
| sbezverk | sdake: if folks who wants to remove tgtdadm is ready to implement it, then great!! | 20:51 |
| inc0 | rwellum: around? | 20:52 |
| sdake | sbezverk you have any idea where to begin that process | 20:52 |
| sbezverk | sdake: for now I would ask inc0 to put -2 on this change | 20:52 |
| rwellum | inc0: yes | 20:52 |
| sdake | sbezverk (file link) | 20:52 |
| inc0 | sbezverk: which change? | 20:52 |
| sdake | sbezverk the change isn't merging if it breaks deployment | 20:52 |
| sdake | no need to -2 it | 20:52 |
| inc0 | yeah please don't break deployment | 20:52 |
| sbezverk | sdake: if it is not tested it can get in | 20:52 |
| sbezverk | kolla-ansible does not test lvm | 20:53 |
| inc0 | rwellum: so, custom image talk, maybe dockerhub publisher + custom images can be done in same talk | 20:53 |
| sbezverk | so nothing to break | 20:53 |
| sdake | it can be manually verified | 20:53 |
| inc0 | sbezverk: gates are not only deployment | 20:53 |
| rwellum | inc0: +1 don't see why not | 20:53 |
| inc0 | also I'd be happy if we could add iscsi to gates too | 20:53 |
| inc0 | it should be fairly easy now | 20:54 |
| *** manheim has quit IRC | 20:54 | |
| *** krtaylor has quit IRC | 20:54 | |
| sbezverk | inc0: there is ps which most likely will break iscsi-lvm deployment, I wanted to make sure it is not merged until proven not breaking actual deployment | 20:55 |
| sdake | i think the expectaton of someone adding a gate to kolla-ansible is a bit high for someone new to the code | 20:55 |
| sdake | sbezverk i will leave a big note in the review stating it shoudl not merge without confirmation | 20:55 |
| sdake | that will do teh rick | 20:55 |
| inc0 | ok, who volunteers to check patch? I don't have Centos around | 20:55 |
| vhosakot | inc0 rwellum I like the idea to combine.. so, can we cover dockerhub publisher from gate, users pulling tested images from dockerhub and deploying kolla, with custom images - all in 45 mins to 1 hour? | 20:55 |
| inc0 | can someone *please* test it out and find whether or not it breaks deploy? | 20:56 |
| inc0 | and after that we'll focus on writing iscsi gates | 20:56 |
| vhosakot | inc0 sbezverk: which patch, I have a Centos box. | 20:56 |
| *** lucasxu has quit IRC | 20:57 | |
| inc0 | we don't release stable today, we have time to fix it in case something breaks and if skips our initial tests | 20:58 |
| sdake | inc0 i know for certain it wont work - it requries both ansible and kuberntes implementation changes | 20:58 |
| inc0 | sbezverk sdake dmsimard here's my plan, we remove iscsi-target-utils and move it to alternative (whatever that is) now | 20:58 |
| inc0 | well, if we'll lose epel, then we need this change | 20:59 |
| inc0 | also having full repo just for single package which can be replaced | 20:59 |
| inc0 | is bad imho | 20:59 |
| sdake | we arent losing epel | 20:59 |
| sdake | we want to lose epel :) | 20:59 |
| sbezverk | inc0: -1 it will break 50% of kolla-k8s gates | 20:59 |
| sdake | we can't blindly merge the patch as it | 20:59 |
| sdake | as is | 20:59 |
| sdake | it ust be tested | 20:59 |
| sdake | kolla-kubernetes implementation must be changed | 21:00 |
| inc0 | sbezverk: then we need to fix it there too, if epel is going to be less maintained it's ticking bomb | 21:00 |
| sdake | kolla-ansible implmeentation needs changing | 21:00 |
| *** vhosakot has quit IRC | 21:00 | |
| sdake | and THEN it can be tested | 21:00 |
| inc0 | I'd rather fix it before it's critical | 21:00 |
| rwellum | vhosakot: I agree. Now will it include a demo? If it does then demoing both ansible and k8s might be overkill - but the process will be different. How should we do that in 45m? | 21:00 |
| sbezverk | inc0: it is wrong approach, to break something before a tested alternative solution is implemented | 21:00 |
| *** vhosakot has joined #openstack-kolla | 21:00 | |
| sdake | sbezverk i'm petty sure nobody said to do that? | 21:01 |
| inc0 | sbezverk: I assume it will be tested when implemented... | 21:01 |
| inc0 | before | 21:01 |
| inc0 | I mean | 21:01 |
| sbezverk | inc0: if somebody implements lio support, great let's drop tgtd, but before that it should be there | 21:01 |
| inc0 | we can't block this change because we don't have gate for it in ansible | 21:01 |
| sdake | inc0 you miss sbezverk 'z point | 21:01 |
| sdake | kolla-kubernetes gates extensively test this code path | 21:01 |
| sbezverk | inc0: I can guarantee you it will break | 21:01 |
| sbezverk | as important binaries will be missing | 21:02 |
| sdake | so even if it breaks lvm - which we may or may not cre about - it will break the gates dramatially | 21:02 |
| sdake | we do care about breaking gates imo :) | 21:02 |
| inc0 | ok, but as far as I understand we need to get rid of this package because epel has issues right? | 21:02 |
| sdake | yes and step 1 in getting rid of the package is implementing support for the otehr "heper" whatever that may be :) | 21:02 |
| jgriffith | sdake what do you mean you "may not care about" | 21:02 |
| jgriffith | that's pretty lame | 21:03 |
| inc0 | is there any way to have both at the same time | 21:03 |
| inc0 | we care about iscsi jgriffith | 21:03 |
| inc0 | don't worry | 21:03 |
| jgriffith | :) | 21:03 |
| inc0 | we do care and because we care we're having this discussion | 21:03 |
| jgriffith | sdake sbezverk I can help you all with the LIO stuff if you want but I don't think it's even remotely critical for you to worry about or deal with | 21:03 |
| jgriffith | and the reality is that most are probably using another backend anyway in production with Kolla | 21:04 |
| inc0 | jgriffith: I think what's critical is that scsi-target-utils in epel is going to be issue in future | 21:04 |
| sdake | jgriffith poor choice of wording - its a matter of priorities -when our gate break - all development stops - if lvm is broken for a little while - its not critial immediately | 21:04 |
| sbezverk | jgriffith : not so sure, last time I tried using lio as iscsi helper nothing worked | 21:04 |
| jgriffith | inc0 yeah, I get that for sure. But that's not like in the next month or something is it? | 21:04 |
| jgriffith | inc0 on the bright side it's almost a direct replacment of how tgt works and is setup | 21:05 |
| inc0 | well I don't want to wait till it happens, I'd rather have it fixed, but we're not on fire | 21:05 |
| eharney | LIO has been the only iscsi helper used on RHEL with LVM for years, i think it works | 21:05 |
| jgriffith | so it shouldn't be a terrible effort, although Ubuntu can be troublesome | 21:05 |
| eharney | we also have a gate job that tests it on Ubuntu... | 21:05 |
| jgriffith | eharney I certainly am not disputing that, it works great! | 21:05 |
| inc0 | is there any way to have both in image for transition time? | 21:05 |
| sbezverk | inc0: all I ask is to let that PS merge only after lio is tested and proven to be working.. | 21:05 |
| sdake | sbezverk that is non-negotiable | 21:06 |
| jgriffith | eharney I just point out Ubuntu because I do run into weirdness sometimes with the lib intiializing properly on install | 21:06 |
| inc0 | sbezverk: ofcourse. That should go without saying | 21:06 |
| jgriffith | but I'm agreeing with you | 21:06 |
| jgriffith | eharney ^^ | 21:06 |
| inc0 | I think we're in violent agreement | 21:06 |
| jgriffith | LIO is certainly a better option in many respects | 21:06 |
| eharney | :) | 21:06 |
| inc0 | yes we want to replace package and ultimatelyu remove epel | 21:06 |
| sdake | ok - well i think we were in agreement when this conversation started | 21:07 |
| jgriffith | performance, flexibility etc | 21:07 |
| inc0 | and yes we want to keep both gates and regular deployments happy | 21:07 |
| sdake | the question is how to implement the support (say starting from kolla-ansible) | 21:07 |
| inc0 | my question is, rolling upgrade while switching these libs will work right? | 21:07 |
| jgriffith | sdake provide an option? | 21:07 |
| jgriffith | sdake and add a second gate job? | 21:07 |
| inc0 | it won't affect communitaction protocol in any way? | 21:07 |
| sdake | jgriffith i'm thinking more along the lines of "how is it configured" | 21:07 |
| jgriffith | then switch the default next cycle? | 21:07 |
| inc0 | yeah jgriffith we want iscsi gate, shouldn't be hard to do | 21:08 |
| jgriffith | same way you do drivers in Kolla today | 21:08 |
| eharney | rolling upgrade from scsi-target-utils to LIO on a live deployment will not work | 21:08 |
| jgriffith | cinder drivers that is | 21:08 |
| inc0 | eharney: ok that's bigger issue then | 21:08 |
| sbezverk | jgriffith : it is not as simple as you try to show ;) we have two containers which helps with what tgtdadm does | 21:09 |
| inc0 | we need to figure out upgrades before making switch | 21:09 |
| jgriffith | eharney BTW, remind me to ask you back in Cinder how RHEL is going to deal with that problem | 21:09 |
| sdake | larsks - can o worms dude :) | 21:09 |
| sbezverk | it would require reworking these containers to support lio | 21:09 |
| jgriffith | inc0 the good news is it's easier in Kolla than it is in traditional BM deployments | 21:09 |
| jgriffith | still not trivial, but not as difficult | 21:09 |
| inc0 | jgriffith: correction, it's possible with kolla and it's often not for BM | 21:10 |
| sdake | inc0 is there any way to turn off the "it didn't build abort" feature of kolla-build | 21:10 |
| inc0 | that being said, we're keeping upgrades. | 21:10 |
| *** mandre is now known as mandre_away | 21:10 | |
| jgriffith | inc0 tomateo/tomato :) | 21:10 |
| inc0 | yeah, well, if that breaks upgrades, I'll hold on till we figure out how to not break upgrades | 21:10 |
| sdake | do we really expect people will be running with iscsi lvm in production | 21:11 |
| inc0 | lvm no, iscsi definetly | 21:11 |
| sdake | i dont know | 21:11 |
| sdake | its an honest question | 21:11 |
| inc0 | netapp and others use iscsi | 21:11 |
| inc0 | many enterprise properitary storage solutions uses iscsi protocol | 21:11 |
| sdake | i got it | 21:11 |
| inc0 | so it defintely has to stay working | 21:12 |
| sdake | is the problem here LIO or lvm or a combo thereof | 21:12 |
| inc0 | well by default I'll assume yes | 21:12 |
| *** rhallisey has quit IRC | 21:12 | |
| inc0 | unless proven that it's not importnant | 21:12 |
| inc0 | also there is intermediate fix | 21:13 |
| inc0 | dockerfile customizations | 21:13 |
| sbezverk | sdake: lio is alternative iscsi helper | 21:13 |
| inc0 | if someone *can't* for some reason use scsi-target-utils, they can build images without it | 21:13 |
| sdake | ok - so this has nothing to do with lvm then? | 21:13 |
| sbezverk | currently we use tgtdadm helper | 21:13 |
| inc0 | while it's not long term, it's possible until we figure out good upgrade strategy and gate migration | 21:14 |
| sbezverk | sdake: the only implementation we have for lvm is with iscsi | 21:14 |
| inc0 | sbezverk: I think question was, netapp+iscsi will be affected right? | 21:15 |
| eharney | sdake: it's just the context of the lvm driver -- which uses one of these iSCSI helpers to handle serving the LVM storage | 21:15 |
| eharney | these helpers aren't used with other cinder iscsi-based drivers | 21:15 |
| inc0 | also jgriffith feel free to correct me if I'm wrong about protocols, I'm storage ignorant | 21:15 |
| sbezverk | inc0: I have no idea, I do not think we even implemented netapp support | 21:15 |
| sdake | ok - so this is lvm only | 21:15 |
| sdake | do people deploy with iscsi ilo with lvm in production | 21:16 |
| inc0 | if it's lvm only then I'm less concerned | 21:16 |
| inc0 | still wouldn't like to break it if possible | 21:16 |
| sdake | inc0 right - my queestion was more around upgrades rather than functionality | 21:16 |
| sdake | i htink you throw upgrades into the mix on a feature virtually nobody uses - pointless waste of effort | 21:16 |
| inc0 | yeah, same here, "break it during upgrade" is what I meant | 21:16 |
| sdake | of course i make some assumptions | 21:16 |
| sdake | i have no ide aif fpeople use this at all | 21:17 |
| sbezverk | I can look into it from kolla-k8s perspective when I understand the difference between tgtd and li approaches | 21:17 |
| inc0 | sdake: I'd be very careful in saying who does and who doesn't use it | 21:17 |
| sdake | inc0 i hve no idea | 21:17 |
| *** manheim has joined #openstack-kolla | 21:17 | |
| inc0 | sbezverk: would you plz? I mean same problem will be with ansible | 21:17 |
| inc0 | you know by far most about our implementation of iscsi | 21:17 |
| sbezverk | inc0: sure but I will need lots of help from jgriffith | 21:18 |
| inc0 | I'd be much happier person if I'd know you're looking at this | 21:18 |
| inc0 | jgriffith: will you help your fellow stackers please? | 21:18 |
| inc0 | sbezverk: let me know what do you find. If we can seamlessly get rid of epel that's great | 21:19 |
| sdake | i would d othe work yself if I had any idea hwo to setup kolla-ansible with lvm with iscsiadm | 21:20 |
| sdake | or whatever magic incantation of stuff is needed to see how it works | 21:20 |
| inc0 | we have docs sdake ;) | 21:22 |
| sdake | cool | 21:22 |
| mgkwill | Congrats sean-k-mooney on https://review.openstack.org/#/c/342354/ merge! Long time coming :) | 21:22 |
| sdake | that woudl be great if the docs were on docs.oo :) | 21:22 |
| inc0 | yeah year without few days | 21:22 |
| inc0 | in queue | 21:22 |
| inc0 | ;) | 21:22 |
| inc0 | and merged | 21:22 |
| mgkwill | inc0: ' | 21:23 |
| mgkwill | inc0: ;) | 21:23 |
| mgkwill | whoops | 21:23 |
| inc0 | hmm I thought we did have these docs | 21:23 |
| inc0 | https://docs.openstack.org/kolla-ansible/latest/cinder-guide.html yes yes we did | 21:24 |
| vhosakot | wasn't there a nova patch like 2 years or something and finally got merged.. lol | 21:25 |
| inc0 | I think that's most of nova patches | 21:25 |
| *** ducttape_ has quit IRC | 21:28 | |
| *** leseb has left #openstack-kolla | 21:28 | |
| openstackgerrit | Marcus Williams proposed openstack/kolla-ansible master: Add OpenDaylight role docs https://review.openstack.org/481289 | 21:29 |
| *** lpetrut has quit IRC | 21:30 | |
| *** ducttape_ has joined #openstack-kolla | 21:33 | |
| *** krtaylor has joined #openstack-kolla | 21:35 | |
| *** japestinho has quit IRC | 21:36 | |
| *** itlinux has quit IRC | 21:40 | |
| *** eharney has left #openstack-kolla | 21:40 | |
| sdake | re the iscsi packaging and EPEL - if nobody else is goingto work on it I guess I'll take a crack | 21:41 |
| sdake | it would be hepful if someone could throw me the slightest bone on how to configure the thing | 21:41 |
| *** itlinux has joined #openstack-kolla | 21:42 | |
| sdake | (with kolla-ansible) | 21:42 |
| sdake | so I have something to test against | 21:42 |
| *** eaguilar has quit IRC | 21:48 | |
| *** jamesbenson has quit IRC | 21:50 | |
| *** jamesbenson has joined #openstack-kolla | 21:52 | |
| *** hieulq has quit IRC | 21:53 | |
| *** daidv has quit IRC | 21:53 | |
| *** duonghq has quit IRC | 21:54 | |
| *** jamesbenson has quit IRC | 21:57 | |
| *** dave-mccowan has quit IRC | 22:05 | |
| *** schwicht has quit IRC | 22:06 | |
| *** ducttap__ has joined #openstack-kolla | 22:08 | |
| *** daidv has joined #openstack-kolla | 22:08 | |
| *** duonghq has joined #openstack-kolla | 22:08 | |
| *** hieulq has joined #openstack-kolla | 22:08 | |
| *** ductta___ has joined #openstack-kolla | 22:11 | |
| *** ducttape_ has quit IRC | 22:11 | |
| *** ducttap__ has quit IRC | 22:13 | |
| *** iniazi_ is now known as iniazi | 22:16 | |
| *** jascott1 has quit IRC | 22:18 | |
| *** jascott1 has joined #openstack-kolla | 22:18 | |
| *** tonanhngo has quit IRC | 22:19 | |
| *** goldyfruit has quit IRC | 22:21 | |
| *** jascott1 has quit IRC | 22:22 | |
| *** goldyfruit has joined #openstack-kolla | 22:24 | |
| *** schwicht has joined #openstack-kolla | 22:28 | |
| *** tonanhngo has joined #openstack-kolla | 22:32 | |
| *** tonanhngo has quit IRC | 22:37 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Reinstate [DEFAULT] firewall_driver option https://review.openstack.org/479313 | 22:44 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add VMware NSXV support to neutron https://review.openstack.org/450092 | 22:46 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add lbaas support to vsphere by vmware-nsx plugin https://review.openstack.org/462844 | 22:46 |
| *** schwicht has quit IRC | 22:56 | |
| inc0 | sdake: in change with scsi package I described upgrade scenario that needs testing too | 23:00 |
| sdake | i really think dealing with upgrades or something nobody usess is overkilll | 23:00 |
| inc0 | I think assuming nobody uses this is premature | 23:00 |
| inc0 | while I dislike lvm myself, I know for a fact that people use it | 23:01 |
| sdake | isnt' it single node | 23:01 |
| inc0 | well data doesn't have HA on it's own | 23:01 |
| sdake | lvm is hosted on the node that runs openstack | 23:02 |
| sdake | single node | 23:02 |
| inc0 | but there are cases when it's not big issue | 23:02 |
| sdake | the only real use csae i see is development | 23:02 |
| inc0 | negative, you can have many lvm nodes | 23:02 |
| inc0 | just data won't get replicated | 23:02 |
| sdake | how is that useful? | 23:02 |
| inc0 | if you for example have database | 23:02 |
| inc0 | and database has replication built into it | 23:02 |
| inc0 | then you don't care that much | 23:02 |
| inc0 | about replication on storage level | 23:02 |
| inc0 | as in, if your app handles HA, you don't need storage HA | 23:03 |
| sdake | how is lvm + iscsi used in a production deployment | 23:03 |
| sdake | you sid people use it | 23:03 |
| sdake | i'd like ot know how | 23:03 |
| inc0 | I just pointed you to one use case | 23:04 |
| inc0 | put db on it | 23:04 |
| sdake | ok let me translate so i understand | 23:04 |
| inc0 | and handle replication within db | 23:04 |
| sdake | you put lvm on 20 nodes | 23:04 |
| sdake | you have those 20 nodes managed by cinder | 23:04 |
| sdake | the 20 nodes provide block storage | 23:04 |
| sdake | is that how its used | 23:04 |
| sdake | ? | 23:04 |
| inc0 | yes | 23:04 |
| inc0 | when you say "I want 20gig of volume" | 23:04 |
| inc0 | cinder scheduler will pick one of 20 nodes and make lvm volume with 20gig on it | 23:05 |
| inc0 | and connect it with iscsi | 23:05 |
| inc0 | if this node dies, your volume is dead | 23:05 |
| inc0 | and data is lost | 23:05 |
| inc0 | but if you have 3 volumes on 3 different nodes and database cluster uses it | 23:05 |
| inc0 | one dead volume will not hurt you because database has 2 more replicas | 23:06 |
| inc0 | so yeah, if database handle replication, iscsi+lvm makes sense | 23:06 |
| inc0 | I'm nto saying it's best of ideas, but it's not horrible idea too | 23:06 |
| inc0 | certinaly less complex than running ceph | 23:07 |
| *** Pavo has joined #openstack-kolla | 23:08 | |
| inc0 | makes sense sdake ? | 23:10 |
| sdake | best practice in this situation is to ping the operator list and ask if they really care for upgrades for this scenario | 23:11 |
| sdake | upgrade sounds impossible | 23:11 |
| inc0 | I'm sure we can think of something | 23:11 |
| kfox1111 | oh, cool. k8s 1.7 statefulset has podManagementPolicy: "Parallel" | 23:12 |
| kfox1111 | makes it much more like a deployment. | 23:13 |
| kfox1111 | but with the nice naming / optional volume support stuff. | 23:13 |
| *** kbyrne has quit IRC | 23:14 | |
| kfox1111 | heh. I'd say its a lot less complex to deploy ceph then to try and make it such that cinder volumes with iscsi lands in just the right way to keep a cluster stable. | 23:14 |
| inc0 | kfox1111: agree, I'd run ceph too, but there are people who don't | 23:15 |
| kfox1111 | you only need a non clustered mysql and a ceph. the other you need to setup the iscsi volume pools in such a way as users can ask for volumes on particular ones, | 23:15 |
| kfox1111 | setup a mysql cluster, carefully allocating a volume per, etc. | 23:15 |
| kfox1111 | yeah, thats fine, but just saying, they cant make the "its simpler" argument. | 23:15 |
| kfox1111 | cause, yeah, no. its not. :) | 23:15 |
| inc0 | it's simpler to understand and dig through | 23:16 |
| inc0 | it gets worse to handle later | 23:16 |
| vhosakot | I always visualize LVM as a software-based way to _spread_ data on multiple physical hard disks, replicated or not, the replication part is added by the storage backend being used with LVM. | 23:16 |
| kfox1111 | heh. digging through lvm/iscsi/tgtd/etc, | 23:16 |
| inc0 | but still yeah, people do that | 23:16 |
| kfox1111 | its not simpler then ceph. ;) | 23:16 |
| inc0 | ok ok we both are ceph fans, I'm poor devils advocate here:( | 23:16 |
| *** manheim has quit IRC | 23:17 | |
| vhosakot | well, ceph has given me many lovable sleepless nights :) | 23:17 |
| kfox1111 | vhosakot: yeah. and ceph is just the same, but across multiple hosts too. :) | 23:17 |
| *** manheim has joined #openstack-kolla | 23:17 | |
| kfox1111 | yeah? I've had the opposite experience. | 23:17 |
| vhosakot | kfox1111: true, I see ceph as the modern cloud-ready way to spread data on disks compared to LVM. | 23:17 |
| kfox1111 | cephs been pretty rock solid. lvm, not as much. :/ | 23:17 |
| kfox1111 | we've built lustres on top of dual attached fiber to try and get ha lustres. | 23:18 |
| kfox1111 | it works, but a lot of work to get to work right. | 23:18 |
| kfox1111 | ceph by comparison's really simple. | 23:18 |
| inc0 | well that being said ceph is not simple to tune in too. It's all rainbows and unicorns until it's rebalancing itself | 23:19 |
| vhosakot | yeah, I did ceph scaling for 5 months, it is awesome! :) no sarcasm there | 23:19 |
| inc0 | can eat a lot of resources | 23:19 |
| kfox1111 | inc0: yeah. we experienced some of that. till we got a script from somewhere that is a little smarter about rebalancing. | 23:19 |
| kfox1111 | bluestore is looking sweet too. | 23:19 |
| kfox1111 | that is the main drawback. | 23:19 |
| kfox1111 | lots of resources. | 23:20 |
| inc0 | nothing's perfect, still would run ceph | 23:20 |
| kfox1111 | we saw something like a 20% performance loss years ago between ceph and iscsi. | 23:20 |
| inc0 | wow that's significant | 23:20 |
| kfox1111 | we deamed it then to not be worth the overhead of having an iscsi storage class too. | 23:20 |
| kfox1111 | the performance hit was fine for our use case and ceph was rock solid. | 23:21 |
| inc0 | well probably due to replication | 23:21 |
| inc0 | copying data to replicas takes time | 23:21 |
| *** goldyfruit has quit IRC | 23:21 | |
| kfox1111 | we didn't do much tuning on it, so the gap could have been lessoned too. | 23:21 |
| kfox1111 | yeah. | 23:21 |
| vhosakot | I saw ceph gets angry when we trigger rebalance of near-full OSDs. Empty OSDs and half-empty OSDs, ceph love to rebalance them. | 23:21 |
| inc0 | yeah, cache tiering can help | 23:21 |
| kfox1111 | we're doing ssd tiers now, and that really has helped. | 23:21 |
| inc0 | that ^ | 23:21 |
| kfox1111 | the things really fast. :) | 23:22 |
| *** manheim has quit IRC | 23:22 | |
| kfox1111 | yeah. the built in balancing needs a little help to not take too much resource. | 23:22 |
| inc0 | I need to experiment with erasure coding | 23:22 |
| kfox1111 | it was basically unusable until recently. | 23:22 |
| kfox1111 | I think luminus has a huge performance tweak in it. | 23:23 |
| *** goldyfruit has joined #openstack-kolla | 23:23 | |
| kfox1111 | so ssd tier with ec behind it works well finally | 23:23 |
| inc0 | anyway, I gtg | 23:23 |
| inc0 | talk to y'all later | 23:24 |
| inc0 | o/ | 23:24 |
| vhosakot | o/ | 23:24 |
| *** daidv has quit IRC | 23:24 | |
| *** hieulq has quit IRC | 23:24 | |
| *** duonghq has quit IRC | 23:24 | |
| kfox1111 | l8r | 23:26 |
| *** manheim has joined #openstack-kolla | 23:27 | |
| *** manheim has quit IRC | 23:27 | |
| *** manheim has joined #openstack-kolla | 23:30 | |
| *** lyang__ has quit IRC | 23:33 | |
| Seyong | rwellum: No, still I am stucked in kolla-k8s.. | 23:36 |
| *** hongbin has quit IRC | 23:38 | |
| *** duonghq has joined #openstack-kolla | 23:40 | |
| *** hieulq has joined #openstack-kolla | 23:40 | |
| *** daidv has joined #openstack-kolla | 23:40 | |
| *** bmace has quit IRC | 23:40 | |
| *** bmace has joined #openstack-kolla | 23:40 | |
| *** itlinux has quit IRC | 23:43 | |
| -openstackstatus- NOTICE: nb03.openstack.org has been cleaned up and rebooted, and should return to building rotation | 23:44 | |
| *** awiddersheim has joined #openstack-kolla | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!