| kfox1111 | heh. helps if ou write it to an actual file.... | 00:00 |
|---|---|---|
| kfox1111 | and look in the right place... | 00:01 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support https://review.openstack.org/446274 | 00:02 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 00:02 |
| inc0 | kfox1111: told you it's going to be quick;) | 00:02 |
| sdake | vhosakot why no workflow on https://review.openstack.org/#/c/447217/ | 00:03 |
| *** eaguilar has quit IRC | 00:03 | |
| vhosakot | sdake: sorry forgot, done! | 00:03 |
| sdake | vhosakot roger thanks :) | 00:03 |
| vhosakot | sdake: np | 00:03 |
| vhosakot | :) | 00:03 |
| kfox1111 | inc0: where's the proof of it working? ;) | 00:04 |
| inc0 | well I did create secret in k8s;) | 00:04 |
| * sdake lols | 00:05 | |
| kfox1111 | inc0: I'm not you. how do I review it? probably shoudl tweak the gate scripts to test it. | 00:05 |
| inc0 | kfox1111: WIP is in title | 00:05 |
| kfox1111 | ok. :) | 00:05 |
| inc0 | I just want to get your opinion on approach | 00:05 |
| kfox1111 | it just sounded like you throught it was done. | 00:05 |
| kfox1111 | k. sec... | 00:05 |
| sdake | inc0 integrating in the gate will be miles harder then doing the implemetation | 00:06 |
| inc0 | sdake: nah | 00:06 |
| sdake | inc0 zhubingbing was planning to rework that cherrypick to integrate with the gate by removing the awful symlinks in teh gate | 00:06 |
| sdake | which are a source of great confusion | 00:06 |
| sdake | i.e. two birds one stone | 00:06 |
| kfox1111 | it doesn't get rid of gencofnig, so won't be too hard I think. | 00:06 |
| inc0 | nah, it assumes genconfig was run and rendered files are available somewhere | 00:07 |
| inc0 | it really just templates configmaps | 00:07 |
| kfox1111 | I think there may be devils in the details. we'll see what falls out of the gate though. | 00:07 |
| kfox1111 | so... how much extra time is this going to add to the gates? | 00:08 |
| kfox1111 | its like a couple minutes just for genconfig right now. | 00:08 |
| inc0 | kfox1111: locally it runs in less than a second | 00:09 |
| kfox1111 | looking through the code, I guess its ok looking to me. it will just shift code from the python command to the ansible one. | 00:09 |
| kfox1111 | ah. cause its ownly handilng the one file. | 00:09 |
| inc0 | yeah....much much simpler ansible one | 00:09 |
| inc0 | and we can examine output at the end | 00:10 |
| kfox1111 | I don't expect that to wokr. most configmaps are multiple files. | 00:10 |
| kfox1111 | might have to tweak that a bit. | 00:10 |
| inc0 | sure, I'll figure it out | 00:10 |
| inc0 | not a hard one to solve too | 00:10 |
| kfox1111 | yeah. | 00:10 |
| inc0 | ok I'm off, tomorrow I'll finish it up and add it to gate | 00:11 |
| kfox1111 | so... most of this could be replaced by just a shell script with | 00:11 |
| kfox1111 | kubectl create configmap .... | 00:11 |
| inc0 | well, rendering template is hard to shell-up | 00:12 |
| kfox1111 | that may be more understandable. | 00:12 |
| inc0 | and also kubectl -f /path/to/dir will check all files in dir with .yaml at the end and add all of them | 00:12 |
| inc0 | so it's one command | 00:12 |
| inc0 | kubectl create -f /tmp/kolla-kubernetes/configmaps | 00:13 |
| inc0 | will add all of them | 00:13 |
| kfox1111 | so is kubectl create configmap --namespace kolla foo-configmap --from-file=/etc/kolla/foo ?? | 00:15 |
| inc0 | yeah | 00:15 |
| inc0 | no | 00:15 |
| inc0 | sorry | 00:15 |
| inc0 | kubectl create --namespace kolla -f /etc/kolla/foo | 00:15 |
| kfox1111 | thats not valid. | 00:16 |
| inc0 | you don't create each configmap | 00:16 |
| inc0 | why? | 00:16 |
| kfox1111 | kubectl create configmap --namespace kolla foo-configmap --from-file=/etc/kolla/foo | 00:16 |
| kfox1111 | is though. | 00:16 |
| inc0 | no, no need | 00:16 |
| kfox1111 | create takes in a k8s yaml file. | 00:16 |
| inc0 | if you create yaml correctly | 00:16 |
| inc0 | this playbook renders yaml file | 00:16 |
| inc0 | for k8s | 00:16 |
| kfox1111 | create configmap generates those objects automatically. | 00:16 |
| inc0 | that's what's its form | 00:16 |
| inc0 | for | 00:16 |
| vhosakot | inc0: yeah, you can add the configmap in YAML spec and add use "kubectl create -f <yaml spec with configmap>" | 00:17 |
| kfox1111 | 90% of what kollakube res create configmap does is kubectl create configmap ... | 00:17 |
| inc0 | and all of it can be replaces with this simple play... | 00:17 |
| kfox1111 | the other 10% is actually rendering templates of its own. | 00:17 |
| kfox1111 | really not sure what ansible's buying us, other hten more ansible. | 00:18 |
| inc0 | kollakube is buggy | 00:18 |
| kfox1111 | inc0: how so? | 00:18 |
| inc0 | kollakube is not super convinient | 00:18 |
| vhosakot | kfox1111: saw your message in the sig-ui slack channel about dashboard. are you able to install the dashboard with service account successfully on kubeadm? I'm seeing errors. | 00:18 |
| inc0 | when I run kollakube today it started to look for neutron in wrong places | 00:19 |
| kfox1111 | vhosakot: I've only tried 1.6 in a static pod configuration. it did work there. | 00:19 |
| inc0 | so I looked at code and cried a little | 00:19 |
| kfox1111 | inc0: then why is it working for everyone else? | 00:19 |
| inc0 | what this gives us is that you'll be able to examine/edit yamls afterwards | 00:19 |
| kfox1111 | yeah, the code isn't very pretty any more. :/ | 00:19 |
| vhosakot | kfox1111: ah, static pod config.. cool.. I'll try service account and will ask you help if needed tomorrow.. | 00:19 |
| kfox1111 | vhosakot: cool. | 00:20 |
| kfox1111 | vhosakot: btw, | 00:20 |
| inc0 | which is additional benefit | 00:20 |
| inc0 | kfox1111: trust me on that one plz, we don't want to have custom cli any more | 00:20 |
| kfox1111 | vhosakot: https://github.com/kubernetes/charts/pull/808 | 00:20 |
| inc0 | rewriting it to proper state will be hard by now | 00:20 |
| kfox1111 | inc0: its on the way out. why spend any more time on it? what does it actually buy us though? | 00:21 |
| inc0 | this ansible gets us there faster and in much more reliable way | 00:21 |
| kfox1111 | users can edit /etc/kolla/* today. | 00:21 |
| inc0 | well, yamls are better for that imho | 00:21 |
| kfox1111 | 99% of the problems in that code path today are genconfig induced breakages. :/ | 00:21 |
| vhosakot | kfox1111: great, thanks for the link. my "helm init" is broken too, after I fix it, I'll install the dashboard chart and post comments in the review. | 00:21 |
| kfox1111 | vhosakot: I'm guessing then its a sdn issue. | 00:22 |
| inc0 | this won't help with genconfig | 00:22 |
| kfox1111 | vhosakot: as helm init requires sdn. | 00:22 |
| kfox1111 | inc0: exactly my point. | 00:22 |
| inc0 | it's meant to solve different problem | 00:22 |
| inc0 | namely kollakube | 00:22 |
| kfox1111 | lets fix the genconfig problem, not the 1% little tiny part in the middle. :/ | 00:22 |
| inc0 | which imho is bad | 00:22 |
| inc0 | but fixing genconfig will be MUCH longer | 00:22 |
| kfox1111 | which fixes itself when the genconfig problem is fixed. | 00:23 |
| inc0 | I mean I did that in an hour. | 00:23 |
| kfox1111 | inc0: yeah, but its far far from done. | 00:23 |
| inc0 | not that far | 00:23 |
| vhosakot | kfox1111: ah, I'm using canal.. is canal just CNI or CNI+SDN? what are you using for pod networking? canal? | 00:23 |
| inc0 | anyway, I'll prove it to you tomorrow | 00:23 |
| kfox1111 | add it to the gate. we'll see. :) | 00:23 |
| kfox1111 | ok. :) | 00:23 |
| inc0 | for now, I'm going home | 00:23 |
| inc0 | have a good one guys | 00:23 |
| kfox1111 | vhosakot: canal. yeah. thats what I was meaning. | 00:23 |
| kfox1111 | inc0: ok. have a good evening. :) | 00:23 |
| *** masber has joined #openstack-kolla | 00:24 | |
| vhosakot | kfox1111: yeah, I'm using canal and tested few apps, so cluster is in working condition. are you able to run "helm init" successfully on kubeadm. sdake and I debugged it last week, and looks like the tiller was in CrashLoopbackook, I'll debug it tomorrow, was pulled into other things today. | 00:25 |
| kfox1111 | vhosakot: yup. | 00:25 |
| kfox1111 | the gate's running it too. | 00:25 |
| vhosakot | kfox1111: cool, thanks for the info. | 00:25 |
| kfox1111 | kubeadm + canal + helm. | 00:25 |
| vhosakot | sweet! | 00:25 |
| vhosakot | yeah, I saw the gate logs today | 00:26 |
| *** jrobinson has joined #openstack-kolla | 00:28 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support https://review.openstack.org/446274 | 00:30 |
| *** vhosakot_ has joined #openstack-kolla | 00:31 | |
| *** Margin has joined #openstack-kolla | 00:33 | |
| *** vhosakot has quit IRC | 00:33 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 00:34 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 00:38 |
| *** Pavo has joined #openstack-kolla | 00:40 | |
| sbezverk | kfox1111: by some reason gate does not like echo "0" /proc/blah commands | 00:43 |
| sbezverk | is there a different way? maybe using sysctl or something? | 00:44 |
| sdake | sbezverk did you try sudo? | 00:44 |
| sdake | vhosakot_ note we ran helm 2.2.2 yesterday - not sure what version kfox1111 is running | 00:45 |
| *** Pavo has quit IRC | 00:45 | |
| *** duonghq has joined #openstack-kolla | 00:45 | |
| sbezverk | sdake: I have sudo in front of this command | 00:45 |
| duonghq | morning guys | 00:45 |
| sdake | sbezverk echo may not be in the sudoers file | 00:45 |
| sdake | sbezverk infra tunes the images so that they are secure from damaging other infra vms | 00:46 |
| sdake | sbezverk an echo to /proc could potentially muck with the infrastructure | 00:46 |
| sdake | sbezverk i'd recommend asking in openstack-infra | 00:46 |
| sdake | sup duonghq | 00:47 |
| vhosakot_ | sdake: right, I saw the issue with 2.1 as well. I see you've pinned k8s version to 1.5.2 and helm version to 2.2.2 in the docs.. cool | 00:48 |
| sdake | kfox1111 FWIW kollakube does not work for bare metal deployments with the symlink workflow | 00:48 |
| sdake | kfox1111 although I agree i struggle with the purpose of ansible in this code path as well | 00:49 |
| kfox1111 | sdake: why not? | 00:49 |
| *** Pavo has joined #openstack-kolla | 00:49 | |
| kfox1111 | sbezverk: oh.... tahts a root level command. | 00:50 |
| *** adrian_otto has quit IRC | 00:50 | |
| *** Margin has quit IRC | 00:50 | |
| vhosakot_ | sdake: thanks for pinning versions in https://review.openstack.org/#/c/447164/1/doc/source/development-environment.rst. less variables in docs now, cool. | 00:50 |
| duonghq | how do you setup vip in multinode setup? | 00:50 |
| *** sayantani01 has joined #openstack-kolla | 00:50 | |
| kfox1111 | try "echo 0 | sudo dd of=/proc...." | 00:50 |
| duonghq | with or without haproxy enable? | 00:50 |
| vhosakot_ | kfox1111: are you using helm 2.2.2? | 00:50 |
| kfox1111 | vhosakot_: yeah. | 00:50 |
| vhosakot_ | kfox1111: cool | 00:50 |
| kfox1111 | oh. gotoa head out. talk to you all later. | 00:50 |
| vhosakot_ | o/ | 00:51 |
| sdake | kfox1111 see this review: https://review.openstack.org/#/c/439740/ | 00:51 |
| *** Margin has joined #openstack-kolla | 00:51 | |
| sdake | kfox1111 the symlinks cause kolla-kube to break wildly when pip install is used | 00:51 |
| sdake | kfox1111 look at the gate results | 00:51 |
| sdake | kfox1111 yet without that patch mariadb configmap cannot be created - this is what triggered this whole discussion | 00:51 |
| kfox1111 | sdake: the gates doing symlinks though. | 00:52 |
| kfox1111 | and it pip installs it I think? | 00:52 |
| sdake | inc0 is reliving the experience I had 2 weeks ago | 00:52 |
| sdake | kfox1111 the gate uses symlinks | 00:52 |
| sdake | I didn't use symlinks | 00:52 |
| masber | how can i find the ports, the kolla containers are listening? | 00:52 |
| sdake | oeprators wont use symlinks | 00:52 |
| kfox1111 | oh. | 00:52 |
| kfox1111 | well, use symlinks :) | 00:52 |
| sdake | wrong answer | 00:52 |
| kfox1111 | I'm an op, and I did. | 00:52 |
| * sdake groans loudly | 00:52 | |
| kfox1111 | right answer is to get out of genconfig. :/ | 00:52 |
| sdake | agreed | 00:52 |
| sdake | short term answer is to get rid of symlinks in the gate | 00:52 |
| kfox1111 | lets stop tilting at the other windmill. | 00:52 |
| kfox1111 | or use a container! :/ | 00:53 |
| sdake | symlinks are not a shiny object | 00:53 |
| kfox1111 | put all the things in a container, symlink away and who cares. :/ | 00:53 |
| sdake | i agree that is a medium term objective | 00:53 |
| kfox1111 | it buys us time to fix it right, and makes it easier for folks to pull a dev/prod env. | 00:53 |
| sdake | getting there is not easy | 00:53 |
| kfox1111 | the dev env already does it. | 00:53 |
| kfox1111 | the works largely been done. | 00:54 |
| masber | I am experiencing this isssue --> https://bugs.launchpad.net/kolla-ansible/+bug/1626339 and http://paste.openstack.org/show/603543/ | 00:54 |
| openstack | Launchpad bug 1626339 in kolla-ansible "Kolla mongodb error: can't find self in the replset config" [High,In progress] - Assigned to Jeffrey Zhang (jeffrey4l) | 00:54 |
| sdake | people dislike the dev env becasue - its based on vagrant | 00:54 |
| *** Pavo has quit IRC | 00:54 | |
| vhosakot_ | ^^^^ | 00:54 |
| kfox1111 | agreed. but the container != vagrent. | 00:54 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 00:54 |
| sdake | kfox1111 we could chase that shiny object for months and still not have an onramp for new devs | 00:54 |
| vhosakot_ | I saw vagrant issues nested-virtualizaion issues for 2 weeks, and now use bare-metal VMs. | 00:55 |
| sdake | kfox1111 ^^ | 00:55 |
| vhosakot_ | sdake: the dev env minus vagrant is awesome. | 00:55 |
| kfox1111 | sdake: or we could rewrite kollakube and spend a bunch of time avoiding a simple symlink. :/ | 00:55 |
| kfox1111 | bigger fish. | 00:55 |
| sdake | i dont think kollakube needs to be rewrritten | 00:56 |
| masber | I was wondering if someone here could give me a workaround, I can't put storage_interface and api_interface on the same interface because my users don't have access to the storage network so I need to separate them if I want them to access Horizon | 00:56 |
| sdake | it is a 2 line patch | 00:56 |
| sdake | https://review.openstack.org/#/c/439740/ | 00:56 |
| sdake | symlinks on the other hand need to be fixed | 00:56 |
| kfox1111 | sdake: seeing a bunch of "simple" patches today that break everything. :) | 00:56 |
| sdake | breaks the gates. | 00:57 |
| sdake | why is that? | 00:57 |
| kfox1111 | though if the gate's fixed, I'm not against it. | 00:57 |
| sdake | there ya go | 00:57 |
| sdake | zhubingbing has agreed to fix the symlink issue this week | 00:57 |
| kfox1111 | sdake: https://review.openstack.org/#/c/439740/1 a lot of red. | 00:57 |
| sdake | so we can have a proper deploy guide | 00:57 |
| sdake | kfox1111 a lot of red all around symlinks | 00:58 |
| sdake | remove the symlinks - use pip install - golden | 00:58 |
| sdake | kfox1111 that patch is actually correct - kolla-kube is defective, the patch needs expansion to fix the gate | 00:58 |
| kfox1111 | fix all the stuff it breaks, and I'll +2 it. :) | 00:58 |
| sdake | kfox1111 zhubingbing will do so :) | 00:59 |
| kfox1111 | ok. | 00:59 |
| sdake | kfox1111 before ou head ou tfor the day you leeft a comment on line 42 | 01:01 |
| *** tonanhngo has quit IRC | 01:01 | |
| sdake | of m=the doc guide | 01:01 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 01:01 |
| sdake | https://review.openstack.org/#/c/447356/5/doc/source/deployment-guide.rst | 01:02 |
| sdake | kfox1111 how precisely do I prune the example | 01:02 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support https://review.openstack.org/446274 | 01:03 |
| kfox1111 | sdake: remove everythign thats a default. | 01:03 |
| sdake | kfox1111 the defaults are in all_values.yaml? | 01:03 |
| kfox1111 | see line 40. | 01:03 |
| kfox1111 | it should spit out all the defaults. | 01:04 |
| kfox1111 | gota head out. l8r. | 01:04 |
| sbezverk | kfox1111: have a good one | 01:05 |
| *** krtaylor has joined #openstack-kolla | 01:06 | |
| openstackgerrit | pangliye proposed openstack/kolla-ansible master: Delete the copy operation of telegraf plugin https://review.openstack.org/447742 | 01:09 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 01:10 |
| duonghq | sdake, how do you setup vip for multinode setup?- | 01:14 |
| *** iceyao has joined #openstack-kolla | 01:16 | |
| duonghq | sbezverk, how do you setup vip for multinode setup? | 01:16 |
| sdake | duonghq do you mean for kolla-kubernetes? | 01:18 |
| duonghq | sdake, yes | 01:18 |
| sdake | duonghq no clue - if you find out do share :) | 01:18 |
| sdake | duonghq i'd like to just get 1 master 1 node working correctly | 01:19 |
| duonghq | so, what IP do you set for VIP? | 01:19 |
| duonghq | which node? | 01:19 |
| sdake | duonghq can you review this: https://review.openstack.org/#/c/447356/ | 01:19 |
| duonghq | sdake, roger | 01:20 |
| sdake | duonghq how are you setting vip in the first place? | 01:20 |
| sdake | duonghq maybe that review contains the answer :) | 01:20 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 01:21 |
| sbezverk | duonghq: yes | 01:21 |
| *** tonanhngo has joined #openstack-kolla | 01:22 | |
| sbezverk | duonghq: sorry I meant to say I was using haproxy | 01:22 |
| duonghq | sbezverk, nice, so I can set vip to any IP in the subnet? | 01:23 |
| sbezverk | which was in front of several api server | 01:23 |
| sbezverk | servers | 01:23 |
| *** tonanhngo has quit IRC | 01:23 | |
| *** eaguilar has joined #openstack-kolla | 01:24 | |
| sbezverk | duonghq: as long as you can stich vip with one of kube-api ip addresses, you should be ok | 01:24 |
| duonghq | sbezverk, understood | 01:24 |
| duonghq | it cannot be arbitrary IP? | 01:24 |
| sbezverk | duonghq: I think it should be from the same subnet | 01:26 |
| sbezverk | never tried fancy stuff here | 01:26 |
| sbezverk | enough problems already | 01:26 |
| duonghq | sbezverk, sure, but if in same subnet, as I tried, seem that it does not show up | 01:26 |
| duonghq | *if just in same subnet | 01:27 |
| duonghq | as in this paste: http://paste.openstack.org/show/603545/ | 01:27 |
| *** goldyfruit has joined #openstack-kolla | 01:27 | |
| sbezverk | duonghq: hmm, I will have to dig up config for haproxy I used with multinode | 01:27 |
| duonghq | I don't know why keystone-admin doesn't got external ip | 01:28 |
| sbezverk | duonghq: what is this ip 192.168.122.248 ? | 01:29 |
| duonghq | this is IP kube master | 01:29 |
| sbezverk | duonghq: you have to explicetely say to use external for admin | 01:29 |
| sbezverk | kubectl get svc -n kolla | grep key | 01:30 |
| sbezverk | keystone-admin 10.57.188.241 192.168.80.232 35357/TCP 11h | 01:30 |
| sbezverk | keystone-internal 10.57.168.139 192.168.80.232 5000/TCP 11h | 01:30 |
| sbezverk | keystone-public 10.57.186.117 192.168.80.232 5000/TCP 11h | 01:30 |
| sbezverk | duonghq: I think you are missing a couple of config settings | 01:30 |
| *** zhubingbing_ has joined #openstack-kolla | 01:30 | |
| duonghq | sbezverk, I have brief meeting now, sorry, brb | 01:30 |
| *** japestinho has joined #openstack-kolla | 01:30 | |
| duonghq | ya, will find it | 01:30 |
| *** iceyao has quit IRC | 01:33 | |
| *** iceyao has joined #openstack-kolla | 01:38 | |
| *** rhallisey has quit IRC | 01:39 | |
| *** Pavo has joined #openstack-kolla | 01:40 | |
| *** iceyao has quit IRC | 01:43 | |
| *** Pavo has quit IRC | 01:43 | |
| *** tonanhngo has joined #openstack-kolla | 01:45 | |
| duonghq | I'm back | 01:45 |
| *** cuongnv has joined #openstack-kolla | 01:45 | |
| *** tonanhngo has quit IRC | 01:46 | |
| *** saneax-_-|AFK is now known as saneax | 01:49 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support https://review.openstack.org/446274 | 01:51 |
| *** crushil has joined #openstack-kolla | 01:51 | |
| duonghq | sbezverk, which tool do you use for setup k!S cluster | 01:51 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 01:53 |
| duonghq | seem that dns in kargo failed me | 01:55 |
| kfox1111 | sbezverk: your about to hit 100 revisions. :) | 01:55 |
| *** zhurong has joined #openstack-kolla | 01:57 | |
| openstackgerrit | pangliye proposed openstack/kolla-ansible master: Delete the copy operation of telegraf plugin https://review.openstack.org/447752 | 01:58 |
| sbezverk | kfox1111: yeah, lots of hick ups along the way | 01:58 |
| *** pramodrj07 has joined #openstack-kolla | 01:59 | |
| sbezverk | duonghq: I do everything maunally using kubeadm | 01:59 |
| kfox1111 | sbezverk: :) its a hard bit of code to get right. but its coming out awesome. great job on it dude. | 02:00 |
| sbezverk | kfox1111: :) thanks.. btw it turned out, gate vm was mnissing br_filter modules | 02:00 |
| sbezverk | that was why echo 0 was failing | 02:00 |
| *** pramodrj07 has quit IRC | 02:01 | |
| kfox1111 | thats ironic. :) | 02:02 |
| *** MasterOfBugs has quit IRC | 02:02 | |
| duonghq | sbezverk, how about network plugin? which one do you use? | 02:03 |
| *** rwellum has joined #openstack-kolla | 02:05 | |
| *** adrian_otto has joined #openstack-kolla | 02:15 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support https://review.openstack.org/446274 | 02:17 |
| *** ipsecguy_ has quit IRC | 02:22 | |
| *** vhosakot_ has quit IRC | 02:22 | |
| *** ipsecguy has joined #openstack-kolla | 02:22 | |
| *** vhosakot has joined #openstack-kolla | 02:23 | |
| *** tonanhngo has joined #openstack-kolla | 02:27 | |
| *** vhosakot has quit IRC | 02:27 | |
| *** tonanhngo has quit IRC | 02:28 | |
| *** saneax is now known as saneax-_-|AFK | 02:29 | |
| *** unicell1 has quit IRC | 02:31 | |
| *** vhosakot has joined #openstack-kolla | 02:33 | |
| *** goldyfruit has quit IRC | 02:35 | |
| *** saneax-_-|AFK is now known as saneax | 02:47 | |
| *** masber has quit IRC | 02:54 | |
| *** zhurong has quit IRC | 03:05 | |
| *** adrian_otto has quit IRC | 03:06 | |
| *** tonanhngo has joined #openstack-kolla | 03:09 | |
| *** tonanhngo has quit IRC | 03:11 | |
| *** masber has joined #openstack-kolla | 03:15 | |
| openstackgerrit | Sayantani Goswami proposed openstack/kolla-ansible master: [WIP] Added playbooks for snap deployment https://review.openstack.org/446165 | 03:19 |
| *** zhurong has joined #openstack-kolla | 03:22 | |
| *** eaguilar has quit IRC | 03:23 | |
| *** dave-mccowan has quit IRC | 03:29 | |
| *** adrian_otto has joined #openstack-kolla | 03:32 | |
| *** saneax is now known as saneax-_-|AFK | 03:40 | |
| *** sayantan_ has joined #openstack-kolla | 03:44 | |
| *** sayantani01 has quit IRC | 03:47 | |
| *** crushil has quit IRC | 03:50 | |
| sdake | sup peeps | 03:53 |
| sdake | sbezverk surprised you can modprobe in the gate | 03:53 |
| masber | hi | 03:59 |
| masber | is the kuryr image ready? | 03:59 |
| masber | deploy is saying that it can't find it | 03:59 |
| *** tovin07_ has joined #openstack-kolla | 04:00 | |
| *** adrian_otto has quit IRC | 04:06 | |
| *** adrian_otto has joined #openstack-kolla | 04:07 | |
| duonghq | sdake, which network plugin are you using for k8s cluster? | 04:08 |
| *** zhurong has quit IRC | 04:12 | |
| *** tonanhngo has joined #openstack-kolla | 04:12 | |
| *** tonanhngo has quit IRC | 04:13 | |
| masber | I amalso getting this error when deploying ironic --> http://paste.openstack.org/show/603557/ | 04:14 |
| sdake | duonghq no clue | 04:24 |
| sdake | masber it was tested by someone in the community although I don't know exactly how :) | 04:25 |
| duonghq | sdake, can you ping to any cluster if or services from within one pod? | 04:25 |
| sdake | duonghq i think canal but can't confirm | 04:25 |
| sdake | duonghq i dont have access to my env atm as it is torn down attempting to test this document :) | 04:25 |
| masber | sdake, no worries, I am just wondering if this was working for someone else or was a mistake I did | 04:26 |
| duonghq | roger | 04:26 |
| masber | by the way I am using kolla-ansible | 04:26 |
| sdake | masber the answer to your quesiton is yes it was working for someone else - the assumption that it is a mistake you did is possibly wrong :) | 04:26 |
| sdake | masber in other words, it may be broken in master of kolla-ansible as the gating of kolla-ansible specifically is pretty weak | 04:26 |
| masber | I see, thanks for the clarification | 04:27 |
| masber | what about kurys? | 04:27 |
| sdake | zhubingbing_ you alive? | 04:27 |
| masber | I can't even pull it | 04:27 |
| masber | *kuryr | 04:27 |
| zhubingbing_ | sdake i am here | 04:27 |
| sdake | masber which version of kolla-ansible are you using? | 04:27 |
| masber | 4.0.0 | 04:28 |
| sdake | masber distro choice? | 04:28 |
| masber | centos binary | 04:28 |
| sdake | kuryr is not available in centos binary | 04:28 |
| sdake | try centos source | 04:28 |
| masber | ok! | 04:29 |
| *** adrian_otto has quit IRC | 04:29 | |
| masber | also I am having this issue https://bugs.launchpad.net/kolla-ansible/+bug/1626339 | 04:29 |
| openstack | Launchpad bug 1626339 in kolla-ansible "Kolla mongodb error: can't find self in the replset config" [High,In progress] - Assigned to Jeffrey Zhang (jeffrey4l) | 04:29 |
| masber | separating api_interface and storage_interface will make mondodb to break while deployment | 04:30 |
| sdake | zhubingbing_ there is renewed vigor in resolving the gating with https://review.openstack.org/#/c/439740/ | 04:30 |
| masber | I tried changing the ip on the ansible command so both points to the same IP but still didn't work | 04:30 |
| sdake | masber no clue on that one :) | 04:31 |
| masber | is there a way I can get in contact with Jeffrey Zhang? | 04:31 |
| masber | I would like to assist him on testing this issue | 04:31 |
| masber | and see if we can get it running soon | 04:31 |
| sdake | masber what i can tell you is the bug is in progress and will always be resolved by Jeffrey4l_ when he takes it on :) | 04:31 |
| *** skramaja has joined #openstack-kolla | 04:32 | |
| masber | sure | 04:32 |
| zhubingbing_ | ok | 04:32 |
| sdake | bmace you alive? | 04:32 |
| masber | I will try to talk to him and see if I can help him testing or something | 04:33 |
| masber | thanks sdake! | 04:33 |
| sdake | masber happy to help | 04:33 |
| zhubingbing_ | sdake i fix it | 04:33 |
| *** manheim has joined #openstack-kolla | 04:33 | |
| openstackgerrit | Surya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Deleting the deprecated option https://review.openstack.org/447789 | 04:35 |
| duonghq | sdake, with the guide, I still got this error: Failed to contact the endpoint at http://keystone-internal:5000/v3 for discovery. Fallback to using that endpoint as the base url. | 04:36 |
| duonghq | Not Found (HTTP 404) | 04:36 |
| duonghq | 04:36 | |
| Jeffrey4l_ | mnaser, i am around ;) | 04:36 |
| sdake | duonghq did you use kargo to deploy? | 04:36 |
| duonghq | yup, guess that it messes up with dns, cannot ping to cluster ip from inside of any pods | 04:37 |
| sdake | duonghq gotcha - can you do this kubectl get pods | grep dns | 04:37 |
| *** saneax-_-|AFK is now known as saneax | 04:37 | |
| sdake | kubectl get pods -n kolla | grep dns | 04:37 |
| *** manheim has quit IRC | 04:37 | |
| Jeffrey4l_ | mnaser, here it the fix https://review.openstack.org/#/c/442999/ | 04:38 |
| sdake | duonghq its good that everything came up :) | 04:38 |
| Jeffrey4l_ | no idea how infra-bot do not add it into bp. | 04:38 |
| duonghq | sdake, http://paste.openstack.org/show/603559/ | 04:38 |
| Jeffrey4l_ | lp* | 04:38 |
| sdake | Jeffrey4l_ sometimes launchpad rate limits and launchpad misses changes | 04:38 |
| Jeffrey4l_ | got. thanks. | 04:39 |
| sdake | Jeffrey4l_ its not an infra problem its a launchpad problem | 04:39 |
| sdake | duonghq moment: | 04:39 |
| masber | sdake, would you recommend using kolla 3.0.2 instead of 4.0.0? | 04:41 |
| sdake | duonghq see this: https://github.com/openstack/kolla-kubernetes/blob/master/tools/setup-resolv-conf.sh#L6 | 04:41 |
| sdake | masber i'd recommend 4.0.0 | 04:41 |
| sdake | masber although 3.0.2 is more stable - depends if your eval or deploying | 04:42 |
| masber | apparently centos-source-kolla-toolbox:4.0.0 is missing from the centos source repository | 04:42 |
| masber | I am deploying | 04:42 |
| sdake | Jeffrey4l_ did you get a chance to push the 4.0.0 images? | 04:42 |
| masber | so I need stability, happy to upgrade once 4.0.0 is stable | 04:42 |
| Jeffrey4l_ | sdake, i have pushed all 4.0.0 images. | 04:42 |
| duonghq | sdake, it can get the ip but not ping to this ip | 04:43 |
| Jeffrey4l_ | except centos+source, iirc you have pushed this, right? | 04:43 |
| sdake | Jeffrey4l_ it is entirely possible i did it wrong - my network was flakey | 04:43 |
| sdake | let me check dockerhub in a minute | 04:43 |
| sdake | masber ^^ | 04:43 |
| masber | mmm I am not sure if I am using docker hub | 04:44 |
| sdake | masber did you build your own images? | 04:44 |
| sdake | duonghq see line 6 of that github resolv.conf? | 04:44 |
| masber | sdake, nop egonzalez gave me another repo | 04:44 |
| duonghq | yup | 04:44 |
| sdake | that needs to be kubedns | 04:44 |
| sdake | kargo uses kubedns - kubeadm uses kube-dns | 04:44 |
| duonghq | I already change kude-dns to kubedns | 04:45 |
| sdake | why have one special snowflake when you can have a blizzard | 04:45 |
| sdake | masber please use dockerhub | 04:45 |
| sdake | masber not sure what egonzales gave you | 04:46 |
| sdake | duonghq did you run setup-resolv-conf.sh? | 04:46 |
| duonghq | yup | 04:46 |
| sdake | when did you run it? | 04:46 |
| sdake | after a clean env setup? | 04:47 |
| duonghq | before 1st helm install | 04:47 |
| duonghq | ya | 04:47 |
| duonghq | should I rerun all of this again? | 04:47 |
| duonghq | cleaning helm | 04:47 |
| duonghq | and reinstall all | 04:47 |
| sdake | moment before you proceed | 04:47 |
| masber | let me double check | 04:47 |
| *** unicell has joined #openstack-kolla | 04:48 | |
| sdake | masber paste your globals.yml | 04:48 |
| masber | http://paste.openstack.org/show/603560/ | 04:49 |
| sdake | masber : docker_registry: "129.94.72.62:4000" | 04:50 |
| sdake | that looks like a local registry of some sort | 04:50 |
| sdake | masber i suspect but can't confirm you didn't push centos source images there | 04:51 |
| sdake | masber this would cause docker pulls to fail as the images are not in the remote registry | 04:51 |
| masber | yes | 04:52 |
| masber | need to fid out which is the remote registry | 04:52 |
| sdake | masber is that some internal registry that egonzalez setup? | 04:53 |
| masber | I followed his isntructions | 04:53 |
| sdake | masber yup - where is that registry stored? | 04:53 |
| masber | http://egonzalez.org/ | 04:53 |
| sdake | imo file a bug - BLOG POST :) | 04:54 |
| masber | wget https://tarballs.openstack.org/kolla/images/centos-binary-registry-ocata.tar.gz | 04:54 |
| masber | looks like this registry is part of kolla | 04:55 |
| sdake | docker_registry: "172.28.128.3:4000" | 04:55 |
| masber | *container docker registry | 04:55 |
| sdake | that needs to be customized to your local environment | 04:55 |
| sdake | Configure kolla globals.yml configuration file with the following content. | 04:55 |
| sdake | Change values when necessary (IP addresses, interface names). | 04:55 |
| sdake | This is a sample minimal configuration. | 04:55 |
| masber | yes, that IP is the host I am running that container | 04:55 |
| masber | I changed taht | 04:56 |
| sdake | # wget https://tarballs.openstack.org/kolla/images/centos-binary-registry-ocata.tar.gz | 04:56 |
| sdake | that will download centos-binary | 04:56 |
| duonghq | sdake, I cannot even ping the nameserver in resolev.conf | 04:56 |
| sdake | masber you need to kill yoru registry and get centos-source-registry-ocata.tar.gz | 04:56 |
| masber | http://paste.openstack.org/show/603561/ | 04:57 |
| masber | ok ok | 04:57 |
| masber | basically my registry does not have the centos source that is why deployment is failing | 04:58 |
| masber | ok | 04:58 |
| masber | so I need to understand, what policy do you follow to push containers to binary or source? | 04:59 |
| masber | are source like for testing? | 04:59 |
| *** zhurong has joined #openstack-kolla | 05:00 | |
| sdake | masber i have other things to do at tthis moment - i can help in the morning perhaps - or ping the emea crew when they arrive :) | 05:00 |
| masber | sure, np | 05:01 |
| sdake | duonghq can you show me your resolv.conf | 05:01 |
| sdake | duonghq in the containers | 05:02 |
| duonghq | sdake, here http://paste.openstack.org/show/603562/ | 05:02 |
| sdake | duonghq give this command a spin outside the container: | 05:05 |
| sdake | kubectl get svc --namespace=kube-system -l k8s-app=kubedns -o jsonpath='{.items[*].spec.clusterIP}' | 05:06 |
| duonghq | sdake, 10.233.0.3 | 05:07 |
| duonghq | sdake, kubectc get svc output: http://paste.openstack.org/show/603563/ | 05:08 |
| sdake | duonghq looks like your dns is setup properl yas keystone-admin is resolving | 05:10 |
| sdake | i think you have a networking problem | 05:10 |
| duonghq | sdake, I can ping pod of other services directly | 05:11 |
| duonghq | but not through cluster ip | 05:11 |
| sdake | define ping pod of other services directly? | 05:12 |
| duonghq | i.e. I can ping mariadb-0 pod, but not mariadb service | 05:12 |
| duonghq | inside keystone pod | 05:12 |
| sdake | what ip is mariadb service? | 05:13 |
| duonghq | mariadb 10.233.25.205 <none> 3306/TCP 22h | 05:13 |
| duonghq | pod: 10.233.97.152 | 05:13 |
| sdake | can you paste kubectl get pod -n kolla | grep mariadb | 05:14 |
| duonghq | mariadb-0 1/1 Running 0 22h | 05:14 |
| sdake | kubectl describe pod mariadb-0 -n kolla? | 05:16 |
| duonghq | http://paste.openstack.org/show/603564/ | 05:16 |
| sdake | IP:10.233.97.152 | 05:22 |
| sdake | you can ping 10.233.97.152? | 05:23 |
| sdake | which IP can't you ping? | 05:23 |
| duonghq | sdake, the IP of mariadb 10.233.25.205 <none> 3306/TCP 22h | 05:25 |
| duonghq | 10.233.25.205 | 05:25 |
| sdake | duonghq not sure why that is | 05:26 |
| duonghq | ya | 05:26 |
| sdake | duonghq gotta hit the rack | 05:26 |
| sdake | duonghq try sbezverk in the morning - he can probably get you going | 05:27 |
| sdake | duonghq if you sort it out - i'd like to know | 05:27 |
| duonghq | sdake, thanks, my keystone cannot even connect to its internal endpoint | 05:27 |
| duonghq | night sdake | 05:28 |
| sdake | night | 05:28 |
| *** satyar has joined #openstack-kolla | 05:29 | |
| *** lazyPower has quit IRC | 05:32 | |
| *** lazyPower has joined #openstack-kolla | 05:32 | |
| *** lazyPower has quit IRC | 05:33 | |
| *** lazyPower has joined #openstack-kolla | 05:33 | |
| *** zhurong has quit IRC | 05:40 | |
| *** tonanhngo has joined #openstack-kolla | 05:46 | |
| *** tonanhngo has quit IRC | 05:48 | |
| *** saneax is now known as saneax-_-|AFK | 05:54 | |
| *** iceyao has joined #openstack-kolla | 05:58 | |
| *** iceyao has quit IRC | 05:59 | |
| *** unicell1 has joined #openstack-kolla | 06:15 | |
| *** unicell has quit IRC | 06:15 | |
| *** unicell1 has quit IRC | 06:22 | |
| *** unicell has joined #openstack-kolla | 06:22 | |
| *** tonanhngo has joined #openstack-kolla | 06:28 | |
| *** tonanhngo has quit IRC | 06:29 | |
| *** jbadiapa has joined #openstack-kolla | 06:30 | |
| *** saneax-_-|AFK is now known as saneax | 06:31 | |
| *** jaosorior has joined #openstack-kolla | 06:31 | |
| *** tonanhngo has joined #openstack-kolla | 07:02 | |
| *** david-lyle has quit IRC | 07:02 | |
| *** vhosakot has quit IRC | 07:03 | |
| *** tonanhngo has quit IRC | 07:03 | |
| *** mpansky has joined #openstack-kolla | 07:04 | |
| *** caowei has quit IRC | 07:08 | |
| *** caowei has joined #openstack-kolla | 07:08 | |
| *** sayantan_ has quit IRC | 07:08 | |
| *** Serlex has joined #openstack-kolla | 07:11 | |
| *** unicell has quit IRC | 07:12 | |
| *** unicell has joined #openstack-kolla | 07:12 | |
| *** iceyao has joined #openstack-kolla | 07:19 | |
| *** iceyao has quit IRC | 07:20 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/kolla master: RHEL: Add mod_ssl for services running over httpd https://review.openstack.org/446911 | 07:26 |
| *** manheim has joined #openstack-kolla | 07:31 | |
| *** manheim has quit IRC | 07:34 | |
| *** skramaja_ has joined #openstack-kolla | 07:34 | |
| *** manheim has joined #openstack-kolla | 07:35 | |
| *** skramaja has quit IRC | 07:37 | |
| *** jrobinson has quit IRC | 07:42 | |
| *** tonanhngo has joined #openstack-kolla | 07:44 | |
| *** tonanhngo has quit IRC | 07:46 | |
| *** jmccarthy has joined #openstack-kolla | 07:47 | |
| *** matrohon has joined #openstack-kolla | 07:51 | |
| *** cu5 has quit IRC | 07:53 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Register Cinder v3 endpoint in keystone https://review.openstack.org/446889 | 07:53 |
| *** cu5 has joined #openstack-kolla | 07:57 | |
| *** skramaja_ is now known as skramaja | 08:00 | |
| *** shardy has joined #openstack-kolla | 08:07 | |
| *** blallau has joined #openstack-kolla | 08:08 | |
| *** matrohon has quit IRC | 08:09 | |
| *** pcaruana has joined #openstack-kolla | 08:11 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/kolla master: RHEL: Add mod_ssl for services running over httpd https://review.openstack.org/446911 | 08:17 |
| *** blallau has quit IRC | 08:23 | |
| *** shardy has quit IRC | 08:24 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Fix error during Grafana container restart https://review.openstack.org/447191 | 08:29 |
| openstackgerrit | Dou Rui Yuan proposed openstack/kolla-ansible master: Doc expressions update for Deployment Philosophy and Production architecture https://review.openstack.org/446882 | 08:29 |
| -openstackstatus- NOTICE: Wiki is broken with database problems, we are working to resolve it | 08:31 | |
| *** ChanServ changes topic to "Wiki is broken with database problems, we are working to resolve it" | 08:31 | |
| *** matrohon has joined #openstack-kolla | 08:32 | |
| *** zhurong has joined #openstack-kolla | 08:34 | |
| *** matrohon has quit IRC | 08:36 | |
| *** ChanServ changes topic to "support: ask.openstack.org | New to Kolla: docs @ http://docs.openstack.org/developer/kolla/ | | Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 08:40 | |
| -openstackstatus- NOTICE: Wiki problems have been fixed, it's up and running | 08:40 | |
| *** mgoddard has joined #openstack-kolla | 08:44 | |
| jaosorior | mandre: finally figured the issue out with the services trying listen on port 443 and failing. | 08:46 |
| jaosorior | mandre: mod_ssl installs a file with defaults to enable the SSL engine, and it has a Listen directive that attempts to listen there. So, I'm removing that directive, since anyway we don't want to add that unless we explicitly specify such a thing in the configurations. | 08:47 |
| openstackgerrit | Dennis Mueller proposed openstack/kolla-ansible master: Added neutron metering-agent support https://review.openstack.org/445858 | 08:49 |
| *** egonzalez has joined #openstack-kolla | 08:54 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/kolla master: RHEL: Add mod_ssl for services running over httpd https://review.openstack.org/446911 | 09:02 |
| *** openstackgerrit has quit IRC | 09:03 | |
| *** athomas has joined #openstack-kolla | 09:06 | |
| *** blallau has joined #openstack-kolla | 09:15 | |
| *** matrohon has joined #openstack-kolla | 09:29 | |
| *** duonghq has quit IRC | 09:52 | |
| *** tovin07_ has quit IRC | 09:58 | |
| *** Serlex has quit IRC | 09:59 | |
| *** Margin has quit IRC | 10:04 | |
| *** mpansky has left #openstack-kolla | 10:04 | |
| *** openstackgerrit has joined #openstack-kolla | 10:05 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/kolla master: RHEL: Add mod_ssl for services running over httpd https://review.openstack.org/446911 | 10:05 |
| *** zhurong has quit IRC | 10:05 | |
| *** cuongnv has quit IRC | 10:10 | |
| *** jaosorior is now known as jaosorior_brb | 10:14 | |
| Daviey_ | please can i get another +2 on a simple change, https://review.openstack.org/#/c/444771/ | 10:17 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Remove ml2_conf.ini from neutron-metadata-agent https://review.openstack.org/447992 | 10:19 |
| *** satyar has quit IRC | 10:24 | |
| *** zhurong has joined #openstack-kolla | 10:24 | |
| *** manheim has quit IRC | 10:40 | |
| *** shardy has joined #openstack-kolla | 10:45 | |
| *** porzech has quit IRC | 10:55 | |
| *** qiliang28 has joined #openstack-kolla | 10:58 | |
| hrw | hi | 10:58 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Remove ml2_conf.ini from some Neutron agents https://review.openstack.org/447992 | 10:59 |
| *** Serlex has joined #openstack-kolla | 10:59 | |
| *** qiliang27 has quit IRC | 10:59 | |
| *** qiliang28 is now known as qiliang27 | 10:59 | |
| *** matrohon has quit IRC | 11:05 | |
| *** iceyao has joined #openstack-kolla | 11:06 | |
| *** eaguilar has joined #openstack-kolla | 11:06 | |
| *** bjolo has joined #openstack-kolla | 11:07 | |
| bjolo | howdy peeps | 11:09 |
| egonzalez | sup bjolo | 11:10 |
| *** iceyao has quit IRC | 11:10 | |
| bjolo | good | 11:11 |
| bjolo | trying to find my way back to IRC | 11:11 |
| bjolo | been so busy on actual work, no time for getting distracted on IRC :) | 11:11 |
| bjolo | how are you?\ | 11:11 |
| hrw | fine | 11:12 |
| hrw | someone knows irc nickname of Paweł Krupa? | 11:13 |
| egonzalez | hrw, no idea | 11:14 |
| hrw | always worth asking ;D | 11:14 |
| *** eaguilar_ has joined #openstack-kolla | 11:23 | |
| *** eaguilar has quit IRC | 11:24 | |
| *** manheim has joined #openstack-kolla | 11:27 | |
| *** jmccarthy1 has joined #openstack-kolla | 11:28 | |
| *** jmccarthy has quit IRC | 11:28 | |
| *** rwellum has quit IRC | 11:29 | |
| *** rwallner has joined #openstack-kolla | 11:31 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Make keystone_user module fully shade https://review.openstack.org/448014 | 11:31 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Make keystone_user module fully shade https://review.openstack.org/448014 | 11:34 |
| *** athomas has quit IRC | 11:44 | |
| *** rmart04 has joined #openstack-kolla | 11:45 | |
| *** eaguilar_ has quit IRC | 11:47 | |
| *** eaguilar has joined #openstack-kolla | 11:48 | |
| *** caowei has quit IRC | 11:53 | |
| *** rwellum has joined #openstack-kolla | 12:02 | |
| *** iceyao has joined #openstack-kolla | 12:06 | |
| openstackgerrit | Dou Rui Yuan proposed openstack/kolla-ansible master: Doc expressions update for Deployment Philosophy and Production architecture https://review.openstack.org/446882 | 12:09 |
| *** eaguilar has quit IRC | 12:10 | |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-ansible master: Fix the typo https://review.openstack.org/448035 | 12:12 |
| *** haplo37 has quit IRC | 12:15 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Remove ml2_conf.ini from some Neutron agents https://review.openstack.org/447992 | 12:17 |
| *** tonanhngo has joined #openstack-kolla | 12:18 | |
| *** srwilkers has joined #openstack-kolla | 12:18 | |
| *** tonanhngo has quit IRC | 12:19 | |
| *** rhallisey has joined #openstack-kolla | 12:19 | |
| *** goldyfruit has joined #openstack-kolla | 12:19 | |
| *** goldyfruit has quit IRC | 12:20 | |
| *** goldyfruit has joined #openstack-kolla | 12:21 | |
| openstackgerrit | Emilien Macchi proposed openstack/kolla master: Switch trunk/cbs/buildlogs to use https https://review.openstack.org/448043 | 12:21 |
| *** iceyao has quit IRC | 12:21 | |
| *** mgoddard has quit IRC | 12:22 | |
| *** iceyao has joined #openstack-kolla | 12:22 | |
| *** haplo37 has joined #openstack-kolla | 12:24 | |
| *** iceyao has quit IRC | 12:25 | |
| *** goldyfruit has quit IRC | 12:25 | |
| *** zhubingbing_ has quit IRC | 12:27 | |
| *** iceyao has joined #openstack-kolla | 12:27 | |
| openstackgerrit | Dou Rui Yuan proposed openstack/kolla-ansible master: Doc syntax error and expressions updates https://review.openstack.org/446882 | 12:28 |
| pbourke_ | egonzalez: can you have a look at https://review.openstack.org/#/c/435024/ | 12:36 |
| pbourke_ | egonzalez: and also its dependant patch | 12:36 |
| *** dave-mccowan has joined #openstack-kolla | 12:37 | |
| mnaser | dang | 12:37 |
| mnaser | gate still broken? | 12:37 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Fix wrong api interface reference in haproxy.cfg for nova serial https://review.openstack.org/448067 | 12:37 |
| *** jaosorior_brb is now known as jaosorior | 12:38 | |
| srwilkers | morning | 12:39 |
| *** zhurong has quit IRC | 12:39 | |
| *** 21WAAVN4H has joined #openstack-kolla | 12:39 | |
| *** matrohon has joined #openstack-kolla | 12:42 | |
| jaosorior | mnaser: are those the failures in centos and ubuntu source? | 12:44 |
| mnaser | jaosorior https://bugs.launchpad.net/kolla/+bug/1674483 yeah | 12:44 |
| openstack | Launchpad bug 1674483 in kolla "Bifrost failing because of missing SUDO_USER" [Undecided,New] | 12:44 |
| mnaser | i have a few $job things i have to take care of so if someone wants to look into it.. :x | 12:44 |
| jaosorior | oh right, I was wondering what that was | 12:44 |
| jaosorior | thanks for the info | 12:45 |
| hrw | hm. hope it is that why bifrost-base fail | 12:45 |
| *** dave-mccowan has quit IRC | 12:49 | |
| *** rhallisey has quit IRC | 12:49 | |
| *** tonanhngo has joined #openstack-kolla | 12:49 | |
| *** rhallisey has joined #openstack-kolla | 12:51 | |
| *** tonanhngo has quit IRC | 12:51 | |
| *** Margin has joined #openstack-kolla | 12:53 | |
| *** leseb has quit IRC | 12:53 | |
| *** kencjohnston_ has quit IRC | 12:56 | |
| *** shardy has quit IRC | 12:57 | |
| *** dave-mccowan has joined #openstack-kolla | 12:57 | |
| *** Margin has quit IRC | 12:59 | |
| *** Margin has joined #openstack-kolla | 13:00 | |
| *** m1dev has joined #openstack-kolla | 13:01 | |
| Margin | hello, I opened a bug https://bugs.launchpad.net/kolla/+bug/1674672 | 13:02 |
| openstack | Launchpad bug 1674672 in kolla "ironic-conductor doesn't work " [Undecided,New] | 13:02 |
| *** shardy has joined #openstack-kolla | 13:03 | |
| thomas_oneill | Margin: is there anything in your /var/lib/docker/volumes/kolla_logs/_data/ironic* about this? | 13:05 |
| *** iceyao has quit IRC | 13:05 | |
| Margin | no ironic-conductor.log file , only ironic-api.log ironic-inspector.log | 13:07 |
| *** tonanhngo has joined #openstack-kolla | 13:07 | |
| thomas_oneill | Margin: what happens when you manually stop and start the container | 13:07 |
| *** kbaegis has joined #openstack-kolla | 13:08 | |
| *** tonanhngo has quit IRC | 13:08 | |
| *** iceyao has joined #openstack-kolla | 13:11 | |
| *** 21WAAVN4H has quit IRC | 13:15 | |
| *** goldyfruit has joined #openstack-kolla | 13:16 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 13:16 |
| Margin | no clue after stop and start the container. | 13:16 |
| *** rhallisey has quit IRC | 13:20 | |
| *** rhallisey has joined #openstack-kolla | 13:21 | |
| thomas_oneill | Margin: it doesn't give you any error? the container is still in a restarting state? | 13:26 |
| thomas_oneill | Margin: check systemctl status docker | 13:26 |
| *** crushil has joined #openstack-kolla | 13:27 | |
| *** jtriley has joined #openstack-kolla | 13:29 | |
| *** mgoddard has joined #openstack-kolla | 13:31 | |
| *** bjolo has quit IRC | 13:31 | |
| *** leseb has joined #openstack-kolla | 13:32 | |
| *** bjolo has joined #openstack-kolla | 13:34 | |
| Margin | no clue | 13:37 |
| *** tonanhngo has joined #openstack-kolla | 13:39 | |
| Margin | sudo " no tty present and no askpass program specified" is clue ? | 13:39 |
| *** eaguilar has joined #openstack-kolla | 13:40 | |
| *** adrian_otto has joined #openstack-kolla | 13:40 | |
| *** tonanhngo has quit IRC | 13:41 | |
| *** rwsu has quit IRC | 13:45 | |
| *** cuongnv has joined #openstack-kolla | 13:48 | |
| *** eaguilar has quit IRC | 13:49 | |
| *** deadnull has joined #openstack-kolla | 13:50 | |
| *** manheim has quit IRC | 13:50 | |
| *** manheim has joined #openstack-kolla | 13:50 | |
| *** tovin07_ has joined #openstack-kolla | 13:55 | |
| *** tovin07_ has left #openstack-kolla | 13:57 | |
| *** adrian_otto has quit IRC | 13:58 | |
| *** tovin07_ has joined #openstack-kolla | 13:58 | |
| *** scsnow_ has joined #openstack-kolla | 14:00 | |
| *** tonanhngo has joined #openstack-kolla | 14:01 | |
| scsnow_ | hi, I'm following kolla-kubernetes quickstart guide. During rabbitmq bootstraping I'm faced with: Failed to pull image "kolla/centos-binary-rabbitmq:auto": Tag auto not found in repository docker.io/kolla/centos-binary-rabbitmq | 14:02 |
| scsnow_ | The same for mariadb bootstraping | 14:02 |
| *** tonanhngo has quit IRC | 14:02 | |
| scsnow_ | Obviously tag auto doesn't exist, but I didn't find a way to configure it | 14:03 |
| sbezverk | scsnow_: you need to provide some variables, one of them is image tag | 14:06 |
| *** jtriley_ has joined #openstack-kolla | 14:07 | |
| sbezverk | the easiest approach would be generate cloud.yaml by using build_example_yaml.py | 14:07 |
| *** rwellum_ has joined #openstack-kolla | 14:08 | |
| sbezverk | scsnow_: and then customize it to your needs | 14:08 |
| *** jtriley has quit IRC | 14:10 | |
| scsnow_ | sbezverk, where should I put generated cloud.yaml? | 14:10 |
| *** satyar has joined #openstack-kolla | 14:12 | |
| *** lamt has joined #openstack-kolla | 14:27 | |
| *** bjolo has quit IRC | 14:31 | |
| *** tonanhngo has joined #openstack-kolla | 14:33 | |
| *** tonanhngo has quit IRC | 14:33 | |
| *** fguillot has joined #openstack-kolla | 14:34 | |
| *** kbaegis has quit IRC | 14:39 | |
| *** kbaegis has joined #openstack-kolla | 14:42 | |
| *** shashank_t_ has joined #openstack-kolla | 14:47 | |
| *** jtriley_ has quit IRC | 14:49 | |
| *** david-lyle has joined #openstack-kolla | 14:49 | |
| *** lamt has quit IRC | 14:52 | |
| *** jtriley has joined #openstack-kolla | 14:52 | |
| *** lamt has joined #openstack-kolla | 14:54 | |
| sbezverk | scsnow_: location is not important, as long as you provide path in --values | 14:56 |
| sbezverk | to it.. example I use helm install blah blah --values examples/cloud.yaml | 14:57 |
| *** blallau has quit IRC | 15:00 | |
| *** dave-mccowan has quit IRC | 15:02 | |
| *** blallau has joined #openstack-kolla | 15:03 | |
| *** Margin has quit IRC | 15:04 | |
| *** lamt has quit IRC | 15:08 | |
| *** david-lyle has quit IRC | 15:09 | |
| *** kbaegis has quit IRC | 15:09 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Update Fluentd config to log in Kibana dashboard https://review.openstack.org/446342 | 15:13 |
| *** david-lyle has joined #openstack-kolla | 15:15 | |
| *** manheim has quit IRC | 15:18 | |
| inc0 | good morning | 15:18 |
| *** manheim has joined #openstack-kolla | 15:19 | |
| srwilkers | morning | 15:19 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix logrotate issue on Octavia and Watcher service https://review.openstack.org/446061 | 15:19 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements https://review.openstack.org/446772 | 15:21 |
| *** dave-mccowan has joined #openstack-kolla | 15:23 | |
| *** shardy is now known as coding_shardy | 15:23 | |
| *** tonanhngo has joined #openstack-kolla | 15:23 | |
| *** manheim has quit IRC | 15:23 | |
| *** tonanhngo has quit IRC | 15:24 | |
| *** sayantani01 has joined #openstack-kolla | 15:27 | |
| *** sayantani01 has quit IRC | 15:29 | |
| *** matrohon has quit IRC | 15:29 | |
| *** sayantani01 has joined #openstack-kolla | 15:29 | |
| sbezverk | inc0: srwilkers: good morning | 15:30 |
| sbezverk | srwilkers: has portdirect talked to you about tomorrow? possibility to have a chat about our session? | 15:31 |
| *** jrist has quit IRC | 15:33 | |
| *** hieulq_ has joined #openstack-kolla | 15:33 | |
| *** hrw has quit IRC | 15:35 | |
| *** hrw has joined #openstack-kolla | 15:36 | |
| *** adrian_otto has joined #openstack-kolla | 15:38 | |
| *** jrist has joined #openstack-kolla | 15:40 | |
| inc0 | sbezverk: question | 15:42 |
| inc0 | https://github.com/openstack/kolla-kubernetes/blob/296fd38507cb0c5a487383db5bf330214de977c8/tools/setup_gate_ceph.sh#L140 what are configmaps -haproxy? | 15:42 |
| inc0 | where do they come from? | 15:43 |
| *** zhubingbing has joined #openstack-kolla | 15:44 | |
| inc0 | ahh...I see it now | 15:44 |
| inc0 | it's in helm | 15:44 |
| *** cuongnv has quit IRC | 15:49 | |
| *** pbandark has joined #openstack-kolla | 15:49 | |
| jmccarthy1 | Hmm I notice services seem to have this pattern with transport_url like transport_url = rabbit://{% for host in groups['rabbitmq'] %}... which puts a list of related hosts into services config - why is this ? Or for example why not use haproxy ? i.e. point to VIP and let it deal with the rest ? | 15:50 |
| *** blallau has quit IRC | 15:52 | |
| *** adrian_otto has quit IRC | 15:52 | |
| *** tonanhngo has joined #openstack-kolla | 15:53 | |
| *** pbandark has quit IRC | 15:53 | |
| *** jmccarthy1 has quit IRC | 15:54 | |
| *** david-lyle has quit IRC | 15:54 | |
| *** jmccarthy has joined #openstack-kolla | 15:54 | |
| *** foliouno has joined #openstack-kolla | 15:57 | |
| sbezverk | inc0: we have configmap kind of hardcoded | 15:57 |
| sbezverk | I mean only for haproxy | 15:57 |
| sbezverk | if you check services/common you will find a template we use for haproxy config map | 15:58 |
| sbezverk | inc0: I do not think we moved them to helm, at least I do not rememeber ;) | 15:58 |
| inc0 | so I don't need to create this configmap from /etc/kolla right? | 15:59 |
| sbezverk | inc0: unfortunately we do need them, but they will not come from kolla | 15:59 |
| inc0 | sbezverk: some of them are there https://github.com/openstack/kolla-kubernetes/blob/8652282f7b32e59965863e07cad7aad511a568f3/helm/test/devenv/templates/nova-api-haproxy.yaml | 16:00 |
| inc0 | how do I generate them? | 16:00 |
| sbezverk | inc0: please ignore those, they were generated manually just for dev env | 16:00 |
| *** david-lyle has joined #openstack-kolla | 16:01 | |
| sbezverk | inc0: they are all hardcoded with values which is completely useless for general deployment | 16:01 |
| *** adrian_otto has joined #openstack-kolla | 16:01 | |
| *** magicboiz has joined #openstack-kolla | 16:01 | |
| inc0 | sure...so how do I generate real ones? | 16:01 |
| sbezverk | inc0: you need to run kollakube tools | 16:02 |
| inc0 | so here's thing...I want to get rid of kollakube | 16:02 |
| inc0 | alltogether | 16:02 |
| inc0 | if we have python generated configs in same place as ansible generated configs, it's worse than I thought | 16:02 |
| inc0 | :( | 16:02 |
| sbezverk | for haproxy kollakube checks etc/kolla-kubernetes/service_resource.yaml | 16:02 |
| sbezverk | where haproxy config maps are defined | 16:03 |
| spsurya | inc0: morning | 16:03 |
| inc0 | good morning spsurya | 16:03 |
| sbezverk | and refer to a template | 16:03 |
| *** lamt has joined #openstack-kolla | 16:03 | |
| inc0 | ok, so I have slightly different idea | 16:03 |
| sbezverk | inc0: yeah we have mixture now | 16:03 |
| inc0 | I need to warn you, it does involve ansible (but not kolla-ansible) | 16:03 |
| sbezverk | inc0: have you sold it to kfox1111 ;)? | 16:04 |
| spsurya | inc0: it would be great if you review this again https://review.openstack.org/#/c/425446/ | 16:04 |
| inc0 | please, let's put religion aside | 16:05 |
| inc0 | we need to move forward without this burden | 16:05 |
| *** lamt has quit IRC | 16:05 | |
| inc0 | if only we agree on having ansible in kolla-k8s, at least for time being | 16:05 |
| inc0 | I can give us separatied configs today | 16:05 |
| sbezverk | inc0: sure, but in this case when some folks will start saying that we are moving away from helm native way, you will need to take heat for it ;) | 16:06 |
| inc0 | single good way to create all the configs with no kolla-ansible involved | 16:06 |
| *** lamt has joined #openstack-kolla | 16:06 | |
| inc0 | well, fuck that | 16:06 |
| inc0 | pardon my language | 16:06 |
| sbezverk | inc0: how do you connect helm/all_values.yaml with ansible generating configmaps? | 16:07 |
| inc0 | negactive | 16:07 |
| inc0 | negative | 16:07 |
| inc0 | I'll just copy ansible dir from kolla-ansible and thin it out to only have configmap | 16:07 |
| *** rmart04 has quit IRC | 16:08 | |
| sbezverk | inc0: ok but helm/all_values.yaml has values which will be needed for configmap generation | 16:08 |
| inc0 | then we can add all the k8s specific configs like this -haproxy crap | 16:08 |
| inc0 | ok...well...they're not used now | 16:08 |
| inc0 | but anyway, can do it | 16:09 |
| inc0 | it's all yaml and ansible understands yaml | 16:09 |
| sbezverk | inc0: currently we have two stages | 16:09 |
| inc0 | kfox1111: around? | 16:09 |
| sbezverk | we create configmaps with some unknown values and second stage gets populated congmaps with "running" info by using init containers | 16:10 |
| Daviey_ | please can i get another +2 on a simple change, https://review.openstack.org/#/c/444771/ | 16:10 |
| inc0 | right because we don't have host specific stuff | 16:10 |
| sbezverk | inc0: so If I undersatnd you right, you plan to replace only the first stage | 16:10 |
| inc0 | sbezverk: for start at least | 16:11 |
| sbezverk | inc0: ok | 16:11 |
| inc0 | but first I need kfox1111 to tell me that he won't go on religious war with me if I do it in ansible that will be optional | 16:11 |
| Daviey_ | inc0: Regarding ansible, i'm assuming you are talking about laying down /etc/kolla on the remote hosts? Surely the cleaner fix is to move this to a docker volume... then ansible isn't needed? | 16:12 |
| sbezverk | inc0: :) yeah curious to see this debate | 16:12 |
| Daviey_ | (and would be cleaner for kolla-ansible aswell) | 16:12 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 16:12 |
| inc0 | Daviey_: issue we're trying to deal with is how to generate config files for kolla-k8s | 16:14 |
| *** gfhellma has joined #openstack-kolla | 16:14 | |
| inc0 | currently it's done with abominable hack | 16:14 |
| Daviey_ | inc0: right... but if we move it to a docker volume.. then it becomes cleaner for kolla-ansible and works for kolla-k8s aswell? | 16:14 |
| inc0 | so I really don't want to drive our arch on premise that "some people are alergic to ansible" | 16:14 |
| openstackgerrit | Merged openstack/kolla-ansible master: Remove heka_socket vol and unwedge some containers https://review.openstack.org/444771 | 16:15 |
| inc0 | Daviey_: well, yes and no, they still need to be created | 16:15 |
| Daviey_ | (for reference, i've been pondering a kolla-nomad project.. but the config problem is what has been bugging me) | 16:15 |
| inc0 | and docker volume is just a directory by default | 16:15 |
| inc0 | all you need to do Daviey_ is to mount them to container somehow | 16:15 |
| Daviey_ | inc0: Right... but as a volume we have less host dependency | 16:16 |
| inc0 | it can be bindmount, it can be configmap it can be docker volume | 16:16 |
| Daviey_ | Both k8 and Nomad support a notion of artifacts... So all the controller server needs to run is a http server where they can be retrieved | 16:16 |
| Daviey_ | The artifacts is a tarball of /etc/kolla | 16:16 |
| Daviey_ | shove that straight into a docker volume, and make it available to multiple containers | 16:16 |
| inc0 | Daviey_: that'd work for nomad, sure | 16:17 |
| Daviey_ | inc0: k8's can have artifacts aswell | 16:17 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements https://review.openstack.org/446772 | 16:17 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix wrong api interface reference in haproxy.cfg for nova serial https://review.openstack.org/448067 | 16:17 |
| inc0 | my point is | 16:17 |
| inc0 | you still need to create file itself:) | 16:17 |
| inc0 | how you mount it to container - that depends on orch engine | 16:17 |
| inc0 | with k8s it's configmap, with nomad it can be served over http | 16:18 |
| Daviey_ | inc0: At the moment, ansible is creating the initial rendering of config right.. for everything? Is ansible the right tool for this? | 16:19 |
| inc0 | know any better? | 16:19 |
| Daviey_ | I'm not suggesting we re-invent the world... but as a simple rendering problem.. maybe a kolla standalone tool for rendering templates.. that is then used by kolla-ansible, kolla-k8's and kolla-anythingelse ? | 16:20 |
| *** hieulq_ has quit IRC | 16:20 | |
| inc0 | Daviey_: https://review.openstack.org/#/c/447733/2 | 16:20 |
| inc0 | here I use ansible as just a rendering toolk | 16:21 |
| Daviey_ | kolla-renderconfig --output configmap ? | 16:21 |
| inc0 | much easier than writing my own from scratch | 16:21 |
| Daviey_ | fair point | 16:21 |
| inc0 | sbezverk: you know what's funny? | 16:22 |
| zhubingbing | around Bertrand Lallau | 16:22 |
| inc0 | today I thought | 16:22 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix Magnum Kolla demo in contrib https://review.openstack.org/442347 | 16:22 |
| inc0 | if we'd just use ansible as templating tool instead of helm | 16:22 |
| inc0 | we'd be done with kolla-k8s by now | 16:22 |
| *** tovin07_ has quit IRC | 16:23 | |
| *** kbaegis has joined #openstack-kolla | 16:23 | |
| sbezverk | inc0: well we had k8s running with jinja | 16:23 |
| Daviey_ | If we used Django as a templating engine.. we'd also might be done.. Doesn't mean it is graceful :) | 16:23 |
| inc0 | ansible is more than just jinja | 16:23 |
| bmace | sdake: sorry i missed your ping yesterday evening. whats up? | 16:24 |
| inc0 | Daviey_: ansible gives you both pieces we need - templating and orchiestration | 16:24 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix Telegraf startup config files issue https://review.openstack.org/447382 | 16:24 |
| inc0 | k8s can just consume resources templated | 16:24 |
| sbezverk | inc0: then community decided to go with helm... I do not think going back or ansible will be taken lightly especially after all pain we went with switching to helm.. | 16:24 |
| inc0 | but whatever, this ship has sailed | 16:24 |
| *** Mech422 has joined #openstack-kolla | 16:24 | |
| Daviey_ | inc0: But does that sound like a clean implementation of k8's? | 16:24 |
| inc0 | nah, I'm not suggesting removing helm and going back to ansible | 16:24 |
| inc0 | Daviey_: why not? | 16:24 |
| inc0 | you'll be closer to native k8s than helm imho | 16:25 |
| inc0 | as ansible will just create resource definition for you | 16:25 |
| Mech422 | SamYaple: I saw your email about rabbitmq, which brought up something I've been wondering about: if OpenStack survives/deals with the occasional lost message - why bother with rabbitmq at all ? | 16:26 |
| Daviey_ | inc0: I eat my words.. it isnt so bad.. https://github.com/mxabierto/k8-ansible | 16:26 |
| sbezverk | inc0: we need rendering engine, there is no way around it and we need packaging.. so far helm seems doing its job.. | 16:26 |
| inc0 | sbezverk: what do you mean by "packaging"? | 16:26 |
| inc0 | Daviey_: it would be better if it would be proper ansible;) | 16:27 |
| inc0 | anyway, again, this is academical by now | 16:27 |
| inc0 | I just know that when I'll write new code for k8s | 16:27 |
| inc0 | I'll use ansible for it | 16:27 |
| inc0 | I think ansible would do great as "orchiestrator" and templating engine for k8s | 16:28 |
| inc0 | rbergeron: ^ | 16:28 |
| *** jaosorior has quit IRC | 16:29 | |
| Mech422 | inc0: btw - saw the discussion about adding etcd to provisioning ... we actually just decided to do somethign similiar for our in-house stuff using consul | 16:29 |
| Daviey_ | inc0: I massively dislike the fire-and-forget of kolla-ansible and the suck-it-and-see approach to changing stuff.. Scares the crap out of me. That is why i like Nomad, as it is pretty workfow independent.. but you can easily see the changed resources before running. | 16:30 |
| inc0 | Mech422: we had some discussion on PTG to use etcd for configs...is that what you're reffering to? | 16:30 |
| inc0 | Daviey_: the way I'd do it is to ask ansible to create bunch of resource files | 16:31 |
| Mech422 | inc0: yeah - though we're prolly gonna end up doing more then configs in-house | 16:31 |
| inc0 | and feed them to k8s on my won | 16:31 |
| inc0 | configs in house are way to go | 16:31 |
| inc0 | kolla-ansible can help you but really if you can handle hand-crafting configs | 16:32 |
| inc0 | that's how I'd do it | 16:32 |
| Mech422 | inc0: big thing that drove the conversation for us was 'e-brake' functionality for our Salt stuff - stopping a upgrade and doing rollbacks if a canary deploy fails | 16:32 |
| inc0 | that's good feature to have | 16:33 |
| Daviey_ | Mech422: But you trust the rollbacks? | 16:33 |
| Mech422 | Daviey_: umm - above my paygrade ? :-P | 16:34 |
| Daviey_ | Mech422: I mean, with k8.. you can bring up a new container... show you are happy.. then divert traffic to it. The rollback is to divert traffic back to the old containers | 16:34 |
| Mech422 | Daviey_: we're charging each product with supplying rollback formulae as part of the production turnover.... | 16:34 |
| Daviey_ | CloudFormation has bitten me so many times, with failed rollbacks and you are left in a state that the tool can't fix | 16:35 |
| Mech422 | inc0: anyway - sounds like it would be a good feature to have... lots of stuff seems to want/need it.. | 16:35 |
| Mech422 | inc0: having a 'standard' one seems like a good thing | 16:35 |
| Daviey_ | Terraform, will just go "oh crap" i broke.. but leave the state defintion of what the current world is.. and allow you to fix it and keep reapplying your config to "roll-forward".. i prefer this model myself | 16:36 |
| inc0 | Mech422: it's ansible, ansible offers lots of these stuff | 16:36 |
| Daviey_ | ansible has "check mode"... to see what it plans to change, but when i last tried it with kolla-ansible it didn't work | 16:37 |
| Mech422 | Daviey_: yeah - we looked at Terraform for some stuff - but the 'state dir' seemed fragile, and we were pretty sure SysOps would end up making changes that broke state | 16:37 |
| Mech422 | Daviey_: "oh,I'll just shut this machine down since its idle" sort of stuff | 16:37 |
| Daviey_ | Mech422: I'm storing the state file in s3, with auto-versioning on s3-side. Hasn't been a problem yet. | 16:37 |
| sbezverk | gents I am debugging baremetal issue I see at the gate and I need to collect baremetal vm logs, (it is regular virsh defined vm) by some reason I cannot get bios related messages and messages which are generated even before os gets loaded, any one has experience in this? | 16:38 |
| Daviey_ | sbezverk: Ask #openstack-infra, they might be willing to pull it for you | 16:38 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 16:38 |
| inc0 | sbezverk: ^ | 16:39 |
| sbezverk | Daviey_: it is not that VM they give us for running a gate job | 16:39 |
| Mech422 | Daviey_: thats good to know.. we were worried about state, and we also liked the fact that consul/etcd let us 'share' the knowledge easily - lots of stuff can talk to them, instead of it being 'locked up' in salt pillars | 16:40 |
| sbezverk | Daviey_: it is VM I defined and use for simulating baremetal provisioning | 16:40 |
| sbezverk | inc0: how this approach would be extensible? example I have a couple of services which are not yet merged like nova placement? would it require adding new service to the list to get configmap generated? | 16:42 |
| Daviey_ | sbezverk: but if you can't see something.. and having out of band ssh to the box allows you to see it.. they might pull it out for you | 16:43 |
| Daviey_ | sbezverk: Are you dumping the qemu console out to txt? Or just the tty console (after the kernel has started)? | 16:44 |
| sbezverk | Daviey_: I mapped console to file | 16:45 |
| Daviey_ | Mech422: ah yes, i've not used the consul support for TF state.. it made me a bit nervous aswell. S3 being flat static versioned file made me feel safer. | 16:45 |
| Daviey_ | sbezverk: right, but which console? tty/kernel or the 'server' console (which would include the bios output) | 16:46 |
| Mech422 | Daviey_: oh - we're not (planning on?) using Terraform atm. | 16:46 |
| Mech422 | Daviey_: first we're just going with a reliable storage setup for 'state-like' stuff | 16:47 |
| Mech422 | Daviey_: we may revisit TF later though | 16:47 |
| Daviey_ | Mech422: so what are you using now? | 16:47 |
| rbergeron | inc0: aye -- i assume you have seen ansible-container -- https://github.com/ansible/ansible-container (may not be exactly what you're looking for) -- but there are kubernetes modules already, and someone was also working on (though i haven't yet seen) a helm module for ansible | 16:47 |
| rbergeron | inc0: https://github.com/kubernetes-incubator/kargo/issues/661#issuecomment-265474063 | 16:48 |
| rbergeron | inc0: just depends on how you want to abstract the abstractions and orchestrations of orchestrators :) | 16:48 |
| *** Daviey_ is now known as Daviey | 16:49 | |
| Mech422 | Daviey_: for most stuff - just plain Salt | 16:49 |
| *** bmace has quit IRC | 16:49 | |
| Daviey | Mech422: ah, cool | 16:49 |
| *** bmace has joined #openstack-kolla | 16:49 | |
| sbezverk | Daviey_: in my local lab I connect with VNC to that VM and see everything even bios related messages. at the gate I do not have luxury of VNC but I need to see bios related messages as I suspect the issue is before OS gets loaded | 16:49 |
| Daviey | -> #o'-infra | 16:50 |
| rbergeron | inc0: also, iirc flaper87 may also be looking at such things | 16:50 |
| *** crushil has quit IRC | 16:51 | |
| *** satyar has quit IRC | 16:51 | |
| * flaper87 looks at many things (including the backlog of this conversation) | 16:51 | |
| *** jaosorior has joined #openstack-kolla | 16:55 | |
| *** coding_shardy is now known as shardy | 16:57 | |
| *** kbaegis has quit IRC | 17:00 | |
| sdake | hey peeps | 17:01 |
| sdake | inc0 we made a decision as a community to dump ansible workflows in kolla-kubernetes | 17:02 |
| sdake | inc0 that was mostly driven by you :) | 17:03 |
| *** egonzalez has quit IRC | 17:04 | |
| *** cu5 has quit IRC | 17:05 | |
| *** yee379 has quit IRC | 17:06 | |
| *** yee379 has joined #openstack-kolla | 17:06 | |
| *** cu5 has joined #openstack-kolla | 17:06 | |
| *** kbaegis has joined #openstack-kolla | 17:07 | |
| *** vhosakot has joined #openstack-kolla | 17:09 | |
| *** srwilkers has quit IRC | 17:10 | |
| *** unicell has quit IRC | 17:10 | |
| *** skramaja has quit IRC | 17:12 | |
| sdake | bmace i wanted to know if you could review https://review.openstack.org/#/c/447356/ | 17:15 |
| sdake | inc0 helm is a package manager - that is how it is being treated for the most part - it may grow workflow features in the future | 17:16 |
| sdake | inc0 first we want to get to helm native packaging of all the stuff - and then sort out the workflow natively | 17:16 |
| sdake | alot of the workflow is driven by how helm packages stuff up | 17:17 |
| bmace | sdake: sure, will look at it in a few | 17:17 |
| kfox1111 | morning. | 17:17 |
| sdake | that packaged up bits we are missing is "config maps" | 17:17 |
| sdake | sup kfox1111 | 17:17 |
| sdake | inc0 creating configmaps != workflow | 17:17 |
| sdake | bmace thanks - hopefully plan to spin that document today with eveyrone's comments fixed | 17:18 |
| *** zhubingbing has quit IRC | 17:22 | |
| *** srwilkers has joined #openstack-kolla | 17:22 | |
| sdake | srwilkers o/ | 17:22 |
| sbezverk | kfox1111: good morning | 17:25 |
| sbezverk | sdake: good morning | 17:25 |
| *** pbourke_ is now known as pbourke | 17:25 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements https://review.openstack.org/446772 | 17:26 |
| sdake | sup sbezverk | 17:26 |
| sdake | I got a late start today ;( | 17:26 |
| sdake | all that travel caught up with me I htink :) | 17:26 |
| sbezverk | kfox1111: gate job driving me mad :( the same damn code work 100% in my test bed, but still failing at the gate. | 17:26 |
| sbezverk | kfox1111: vm does not call back to ironic-api | 17:27 |
| *** kbaegis has quit IRC | 17:27 | |
| kfox1111 | sbezverk: hmm... | 17:27 |
| kfox1111 | iptables? | 17:27 |
| *** kbaegis has joined #openstack-kolla | 17:27 | |
| kfox1111 | try turning it off. | 17:27 |
| sdake | i think flushing the rules first makes more sens ethen turning it off - I suspect but odn't knwo that k8s and openstack use iptables in some way | 17:28 |
| kfox1111 | sdake: yeah. thanks. that s what I meant. | 17:28 |
| kfox1111 | flush it at the very begining of the gate job, then see if it works better. | 17:28 |
| sdake | sbezverk iptables -f I think does that | 17:28 |
| kfox1111 | I know they have some very custom iptables rules on the vms. | 17:29 |
| srwilkers | hey sdake o/ | 17:29 |
| *** lamt has quit IRC | 17:29 | |
| sdake | sbezverk although best to reference stackoverflow | 17:30 |
| sdake | everytime i want to flush rules, i have to look it up:) | 17:30 |
| *** zhubingbing has joined #openstack-kolla | 17:31 | |
| sbezverk | kfox1111: I finally get config to log all VM messages even when it is in bios | 17:32 |
| kfox1111 | sbezverk: awesome. :) | 17:32 |
| kfox1111 | what did you do? | 17:32 |
| sbezverk | hopefully that could help to identtify at what point it get stuck | 17:32 |
| sbezverk | kfox1111: there is an option in virsh | 17:32 |
| kfox1111 | ah. cool. | 17:32 |
| sbezverk | kfox1111: <bios useserial='yes'/> | 17:33 |
| *** tonanhngo has quit IRC | 17:33 | |
| kfox1111 | nice. | 17:33 |
| sbezverk | kfox1111: if not use then only OS messages are logged and since I was not reaching OS stage, nothing got logged | 17:33 |
| kfox1111 | yeah. that makes sense. | 17:34 |
| *** shashank_t_ has quit IRC | 17:35 | |
| inc0 | sdake: I'm only talking of configmaps tnow | 17:36 |
| *** rwsu has joined #openstack-kolla | 17:36 | |
| inc0 | now | 17:36 |
| *** david-lyle_ has joined #openstack-kolla | 17:37 | |
| inc0 | kfox1111: sbezverk where are files that genconfig created in gates? | 17:37 |
| inc0 | files normally would land in /etc/kolla | 17:37 |
| kfox1111 | /etc/kolla | 17:37 |
| sdake | copying over the genconfig code from kolla-ansible and landing just those bits would make the ejection of kolla-ansible as a dep go faster | 17:37 |
| sdake | as in immediate | 17:37 |
| inc0 | yup | 17:38 |
| inc0 | that's what I wanted to doscuss | 17:38 |
| sdake | genpwd needs to be copied | 17:38 |
| sdake | as well | 17:38 |
| *** david-lyle has quit IRC | 17:38 | |
| sdake | i was planning to do that this week when i got to it :) | 17:38 |
| inc0 | I can make this patchset probably...idk today? without gates | 17:38 |
| sdake | (genpwd copying) | 17:38 |
| kfox1111 | hmm... so, fork genconfig and strip it? | 17:38 |
| sdake | kfox1111 that is the proposal on the table :) | 17:38 |
| inc0 | kfox1111: exactly | 17:38 |
| kfox1111 | interesting. | 17:38 |
| inc0 | add all the missing templates | 17:38 |
| kfox1111 | that should be very quick to do, and greatly simplify things... | 17:39 |
| *** unicell has joined #openstack-kolla | 17:39 | |
| inc0 | combine with with logic to create k8s resources instead of only configs | 17:39 |
| sdake | its a winner | 17:39 |
| inc0 | kk I'm on it | 17:39 |
| sdake | we still need external config maps after tha tstep :) | 17:39 |
| kfox1111 | +1 | 17:39 |
| *** eaguilar has joined #openstack-kolla | 17:39 | |
| inc0 | well it won't on it's own solve per-node overrides | 17:39 |
| sdake | ejecting kolla-ansible as a dep woudl greatly simplify kolla-kubernetes dev | 17:39 |
| sdake | ya per node overrides can be done with helm configmaps I am pretty sure | 17:40 |
| inc0 | we need to bake this into helm configmap mounting logic | 17:40 |
| inc0 | so we can *create* per node configmap with ansible | 17:40 |
| sdake | just add multiple config maps | 17:40 |
| inc0 | to some extend | 17:40 |
| *** EmilienM has joined #openstack-kolla | 17:40 | |
| sdake | and run nova config1 config2 etc | 17:40 |
| inc0 | as in, if we provide all the interface/ips and stuff | 17:40 |
| EmilienM | sdake: the -1 sounds unfair on https://review.openstack.org/#/c/448043 tbh | 17:41 |
| EmilienM | inc0: ^ | 17:41 |
| sdake | EmilienM i didn't -1 it :) | 17:41 |
| sdake | EmilienM moment | 17:41 |
| * EmilienM trying to be helpful | 17:41 | |
| inc0 | EmilienM: gates are red... | 17:41 |
| EmilienM | sdake: I know but caoyuan did, not sure if this person is here | 17:41 |
| EmilienM | inc0: yeah no worries | 17:41 |
| sdake | EmilienM lemme recheck it | 17:42 |
| EmilienM | I just found useful to show this kind of useless -1 review | 17:42 |
| inc0 | and that was his comment, so comment was...redundant;) | 17:42 |
| inc0 | yeah, I'll add this to review | 17:42 |
| sdake | EmilienM even though our gates are nonvoting we do want them to be green :) | 17:42 |
| inc0 | voting gates are red too | 17:42 |
| inc0 | centos source namely | 17:42 |
| sdake | inc0 the gates are jammed up atm I believe | 17:42 |
| sdake | inc0 the ironic bug | 17:43 |
| kfox1111 | inc0: your very concerned with per node config. but k8s is all about not having to do that. | 17:43 |
| sdake | with inspector | 17:43 |
| sdake | wrather with biforst | 17:43 |
| sdake | too many components :) | 17:43 |
| inc0 | kfox1111: well...neutron still needs it's interfaces | 17:43 |
| inc0 | and provider_network still needs it's vlan tags | 17:43 |
| kfox1111 | inc0: yes, but you very very veary rarely have a situation where you have every machine needs a different interface. | 17:43 |
| inc0 | not to mention all sorts of host aggregatey stuff | 17:43 |
| kfox1111 | usualy its a few dozen hardware types. or less. | 17:44 |
| sdake | EmilienM change looks good to me | 17:44 |
| sdake | EmilienM i think our gates are temporarily broken | 17:44 |
| inc0 | kfox1111: yeah, usually it's "just" 2 or 3;) | 17:44 |
| kfox1111 | so, a k8s host aggregate like thing, not a per host thing. | 17:44 |
| EmilienM | yep, I figured, no worries | 17:44 |
| sdake | EmilienM lemme dig into the gate and see what is busted | 17:44 |
| kfox1111 | so, deploy multiple instances of nova-openvswitch-agent with node selectors and a configmap. | 17:44 |
| sdake | EmilienM (if its still the bifrost thing) | 17:44 |
| inc0 | kfox1111: so create configmap1, deploy on aggregate 1, configmap2 deploy on aggregate 2 | 17:45 |
| kfox1111 | inc0: yeah. | 17:45 |
| inc0 | let me put it this way, config overrides per agregate wfm | 17:45 |
| inc0 | then you have configmap-aggregate1 and helm/k8s will mount it | 17:45 |
| kfox1111 | right. | 17:46 |
| *** Serlex has quit IRC | 17:46 | |
| inc0 | that's totally fine | 17:46 |
| inc0 | then in ansible you can create configmaps per aggregate | 17:46 |
| kfox1111 | right. | 17:46 |
| inc0 | for example -e aggregate=aggregate1 | 17:46 |
| inc0 | cool, totally doable then | 17:46 |
| kfox1111 | though I don't think we want to spend a lot of time making that work very smoothly before doing it in helm. | 17:47 |
| sdake | EmilienM enjoy :) | 17:47 |
| inc0 | kfox1111: thing is, I think we will already have this mechanism | 17:47 |
| kfox1111 | helm install neutron-openvswitch-configmap --set interface=eth0,name=aggregate1 or something like that. | 17:47 |
| inc0 | with our current kolla-ansible arch;) | 17:47 |
| kfox1111 | ah. ok. | 17:47 |
| inc0 | maybe with one more line per task | 17:47 |
| sdake | i guess that is a bit of scope creep for 1.0.0 | 17:48 |
| inc0 | then you can create variable "kubernetes_aggregates: ag1, ag2" | 17:48 |
| *** iceyao has quit IRC | 17:48 | |
| inc0 | sdake: it's few hrs of work | 17:48 |
| inc0 | and we get whole override mechanism | 17:48 |
| sdake | you indicated you wouldn't get married to the code ;) | 17:48 |
| inc0 | its *much* easier than writing config mechanism to helm | 17:48 |
| sdake | when we go to helm i don't think it will be a few hours of work | 17:48 |
| inc0 | my point exactly | 17:49 |
| *** tonanhngo has joined #openstack-kolla | 17:49 | |
| inc0 | helm might be bad tool for this task | 17:49 |
| sdake | disagree ;) | 17:49 |
| sdake | although ejecting kolla-ansible is good objective | 17:49 |
| inc0 | anyway | 17:49 |
| inc0 | if we have fully feature complete config generation in kolla-k8s | 17:49 |
| inc0 | if someone wants to go lenghts to rewrite it to helm | 17:49 |
| inc0 | fine by me | 17:49 |
| inc0 | but I'd change 1.0 scope to remove requirement of this being in helm | 17:50 |
| sdake | cool - as long as your not married to it like I said :) | 17:50 |
| inc0 | I'm always about what makes sense | 17:50 |
| *** tonanhngo has quit IRC | 17:50 | |
| inc0 | I have no emotional attachments to any code | 17:50 |
| *** zhubingbing has quit IRC | 17:50 | |
| sdake | lies :) | 17:50 |
| inc0 | and I'm slowly startking to challenge helm being good tool for this job | 17:51 |
| inc0 | but that's me | 17:51 |
| inc0 | I'm entitled to have my own opinions;) | 17:51 |
| sdake | agreed | 17:51 |
| kfox1111 | yup. | 17:51 |
| kfox1111 | thats why we did microservices. | 17:51 |
| kfox1111 | to allow anyone to do config, however is best for them. | 17:51 |
| kfox1111 | if ansible's the right solution for you, great. | 17:51 |
| inc0 | beautiful thing about moving this ansible to kolla-k8s | 17:51 |
| kfox1111 | if its helm, or chef or shell, great. | 17:51 |
| inc0 | is that it's totally optional | 17:52 |
| inc0 | all we need is config files in a dir | 17:52 |
| inc0 | and we ship one way to generate them | 17:52 |
| inc0 | namely ansible | 17:52 |
| *** satyar has joined #openstack-kolla | 17:52 | |
| inc0 | this approach is acceptable? | 17:52 |
| *** crushil has joined #openstack-kolla | 17:53 | |
| inc0 | that will also make poritng good changes from kolla-ansible to kolla-k8s much easier | 17:53 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 17:54 |
| sbezverk | inc0: guys, are you kidding we spent whole cycle helming everything and now you want to change again!?!?!?! | 17:56 |
| *** sayantani01 has quit IRC | 17:56 | |
| kfox1111 | I'm ok with it as a transition thing. as of long term, less convinced. | 17:57 |
| kfox1111 | config generation is very religious. | 17:57 |
| *** sayantani01 has joined #openstack-kolla | 17:57 | |
| kfox1111 | sbezverk: only talking configmaps. | 17:57 |
| *** jaosorior has quit IRC | 17:57 | |
| bmace | sdake: so, is that doc going to get a lot of massage? it is kind of a mess right now. | 17:58 |
| sbezverk | kfox1111: ok if it is just configmaps, it scared me, so much time invested into helm.. | 17:58 |
| sdake | bmace its getting a full spa treatment after you review it :) | 17:58 |
| *** srwilkers has quit IRC | 17:58 | |
| *** rwsu has quit IRC | 17:59 | |
| bmace | sdake: kk, it is just a little chaotic at this point to get a feel for what really needs to be fixed.. depending on what you are going to yank out completely, which i hope is a lot, like all the in-line comments, etc? | 17:59 |
| inc0 | kfox1111: yeah...well...again optional;) | 18:00 |
| inc0 | anyway. it's quck win | 18:00 |
| inc0 | quick win | 18:00 |
| inc0 | so let's take it | 18:00 |
| *** rwsu has joined #openstack-kolla | 18:00 | |
| inc0 | kfox1111: http://logs.openstack.org/33/447733/3/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/7aaa270/console.html#_2017-03-21_17_00_24_616166 you sure config files are in /etc/kolla/? | 18:00 |
| *** lamt has joined #openstack-kolla | 18:01 | |
| *** scsnow_ has quit IRC | 18:01 | |
| *** mgoddard has quit IRC | 18:01 | |
| kfox1111 | sbezverk: yeah. not backing out of helm. its really the right solultion I think for the microservices. | 18:03 |
| inc0 | yeah, I agree | 18:03 |
| *** zhubingbing has joined #openstack-kolla | 18:04 | |
| kfox1111 | inc0: https://github.com/openstack/kolla-kubernetes/blob/master/tests/bin/gate_capture_logs.sh#L101 | 18:04 |
| kfox1111 | inc0: http://logs.openstack.org/33/447733/3/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/7aaa270/logs/kolla/ | 18:05 |
| kfox1111 | copyied verbatium into the g ate logs, for your inspection. :) | 18:05 |
| inc0 | hmm | 18:05 |
| *** scsnow_ has joined #openstack-kolla | 18:06 | |
| inc0 | right... | 18:06 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support https://review.openstack.org/446274 | 18:07 |
| inc0 | kfox1111: do we copy kubectl describe po too? | 18:09 |
| kfox1111 | yes | 18:09 |
| kfox1111 | http://logs.openstack.org/33/447733/3/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/7aaa270/logs/pods/ | 18:09 |
| kfox1111 | anything with a -containername.txt is logs form the container, | 18:09 |
| kfox1111 | and the other is the describe pod. | 18:09 |
| kfox1111 | so, like: http://logs.openstack.org/33/447733/3/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/7aaa270/logs/pods/kolla-ceph-rbd-351jn-main.txt | 18:10 |
| kfox1111 | logs for the container. | 18:10 |
| kfox1111 | http://logs.openstack.org/33/447733/3/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/7aaa270/logs/pods/kolla-ceph-rbd-351jn.txt describe for the pod | 18:10 |
| *** klindgren has quit IRC | 18:10 | |
| inc0 | ahh...kollakube does secrets too? | 18:10 |
| kfox1111 | kollakube did it all, pre helm. | 18:10 |
| kfox1111 | there is still a few non configmap things in it. | 18:11 |
| inc0 | https://review.openstack.org/#/c/447733/3/tools/setup_gate_ceph.sh | 18:11 |
| inc0 | here, gates failed because of no secret created | 18:11 |
| kfox1111 | which secret? | 18:12 |
| inc0 | http://logs.openstack.org/33/447733/3/check/gate-kolla-kubernetes-deploy-ubuntu-binary-2-ceph-nv/1ebe874/logs/pods/kolla-mariadb-init-element-n228k.txt | 18:12 |
| inc0 | nvm | 18:12 |
| inc0 | I misread it | 18:12 |
| kfox1111 | k. :) | 18:12 |
| inc0 | riiiiight | 18:13 |
| inc0 | I know what happened | 18:13 |
| inc0 | --namespace kolla :( | 18:13 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 18:14 |
| bmace | sdake: i don't see anything super obvious.. would love to see a post-spa version of it and then perhaps when it is more readable it will be easier to find issues. | 18:14 |
| *** scsnow_ has quit IRC | 18:18 | |
| *** shardy has quit IRC | 18:18 | |
| *** manheim has joined #openstack-kolla | 18:20 | |
| *** tonanhngo has joined #openstack-kolla | 18:21 | |
| *** tonanhngo has quit IRC | 18:21 | |
| jascott1 | kfox1111 ping | 18:22 |
| kfox1111 | jascott1: pong | 18:22 |
| jascott1 | hey whats up | 18:22 |
| jascott1 | got an import values question | 18:22 |
| kfox1111 | doing good. you? | 18:22 |
| kfox1111 | sure | 18:22 |
| jascott1 | great, signing papers today for house im buying! | 18:23 |
| kfox1111 | congrats. :) | 18:23 |
| jascott1 | thanks! | 18:23 |
| dims | jascott1 : nice! | 18:23 |
| jascott1 | so yeah | 18:23 |
| jascott1 | thanks dims | 18:24 |
| jascott1 | wrt to import values | 18:24 |
| jascott1 | i dont think we talked about overwriting | 18:24 |
| jascott1 | if parent value already exists | 18:24 |
| jascott1 | we use imported or parent? | 18:24 |
| kfox1111 | parent always wins. | 18:24 |
| kfox1111 | and users values wins over parent. | 18:25 |
| jascott1 | right | 18:25 |
| *** manheim has quit IRC | 18:25 | |
| jascott1 | yeah I didnt do that | 18:26 |
| jascott1 | but will fix up in jiffy | 18:26 |
| kfox1111 | ah. ok. cool. | 18:26 |
| jascott1 | idk what i was thinking | 18:26 |
| kfox1111 | no wories. :) | 18:26 |
| jascott1 | hey did you get your data center real world problems all sorted? | 18:27 |
| kfox1111 | limping along stablily now. :) | 18:28 |
| *** jmccarthy has quit IRC | 18:28 | |
| jascott1 | nice | 18:28 |
| kfox1111 | we're maybe a week or two, or alternately months from getting that ups back. | 18:28 |
| *** zhubingbing has quit IRC | 18:28 | |
| *** mgoddard has joined #openstack-kolla | 18:37 | |
| *** pcaruana has quit IRC | 18:39 | |
| *** rwsu has quit IRC | 18:42 | |
| *** haplo37 has quit IRC | 18:44 | |
| *** david-lyle_ has quit IRC | 18:46 | |
| *** haplo37 has joined #openstack-kolla | 18:49 | |
| *** eaguilar has quit IRC | 18:52 | |
| *** lamt has quit IRC | 19:06 | |
| *** shashank_t_ has joined #openstack-kolla | 19:07 | |
| *** tonanhngo has joined #openstack-kolla | 19:09 | |
| *** tonanhngo has quit IRC | 19:10 | |
| *** satyar has quit IRC | 19:10 | |
| *** rwsu has joined #openstack-kolla | 19:12 | |
| *** rwsu has quit IRC | 19:17 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 19:25 |
| inc0 | kfox1111: parent wins over child? | 19:27 |
| inc0 | hmm...I'd rather go child winds over parent and don't add defaults unless need to | 19:28 |
| inc0 | like, neutron image == 3.0.2 where all other images == 3.0.3 | 19:28 |
| inc0 | but I guess that could be done by redefinition of new variable | 19:29 |
| *** MasterOfBugs has joined #openstack-kolla | 19:36 | |
| *** kbaegis has quit IRC | 19:37 | |
| *** kbaegis has joined #openstack-kolla | 19:38 | |
| *** kbaegis has quit IRC | 19:40 | |
| *** kbaegis has joined #openstack-kolla | 19:41 | |
| *** kbaegis has quit IRC | 19:43 | |
| *** mgoddard has quit IRC | 19:44 | |
| *** kbaegis has joined #openstack-kolla | 19:44 | |
| *** rhallisey has quit IRC | 19:45 | |
| *** rwallner has quit IRC | 19:46 | |
| *** rwallner has joined #openstack-kolla | 19:47 | |
| *** tonanhngo has joined #openstack-kolla | 19:47 | |
| *** rhallisey has joined #openstack-kolla | 19:48 | |
| *** tonanhngo has quit IRC | 19:49 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP Move config generation to kolla-k8s https://review.openstack.org/448258 | 19:50 |
| inc0 | kfox1111: sbezverk ^ | 19:50 |
| inc0 | just neutron | 19:50 |
| inc0 | but check approach | 19:50 |
| *** MasterOfBugs has quit IRC | 19:52 | |
| *** lamt has joined #openstack-kolla | 19:53 | |
| *** kbaegis has quit IRC | 19:55 | |
| *** kbaegis has joined #openstack-kolla | 19:57 | |
| *** kbaegis has quit IRC | 20:00 | |
| *** dave-mccowan has quit IRC | 20:02 | |
| *** manheim has joined #openstack-kolla | 20:02 | |
| *** matrohon has joined #openstack-kolla | 20:02 | |
| sbezverk | kfox1111: could you check this error, while centos is always passing this point, ubuntu always fails http://logs.openstack.org/50/441550/102/experimental/gate-kolla-kubernetes-deploy-ubuntu-source-3-ironic-nv/4bdd216/console.html#_2017-03-21_19_47_33_854459 | 20:03 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 20:03 |
| *** rwallner has quit IRC | 20:03 | |
| *** rwellum has quit IRC | 20:03 | |
| *** rwellum_ is now known as rwellum | 20:03 | |
| *** rwallner has joined #openstack-kolla | 20:04 | |
| *** kbaegis has joined #openstack-kolla | 20:05 | |
| inc0 | lmao check "conflicts with" https://review.openstack.org/#/c/448258/ | 20:05 |
| sbezverk | inc0: could you comment on one point. ansible bases its actions on an inventory file, in kolla kubernetes to have inventory file is against the nature of kubernetes cluster where you have no knowledge about your inventory. how it is going to work? | 20:06 |
| inc0 | sbezverk: well, it *may* use inventory file | 20:06 |
| inc0 | https://review.openstack.org/#/c/448258/1/ansible/site.yml | 20:06 |
| sbezverk | inc0: Ah so inventory is not a mandatory? | 20:07 |
| inc0 | hosts: localhost connection:local | 20:07 |
| sbezverk | inc0: got it | 20:07 |
| inc0 | you can specify this in play | 20:07 |
| inc0 | and in our case, that's exactly what we're going to do | 20:07 |
| *** rwallner has quit IRC | 20:08 | |
| sbezverk | inc0: are you going to clean up this ps, it seems it has bunch of unrelated staff, things like reconfig and etc? | 20:10 |
| inc0 | sbezverk: sure | 20:10 |
| sbezverk | inc0: cause now it looks scary ;) | 20:10 |
| *** MasterOfBugs has joined #openstack-kolla | 20:11 | |
| sdake | dims is go supported by the infra team currently? | 20:13 |
| sdake | dims I ask as your leading the go effort :) | 20:13 |
| dims | sdake : yes | 20:13 |
| sdake | TIL :) | 20:13 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP Move config generation to kolla-k8s https://review.openstack.org/448258 | 20:14 |
| inc0 | sbezverk: ^ how about now? | 20:14 |
| *** pbourke has quit IRC | 20:16 | |
| sbezverk | inc0: man that looks way more human :) | 20:18 |
| sdake | inc0 i don't htink we nbeed most of the library code | 20:18 |
| inc0 | sdake: slowly | 20:18 |
| sdake | and also need the templates for more then neutron | 20:18 |
| inc0 | I think w ecan thin it up more | 20:18 |
| inc0 | but I want to get it to work and test first | 20:18 |
| inc0 | obviously... | 20:19 |
| inc0 | I want to do one service at the time tho as it's easier to handle | 20:19 |
| sdake | wfm | 20:19 |
| *** tonanhngo has joined #openstack-kolla | 20:19 | |
| sbezverk | inc0: probably if you get config generation for one service as PoC then the rest of services should be easy to add | 20:19 |
| inc0 | that's the plan | 20:19 |
| inc0 | and since it's kinda tradition in kk8s | 20:20 |
| sbezverk | inc0: great | 20:20 |
| inc0 | service is going to be neutron | 20:20 |
| *** tonanhngo has quit IRC | 20:20 | |
| *** jemcevoy has quit IRC | 20:20 | |
| sbezverk | inc0: cool, you can drop openvswitch things as they are not considerd under nuetron | 20:21 |
| mnaser | is kolla-ansible and kolla-k8s maintaining their own subtrees of ansible code? | 20:22 |
| sdake | inc0 i think a path forward is your work -> generate helm charts for the cnofigmaps | 20:22 |
| *** rhallisey has quit IRC | 20:22 | |
| inc0 | mnaser: not yet, but that's poc I'm working on now | 20:22 |
| sdake | mnaser only genconfig - not full workflow | 20:22 |
| mnaser | oh okay | 20:22 |
| inc0 | for k8s tho it will be *MUCH* simpler one | 20:22 |
| sdake | mnaser and idea is to eject ansible/genconfig when we find a suitable replacement | 20:22 |
| inc0 | we'll see about helm:P | 20:23 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Use service_name variable when define in neutron role https://review.openstack.org/448269 | 20:23 |
| *** lamt has quit IRC | 20:23 | |
| inc0 | I'd like to have discussion | 20:23 |
| mnaser | whats genconfig | 20:23 |
| inc0 | if I get this to work | 20:23 |
| inc0 | mnaser: kolla-ansible genconfig | 20:23 |
| inc0 | so "just render me a damn config files" | 20:23 |
| sdake | mnaser that is the thing that sets the default configuration in /etc/kolla for all of the services | 20:23 |
| mnaser | ah okay okay | 20:23 |
| mnaser | gotch | 20:23 |
| mnaser | +a | 20:23 |
| sdake | mnaser e.g. kolla-ansible genconfig will write out /etc/kolla/nova.conf | 20:23 |
| mnaser | i never used that feature but | 20:24 |
| inc0 | that's how we create config files in kk8s today | 20:24 |
| mnaser | that would have been nice to know | 20:24 |
| mnaser | when we were doing our migration :P | 20:24 |
| mnaser | TIL | 20:24 |
| sdake | mnaser its not really a feature we want t o keep in kolla-ansible ;) | 20:24 |
| sdake | although we are probably stuck with it if operators use it | 20:24 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP Move config generation to kolla-k8s https://review.openstack.org/448258 | 20:24 |
| inc0 | it's a good feature | 20:25 |
| inc0 | if you want to ship hand crafted configs | 20:25 |
| *** jemcevoy has joined #openstack-kolla | 20:25 | |
| inc0 | to carefully compare with what's going to happen once you run k-ans | 20:25 |
| sdake | inc0 at some point you want to integrate with clouds.yaml | 20:26 |
| sdake | inc0 what that integration means is up to you :) | 20:27 |
| *** crushil has quit IRC | 20:27 | |
| inc0 | well, it's different tho (if we're talking of clouds.yaml as replacement of openrc) | 20:27 |
| sdake | sbezverk can explain the purpose of cloud.yaml much more eloquently then I can - I barely understand it - however its an override mechanism for config - not a replacement for openrc | 20:28 |
| inc0 | sbezverk: please... | 20:28 |
| *** lamt has joined #openstack-kolla | 20:28 | |
| *** lamt has quit IRC | 20:29 | |
| sbezverk | inc0: cloud yaml allows to describe your environment completely in terms of settings/values etc | 20:30 |
| mnaser | https://github.com/openstack/os-client-config/blob/master/README.rst | 20:30 |
| sbezverk | basically once you build your cloud yaml, rebuilding the environment becomes a very easy task | 20:30 |
| mnaser | if you're up for a fun read | 20:30 |
| mnaser | unless we're talking | 20:31 |
| mnaser | about another clouds.yaml.. | 20:31 |
| inc0 | sbezverk: cloud.yaml == globals.yaml for us | 20:31 |
| sbezverk | inc0: yes | 20:31 |
| inc0 | want to change name from globals to clouds?:P | 20:31 |
| inc0 | because we can do it:P | 20:31 |
| inc0 | mnaser: I know of clouds.yaml as in "my endpoints are there, and my passwords" | 20:32 |
| sbezverk | inc0: not sure, cloud yaml is PERSONAL configuration | 20:32 |
| inc0 | aka brave new openrc | 20:32 |
| mnaser | same inc0 | 20:32 |
| sbezverk | global.yaml is just a list of default variables | 20:32 |
| mnaser | ok so | 20:32 |
| mnaser | i see where sbezverk is coming from and | 20:32 |
| mnaser | we did something simila | 20:32 |
| inc0 | sbezverk: negative, globals are personal, defaults are in ansible dir | 20:32 |
| *** dave-mccowan has joined #openstack-kolla | 20:32 | |
| mnaser | we maintain per-env folder that contains everything that /etc/kolla would contain for a specific env | 20:32 |
| sbezverk | inc0: like for me to rebuild kolla-kube takes very little effort | 20:33 |
| mnaser | and we run tools/run-kolla.sh <envname> deploy | 20:33 |
| sbezverk | as my cloud.yaml is exactly what I need it to be | 20:33 |
| inc0 | well...again, you're talking about globals.yaml in kolla-ansible nomenclature;) | 20:33 |
| mnaser | which wraps around kolla-ansible and reconfigures the config dir to a local dir | 20:33 |
| inc0 | mnaser: so if you run kolla-ansible | 20:34 |
| inc0 | it will output ansible-playbook .... | 20:34 |
| inc0 | (pretty long command) | 20:34 |
| inc0 | as first line | 20:34 |
| inc0 | kolla-ansible only builds this command | 20:34 |
| mnaser | oh yeah | 20:34 |
| mnaser | but i figured its safer to keep kolla-ansible in there | 20:34 |
| inc0 | so you can just wrap this one | 20:34 |
| mnaser | just in case some things change | 20:34 |
| inc0 | well I don't think we'll ever change this behavior | 20:35 |
| inc0 | I wouldn't be ok with it | 20:35 |
| mnaser | (hell at first we integrated it by including the roles in role_path) | 20:35 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 20:35 |
| inc0 | as kolla-ansible is just glorified alias | 20:35 |
| mnaser | so our very own site.yml would run the roles/include the site.yml from kolla | 20:35 |
| mnaser | but that was starting to push some boundaries lo | 20:35 |
| inc0 | yeah, site.yaml might change | 20:35 |
| inc0 | kolla-ansible - unlikely | 20:35 |
| inc0 | ansible-playbook allows you much more runtime modifications than kolla-ansible uses | 20:36 |
| inc0 | and we want to keep this | 20:36 |
| *** shashank_t_ has quit IRC | 20:37 | |
| sbezverk | inc0: hmm I thought helm/all_values.yaml is more like global.yaml or maybe it is kind of mixture of both.. | 20:37 |
| *** tonanhngo has joined #openstack-kolla | 20:37 | |
| *** shashank_t_ has joined #openstack-kolla | 20:37 | |
| *** tonanhngo has quit IRC | 20:38 | |
| inc0 | helm/all_values would be equivalent to this: https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml | 20:38 |
| *** shashank_t_ has quit IRC | 20:38 | |
| inc0 | *this* is defaults file | 20:38 |
| *** shashank_t_ has joined #openstack-kolla | 20:39 | |
| *** manheim has quit IRC | 20:39 | |
| sbezverk | inc0: right | 20:39 |
| *** shashank_t_ has quit IRC | 20:43 | |
| inc0 | sbezverk: you sir are wrong;) kubectl works with folder all righnt | 20:45 |
| inc0 | if you run kubectl create -f /path/to/dir | 20:45 |
| *** jemcevoy has quit IRC | 20:45 | |
| inc0 | it will try to create everything in dir that ends with .yaml or .json | 20:45 |
| inc0 | it's neat | 20:45 |
| sbezverk | inc0: you do not want to combine them in a single yaml file? | 20:46 |
| inc0 | no need imho | 20:47 |
| sbezverk | inc0: cause then you will need to loop through them | 20:47 |
| inc0 | no | 20:47 |
| inc0 | I'm saying | 20:47 |
| inc0 | try to create multiple files in same dir | 20:47 |
| inc0 | and run kubectl create on whole dir | 20:47 |
| *** klindgren has joined #openstack-kolla | 20:48 | |
| inc0 | it will pick up all the files on it's own | 20:48 |
| *** jemcevoy has joined #openstack-kolla | 20:48 | |
| sbezverk | inc0: I have never seen kubectl capable of creating object for all files in a directory | 20:48 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 20:49 |
| inc0 | sbezverk: well...it is capable:) | 20:49 |
| inc0 | check it out! | 20:49 |
| sbezverk | + kubectl create --namespace=kolla -f /tmp/kolla-kubernetes/configmaps | 20:50 |
| sbezverk | 2017-03-21 20:16:27.061001 | error: You must provide one or more resources by argument or filename. | 20:50 |
| inc0 | ahh it fails on centos for different reason | 20:50 |
| inc0 | which I just fixed (I hope) | 20:50 |
| sbezverk | inc0: if it is working, great | 20:50 |
| inc0 | in ubuntu it works, in centos I have to install libselinux-python | 20:50 |
| inc0 | which I just did | 20:51 |
| inc0 | ;_) | 20:51 |
| *** athomas has joined #openstack-kolla | 21:05 | |
| *** shashank_t_ has joined #openstack-kolla | 21:07 | |
| *** crushil has joined #openstack-kolla | 21:09 | |
| *** lamt has joined #openstack-kolla | 21:10 | |
| *** shashank_t_ has quit IRC | 21:11 | |
| *** kbaegis has quit IRC | 21:17 | |
| *** kbaegis has joined #openstack-kolla | 21:17 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 21:20 |
| *** david-lyle has joined #openstack-kolla | 21:23 | |
| *** jtriley has quit IRC | 21:25 | |
| *** tonanhngo has joined #openstack-kolla | 21:25 | |
| *** jrobinson has joined #openstack-kolla | 21:25 | |
| *** tonanhngo has quit IRC | 21:27 | |
| *** Pavo has joined #openstack-kolla | 21:27 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP Move config generation to kolla-k8s https://review.openstack.org/448258 | 21:32 |
| *** shashank_t_ has joined #openstack-kolla | 21:33 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 21:34 |
| *** Jeffrey4l_ has quit IRC | 21:35 | |
| *** crushil has quit IRC | 21:36 | |
| inc0 | kfox1111: around? | 21:37 |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Use service_name variable when defined in neutron role https://review.openstack.org/448269 | 21:38 |
| *** Pavo has quit IRC | 21:40 | |
| kfox1111 | inc0: hi | 21:41 |
| inc0 | I need halp:( | 21:41 |
| kfox1111 | back | 21:41 |
| kfox1111 | whats up? | 21:41 |
| inc0 | https://review.openstack.org/#/c/447733/7/tools/setup_gate_common.sh looking at gate logs, line 23 doesn't seem to be run | 21:41 |
| kfox1111 | inc0: you looking at the isci or ceph gate logs? | 21:43 |
| inc0 | http://logs.openstack.org/33/447733/6/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-nv/6695c7b/console.html#_2017-03-21_21_00_48_519453 <- I'm trying to debug thios | 21:43 |
| kfox1111 | we're disabling selinux too. | 21:43 |
| kfox1111 | grep for sudo yum install -y libselinux-python | 21:45 |
| kfox1111 | http://logs.openstack.org/33/447733/6/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-nv/6695c7b/console.html#_2017-03-21_20_50_23_861603 | 21:45 |
| kfox1111 | says its already installed. | 21:45 |
| kfox1111 | maybe ansible's trying to talk to selinux even though its disabled? | 21:45 |
| inc0 | omg | 21:45 |
| inc0 | dmsimard: arouind by any chance? | 21:45 |
| kfox1111 | sbezverk: around? | 21:46 |
| kfox1111 | still need me to look at something? | 21:46 |
| inc0 | kfox1111: also check this one out https://review.openstack.org/#/c/448258/ | 21:46 |
| kfox1111 | inc0: context for the conversation with jascott1 was for library helm charts. parent needs to win. totally seperate thing from the global api, in which case the more specific wins over the more generic. | 21:47 |
| *** Jeffrey4l_ has joined #openstack-kolla | 21:48 | |
| *** haplo37 has quit IRC | 21:52 | |
| dmsimard | inc0: on my phone right now, sup? | 21:53 |
| inc0 | dmsimard: seen this in gates? http://logs.openstack.org/33/447733/6/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-nv/6695c7b/console.html#_2017-03-21_21_00_48_519453 | 21:54 |
| *** lamt has quit IRC | 21:54 | |
| sbezverk | kfox1111: hey | 21:54 |
| dmsimard | Doesn't ring me a bell but I've seen that problem before | 21:55 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 21:55 |
| dmsimard | inc0: https://dmsimard.com/2016/01/08/selinux-python-virtualenv-chroot-and-ansible-dont-play-nice/ | 21:55 |
| sbezverk | kfox1111: yeah if you could look at the link I posted http://logs.openstack.org/50/441550/102/experimental/gate-kolla-kubernetes-deploy-ubuntu-source-3-ironic-nv/4bdd216/console.html#_2017-03-21_19_47_33_854459 | 21:55 |
| inc0 | yeah, I saw this, hence my asking...problem is I don't think this uses virtualenv | 21:56 |
| inc0 | or chroot for that matter | 21:56 |
| dmsimard | tl;dr certain tasks require selinux bindings even on the control host because some operations happen there | 21:56 |
| *** fguillot has quit IRC | 21:56 | |
| inc0 | yeah this play runs on local | 21:56 |
| *** lamt has joined #openstack-kolla | 21:56 | |
| sbezverk | kfox1111: ubuntu uis misbehaving when centos is perfectly ok.. | 21:56 |
| dmsimard | inc0: doesn't sound like something that'd happen all of a sudden | 21:57 |
| dmsimard | There's usually an ansible fact for python interpreter | 21:57 |
| inc0 | yeah it's strange especially that kolla-ansible works, ansible-playbook doesnt | 21:58 |
| inc0 | and kolla ansible is just wrapper around ansible playbook command | 21:58 |
| inc0 | hmm that might be it | 21:58 |
| inc0 | I disabled fact gathering | 21:58 |
| dmsimard | Is there an ara report in there ? Reading raw ansible output hurts my eyes now :P | 21:59 |
| *** matrohon has quit IRC | 22:00 | |
| inc0 | lol | 22:00 |
| inc0 | nope, no ara | 22:00 |
| *** haplo37 has joined #openstack-kolla | 22:01 | |
| *** shashank_t_ has quit IRC | 22:09 | |
| *** vhosakot has quit IRC | 22:09 | |
| *** manheim has joined #openstack-kolla | 22:10 | |
| *** tonanhngo has joined #openstack-kolla | 22:12 | |
| *** tonanhngo has quit IRC | 22:13 | |
| *** athomas has quit IRC | 22:16 | |
| *** crushil has joined #openstack-kolla | 22:18 | |
| *** foliouno has quit IRC | 22:19 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP Move config generation to kolla-k8s https://review.openstack.org/448258 | 22:21 |
| *** klindgren has quit IRC | 22:22 | |
| *** Margin has joined #openstack-kolla | 22:26 | |
| *** Margin has quit IRC | 22:26 | |
| *** Margin has joined #openstack-kolla | 22:27 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-kubernetes master: WIP: Playbook to add configmaps and secrets https://review.openstack.org/447733 | 22:27 |
| *** lamt has quit IRC | 22:29 | |
| kfox1111 | sbezverk: so, the issue is: qemu-system-x86_64: could not acquire pid file: Permission denied? | 22:31 |
| kfox1111 | http://logs.openstack.org/50/441550/102/experimental/gate-kolla-kubernetes-deploy-ubuntu-source-3-ironic-nv/4bdd216/logs/libvirt-vm-logs.txt looks ok I think. | 22:33 |
| sbezverk | kfox1111: it looks like.. funny thing I have not seen similar error with newton | 22:36 |
| kfox1111 | sbezverk: maybe there is an el7 specific feature in the xml? | 22:37 |
| kfox1111 | or the feature uses a file location different between ubuntu and el7 | 22:37 |
| sbezverk | kfox1111: not sure, I mean I used only generic settings | 22:37 |
| kfox1111 | did it get past this point in ubuntu before you added the bios file thingy? | 22:37 |
| sbezverk | kfox1111: nope I started seeing as soon as I switch to ocata | 22:38 |
| sbezverk | btw check this http://logs.openstack.org/50/441550/104/experimental/gate-kolla-kubernetes-deploy-centos-source-3-ironic-nv/9940aa3/console.html#_2017-03-21_22_10_11_247633 | 22:38 |
| sbezverk | now I see where VM got stuck | 22:38 |
| kfox1111 | nice. :) | 22:38 |
| kfox1111 | can you copy that into $logs/... on failure? | 22:39 |
| kfox1111 | only other thing I can think of on the ubuntu side is to setup some strace's and see if you can determine the file it cant open. :/ | 22:40 |
| sbezverk | kfox1111: not sure because it gets kept in the container, if the container is still running when failure occur, then yes | 22:41 |
| sbezverk | kfox1111: ok I will look into ubuntu after centos is operational | 22:41 |
| *** crushil has quit IRC | 22:42 | |
| kfox1111 | sbezverk: I think you can just move the command to cat the file to the gate_log script and if the containers still running, it will capture it. the script does a lot of things like that. | 22:43 |
| kfox1111 | k. | 22:44 |
| *** tonanhngo has joined #openstack-kolla | 22:44 | |
| *** adrian_otto has quit IRC | 22:44 | |
| *** tonanhngo has quit IRC | 22:45 | |
| sbezverk | kfox1111: cool will add it. | 22:45 |
| *** fooliouno has joined #openstack-kolla | 22:56 | |
| *** david-lyle has quit IRC | 22:57 | |
| *** cloudkiller has joined #openstack-kolla | 23:00 | |
| *** cloudkiller has quit IRC | 23:00 | |
| *** sayantani01 has quit IRC | 23:02 | |
| *** sayantani01 has joined #openstack-kolla | 23:03 | |
| *** sayantani01 has quit IRC | 23:03 | |
| *** cloudkiller has joined #openstack-kolla | 23:04 | |
| *** sayantani01 has joined #openstack-kolla | 23:04 | |
| *** eglute_s has joined #openstack-kolla | 23:04 | |
| *** sayantani01 has quit IRC | 23:06 | |
| *** harlowja has quit IRC | 23:06 | |
| *** sayantani01 has joined #openstack-kolla | 23:06 | |
| *** eglute_s has quit IRC | 23:07 | |
| *** cloudkiller has quit IRC | 23:07 | |
| *** goldyfruit has quit IRC | 23:09 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 23:16 |
| *** Margin has quit IRC | 23:17 | |
| *** sayantani01 has quit IRC | 23:20 | |
| *** sayantani01 has joined #openstack-kolla | 23:21 | |
| *** rwsu has joined #openstack-kolla | 23:22 | |
| *** gfhellma has quit IRC | 23:26 | |
| spsurya | morning all | 23:29 |
| *** jtriley has joined #openstack-kolla | 23:32 | |
| *** cloudkiller has joined #openstack-kolla | 23:34 | |
| *** eglute_s has joined #openstack-kolla | 23:34 | |
| *** eglute_s has quit IRC | 23:46 | |
| *** cloudkiller has quit IRC | 23:46 | |
| *** cloudkiller has joined #openstack-kolla | 23:47 | |
| *** eglute_s has joined #openstack-kolla | 23:47 | |
| masber | hi | 23:48 |
| *** eglute has quit IRC | 23:50 | |
| *** cloudnull has quit IRC | 23:50 | |
| *** cloudkiller is now known as cloudnull | 23:50 | |
| *** manheim has quit IRC | 23:53 | |
| masber | I am testing kolla-ansible installation, I tried different interfaces configuration but my vms doesn't seems like they have internet access. According to the documentation neutron_external_interface is the one connected to br-ex so I am assuming the external traffic from/to my vms will go through that interface is that right? | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!