| *** goldyfruit has joined #openstack-kolla | 00:04 | |
| *** dave-mccowan has quit IRC | 00:07 | |
| sbezverk | kfox1111: I might need your help with global api. by some reason a couple of variable cannot be found.. I will try a couple of more things, if does not work, I will have to go to the source ;-) | 00:09 |
|---|---|---|
| *** harlowja has quit IRC | 00:09 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 00:09 |
| *** imcsk8 has quit IRC | 00:09 | |
| kfox1111 | ok. let me know and I'll help however I can. | 00:09 |
| *** imcsk8 has joined #openstack-kolla | 00:09 | |
| kfox1111 | sbezverk: is gate-kolla-kubernetes-deploy-ubuntu-binary-iscsi-nv just generally broken? I don't know if I've ever seen it pass. | 00:10 |
| *** peterjenkins_ has quit IRC | 00:11 | |
| *** peterjenkins_ has joined #openstack-kolla | 00:11 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service https://review.openstack.org/417474 | 00:12 |
| *** macsz has quit IRC | 00:12 | |
| *** cliles has quit IRC | 00:12 | |
| portdirect | kfox1111: overlay2 requires kernel 4.2 or greater - they have not backported it :( | 00:12 |
| *** macsz has joined #openstack-kolla | 00:12 | |
| sbezverk | kfox1111: yes, there is a bug in ubuntu cinder. they use older than centos version of cinder | 00:12 |
| *** alanmeadows has quit IRC | 00:12 | |
| *** afranc has quit IRC | 00:12 | |
| kfox1111 | portdirect: yeah. I didn't think so. just doing some sanity checking. | 00:13 |
| kfox1111 | portdirect: redhat's kernels are very odd, with all the backporting they do. :) | 00:13 |
| *** cliles has joined #openstack-kolla | 00:13 | |
| kfox1111 | sbezverk: ah. ok. I'll ignore it then. | 00:13 |
| sbezverk | as soon as ubuntu upstream updates their cinder | 00:13 |
| *** shasha_t_ has quit IRC | 00:13 | |
| kfox1111 | maybe it will work with 3.0.2 | 00:13 |
| *** jroll has quit IRC | 00:14 | |
| sbezverk | kfox1111: not sure.. as we use binary so if upstream has not been updated then no change | 00:15 |
| kfox1111 | yeah. | 00:16 |
| kfox1111 | sbezverk: so, which vars arn't showing? | 00:16 |
| sbezverk | kfox1111: when we start using master source then it mioght change for better or worse ;-) | 00:16 |
| sbezverk | kfox1111: I will tell you when I get gate's results.. | 00:17 |
| kfox1111 | trunk's always worse. ;) | 00:17 |
| *** afranc has joined #openstack-kolla | 00:18 | |
| kfox1111 | ok cool. | 00:18 |
| *** jroll has joined #openstack-kolla | 00:25 | |
| *** harlowja has joined #openstack-kolla | 00:25 | |
| *** gfidente|afk has quit IRC | 00:26 | |
| sdake | kfox1111 yup those are the two fses that can be used with centos/rhel 7 | 00:29 |
| sdake | overlay sort of works | 00:29 |
| sdake | but i find it dangerous | 00:29 |
| sdake | although it is really fast :) | 00:29 |
| sdake | so in the future it will be the go to solution | 00:29 |
| kfox1111 | sbezverk: portdirect: srwilkers: https://review.openstack.org/#/c/418556/ shouild be ready for review please. | 00:32 |
| *** sdake has quit IRC | 00:34 | |
| *** Pavo has joined #openstack-kolla | 00:35 | |
| *** Pavo has quit IRC | 00:36 | |
| *** awiddersheim has joined #openstack-kolla | 00:39 | |
| kfox1111 | sbezverk: there are still some .Values.port references in the template. | 00:40 |
| *** adrian_otto has joined #openstack-kolla | 00:44 | |
| *** duonghq has joined #openstack-kolla | 00:46 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 00:48 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service https://review.openstack.org/417474 | 00:50 |
| sbezverk | kfox1111: yeah I noticed :) and fixed.. | 00:51 |
| duonghq | morning guys | 00:52 |
| duonghq | morning sbezverk | 00:52 |
| kfox1111 | duonghq: hey | 00:52 |
| duonghq | hi kfox1111 | 00:52 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service https://review.openstack.org/417474 | 00:53 |
| openstackgerrit | jimmygc proposed openstack/kolla: Use install-pip macro in most source images https://review.openstack.org/413861 | 00:58 |
| sbezverk | duonghq: good morning | 01:03 |
| *** Pavo has joined #openstack-kolla | 01:06 | |
| *** zhubingbing has joined #openstack-kolla | 01:06 | |
| *** ssurana has quit IRC | 01:09 | |
| *** adrian_otto has quit IRC | 01:09 | |
| duonghq | sbezverk, still here? | 01:10 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 01:10 |
| duonghq | for your dev environment, you still you halcyon? | 01:11 |
| *** dave-mccowan has joined #openstack-kolla | 01:12 | |
| *** ssurana has joined #openstack-kolla | 01:13 | |
| duonghq | *use | 01:13 |
| *** ssurana has left #openstack-kolla | 01:14 | |
| *** tonanhngo has quit IRC | 01:14 | |
| sbezverk | duonghq: yep | 01:15 |
| sbezverk | what is up? | 01:15 |
| sbezverk | kfox1111: qq when I post info to api server I get "method not allowed" any idea why it could be the case? | 01:15 |
| duonghq | sbezverk, you install microservice directly from repo, or you setup helm repo server? | 01:16 |
| kfox1111 | sbezverk: which api server? | 01:16 |
| sbezverk | duonghq: atm directly from local repo which is automatic | 01:16 |
| duonghq | ah, I mean the git dir? hay local repo server? | 01:17 |
| sbezverk | kfox1111: kube api.. I need to update the state of 3rd party object | 01:17 |
| kfox1111 | ah. | 01:17 |
| kfox1111 | is it post put or patch? | 01:17 |
| kfox1111 | I'm not sure which is the right one to use in this case. | 01:18 |
| sbezverk | duonghq: you do not need to do any special 1. git clone kolla-kubernetes, 2. tools/helm_prebuild.microservices.py and you are good to go | 01:18 |
| Pavo | anyone ever see an error when trying to spin up an instance where it says image unacceptable, could not convert to raw when the image is qcow? | 01:19 |
| sbezverk | helm install helm/microservice/your-favorit-microservice --namespace kolla | 01:19 |
| duonghq | sbezverk, thank you | 01:19 |
| sbezverk | kfox1111: I think post | 01:19 |
| *** tonanhngo has joined #openstack-kolla | 01:22 | |
| kfox1111 | I think post is for creation and patch maybe for updates? | 01:23 |
| kfox1111 | not really sure though. | 01:23 |
| kfox1111 | that part of rest isn't very well defined. | 01:23 |
| sbezverk | kfox1111: I see, probably you are right, since the object already exist | 01:24 |
| kfox1111 | might do a kubectl update -f xxx and strace it? | 01:24 |
| sbezverk | kfox1111: ahh good idea | 01:26 |
| duonghq | sbezverk, do we have plan move some settings from all_values.yml to /etc/...? | 01:28 |
| kfox1111 | duonghq: the user can have their own values.yaml file, passed during helm release. | 01:29 |
| kfox1111 | like: | 01:29 |
| kfox1111 | helm install kolla/foo --values /etc/kolla-kubernetes/values.yaml | 01:29 |
| kfox1111 | or wherever you want to store it. | 01:29 |
| duonghq | ok | 01:29 |
| *** Pavo has quit IRC | 01:30 | |
| sbezverk | kfox1111: do you mind to take a look at strace? I could see only GET and OK :( | 01:31 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 01:31 |
| kfox1111 | did you set up the string lenghts? | 01:31 |
| kfox1111 | often you need something like -s 1000 to catch it. | 01:31 |
| kfox1111 | or alternately, tcpdump might be easier if you can get the unencrypted traffic. | 01:31 |
| kfox1111 | wireshark will make it easy to watch. | 01:31 |
| sbezverk | kfox1111: http://paste.openstack.org/show/594489/ | 01:32 |
| sbezverk | well it is all in one and cli only box | 01:32 |
| kfox1111 | got the review handy? | 01:34 |
| kfox1111 | oh.... I might not have easy access to an unencyrpted endpoint. :/ | 01:34 |
| kfox1111 | can you please set -s 1000 on the strace cli and try again? | 01:35 |
| sbezverk | sure thing | 01:35 |
| duonghq | sbezverk, you halcyon setup still usable after restart machine? | 01:37 |
| duonghq | (the physical one) | 01:37 |
| sbezverk | kfox1111: here you go: kfox1111 | 01:37 |
| sbezverk | duonghq: you'd better ask portdirect | 01:38 |
| duonghq | sbezverk, ah, forgot it | 01:38 |
| duonghq | portdirect, ping | 01:38 |
| sbezverk | kfox1111: opps ;) http://paste.openstack.org/show/594490/ | 01:38 |
| portdirect | pong | 01:38 |
| portdirect | duonghq: probably not :( | 01:38 |
| portdirect | I cant rememeber exactly - but i think that resolv.conf will be updated at restart | 01:39 |
| duonghq | portdirect, roger, in my setup kubernetes cluster is splitted | 01:40 |
| duonghq | kubectl cannot get nodes | 01:40 |
| portdirect | so you will need to update that again i think - or disable that | 01:40 |
| kfox1111 | sbezverk: thats a strange looking strace. it feels like there are things missing. could you add '-f -F' to the strace as well? | 01:40 |
| duonghq | portdirect, I'll try that | 01:40 |
| duonghq | nice | 01:40 |
| portdirect | i think (hope) that should get you back up and running - if not I'll be around for then next hour or so | 01:41 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service https://review.openstack.org/417474 | 01:41 |
| sbezverk | kfox1111: one sec | 01:42 |
| duonghq | portdirect, I'll try your idea next time I restart my machine, still keep it running for a few days, due to unstable network here, | 01:42 |
| duonghq | portdirect, thank you | 01:42 |
| kfox1111 | sbezverk: I think I figured out how to get an unencrypted endpoint in minikube. | 01:43 |
| sbezverk | kfox1111: they use PUT | 01:43 |
| kfox1111 | ah. | 01:44 |
| portdirect | duonghq: if you make any discoveries or anything else of note in your adventures please give me a shout - or put in a pr at halcyon :) | 01:44 |
| kfox1111 | interesting. | 01:44 |
| kfox1111 | put's a bit less defined in rest. so that kind of makes sense. | 01:44 |
| duonghq | portdirect, sure | 01:44 |
| *** wcb has quit IRC | 01:45 | |
| *** Pavo has joined #openstack-kolla | 01:45 | |
| sbezverk | kfox1111: email it, it is so huge it does not fit in paste | 01:45 |
| kfox1111 | sbezverk: hows this one look to you: https://review.openstack.org/#/c/418556/ ? | 01:45 |
| kfox1111 | kk | 01:45 |
| *** williamcaban has joined #openstack-kolla | 01:46 | |
| sbezverk | kfox1111: looks good | 01:46 |
| kfox1111 | yeah. looks like a put to me too. :) | 01:46 |
| *** v1k0d3n has quit IRC | 01:47 | |
| kfox1111 | a little odd to have so many pids. I guess they are maping goroutines to pids though. | 01:47 |
| *** v1k0d3n has joined #openstack-kolla | 01:47 | |
| kfox1111 | thanks for the review. | 01:47 |
| *** v1k0d3n has quit IRC | 01:47 | |
| *** v1k0d3n has joined #openstack-kolla | 01:48 | |
| *** saneax is now known as saneax-_-|AFK | 01:48 | |
| sbezverk | kfox1111: as long as it ia green ;-) | 01:59 |
| *** v1k0d3n has quit IRC | 02:00 | |
| *** v1k0d3n has joined #openstack-kolla | 02:00 | |
| *** tovin07 has joined #openstack-kolla | 02:00 | |
| *** unicell has quit IRC | 02:01 | |
| kfox1111 | :) | 02:02 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 02:04 |
| *** celo_r has quit IRC | 02:04 | |
| *** sacharya has quit IRC | 02:07 | |
| *** celo_r has joined #openstack-kolla | 02:07 | |
| *** alanmeadows has joined #openstack-kolla | 02:09 | |
| *** sayantani01 has joined #openstack-kolla | 02:10 | |
| *** sayantani01 has quit IRC | 02:15 | |
| *** yingjun has joined #openstack-kolla | 02:16 | |
| *** caowei has joined #openstack-kolla | 02:16 | |
| *** Pavo has quit IRC | 02:18 | |
| *** jianyi has joined #openstack-kolla | 02:19 | |
| *** tonanhngo has quit IRC | 02:22 | |
| duonghq | sbezverk, we use image from newton? I see default tag on all_values.yml is 3.0 | 02:22 |
| *** irtermite has left #openstack-kolla | 02:22 | |
| openstackgerrit | jimmygc proposed openstack/kolla: Use install-pip macro in most source images https://review.openstack.org/413861 | 02:24 |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move solum precheck into its own role https://review.openstack.org/418256 | 02:26 |
| *** yingjun has quit IRC | 02:26 | |
| *** yingjun has joined #openstack-kolla | 02:28 | |
| *** tonanhngo has joined #openstack-kolla | 02:33 | |
| duonghq | kfox1111, we use image from newton or ocata? | 02:45 |
| *** Pavo has joined #openstack-kolla | 02:50 | |
| *** sayantani01 has joined #openstack-kolla | 02:50 | |
| kfox1111 | duonghq: mitaka and newton so far. | 02:51 |
| kfox1111 | inc0 is getting ocata images in gate going. | 02:51 |
| duonghq | kfox1111, ack | 02:52 |
| duonghq | kfox1111, where can I get kubernetes_entrypoint? | 02:56 |
| duonghq | kfox1111, see it | 02:56 |
| duonghq | sorry | 02:56 |
| kfox1111 | there's abuild on the hub. | 02:56 |
| kfox1111 | only centos-binary at the moment. | 02:56 |
| kfox1111 | 3.0.2 should come out tomorrow with all the builds. | 02:57 |
| duonghq | but how can I build? | 02:57 |
| duonghq | manually? | 02:57 |
| kfox1111 | can do with the kolla repo. | 02:57 |
| duonghq | ah, ok | 02:57 |
| duonghq | nice | 02:57 |
| Jeffrey4l_ | 3.0.2 tag patch is passed. | 03:01 |
| *** celo_r has quit IRC | 03:01 | |
| Pavo | anyone ever see an error when trying to spin up an instance where it says image unacceptable, could not convert to raw when the image is qcow? | 03:02 |
| *** sacharya has joined #openstack-kolla | 03:02 | |
| Pavo | wait what | 03:03 |
| Pavo | new kolla pip package coming tomorrow? | 03:03 |
| Jeffrey4l_ | for newton release, yes. | 03:03 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move rabbitmq precheck into its own role https://review.openstack.org/417731 | 03:03 |
| Pavo | does it include all the murano and magmum fixes? | 03:03 |
| Jeffrey4l_ | i am not sure. it these fixed is backported into newton, so it is included. | 03:04 |
| Jeffrey4l_ | it these/if these | 03:04 |
| Pavo | how can I findout what is going to be updated in this | 03:04 |
| Jeffrey4l_ | Pavo, check this list https://launchpad.net/kolla/+milestone/3.0.2 | 03:04 |
| kfox1111 | Jeffrey4l_: nice. gona build the contaienrs tonight or tomorrow? | 03:05 |
| Jeffrey4l_ | kfox1111, the issue is: *who* gonna build | 03:05 |
| kfox1111 | Jeffrey4l_: I've got 2 ps's to go in tomorrow morning once we have 3.0.2 containers and then we can cut the kolla-kubernetes release. | 03:05 |
| Jeffrey4l_ | nice. | 03:06 |
| kfox1111 | Jeffrey4l_: oh. not you then? | 03:06 |
| Pavo | Jeffrey4l_ where on this tells me what is going to be included, all I see are a bunch of bugs | 03:06 |
| Jeffrey4l_ | hrm. it is a little hard for me ( who located in China, has a bad network access ) | 03:06 |
| kfox1111 | well, we'll see if we can find someone tomorrow morning. I've got to head out. | 03:06 |
| kfox1111 | maybe inc0 or portdirect? | 03:07 |
| Jeffrey4l_ | i made some change in my local env. | 03:07 |
| Jeffrey4l_ | yep. | 03:07 |
| *** sacharya has quit IRC | 03:07 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move rabbitmq precheck into its own role https://review.openstack.org/417731 | 03:08 |
| kfox1111 | k. I'll check back in the morning. good night. | 03:08 |
| Jeffrey4l_ | good night. | 03:08 |
| Pavo | nite | 03:08 |
| *** celo_r has joined #openstack-kolla | 03:08 | |
| portdirect | hey - i can build containers tommorow if needed | 03:10 |
| Jeffrey4l_ | portdirect, cool. do u have write permission for hub.docker.com/u/kolla? | 03:11 |
| portdirect | yes | 03:13 |
| portdirect | I'm busting the master images there | 03:13 |
| Jeffrey4l_ | cool. please build and push the images, when you are free | 03:13 |
| portdirect | *pushing, but busting works too :) | 03:13 |
| portdirect | will be able to tomorrow morning my time - so about 9 hours | 03:14 |
| portdirect | is 3.0.2 out? | 03:15 |
| Jeffrey4l_ | roger. thanks a lot. | 03:15 |
| Jeffrey4l_ | yep , https://review.openstack.org/417442 | 03:15 |
| openstackgerrit | Li Yingjun proposed openstack/kolla-kubernetes: Fix different repo issue for documentation https://review.openstack.org/405054 | 03:15 |
| Jeffrey4l_ | it is already pushed to pypi https://pypi.python.org/pypi/kolla | 03:15 |
| portdirect | whoop - congrats guys :) | 03:16 |
| Pavo | do I need to rebuild images if I update to 3.0.2?? | 03:16 |
| *** dave-mccowan has quit IRC | 03:17 | |
| Pavo | and will a kolla-ansible update work? | 03:17 |
| Jeffrey4l_ | Pavo, yes. | 03:17 |
| Jeffrey4l_ | Pavo, need re-build the images. | 03:17 |
| Pavo | roger | 03:17 |
| Pavo | and will a kolla-ansible update work? | 03:17 |
| Jeffrey4l_ | some fixes are related to Dockerfile changes. | 03:17 |
| Jeffrey4l_ | Pavo, i am not sure. bug guess no for some service. | 03:17 |
| Jeffrey4l_ | bug/but* | 03:18 |
| Pavo | ok | 03:18 |
| Pavo | will test it out | 03:18 |
| *** absubram has quit IRC | 03:18 | |
| Pavo | found out today that when doing a kolla-ansible reconfigure make sure you have all instances shutdown | 03:18 |
| Pavo | nova compute goes haywire if not | 03:19 |
| Pavo | also nova-libvirt | 03:19 |
| *** tonanhngo has quit IRC | 03:19 | |
| Pavo | assuming update is the same | 03:19 |
| Jeffrey4l_ | Pavo, what happen when all instances are not shutdown' | 03:20 |
| Pavo | nova-libvirt crashed, nova-compute just exited | 03:20 |
| Jeffrey4l_ | hrm. crashed.. never see this. | 03:21 |
| Jeffrey4l_ | did u catch the related logs? | 03:21 |
| Pavo | no but I am sure I can go back and look tomorrow | 03:21 |
| Jeffrey4l_ | cool | 03:22 |
| *** absubram has joined #openstack-kolla | 03:25 | |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move solum precheck into its own role https://review.openstack.org/418256 | 03:25 |
| openstackgerrit | Surya Prakash Singh proposed openstack/kolla-ansible: Docs update related to hostname resolution in RabbitMQ https://review.openstack.org/418252 | 03:25 |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move solum precheck into its own role https://review.openstack.org/418256 | 03:27 |
| *** absubram has quit IRC | 03:31 | |
| *** salv-orl_ has quit IRC | 03:32 | |
| *** salv-orlando has joined #openstack-kolla | 03:32 | |
| *** prameswar has joined #openstack-kolla | 03:32 | |
| duonghq | kfox1111, so I grab the ocata repo for kubernetes_entrypoint and tag it as 3.0.x? | 03:34 |
| sp_ | pbourke: zhubingbing I have incorporated the comment. It will be great if you review it again. https://review.openstack.org/418252 | 03:38 |
| zhubingbing | hi | 03:38 |
| zhubingbing | sure | 03:38 |
| zhubingbing | i review it now | 03:38 |
| zhubingbing | thanks your commit | 03:38 |
| *** kbyrne has quit IRC | 03:45 | |
| *** shasha_t_ has joined #openstack-kolla | 03:45 | |
| *** yingjun has quit IRC | 03:46 | |
| *** kbyrne has joined #openstack-kolla | 03:48 | |
| *** jrobinson is now known as jrobinson-afk | 03:48 | |
| *** jrobinson-afk has quit IRC | 03:52 | |
| openstackgerrit | zhubingbing proposed openstack/kolla-ansible: [WIP] Add fluentd role https://review.openstack.org/407392 | 03:53 |
| *** sdake has joined #openstack-kolla | 03:55 | |
| *** jrobinson has joined #openstack-kolla | 03:55 | |
| *** jrobinson is now known as jrobinson-afk | 03:56 | |
| *** sdake has quit IRC | 04:00 | |
| *** sdake has joined #openstack-kolla | 04:01 | |
| sdake | hello peoples | 04:05 |
| *** Pavo has quit IRC | 04:05 | |
| *** newmember has quit IRC | 04:08 | |
| sbezverk | kfox1111: ping | 04:12 |
| sbezverk | kfox1111: all green when you have time please ack: https://review.openstack.org/417474 | 04:13 |
| *** sacharya has joined #openstack-kolla | 04:15 | |
| *** sdake has quit IRC | 04:18 | |
| *** sacharya has quit IRC | 04:20 | |
| *** salv-orl_ has joined #openstack-kolla | 04:23 | |
| *** salv-orlando has quit IRC | 04:26 | |
| *** jrobinson-afk is now known as jrobinson | 04:29 | |
| *** mdnadeem has joined #openstack-kolla | 04:35 | |
| *** sayantani01 has quit IRC | 04:51 | |
| *** sayantani01 has joined #openstack-kolla | 04:51 | |
| *** sayantani01 has quit IRC | 04:52 | |
| *** sayantan_ has joined #openstack-kolla | 04:52 | |
| *** goldyfruit has quit IRC | 04:52 | |
| *** absubram has joined #openstack-kolla | 04:55 | |
| *** absubram_ has joined #openstack-kolla | 04:59 | |
| *** absubram has quit IRC | 04:59 | |
| *** absubram_ is now known as absubram | 04:59 | |
| *** ccesario has quit IRC | 05:02 | |
| *** prameswar has quit IRC | 05:03 | |
| *** caowei has quit IRC | 05:03 | |
| *** sdake has joined #openstack-kolla | 05:07 | |
| *** ccesario has joined #openstack-kolla | 05:14 | |
| *** adrian_otto has joined #openstack-kolla | 05:42 | |
| *** adrian_otto has quit IRC | 05:46 | |
| *** adrian_otto has joined #openstack-kolla | 05:47 | |
| *** caowei has joined #openstack-kolla | 05:52 | |
| *** adrian_otto has quit IRC | 05:58 | |
| *** sayantan_ has quit IRC | 06:04 | |
| *** skramaja has joined #openstack-kolla | 06:07 | |
| *** unicell has joined #openstack-kolla | 06:07 | |
| *** dims has quit IRC | 06:13 | |
| *** unicell has quit IRC | 06:14 | |
| *** unicell has joined #openstack-kolla | 06:14 | |
| *** prameswar has joined #openstack-kolla | 06:15 | |
| *** sacharya has joined #openstack-kolla | 06:16 | |
| *** sacharya has quit IRC | 06:20 | |
| *** prameswar has quit IRC | 06:21 | |
| *** dturner has quit IRC | 06:24 | |
| *** pcaruana has joined #openstack-kolla | 06:25 | |
| duonghq | sup sdake | 06:29 |
| sdake | ready for bed :) | 06:29 |
| *** satyar has joined #openstack-kolla | 06:31 | |
| *** prameswar has joined #openstack-kolla | 06:33 | |
| *** sdake has quit IRC | 06:45 | |
| *** saneax-_-|AFK is now known as saneax | 06:45 | |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move solum precheck into its own role https://review.openstack.org/418256 | 06:50 |
| *** prameswar has quit IRC | 06:54 | |
| *** hfu has joined #openstack-kolla | 06:56 | |
| *** prameswar has joined #openstack-kolla | 07:07 | |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move solum precheck into its own role https://review.openstack.org/418256 | 07:09 |
| *** unicell has quit IRC | 07:10 | |
| *** unicell has joined #openstack-kolla | 07:10 | |
| *** bjolo has joined #openstack-kolla | 07:13 | |
| bjolo | morning | 07:14 |
| *** caowei has quit IRC | 07:16 | |
| *** caowei has joined #openstack-kolla | 07:16 | |
| *** msimonin has quit IRC | 07:17 | |
| *** msimonin has joined #openstack-kolla | 07:17 | |
| satyar | hi jeffrey4l_ | 07:20 |
| satyar | do we need to fix https://bugs.launchpad.net/kolla/+bug/1583655 in netwon also? | 07:20 |
| openstack | Launchpad bug 1583655 in kolla "rabbitmq setup from secondary IP" [Wishlist,In progress] - Assigned to Satya Sanjibani Routray (satroutr) | 07:20 |
| *** msimonin has quit IRC | 07:22 | |
| Jeffrey4l_ | sup satyar | 07:22 |
| *** msimonin has joined #openstack-kolla | 07:22 | |
| duonghq | I think it's a feature? | 07:23 |
| duonghq | we donot backport feature? | 07:23 |
| *** msimonin has quit IRC | 07:23 | |
| *** msimonin has joined #openstack-kolla | 07:23 | |
| openstackgerrit | zhubingbing proposed openstack/kolla-ansible: [WIP] Add fluentd role https://review.openstack.org/407392 | 07:23 |
| Jeffrey4l_ | yep. it is a feature. | 07:24 |
| satyar | so we can remove the affects newton from there? | 07:24 |
| Jeffrey4l_ | i think so . | 07:24 |
| *** msimonin has quit IRC | 07:24 | |
| *** msimonin has joined #openstack-kolla | 07:24 | |
| satyar | should i mark invalid for newton? | 07:24 |
| *** msimonin has quit IRC | 07:25 | |
| *** msimonin has joined #openstack-kolla | 07:25 | |
| *** msimonin has quit IRC | 07:26 | |
| *** msimonin has joined #openstack-kolla | 07:26 | |
| Jeffrey4l_ | won't feature? | 07:26 |
| Jeffrey4l_ | won't fix? | 07:26 |
| *** msimonin has quit IRC | 07:26 | |
| openstackgerrit | Merged openstack/kolla-ansible: Move gnocchi precheck into its own role https://review.openstack.org/415326 | 07:30 |
| duonghq | satyar, invalid is reasonable imo | 07:30 |
| openstackgerrit | Merged openstack/kolla-ansible: Remove precheck.yml in certificates https://review.openstack.org/418431 | 07:30 |
| satyar | ok got it :) jeffrey4l: duonghq | 07:31 |
| Jeffrey4l_ | duonghq, satyar no. it is an issue. but we do not fix on stable branch. so won't fix is more reasonable. | 07:32 |
| satyar | :( i am not getting the wont fix option there | 07:32 |
| duonghq | satyar, done | 07:33 |
| satyar | Thanks :) duonghq | 07:33 |
| *** jrobinson has quit IRC | 07:46 | |
| *** hogepodge has quit IRC | 07:56 | |
| *** sacharya has joined #openstack-kolla | 08:00 | |
| *** britthouser has quit IRC | 08:03 | |
| *** bmace has quit IRC | 08:03 | |
| *** britthouser has joined #openstack-kolla | 08:03 | |
| *** jianyi has quit IRC | 08:04 | |
| *** sacharya has quit IRC | 08:06 | |
| openstackgerrit | Merged openstack/kolla-ansible: Move mistral precheck into its own role https://review.openstack.org/415515 | 08:10 |
| *** strigazi_AFK is now known as strigazi | 08:13 | |
| openstackgerrit | HongZhi XU proposed openstack/kolla-ansible: Move gnocchi precheck into its own role https://review.openstack.org/418762 | 08:18 |
| *** liyifeng has joined #openstack-kolla | 08:18 | |
| *** salv-orl_ has quit IRC | 08:21 | |
| liyifeng | hi, I have a problem, when I deploy kolla, exec task: Fetching ceph keyring, there has an error, No Json object could be decoded,what is the reason?thanks | 08:23 |
| liyifeng | this deploy is redeploy on the same host,I del two osd disk,and add a new disk as osd. | 08:25 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move grafana precheck into its own role https://review.openstack.org/416055 | 08:25 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move murano precheck into its own role https://review.openstack.org/415113 | 08:29 |
| *** ocaner has joined #openstack-kolla | 08:30 | |
| *** l4yerffeJ has joined #openstack-kolla | 08:31 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move nova precheck into its own role https://review.openstack.org/415564 | 08:31 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move nova precheck into its own role https://review.openstack.org/415564 | 08:34 |
| *** msimonin has joined #openstack-kolla | 08:37 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move etcd precheck into its own role https://review.openstack.org/416063 | 08:38 |
| *** l4yerffeJ has quit IRC | 08:40 | |
| *** l4yerffeJ has joined #openstack-kolla | 08:40 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move searchlight precheck into its own role https://review.openstack.org/415135 | 08:41 |
| *** egonzalez has joined #openstack-kolla | 08:45 | |
| egonzalez | morning | 08:45 |
| Jeffrey4l_ | morning | 08:46 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move rabbitmq precheck into its own role https://review.openstack.org/417731 | 08:50 |
| *** haplo37_ has quit IRC | 08:54 | |
| *** haplo37_ has joined #openstack-kolla | 08:57 | |
| sp_ | egonzalez: pbourke: satyar: zhubingbing: please review this time https://review.openstack.org/#/c/418252/ | 08:59 |
| egonzalez | sp_: Didn't realize it was a spec before, Jeffrey4l_ commented that we do not maintain spec files | 09:01 |
| *** l4yerffeJ has quit IRC | 09:01 | |
| *** l4yerffeJ has joined #openstack-kolla | 09:01 | |
| egonzalez | Specifying hostname resolution can be in other doc file | 09:01 |
| *** prameswar has quit IRC | 09:02 | |
| *** Serlex has joined #openstack-kolla | 09:02 | |
| sp_ | egonzalez: so what next ? | 09:03 |
| sp_ | egonzalez: ok ok, missed yr last text | 09:04 |
| egonzalez | sp_: what about moving to advanced-configuration or networking-guide? | 09:05 |
| egonzalez | or even multinode guide, not sure on which doc should be | 09:06 |
| sp_ | egonzalez: i think networking-guide's "IP Address Constrained Environments" section would be fine. | 09:07 |
| sp_ | pbourke: around ?? | 09:08 |
| sp_ | Jeffrey4l_: ping.. | 09:11 |
| Jeffrey4l_ | sp_, pong | 09:11 |
| sp_ | Jeffrey4l_: which doc would be suitable to update regarding this. https://review.openstack.org/#/c/418252 | 09:12 |
| sp_ | Jeffrey4l_: please suggest | 09:12 |
| *** shardy has joined #openstack-kolla | 09:13 | |
| Jeffrey4l_ | some place in doc folder i think. | 09:13 |
| sp_ | Jeffrey4l_: thanks | 09:14 |
| Jeffrey4l_ | np ;) | 09:14 |
| openstackgerrit | Merged openstack/kolla-ansible: Move grafana precheck into its own role https://review.openstack.org/416055 | 09:14 |
| sp_ | egonzalez: doc/advanced-configuration.rst's "IP Address Constrained Environments" section | 09:17 |
| egonzalez | zhubingbing: ping | 09:17 |
| *** gfidente has joined #openstack-kolla | 09:18 | |
| *** kproskurin has joined #openstack-kolla | 09:18 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move searchlight precheck into its own role https://review.openstack.org/415135 | 09:21 |
| Mech422 | I can't believe I've spent over an hour just looking for the names of the ldap support packages required for openstack :-P | 09:21 |
| Mech422 | 500 articles on openstack+ldap, and no-one bothers to mention which packages you need :-P | 09:22 |
| egonzalez | zhubingbing: in the firsts fluentd PS, cron logrotate tasks were removed from common role and now are present. Is fluentd responsible of log rotation now? | 09:23 |
| *** DanyC has joined #openstack-kolla | 09:24 | |
| *** matrohon has joined #openstack-kolla | 09:25 | |
| *** DanyC has left #openstack-kolla | 09:25 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move swift precheck into its own role https://review.openstack.org/418762 | 09:28 |
| zhubingbing | hi | 09:30 |
| zhubingbing | egonzalez yes | 09:31 |
| egonzalez | zhubingbing: OK, i'll add a comment to remove cron files | 09:32 |
| zhubingbing | roger | 09:32 |
| zhubingbing | https://review.openstack.org/#/c/407392/ | 09:33 |
| zhubingbing | you can write down your comments in this patch review | 09:33 |
| zhubingbing | ；） | 09:33 |
| egonzalez | zhubingbing: yup, I'm doing | 09:37 |
| zhubingbing | thanks you very much | 09:38 |
| openstackgerrit | Surya Prakash Singh proposed openstack/kolla-ansible: Docs update related to hostname resolution in RabbitMQ https://review.openstack.org/418252 | 09:38 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add miss chmod 755 /usr/local/bin/kolla_extend_start in grafana https://review.openstack.org/418799 | 09:45 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix grafana container https://review.openstack.org/418799 | 09:47 |
| *** l4yerffeJ has quit IRC | 09:47 | |
| *** l4yerffeJ has joined #openstack-kolla | 09:47 | |
| *** hfu has quit IRC | 09:51 | |
| *** prameswar has joined #openstack-kolla | 09:54 | |
| *** DanyC has joined #openstack-kolla | 09:55 | |
| *** DanyC has quit IRC | 09:55 | |
| *** DanyC has joined #openstack-kolla | 09:56 | |
| sp_ | egonzalez: corrected the location as per Jeffre comment https://review.openstack.org/418252 please check now | 09:56 |
| *** DanyC has quit IRC | 09:58 | |
| *** DanyC has joined #openstack-kolla | 09:58 | |
| *** DanyC has quit IRC | 10:00 | |
| *** DanyC has joined #openstack-kolla | 10:01 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix telegraf contaiers https://review.openstack.org/418811 | 10:02 |
| *** bjolo has quit IRC | 10:02 | |
| *** bjolo has joined #openstack-kolla | 10:03 | |
| *** DanyC has quit IRC | 10:05 | |
| zhubingbing | around egonzalez | 10:05 |
| zhubingbing | Ask a question , now docker version is 1.12 so we should change docker version 1.11 into 1.20 ? | 10:06 |
| *** l4yerffeJ has quit IRC | 10:08 | |
| *** l4yerffeJ has joined #openstack-kolla | 10:08 | |
| openstackgerrit | Merged openstack/kolla-ansible: Move magnum precheck into its own role https://review.openstack.org/415513 | 10:13 |
| openstackgerrit | Surya Prakash Singh proposed openstack/kolla: Cover environment for kolla https://review.openstack.org/411313 | 10:17 |
| sp_ | Jeffrey4l_: incorporated your comment. It would be great if you review it now. Thanks | 10:19 |
| openstackgerrit | Merged openstack/kolla-ansible: Move searchlight precheck into its own role https://review.openstack.org/415135 | 10:24 |
| *** mdnadeem has quit IRC | 10:24 | |
| egonzalez | zhubingbing: where? in docs or requirements? | 10:25 |
| zhubingbing | flunted ? | 10:25 |
| zhubingbing | https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/baremetal/defaults/main.yml#L22 | 10:26 |
| *** liyifeng has quit IRC | 10:27 | |
| *** mannamne has joined #openstack-kolla | 10:27 | |
| *** satyar has quit IRC | 10:29 | |
| *** DanyC has joined #openstack-kolla | 10:34 | |
| *** cristicalin has joined #openstack-kolla | 10:34 | |
| *** DanyC has quit IRC | 10:37 | |
| *** DanyC has joined #openstack-kolla | 10:37 | |
| *** DanyC has quit IRC | 10:39 | |
| *** mdnadeem has joined #openstack-kolla | 10:40 | |
| *** DanyC has joined #openstack-kolla | 10:42 | |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move kibana precheck into its own role https://review.openstack.org/418834 | 10:44 |
| *** pbourke has quit IRC | 10:44 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-kubernetes: Helm Ceph RGW https://review.openstack.org/418835 | 10:44 |
| *** pbourke has joined #openstack-kolla | 10:45 | |
| *** rmart04 has joined #openstack-kolla | 10:46 | |
| openstackgerrit | avnish proposed openstack/kolla: Removed unnecessary utf-8 encoding https://review.openstack.org/418837 | 10:46 |
| des_consolado | hmmm I'm getting this on CentOS 7... https://gist.github.com/thomasodot/d936e4bad145e5cd22f5fe3a7de61df6 | 10:46 |
| *** gfidente has quit IRC | 10:46 | |
| *** zhurong has joined #openstack-kolla | 10:46 | |
| *** duonghq has quit IRC | 10:46 | |
| egonzalez | des_consolado: pbr need git installed to check version | 10:48 |
| *** sayantani01 has joined #openstack-kolla | 10:49 | |
| des_consolado | git is installed, on the CentOS at least... And there isn't a pip module for git I don't think | 10:49 |
| des_consolado | git 1.8.3.1 | 10:50 |
| des_consolado | oh there are lots of pip modules when you search for git actually, just not one that's called git specifically | 10:52 |
| *** zhubingbing has quit IRC | 10:53 | |
| *** msimonin has quit IRC | 10:53 | |
| *** DanyC has quit IRC | 10:53 | |
| egonzalez | saw some other bugs complaining about it, more of the answers said to upgrade setuptools | 10:53 |
| *** DanyC has joined #openstack-kolla | 10:54 | |
| *** sayantani01 has quit IRC | 10:54 | |
| des_consolado | yeah I did google and pip install --upgrade setuptools distutils pbr, it's all up-to-date D: | 10:54 |
| *** DanyC has quit IRC | 10:54 | |
| des_consolado | oh not distutils | 10:55 |
| des_consolado | distutils isn't a pip package, only setup tools | 10:55 |
| des_consolado | there was distutils2, tried installing that, same result | 10:56 |
| des_consolado | ah my bad | 10:59 |
| des_consolado | I'd only installed kolla-ansible in pip, I hadn't installed kolla | 10:59 |
| des_consolado | so I guess maybe that's a kolla-ansible bug? I mean kolla-ansible does install the kolla-genpwd binary, so it should make it so it works right? | 11:01 |
| des_consolado | is there a document somewhere outlining the roadmap for the split between the kolla and kolla-ansible repo's anyway? | 11:01 |
| egonzalez | maybe there is a missing requirement, Jeffrey4l_ any idea? ^^ | 11:01 |
| *** sacharya has joined #openstack-kolla | 11:02 | |
| *** DanyC has joined #openstack-kolla | 11:04 | |
| *** sacharya has quit IRC | 11:07 | |
| *** matrohon has quit IRC | 11:09 | |
| *** bjolo has quit IRC | 11:13 | |
| egonzalez | is ask.openstack.org down in your location? | 11:23 |
| openstackgerrit | qiankun.zhang proposed openstack/kolla-ansible: Move solum precheck into its own role https://review.openstack.org/418256 | 11:27 |
| *** Serlex has quit IRC | 11:29 | |
| *** zhurong has quit IRC | 11:35 | |
| *** dims has joined #openstack-kolla | 11:39 | |
| *** JoseMello has joined #openstack-kolla | 11:40 | |
| *** haplo37_ has quit IRC | 11:40 | |
| openstackgerrit | yangyang proposed openstack/kolla-ansible: Move Influxdb precheck into its own role https://review.openstack.org/418875 | 11:40 |
| *** cristicalin has quit IRC | 11:41 | |
| *** Bico_Fino has joined #openstack-kolla | 11:42 | |
| mliima | morning guys | 11:42 |
| *** haplo37_ has joined #openstack-kolla | 11:43 | |
| des_consolado | egonzalez: yes actuqally | 11:43 |
| des_consolado | actually | 11:43 |
| des_consolado | it tries ipv6 port 80 and gets connection refused :O | 11:44 |
| egonzalez | des_consolado: thanks | 11:44 |
| egonzalez | morning mliima | 11:44 |
| des_consolado | curl with -4 also fails with the same shit... openstack.org is fine tho isn't it | 11:44 |
| des_consolado | same on 443 if https specified... someone made a firewall or web server change or something maybe :P | 11:45 |
| *** bjolo has joined #openstack-kolla | 11:53 | |
| openstackgerrit | Merged openstack/kolla-ansible: cinder.conf either supports lvm or ceph backend, not both https://review.openstack.org/417890 | 11:55 |
| *** DanyC has quit IRC | 12:01 | |
| *** Bico_Fino_ has joined #openstack-kolla | 12:09 | |
| *** Bico_Fino has quit IRC | 12:09 | |
| *** Bico_Fino_ is now known as Bico_Fino | 12:09 | |
| des_consolado | oh it's back up again now | 12:14 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla-ansible: Allow cinder-volume to be configured to use HNAS nfs https://review.openstack.org/417094 | 12:18 |
| *** gfidente has joined #openstack-kolla | 12:22 | |
| *** gfidente has joined #openstack-kolla | 12:22 | |
| *** nick-ma_ has quit IRC | 12:22 | |
| *** Serlex has joined #openstack-kolla | 12:34 | |
| *** shardy is now known as shardy_lunch | 12:43 | |
| *** saneax is now known as saneax-_-|AFK | 12:45 | |
| *** mdnadeem has quit IRC | 12:50 | |
| *** msimonin has joined #openstack-kolla | 12:50 | |
| *** sacharya has joined #openstack-kolla | 13:03 | |
| *** mgiles has joined #openstack-kolla | 13:03 | |
| *** liyifeng has joined #openstack-kolla | 13:05 | |
| *** sacharya has quit IRC | 13:08 | |
| *** tovin07_ has joined #openstack-kolla | 13:09 | |
| *** matrohon has joined #openstack-kolla | 13:09 | |
| egonzalez | Jeffrey4l_: is octavia bp finished? https://blueprints.launchpad.net/kolla-ansible/+spec/octavia-ansible-role | 13:10 |
| *** msimonin has quit IRC | 13:12 | |
| *** shardy_lunch has quit IRC | 13:12 | |
| *** srwilkers has joined #openstack-kolla | 13:13 | |
| *** msimonin has joined #openstack-kolla | 13:13 | |
| *** tovin07 has quit IRC | 13:15 | |
| *** tovin07_ is now known as tovin07 | 13:15 | |
| *** tovin07_ has joined #openstack-kolla | 13:15 | |
| *** msimonin has quit IRC | 13:18 | |
| *** msimonin has joined #openstack-kolla | 13:19 | |
| *** shardy_lunch has joined #openstack-kolla | 13:19 | |
| *** srwilkers has quit IRC | 13:21 | |
| des_consolado | I've documented what I guess is basically a "for idiots" guide to deploying OpenStack with kolla(-ansible), needed to do this for work anyway. I've stuck it on gist for now, I don't know if it'd be of use to anyone else... I don't see a lot of other noobs around here tbh, not sure how it's worked out that the only people using OpenStack are its pro dev's O.o | 13:21 |
| des_consolado | https://gist.github.com/thomasodot/b866989b1ed523e9687ba0e461a67264 | 13:21 |
| *** yangzb09 has joined #openstack-kolla | 13:22 | |
| egonzalez | des_consolado: the first thing I'm seeing there is that you do pip install -r requirements | 13:23 |
| egonzalez | if installing kolla from pip is not needed at all | 13:23 |
| Jeffrey4l_ | egonzalez, there are still some nit, i will push a fix. and then it can be marked as implemented. | 13:23 |
| *** srwilkers has joined #openstack-kolla | 13:23 | |
| des_consolado | oh right | 13:24 |
| egonzalez | if not using pip, using git repo, need to add test-requirements too | 13:24 |
| egonzalez | Jeffrey4l_: cool | 13:24 |
| *** hfu has joined #openstack-kolla | 13:26 | |
| egonzalez | des_consolado: could you fill a bug in kolla-ansible lp for the issue with generate-passwords dependencies? | 13:26 |
| des_consolado | egonzalez: sure, give me a min | 13:26 |
| Jeffrey4l_ | egonzalez, have you found anything about the mitaka branch failure? | 13:27 |
| egonzalez | Jeffrey4l_: not | 13:28 |
| *** shardy_lunch is now known as shardy | 13:28 | |
| egonzalez | maybe is a bug with selinux-policies packages, but gates use the same packages up-to-date right? | 13:28 |
| *** shardy has quit IRC | 13:28 | |
| *** shardy has joined #openstack-kolla | 13:29 | |
| Jeffrey4l_ | egonzalez, yes. all branch gates are using the same images and should be the latest packages. | 13:29 |
| egonzalez | des_consolado: recomended method to use stable/branch is installing from pip, not using git repository | 13:29 |
| des_consolado | https://bugs.launchpad.net/kolla-ansible/+bug/1655653 | 13:29 |
| openstack | Launchpad bug 1655653 in kolla-ansible "kolla-genpwd pbr versioning exception" [Undecided,New] | 13:29 |
| des_consolado | yeah I'll change that, thanks man | 13:32 |
| egonzalez | maybe the issue is from there, that missed test-requirements | 13:33 |
| *** jheroux has joined #openstack-kolla | 13:37 | |
| des_consolado | kolla-ansible provides kolla-genpwd in a non-working state though doesn't it I think | 13:38 |
| *** sdake has joined #openstack-kolla | 13:39 | |
| *** satyar has joined #openstack-kolla | 13:40 | |
| *** hfu has quit IRC | 13:41 | |
| des_consolado | yeah I just reproduced it on my compute node by installing kolla-ansible and trying kolla-genpwd again (all sudo) | 13:42 |
| *** sdake_ has joined #openstack-kolla | 13:44 | |
| *** Jeffrey4l_ has quit IRC | 13:45 | |
| *** sdake has quit IRC | 13:46 | |
| *** duonghq has joined #openstack-kolla | 13:47 | |
| sean-k-mooney | des_consolado: should there not be a dependcy between kolla-ansible on kolla | 13:49 |
| *** ccesario has quit IRC | 13:50 | |
| *** rhallisey has joined #openstack-kolla | 13:50 | |
| sean-k-mooney | kolla is missing form the requirements file for kolla-ansible | 13:50 |
| sean-k-mooney | that is why https://bugs.launchpad.net/kolla-ansible/+bug/1655653 is happening | 13:51 |
| openstack | Launchpad bug 1655653 in kolla-ansible "kolla-genpwd pbr versioning exception" [Undecided,Triaged] | 13:51 |
| *** tyrola has quit IRC | 13:53 | |
| openstackgerrit | Merged openstack/kolla: Fix grafana container https://review.openstack.org/418799 | 13:53 |
| openstackgerrit | Merged openstack/kolla: Configure local registries to proxy docker hub https://review.openstack.org/413806 | 13:54 |
| openstackgerrit | Merged openstack/kolla: Fix mitaka branch gate https://review.openstack.org/412470 | 13:54 |
| *** tyrola has joined #openstack-kolla | 13:54 | |
| *** david-lyle has quit IRC | 13:56 | |
| *** zhubingbing has joined #openstack-kolla | 13:57 | |
| *** david-lyle has joined #openstack-kolla | 13:59 | |
| *** athomas has quit IRC | 14:01 | |
| *** lamt has joined #openstack-kolla | 14:01 | |
| rhallisey | sdake_, hey | 14:05 |
| sdake_ | sup dude | 14:05 |
| rhallisey | had an idea from the ml thread | 14:05 |
| rhallisey | what if bootstrap + deploy + upgrade + reconfig were their own playbooks | 14:05 |
| rhallisey | they are all currently in one playbook | 14:05 |
| rhallisey | and they are connected by the deploy script | 14:06 |
| rhallisey | you can call other playbooks from within playbooks | 14:06 |
| Mech422 | rhallisey: I sorta do the bootstrap thing... | 14:07 |
| rhallisey | Mech422, you do the bootstrap yourself by hand? | 14:07 |
| Mech422 | rhallisey: I had a bunch of playbooks that bootstrapped the hosts, then 'drove' kolla | 14:07 |
| rhallisey | interesting | 14:07 |
| Mech422 | rhallisey: I had ansible inception going - ansible driving ansible :-P | 14:07 |
| rhallisey | what is your opinion about it? Did you find it worked well? | 14:08 |
| Mech422 | rhallisey: also, see the email from 'kris lindgren' ... | 14:08 |
| rhallisey | ya I just read it | 14:08 |
| Mech422 | rhallisey: he has a lot of the same issues I did | 14:08 |
| Mech422 | aka - not a greenfield deployment, must work within existing designs/reporting structure etc | 14:08 |
| *** msimonin has quit IRC | 14:08 | |
| rhallisey | Mech422, so would breaking the playbooks apart help you? | 14:08 |
| *** athomas has joined #openstack-kolla | 14:09 | |
| Mech422 | rhallisey: Umm - not sure it would - I needed 'prelude' and 'epilogue' stuff wrapped around kolla | 14:09 |
| Mech422 | but kolla itself could run as a 'chunk' | 14:09 |
| Mech422 | rhallisey: stuff I had to deal with was: no touchey my OVS setup! externally created/managed ceph cluster (think thats been addressed though), etc etc | 14:10 |
| rhallisey | Mech422, the anisble variable file is helpful, but I don't think it goes far enough. Like you said, it touches a like of things you may not want | 14:11 |
| Mech422 | yeah - it restarted ovs, which caused me to lose connection to all my cluster hosts - time to drive to the D.C. :-P | 14:12 |
| Mech422 | rhallisey: also, I tend to like a LOT of networks 'just in case' - so ceph replication network, mysql/mongo replication network, etc etc | 14:13 |
| Mech422 | rhallisey: I'm using vlans over bonded nics, so I tend to throw anything I think 'might' grow to significant traffic on its on vlan/ovs interface | 14:15 |
| rhallisey | Mech422, it would be nice to add another "layer", where playbooks can be used or not. | 14:15 |
| rhallisey | custom playbooks plugged in or removed etc.. | 14:15 |
| *** david-lyle has quit IRC | 14:16 | |
| Mech422 | rhallisey: I really don't know - I've been trying to figure out the 'right' way to do for our installs for a couple of months now - but I keep hitting issues around ansible | 14:17 |
| *** fguillot has joined #openstack-kolla | 14:17 | |
| Mech422 | rhallisey: roles arent really easily composable, tons of [ hostvars[ groups[ 'cinder'][0] ].endpoint_url type stuff | 14:18 |
| *** msimonin has joined #openstack-kolla | 14:18 | |
| *** Pavo has joined #openstack-kolla | 14:19 | |
| Mech422 | rhallisey: I can imagine it would only be worse for something a complex as kolla | 14:19 |
| rhallisey | I could see that | 14:19 |
| rhallisey | it could make things less readable | 14:19 |
| rhallisey | kolla could use 'tags' everywhere too | 14:20 |
| rhallisey | that could empower the anisble var file without splitting the playbook | 14:20 |
| rhallisey | so there would be a bootstrap tag | 14:21 |
| *** sdake has joined #openstack-kolla | 14:21 | |
| rhallisey | and you could turn it off in the ansible var file | 14:21 |
| Mech422 | Really stupid example of the 'fun' I have with ansible... | 14:21 |
| Mech422 | we require external acces via a http proxy | 14:21 |
| Mech422 | so all the boxes have http_proxy env vars defined | 14:21 |
| Mech422 | but that hoses openstack when it wants to talk to api endpoints | 14:22 |
| Mech422 | so I have a 'pkgs' tag in my playbooks, that downloads/installs all the packages across all playbooks... | 14:22 |
| Mech422 | after thats step, I can undefine the http_proxy vars :-P | 14:22 |
| *** sdake_ has quit IRC | 14:23 | |
| rhallisey | neat | 14:23 |
| rhallisey | idk it's a thought - more brownfield friendly playbooks would be good | 14:24 |
| Mech422 | rhallisey: always :-) | 14:24 |
| Mech422 | rhallisey: the only thing I'm really happy with so far with my stuff is that pkgs thing - get everything downloaded on the box locally before mucking with any network settings, upgrading kernels, etc. | 14:25 |
| *** tovin07 has left #openstack-kolla | 14:27 | |
| *** saneax-_-|AFK is now known as saneax | 14:27 | |
| dmsimard | mandre, sdake, sean-k-mooney: so I know I'm annoying with this but Ansible version upgrade for Ocata - Ansible just pushed out release candidates that includes a CVE fix for both the 2.2 and 2.1 branches, nothing in sight for 2.0 | 14:28 |
| Mech422 | rhallisey: I wonder how much of this is kolla's 'problem' though - if kolla is just making images for consumption by other projects (like kolla-kube) Maybe these things should be handled by the other projects? | 14:30 |
| *** skramaja has quit IRC | 14:30 | |
| sean-k-mooney | dmsimard: so master has a minium of 2.0 but i use 2.2 and it works fine | 14:30 |
| rhallisey | Mech422, well I'd say it's kolla-ansible's problem. The containers wouldn't change | 14:31 |
| dmsimard | sean-k-mooney: I guess I'm asking to bump the minimum to >2.1 | 14:31 |
| sean-k-mooney | i think that is resonable to do | 14:31 |
| *** dave-mccowan has joined #openstack-kolla | 14:31 | |
| dmsimard | sean-k-mooney: ansible is often not very backwards compat between versions though | 14:31 |
| dmsimard | so there might be gotchas | 14:32 |
| sean-k-mooney | is the gate using the kolla-host playbook to bootsrap yet? | 14:32 |
| sean-k-mooney | dmsimard: 2.2 seams to work fine 2.1.0 had issues i belive | 14:32 |
| Mech422 | rhallisey: yeah - the containers won't change - but like that email asked about taking nodes out of service and updating the haproxies - doesn't kubernetes do that for you ? | 14:32 |
| sean-k-mooney | so i would bump minium to 2.2 | 14:32 |
| Mech422 | rhallisey: so maybe it makes sense to let k8s worry about that and not kolla-ansible? | 14:33 |
| sbezverk | good morning | 14:33 |
| rhallisey | ya not everything was perfectly solvable | 14:33 |
| rhallisey | depends on what playbooks you have available to assist operators | 14:34 |
| rhallisey | or done automatically with k8s | 14:34 |
| Mech422 | sbezverk: morning | 14:34 |
| sean-k-mooney | dmsimard: in any case i would be infavor of the bump to 2.2 and raising the minium | 14:34 |
| sean-k-mooney | i have a meeting to run too sorry. will be back soon | 14:35 |
| dmsimard | bumping to 2.2 would allow us to land https://review.openstack.org/#/c/411930/ with blocks :) | 14:35 |
| sean-k-mooney | dmsimard: maybe submit a patch to the quickstart and we could continue the discustion there | 14:35 |
| *** lamt has quit IRC | 14:40 | |
| *** goldyfruit has joined #openstack-kolla | 14:41 | |
| *** jtriley has joined #openstack-kolla | 14:42 | |
| *** bjolo has quit IRC | 14:46 | |
| sdake | sup peeps | 14:49 |
| sdake | ehy sbezverk i am actually awake this morning :) | 14:49 |
| Serlex | hello | 14:49 |
| sdake | mgiles - are you all unblocked and whatnot | 14:49 |
| mgiles | sdake yes, all set - thanks | 14:50 |
| sdake | mgiles cool | 14:50 |
| *** sdake_ has joined #openstack-kolla | 14:51 | |
| *** msimonin has quit IRC | 14:51 | |
| *** sdake has quit IRC | 14:54 | |
| *** msimonin has joined #openstack-kolla | 14:58 | |
| *** sacharya has joined #openstack-kolla | 15:04 | |
| *** lrensing has joined #openstack-kolla | 15:05 | |
| *** sacharya has quit IRC | 15:08 | |
| duonghq | Evening guys, my network is very horrible so I must use my phone w/ 3G | 15:08 |
| duonghq | (Data plan) | 15:09 |
| duonghq | So, do we still have meeting this week | 15:09 |
| duonghq | ? | 15:09 |
| egonzalez | duonghq: if nothing changed yes | 15:11 |
| duonghq | Thank egonzalez | 15:11 |
| openstackgerrit | Flavio Percoco proposed openstack/kolla: Fix string formatting on error log https://review.openstack.org/418992 | 15:13 |
| *** david-lyle has joined #openstack-kolla | 15:17 | |
| *** DanyC has joined #openstack-kolla | 15:19 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move keystone precheck into its own role https://review.openstack.org/417723 | 15:21 |
| *** david-lyle has quit IRC | 15:21 | |
| *** sdake_ has quit IRC | 15:22 | |
| *** sdake has joined #openstack-kolla | 15:22 | |
| *** hogepodge has joined #openstack-kolla | 15:22 | |
| *** DanyC has left #openstack-kolla | 15:25 | |
| *** sacharya has joined #openstack-kolla | 15:25 | |
| openstackgerrit | Flavio Percoco proposed openstack/kolla: Fix string formatting on error log https://review.openstack.org/418992 | 15:27 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move iscsi precheck into its own role https://review.openstack.org/417725 | 15:27 |
| openstackgerrit | Merged openstack/kolla-ansible: Move nova precheck into its own role https://review.openstack.org/415564 | 15:40 |
| *** sp__ has joined #openstack-kolla | 15:40 | |
| *** Jeffrey4l has joined #openstack-kolla | 15:40 | |
| kfox1111 | morning. | 15:41 |
| *** shardy has quit IRC | 15:43 | |
| sbezverk | kfox1111: morning, helm-repo ps is all green, please check and ack if you have time | 15:43 |
| kfox1111 | k. | 15:43 |
| kfox1111 | did anyone get a chance to build the 3.0.2 containers last night? | 15:44 |
| sbezverk | kfox1111: not that I know of | 15:44 |
| kfox1111 | k.... | 15:44 |
| kfox1111 | portdirect: are you awake? | 15:45 |
| Jeffrey4l | egonzalez, could u review this https://review.openstack.org/#/c/415504/ ? | 15:45 |
| *** TxGirlGeek has joined #openstack-kolla | 15:45 | |
| *** DanyC has joined #openstack-kolla | 15:45 | |
| portdirect | physically, yes. mentally, never. sup kfox1111 ? | 15:45 |
| *** liyifeng has quit IRC | 15:45 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the variable name mismatch https://review.openstack.org/414804 | 15:45 |
| *** DanyC has quit IRC | 15:46 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Install the required packages for ceph-base image https://review.openstack.org/414805 | 15:46 |
| *** DanyC has joined #openstack-kolla | 15:46 | |
| kfox1111 | portdirect: do you think you could build the newly released 3.0.2 and push to the hub? | 15:47 |
| *** inc0 has joined #openstack-kolla | 15:48 | |
| portdirect | yea - but I've been caught up - 6 hours and they will be up :/ | 15:48 |
| portdirect | (cannot ssh into the machine i can push from) | 15:49 |
| kfox1111 | ok. | 15:49 |
| kfox1111 | maybe inc0 can do it? | 15:49 |
| kfox1111 | inc0: could you build the 3.0.2 release and push to the hub? | 15:49 |
| inc0 | hey, what's up? | 15:49 |
| inc0 | yeah I should | 15:49 |
| kfox1111 | the release was cut, but no one's lined up to build it. | 15:49 |
| kfox1111 | ok. cool. thanks. | 15:50 |
| inc0 | just let me wake up, I have some sleep depravation today | 15:50 |
| inc0 | I could...well...proves my point | 15:50 |
| kfox1111 | if you could do the centos-binary and the ubuntu-binary first, that would give us a little more chance to do the kolla-kubernetes testing earlier so we can cut that release? | 15:50 |
| inc0 | I'll do it later today | 15:50 |
| inc0 | however I have issues building centos-binary | 15:50 |
| kfox1111 | k. thanks. | 15:50 |
| inc0 | or had | 15:51 |
| inc0 | as I don't have centos box handy | 15:51 |
| *** DanyC has quit IRC | 15:51 | |
| kfox1111 | do you need a centos box to build the containers? | 15:51 |
| inc0 | trying to build centos-bin on top of xenial caused issues | 15:51 |
| kfox1111 | ah. | 15:51 |
| sean-k-mooney | inc0: i have done that in the past and i think it worked fine | 15:52 |
| kfox1111 | portdirect: review please: https://review.openstack.org/#/c/418556/ | 15:52 |
| sean-k-mooney | im not sure how well deploying centos on ubuntu would go in the general case but i have deployed a subset in the past too | 15:53 |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move murano precheck into its own role https://review.openstack.org/415113 | 15:54 |
| sdake | morning peeps | 15:56 |
| sp__ | sdake: mrg | 15:56 |
| inc0 | meeting in 4 | 15:56 |
| zhubingbing | ;) | 15:57 |
| inc0 | I think we have agenda mixup | 15:57 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: TEST NEWTON BRANCH https://review.openstack.org/417033 | 15:58 |
| portdirect | kfox1111: that looks really impressive | 15:58 |
| inc0 | berendt, you added debian deprecation right? | 15:58 |
| zhubingbing | go!! | 15:58 |
| berendt | inc0: yes | 15:58 |
| portdirect | but couple of q's - you are talking to docker from within a pod!? | 15:58 |
| inc0 | other topics seems from last week | 15:58 |
| *** sayantani01 has joined #openstack-kolla | 15:59 | |
| *** adrian_otto has joined #openstack-kolla | 16:00 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move rabbitmq precheck into its own role https://review.openstack.org/417731 | 16:00 |
| kfox1111 | portdirect: its just moving the existing stuff over. | 16:01 |
| kfox1111 | portdirect: you talking about the ceph-rbd trick? :) | 16:01 |
| kfox1111 | basicaly its a way not to have to install ceph on the host. | 16:02 |
| kfox1111 | kubelet only uses the rbd command. | 16:02 |
| kfox1111 | so the pod lays down a shell script rbd command that calls back into the ceph-rbd container to run its copy of rbd. | 16:02 |
| sdake | pbourke kfox1111 team meeting in openstack-meeting-4 if you have time :) | 16:02 |
| kfox1111 | seems to work really well. :) | 16:02 |
| sdake | portdirect :) | 16:02 |
| kfox1111 | yup. gota change a diper, then be right there. | 16:02 |
| pbourke | sdake: thanks! | 16:02 |
| *** lamt has joined #openstack-kolla | 16:03 | |
| portdirect | kfox1111: I'll have a look after meting - I knew there would be method to your madness :D | 16:04 |
| *** wirehead_1 is now known as wirehead_ | 16:05 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 16:07 |
| *** salv-orlando has joined #openstack-kolla | 16:09 | |
| *** ccesario has joined #openstack-kolla | 16:13 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move mongodb precheck into its own role https://review.openstack.org/416060 | 16:13 |
| *** matrohon_ has joined #openstack-kolla | 16:16 | |
| *** matrohon has quit IRC | 16:19 | |
| *** salv-orl_ has joined #openstack-kolla | 16:22 | |
| *** salv-orlando has quit IRC | 16:25 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible: Move kuryr precheck into its own role https://review.openstack.org/419022 | 16:27 |
| *** TxGirlGeek has quit IRC | 16:29 | |
| *** shardy has joined #openstack-kolla | 16:38 | |
| Pavo | anyone know why when starting an instance why it takes to spawn the instance? | 16:40 |
| Pavo | seems like its not caching the images on the compute nodes | 16:40 |
| Mech422 | wow...trumps press conference has me reeling...I keep expecting someone to yell 'April Fools'! | 16:54 |
| inc0 | Pavo, you're not using ceph right? | 16:54 |
| sdake | Mech422 that is what you get for watching tv during the workday ;) | 16:57 |
| Mech422 | sdake; I worked all night anyway | 16:58 |
| sdake | Mech422 fair enough :) | 16:58 |
| sdake | Mech422 i have some crazy schedule too :) | 16:58 |
| Mech422 | sdake: maybe this is all a dream? trump donating all revenues from international holdings to the US treasury ? | 16:59 |
| mliima | :0 | 17:00 |
| sdake | Mech422 not watching | 17:00 |
| sdake | Mech422 I don't keep up with global politics - its hardenough dealing with the politics of open source + my company :) | 17:00 |
| zhubingbing | hi | 17:01 |
| Mech422 | sdake: yeah - well, its all surreal... I expected him to fight tooth and nail to hold onto his businesses | 17:02 |
| zhubingbing | i hope all can help to see fluentd | 17:02 |
| *** sayantani01 has quit IRC | 17:02 | |
| Mech422 | anyway, I'll shutup now...just a lil shocked I guess | 17:02 |
| inc0 | so wall is still happening? | 17:02 |
| inc0 | how fun! | 17:03 |
| *** sayantani01 has joined #openstack-kolla | 17:03 | |
| *** gema has joined #openstack-kolla | 17:03 | |
| *** matrohon_ has quit IRC | 17:03 | |
| Serlex | Wall is definitely happening | 17:05 |
| inc0 | I hope they won't ruin big bend, it's beautiful without wall in it;) | 17:06 |
| Mech422 | yeah - that part was sorta a bummer... | 17:07 |
| *** dmsimard is now known as dmsimard|afk | 17:08 | |
| Serlex | he probably has never been, so probably say goodbye | 17:11 |
| *** pcaruana has quit IRC | 17:12 | |
| *** rmart04 has quit IRC | 17:13 | |
| *** vhosakot has joined #openstack-kolla | 17:16 | |
| sdake | i find it highly unlikely he can get a wall thoruhg congress because the US has the OUT OF (F) MONEY ERROR | 17:16 |
| sdake | and congress controls the money :) | 17:17 |
| Serlex | I know nothing about politics but isnt congress mostly made of his party? | 17:18 |
| *** egonzalez has quit IRC | 17:18 | |
| kfox1111 | yes/no. | 17:19 |
| kfox1111 | not sure trump has a party. ;) | 17:19 |
| Serlex | you know what I mean | 17:19 |
| Serlex | :) | 17:19 |
| kfox1111 | he's more of an independent. | 17:19 |
| gema | inc0: o/ I am looking to discuss with someone the possibility of adding ARM64 containers for the different services and was wondering if you are the right person | 17:19 |
| *** bmace has joined #openstack-kolla | 17:19 | |
| *** MasterOfBugs has quit IRC | 17:20 | |
| *** pramodrj07 has joined #openstack-kolla | 17:20 | |
| inc0 | gema, yeah, I guess I am:) | 17:20 |
| sbezverk | kfox1111: how about my review? | 17:20 |
| inc0 | so arm containers are going to be pain, as I don't think we have any access to ARM64 on gates | 17:21 |
| gema | inc0: that's where I can help | 17:21 |
| gema | inc0: I am also talking to infra about this | 17:21 |
| gema | inc0: I work for Linaro and we have member hardware ready to be used for upstream testing | 17:21 |
| inc0 | we could create 3rd party CI for ARM64 | 17:21 |
| gema | inc0: easily, yep | 17:22 |
| *** sdake_ has joined #openstack-kolla | 17:22 | |
| kfox1111 | sbezverk: oh, yeah. sorry. go a broken machine at work I'm working on. give me a few and I'll get back to it. | 17:22 |
| inc0 | I think containers wouldn't be big issue, just make sure packages are installed for correct arch | 17:22 |
| gema | inc0: the part that worries me at the moment is the ramp up and whether the use case of the containers you produce is the same we have | 17:22 |
| inc0 | what do you mean by use case? | 17:22 |
| gema | inc0: who uses kolla? | 17:22 |
| *** unicell has quit IRC | 17:23 | |
| sdake_ | gema openstack operators | 17:24 |
| inc0 | well, people who deploys openstack? I mean, use case of kolla is to deploy openstack | 17:24 |
| gema | inc0: our use case is starting small, generate a small set of containers that are known to work, i.e. pass testing and are capable of standing up a cloud for more extensive testing | 17:24 |
| inc0 | if it's meant to be deployed on arm, so be itr | 17:24 |
| gema | inc0: and do we need to produce containers for all the projects or can we start with a few and then expand? | 17:24 |
| inc0 | gema, so I'd rather see modification of current container than full set of new ones | 17:24 |
| gema | inc0: absolutely agree | 17:24 |
| gema | inc0: else we wouldn't be talking at all :D | 17:25 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator https://review.openstack.org/418073 | 17:25 |
| *** sdake has quit IRC | 17:25 | |
| *** pramodrj07 has quit IRC | 17:25 | |
| inc0 | templating allows you to tackle it container by container | 17:25 |
| inc0 | however for testing and proof points you can use our override mechanism | 17:25 |
| gema | inc0: can you point me to the test cases? | 17:25 |
| inc0 | I'm just not sure exactly how much work that would require | 17:25 |
| inc0 | of override mechanism? | 17:26 |
| gema | yeah, but also of the test cases you run for the existing containers | 17:26 |
| inc0 | http://docs.openstack.org/developer/kolla/image-building.html#dockerfile-customisation <- this is overrides | 17:26 |
| inc0 | our test case for existing containers are quite simple - we buold them on 3 distros so far (ubuntu xenial, centos 7 and oraclelinux) | 17:27 |
| inc0 | and then deploy with ansible in kolla-ansible | 17:27 |
| gema | and then you run something to make sure the openstack you've deployed runs? | 17:27 |
| gema | is operational, I mean | 17:27 |
| *** msimonin has quit IRC | 17:28 | |
| inc0 | yes, we spawn vm at the end | 17:28 |
| *** Bico_Fino has quit IRC | 17:28 | |
| gema | ok | 17:28 |
| *** msimonin has joined #openstack-kolla | 17:28 | |
| *** msimonin has quit IRC | 17:28 | |
| gema | seems doable | 17:28 |
| gema | your testing is based on ubuntu hosts, or coreos or ? | 17:28 |
| *** strigazi is now known as strigazi_AFK | 17:29 | |
| inc0 | infra has ubuntu and centos vms | 17:29 |
| gema | the reason I ask is because currently we run our testing on debian and centos | 17:29 |
| gema | ok, fair enough | 17:30 |
| gema | we'd need to test on HW as well | 17:30 |
| gema | but that can be transparent to you | 17:30 |
| portdirect | kfox1111: ping | 17:30 |
| inc0 | yeah, as far as I know infra won't be able to use it's default gating on that | 17:30 |
| kfox1111 | portdirect: hi | 17:30 |
| inc0 | so 3rd party CI is required | 17:30 |
| gema | inc0: yep, agreed | 17:30 |
| portdirect | kfox1111: another question about that ps :) why admin-pod - what's its purpose? | 17:31 |
| kfox1111 | batteries included option. | 17:31 |
| gema | inc0: I will start reading and planning and maybe we can touch base at PTG ? | 17:31 |
| portdirect | kfox1111: fyi thats a really neat trick with rbd :) | 17:31 |
| kfox1111 | lets shell scripts get to an admin client to do stuff like setup volumes/create users/etc. | 17:31 |
| inc0 | that's my requirement, if you guys can provide that, maintiain quality, there is no reason not to include arm arch packages | 17:31 |
| openstackgerrit | Merged openstack/kolla: Fix string formatting on error log https://review.openstack.org/418992 | 17:31 |
| gema | inc0: +1 | 17:31 |
| kfox1111 | portdirect: thanks. :) | 17:31 |
| inc0 | yeah, sure, feel free to tag along any kolla sessions, I'll the loud one | 17:31 |
| portdirect | why a pod - and not a rc? | 17:32 |
| gema | inc0: perfect, thanks for your help and all the pointers! | 17:32 |
| kfox1111 | portdirect: ceph-admin's a batteries included thing. | 17:32 |
| inc0 | any time | 17:32 |
| kfox1111 | portdirect: it was intended just as a helper for onetime setup, then can get destroyed. | 17:32 |
| sbezverk | kfox1111: ? | 17:32 |
| kfox1111 | so didn't put much effort into it to make it stick around. | 17:32 |
| kfox1111 | sbezverk: what's your question? | 17:33 |
| sbezverk | kfox1111: need you to ack my PS | 17:33 |
| kfox1111 | let me look | 17:33 |
| portdirect | kfox1111: I'll jump in with one - is ceph-admin used during deployment at all? | 17:34 |
| portdirect | kfox1111: but help sbezverk first :) | 17:34 |
| *** vhosakot has quit IRC | 17:34 | |
| kfox1111 | portdirect: only during deployment. then it can be deleted. | 17:34 |
| kfox1111 | portdirect: and only if you don't want to create the user accounts yourself. | 17:34 |
| portdirect | ok - so why not have a job to do that? | 17:35 |
| mliima | need act here https://review.openstack.org/#/c/418467/ and https://review.openstack.org/#/c/417094/ | 17:35 |
| kfox1111 | portdirect: have a loook at tools/simple_setup_ceph_users | 17:35 |
| *** sayantani01 has quit IRC | 17:36 | |
| kfox1111 | sbezverk: looking really good. | 17:36 |
| kfox1111 | have you tried turning off persistence though? I'm guessing it wont work as its a string. | 17:36 |
| *** sayantani01 has joined #openstack-kolla | 17:36 | |
| kfox1111 | probably need to pull it with kolla_val_get_raw instead. | 17:37 |
| kfox1111 | other then that, I think its good to go. | 17:37 |
| sbezverk | kfox1111: right without persistence it will not work as there is no packages in the image | 17:38 |
| sbezverk | so for now I think it should be ok | 17:38 |
| portdirect | kfox1111: should https://github.com/openstack/kolla-kubernetes/blob/master/tools/setup_simple_ceph_users.sh not be made into a job? | 17:38 |
| kfox1111 | portdirect: I guess it could be a set of jobs. I just wanted to minimize the mount of stuff with a ceph admin credential. | 17:38 |
| kfox1111 | it gives you a good place to run commands manually too. | 17:39 |
| kfox1111 | I had to use it a lot for debugging ceph. | 17:39 |
| * kfox1111 shrugs. I wouldn't be apposed to rewriting it as jobs. | 17:40 | |
| portdirect | kfox1111: yeah - I'm not opposed to the ceph-amin pod (i see its value for that) | 17:40 |
| kfox1111 | but can't for 0.4.0. | 17:40 |
| portdirect | hmmm | 17:40 |
| portdirect | do you promise to put it on the todo list? ;D | 17:40 |
| kfox1111 | heh. not sure it will be towards the top. | 17:41 |
| portdirect | :( grumpy cat | 17:41 |
| kfox1111 | I won't block someone else from rewriting it sooner then I get to it though. :) | 17:41 |
| portdirect | I wont block as its a sound conversion from jinja | 17:41 |
| kfox1111 | this ps is just a jinja2 -> helm conversion. so its really waht has been happening for a while now. | 17:42 |
| portdirect | but think that we really should not be combining tacktics like this moving forward :) | 17:42 |
| kfox1111 | agreed. | 17:42 |
| * portdirect runs back off to gerrit | 17:42 | |
| sbezverk | kfox1111: portdirect: when you stop discussing ceph. can you check this ps: https://review.openstack.org/418073 please ignore any programming issues, my goal is to make sure it has a sound logic. | 17:43 |
| kfox1111 | sbezverk: if persistent is not set, it will still work though, as it will just pull/build every time? | 17:43 |
| portdirect | sbezverk: we will never stop discussing ceph | 17:44 |
| *** sp__ has quit IRC | 17:44 | |
| portdirect | sbezverk: but I'll have a look now | 17:44 |
| sbezverk | kfox1111: no because you wanted to have them in docker image | 17:44 |
| *** jemcevoy has joined #openstack-kolla | 17:44 | |
| *** sp__ has joined #openstack-kolla | 17:44 | |
| sbezverk | so wehn I detect that folder is not empty, I just ignore any prep steps | 17:44 |
| sbezverk | assuming that packages came from docker image | 17:45 |
| kfox1111 | sbezverk: ah. ok. yeah. | 17:45 |
| kfox1111 | sbezverk: ok. I guess I'm ok with it being kind of broken for now, as we don't have a way to test the other case anyway. | 17:45 |
| kfox1111 | sbezverk: we can fix it in a followup ps. | 17:46 |
| sbezverk | kfox1111: right and there was some opposition to including helm packages into docker image | 17:46 |
| sbezverk | from sdake | 17:46 |
| sbezverk | I need this PS to continue work of operator's gate | 17:47 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Convert ceph-rbd and ceph-admin to microservice packages. https://review.openstack.org/418556 | 17:48 |
| kfox1111 | k.... this is going to make the -type ps a bit of a pain to merge.... | 17:48 |
| kfox1111 | but lets just do it. | 17:48 |
| *** mannamne has quit IRC | 17:49 | |
| jemcevoy | Pavo: Morning... How would I go about patching nova-spicehtml5proxy on centOS 7 v 3.0.1 to include the missing numpy package? bug/1638801 | 17:49 |
| kfox1111 | I'm really starting to dislike all the varient forks in the gate. :/ | 17:49 |
| kfox1111 | way too much code to edit. :/ | 17:49 |
| sbezverk | kfox1111: well I do not mind before continue operator gate to work on modularization unless you have it on your to do list at the top | 17:50 |
| kfox1111 | wf'ed. | 17:50 |
| kfox1111 | I | 17:50 |
| kfox1111 | I'm hoping after the next 2 ps's we won't be touching the gate so much, so won't be quite as important. | 17:51 |
| kfox1111 | but if you have some time to pull out the common bits at the top of *workflow* to a common file, that would really be nice. | 17:51 |
| sbezverk | kfox1111: ok sounds good.. | 17:51 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Helm repository container and service https://review.openstack.org/417474 | 17:53 |
| *** MasterOfBugs has joined #openstack-kolla | 17:55 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 17:57 |
| kfox1111 | sdake_: inc0: hows the container building going? | 17:57 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to nova-api container https://review.openstack.org/418131 | 17:58 |
| *** bjolo has joined #openstack-kolla | 17:58 | |
| sdake_ | kfox1111 haven't got to it yet - but its on deck for today | 18:00 |
| mliima | need act here https://review.openstack.org/#/c/418467/ | 18:01 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to nova-api container https://review.openstack.org/418131 | 18:01 |
| kfox1111 | sdake_: ok. thanks. | 18:01 |
| *** saneax is now known as saneax-_-|AFK | 18:03 | |
| *** prameswar has quit IRC | 18:04 | |
| kfox1111 | I think we're down to 2 ps's before 0.4.0. | 18:05 |
| kfox1111 | https://review.openstack.org/418587 | 18:05 |
| kfox1111 | and one to update to 3.0.2 and rename packages 0.4.0. | 18:05 |
| kfox1111 | almost there... | 18:06 |
| *** sp__ has quit IRC | 18:06 | |
| sdake_ | kfox1111 cool - let me finish this review then i'll start the build | 18:06 |
| sdake_ | (and push) | 18:06 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to cinder-api container https://review.openstack.org/419076 | 18:07 |
| kfox1111 | sdake_: cool. thanks. | 18:08 |
| *** athomas has quit IRC | 18:10 | |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to aodh-api container https://review.openstack.org/419077 | 18:10 |
| *** adrian_otto has quit IRC | 18:13 | |
| sdake_ | inc0 would you mind joining #openstack-tripleo | 18:14 |
| sdake_ | inc0 and #rdo | 18:14 |
| *** tonanhngo has joined #openstack-kolla | 18:15 | |
| *** adrian_otto has joined #openstack-kolla | 18:15 | |
| inc0 | omw | 18:15 |
| kfox1111 | join #tripleo | 18:15 |
| *** david-lyle has joined #openstack-kolla | 18:15 | |
| *** david-lyle has quit IRC | 18:16 | |
| *** david-lyle has joined #openstack-kolla | 18:16 | |
| *** unicell has joined #openstack-kolla | 18:17 | |
| sdake_ | inc0 ya sorry wrong channel - I fial at irc :) | 18:21 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to gnocchi-api container https://review.openstack.org/419080 | 18:23 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to aodh-api container https://review.openstack.org/419077 | 18:23 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to cinder-api container https://review.openstack.org/419076 | 18:24 |
| *** adrian_otto has quit IRC | 18:25 | |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to nova-api container https://review.openstack.org/418131 | 18:25 |
| sdake_ | wow something is causing music to come out of my computer | 18:28 |
| sdake_ | time to close chrome - hope that fixes it :) | 18:28 |
| sdake_ | it was itunes- thank god ;) | 18:29 |
| sdake_ | last thing I need now is a mac virus | 18:29 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: Set versions to 0.4.0 and 3.0.2 https://review.openstack.org/419082 | 18:30 |
| sbezverk | I bet kfox1111 will go into the Guiness book, for the longest PS ever ;-) | 18:32 |
| *** TxGirlGeek has joined #openstack-kolla | 18:32 | |
| kfox1111 | sed -i... :) | 18:33 |
| kfox1111 | this is one of my complaints about helm though. makes touching a lot of files needed. :/ | 18:34 |
| kfox1111 | there really needs to be templating of Values and Chart yaml's. :/ | 18:34 |
| sbezverk | kfox1111: I hope so, othewise it would take very very long time | 18:34 |
| kfox1111 | I put $VERSION into a variable in the gate scripts too, so next time we change that, it should only be in a few places, rather then all over the file. | 18:36 |
| *** absubram has quit IRC | 18:36 | |
| kfox1111 | after these two changes, I really hope thats the last of the sweeping changes. | 18:39 |
| *** absubram has joined #openstack-kolla | 18:39 | |
| kfox1111 | (famious last words) | 18:39 |
| sbezverk | :) | 18:41 |
| sbezverk | kfox1111: have you had a chance to check the operator lif cycle? | 18:41 |
| kfox1111 | oh. thanks for the reminder. I started looking at it but got destracted. | 18:43 |
| *** shardy has quit IRC | 18:48 | |
| satyar | Hi jeffrey4l | 18:49 |
| kfox1111 | sbezverk: I think the logic looks ok, if I'm following it correctly. | 18:50 |
| kfox1111 | sbezverk: though I think the delete logic might be a bit off. the user will just remove the 3rd party resource in that case, and then the the operator should do the same with the helm packages. rather then the user uploading a state=delete? | 18:52 |
| sbezverk | kfox1111: when operator get delete state, it will only remove earlier deployed components but do not removed itself | 18:53 |
| sbezverk | gets back into standby state waiting for another command fromn the user | 18:53 |
| kfox1111 | oh. I was thinking the state was in 3rd party resource. let me go look through again. | 18:54 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: Set versions to 0.4.0 and 3.0.2 https://review.openstack.org/419082 | 18:54 |
| *** dave-mcc_ has joined #openstack-kolla | 18:54 | |
| *** dave-mccowan has quit IRC | 18:54 | |
| kfox1111 | oh. your storing the state in the 3rd party resource type creation object. | 18:57 |
| kfox1111 | well... | 18:57 |
| kfox1111 | no... | 18:57 |
| kfox1111 | ok... lets see.. | 18:57 |
| sbezverk | kfox1111: yes | 18:58 |
| kfox1111 | operators/memcached/memcached_3rd_party.yaml defines the 3rd party resource type. | 18:58 |
| sdake_ | kfox1111 that is the purpose of a 3rd party resource - it is like a context operator | 18:58 |
| sdake_ | rather context variable | 18:58 |
| sdake_ | in c | 18:58 |
| kfox1111 | right? | 18:58 |
| rhallisey | kfox1111, what are your thought about having the operator polling | 18:58 |
| kfox1111 | and the memcached_operator.yaml is the 3rd party resource instance? | 18:59 |
| * rhallisey just glanced at sbezverk patch briefly | 18:59 | |
| kfox1111 | rhallisey: should use longpolling if possible. | 18:59 |
| sbezverk | kfox1111: ok let me walk you through what I have | 18:59 |
| *** pramodrj07 has joined #openstack-kolla | 18:59 | |
| kfox1111 | sbezverk: yes please. :) | 18:59 |
| sbezverk | 1. yaml defining new kind | 19:00 |
| sbezverk | 2. object of new kind | 19:00 |
| sbezverk | 3. operator code | 19:00 |
| sbezverk | new kind gets created once and forever | 19:00 |
| kfox1111 | right. and #1 is operators/memcached/memcached_3rd_party.yaml and #2 is operators/memcached/memcached_operator.yaml | 19:00 |
| sbezverk | operator object of new kind could be created delete | 19:00 |
| sbezverk | but not necessarily | 19:01 |
| openstackgerrit | Dan Prince proposed openstack/kolla: Add Apache packages to ironic-pxe container https://review.openstack.org/419092 | 19:01 |
| *** satyar has quit IRC | 19:01 | |
| kfox1111 | yeah, #2 is what a user would upload to get the operator to do stuff? | 19:01 |
| sbezverk | it used to store some info and also to communicate to operator some actions | 19:01 |
| sbezverk | kfox1111: correct | 19:01 |
| sbezverk | it also used to pass some parameters for deployment | 19:02 |
| sbezverk | like in my example port info | 19:02 |
| kfox1111 | right. | 19:02 |
| sbezverk | once deployed the operator goes into standby mode | 19:02 |
| sbezverk | and wait for other instruction from the use via operator object | 19:02 |
| kfox1111 | goes back into standby mode? | 19:02 |
| sbezverk | yes | 19:03 |
| kfox1111 | yeah. | 19:03 |
| sbezverk | if user decides to remove deployment | 19:03 |
| sbezverk | it send delete command, operator does it and then goes into standby mode again | 19:04 |
| kfox1111 | yeah. so, kubectl delete my3rdpartythingy | 19:04 |
| kfox1111 | operator notices the resource goes away and helm deletes stuff. | 19:04 |
| kfox1111 | then goes back to standby? | 19:04 |
| sbezverk | kfox1111: at this point you need to do kubectl replace -f my3rdpartythingy with new state value | 19:05 |
| kfox1111 | seems like I'm missing a bit though. state's stored in the 3rd party object, but there can be more then one 3rd party object. | 19:05 |
| sbezverk | I mean it can be nicely wrapped into somehting | 19:05 |
| kfox1111 | seems like maybe the wrong ui to me then. | 19:06 |
| kfox1111 | maybe we're missing a piece of state. | 19:06 |
| sbezverk | kfox1111: operator works with 1 object | 19:06 |
| kfox1111 | the 3rd party resource is the requested state from the user. | 19:06 |
| *** zhubingbing has quit IRC | 19:06 | |
| kfox1111 | then there's state in the k8s system. | 19:06 |
| kfox1111 | and when they differ, the operator should try and make the k8s system match the request. | 19:06 |
| sbezverk | kfox1111: requested state for the deployment and not the actual 3rdparty operator | 19:07 |
| kfox1111 | so user creates a new 3rd party object, the opeartor launches helm packages. | 19:07 |
| kfox1111 | user updates the 3rd party object, the operator helm upgrades with new values? | 19:07 |
| kfox1111 | and user deletes the 3rd party object, the operator helm deletes the objects | 19:07 |
| rhallisey | sbezverk, if the user requests deploy, the thirdpartyresource will remain in place while the operator reads it. Would the operator have to track the previous state? | 19:07 |
| rhallisey | so as to not continuously deploy | 19:08 |
| *** Serlex has quit IRC | 19:08 | |
| sbezverk | kfox1111: 3rd part is only the commuication mechamnism between a user and operator | 19:08 |
| kfox1111 | yeah. I think the operator will need to keep track of whatever it needs to track resources requested. | 19:08 |
| rhallisey | kfox1111, but to that point, should it at all? | 19:08 |
| kfox1111 | but the user could delete the 3rd party resource at any time. | 19:08 |
| sbezverk | kfox1111: 3rd party operator has state in its definition | 19:08 |
| rhallisey | that could be trouble because you want the opereator to be dumb | 19:08 |
| kfox1111 | so if state is stored in that 3rd party resource, it needs a way to recover from that case. | 19:09 |
| *** gfidente has quit IRC | 19:09 | |
| sbezverk | kfox1111: in this case operator will just sit idle until new 3rd party object is created | 19:09 |
| kfox1111 | label sould be one way to solve it, if helm supported them. :/ | 19:09 |
| rhallisey | sbezverk, no the thridpartyresource is attached to the container iirx | 19:09 |
| sbezverk | I am planning to use labels | 19:09 |
| rhallisey | iirc | 19:09 |
| rhallisey | unless you have it read another resource | 19:10 |
| sbezverk | rhallisey: nope | 19:10 |
| kfox1111 | user creates myresource. | 19:10 |
| sbezverk | 3rd party has no associate with container at all | 19:10 |
| kfox1111 | operator launches package foo operator_tag=myresource | 19:10 |
| kfox1111 | when user deletes myresource | 19:10 |
| sbezverk | there is no pod associated with 3rd part | 19:10 |
| kfox1111 | operator notices myresource doesn't exist, and deletes all operator_tag=myresource things. | 19:10 |
| rhallisey | then in what container will this script run in: https://review.openstack.org/#/c/418073/9/operators/memcached/memcached_operator.py | 19:11 |
| sbezverk | kfox1111: operator react only to user command | 19:11 |
| sbezverk | it does not notice anything or take any action | 19:11 |
| rhallisey | sbezverk, ok I think your model for an operator is a little different than mine | 19:11 |
| kfox1111 | sbezverk: users don't command though. they request. | 19:11 |
| kfox1111 | "here's the desired state" "do whatever you need to do to make it happen" | 19:12 |
| sbezverk | rhallisey: have you get it working a bit, cause after all research I did I could see the only way | 19:12 |
| kfox1111 | declarative vs imparative | 19:12 |
| *** ccesario has quit IRC | 19:12 | |
| kfox1111 | the 3rd party resource is the requested desired state. | 19:12 |
| rhallisey | I associate the operator with the thirdpartyresource: https://github.com/rthallisey/nave/blob/master/vessels/mariadb/mariadb-vessel.yml#L5 | 19:13 |
| sbezverk | rhallisey: I would be really interested to see other approaches | 19:13 |
| kfox1111 | "I want a memcached on port 372". | 19:13 |
| kfox1111 | the operator takes that, and decides, I need a svc, a deployment, etc and launches. | 19:13 |
| kfox1111 | if the user updates it to be "I want a memcached on port 724", it looks at the existing deployment, | 19:13 |
| rhallisey | sbezverk, don't get me wrong, I don't think your approach is wrong. It's just not the approach I took | 19:13 |
| kfox1111 | decides it only needs to change the svc, and redeploys that. | 19:13 |
| sbezverk | rhallisey: absolutely that is why I asked all you guys to take a stab it it | 19:14 |
| kfox1111 | user then uploads a second 3rd party resource "I want a memcached on port 772" and it decides to do another deployment of svc and deployment. | 19:14 |
| rhallisey | the way I look at it, an operator is associated with a thirdpartyresource. It performs the action as described to it by the template | 19:14 |
| sbezverk | I really wanted to see different approches | 19:15 |
| rhallisey | that action could be acting as a dameon or a one time operation | 19:15 |
| rhallisey | the temple could be ammended with a kubectl replace | 19:15 |
| sbezverk | rhallisey: the yaml you showed, can you launch it and it runs as a container of new kind? | 19:15 |
| *** dave-mcc_ has quit IRC | 19:15 | |
| rhallisey | ya I've launched the container, but it doesn't do anything atm | 19:16 |
| sbezverk | rhallisey: cool cause nobody at kube community did not mention that it is possible to do it :( | 19:17 |
| sbezverk | kfox1111: I thought to use operator per specific deployment | 19:17 |
| sbezverk | if you want to have 5 memacehd with different ports that would require 5 operators managing these deployments | 19:18 |
| rhallisey | I'll need to mess around with it some more. Iirc it was fine a month ago | 19:18 |
| rhallisey | but the code that the container runs is where I need a ton more work | 19:19 |
| sbezverk | rhallisey: appreciate if you could keep me in the loop :) | 19:19 |
| rhallisey | for sure | 19:19 |
| kfox1111 | sbezverk: no, I think we want an operator instance per deployment type. | 19:19 |
| kfox1111 | sbezverk: then a 3rd party resource per deployment. | 19:20 |
| kfox1111 | so, for memcached, say you have one memcached per openstack service (neutron/nova/etc) | 19:20 |
| kfox1111 | you would launch the memcached-operator | 19:20 |
| kfox1111 | then upload a 3rd party resource for neutron, one for nova, etc. | 19:20 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: Set versions to 0.4.0 and 3.0.2 https://review.openstack.org/419082 | 19:20 |
| sbezverk | kfox1111: I thought 1 operator per service in the simplest form | 19:21 |
| sbezverk | if you need multiple instance of mariadb then extra instance would require its own operator | 19:21 |
| kfox1111 | thats not usuallly how operators work. | 19:21 |
| kfox1111 | you don't have to launch a new kube-controller-manager each time you want a new deployment object. :) | 19:22 |
| sbezverk | kfox1111: controller manager is completely different type of code | 19:22 |
| kfox1111 | not really. | 19:23 |
| kfox1111 | controller-manager talks to k8s and looks for new deployments/daemonset/etc objects, | 19:23 |
| kfox1111 | and then converts those objects to new object types it pushes back into k8s. | 19:23 |
| kfox1111 | (usually pods) | 19:23 |
| sdake_ | sbezverk the work looks good to me | 19:23 |
| sdake_ | sbezverk I think it will evolve over itme | 19:23 |
| kfox1111 | kolla-kubernetes operators would do the same. | 19:23 |
| kfox1111 | take in a resource type, decide what other objects are required to realize the request, | 19:24 |
| sbezverk | controller manager is just an umbrealla for bunch of separate controllers per type | 19:24 |
| kfox1111 | then push them back into k8s for creation. | 19:24 |
| *** satyar has joined #openstack-kolla | 19:24 | |
| kfox1111 | sbezverk: thats oversimplifying what it does. it does aggregate a bunch of operators into one big one. | 19:24 |
| kfox1111 | but that is largely unimportant. | 19:25 |
| kfox1111 | the logic of getting in a declarative request, and pushing in new objects to realize it is the important bit. | 19:26 |
| mliima | sdake need act here https://review.openstack.org/#/c/418467/ | 19:26 |
| rhallisey | sbezverk, getting back to the thirdpartyresource, however it is read it needs to be read once | 19:27 |
| rhallisey | and react to a change, which requires state | 19:28 |
| sbezverk | kfox1111: ok possibly multiple ways exist, atm I do not see other life cycle. I guess I will have to see other proposeal more carefully.. | 19:28 |
| kfox1111 | https://review.openstack.org/#/c/418587/ is ready. | 19:28 |
| kfox1111 | so, lets take a use case and try and isolate it. | 19:28 |
| kfox1111 | an rc, say. | 19:29 |
| sbezverk | rhallisey: I think about this resource more like a communication pipe :-) | 19:29 |
| kfox1111 | k8s extends the system with knowlege of how to process rc's. ie, launching an instance of kube-controller-manager. | 19:29 |
| sbezverk | user via this pipe tells operator what to do and it does it. | 19:29 |
| kfox1111 | kube-controller-manager starts up, and watches for any changes to kind=rc | 19:30 |
| rhallisey | sbezverk, I agree with that. But the current idea is to poll the resource and unless it gets deleted it would end up continually polling the same command | 19:30 |
| kfox1111 | user uploads a request for kind=rc. | 19:30 |
| kfox1111 | the operator picks it up, notices its new. | 19:30 |
| kfox1111 | it inspects its properties for stuff specific for it. in this case, number of replica's. | 19:30 |
| kfox1111 | it then creates 3 (requested number) pod.yaml's based on the stuff in the request and uploads them to k8s. | 19:31 |
| kfox1111 | user then updates the request to set replicas=2. | 19:31 |
| sbezverk | kfox1111: I know how rc works more or less, I was not aware that we actually want to write something of the same magnitude | 19:32 |
| kfox1111 | operator notices resource changed. checks new state against old and notices replicas went from 3 to 2. | 19:32 |
| rhallisey | couldn't we version the thirdpartyresource so that we could just check the version and react | 19:32 |
| kfox1111 | it then decides to delete one of the existing pods it credated to realize the request. | 19:32 |
| *** yangzb09 has quit IRC | 19:32 | |
| sdake_ | kfox1111 that sounds g ood in teh long term - in the short term I think we want something that does a basic standing up of kolla microservices that helm implement | 19:32 |
| sdake_ | kfox1111 so I wouldn't block on more advanced logic | 19:33 |
| sdake_ | (block the review that is) | 19:33 |
| rhallisey | sbezverk, well in the case of a complex application, deleting may not be enough | 19:33 |
| sbezverk | rhallisey: sure I think it can easily be extended to reconfigure upgrade | 19:33 |
| kfox1111 | sdake_: yeah. we need to make sure we have the right logic to allow it to be added later and not blocked by early assumptions. | 19:33 |
| kfox1111 | doesn't have to be fully implemented right away though. | 19:34 |
| sdake_ | kfox1111 i expect the operators will go thorugh numerous iterations befoer they are fully baked ;) | 19:35 |
| rhallisey | super simple case: deploy mariadb | 19:35 |
| rhallisey | even simpler | 19:35 |
| rhallisey | lets say keystone | 19:35 |
| kfox1111 | sdake_: I agree. just want to make sure we don't block more advanced usages. | 19:35 |
| rhallisey | take a crack at that | 19:35 |
| sbezverk | rhallisey: were refering to me to take a crack at it? | 19:36 |
| sdake_ | ya keystone should be simple | 19:36 |
| sdake_ | memcache should be simpel too | 19:36 |
| rhallisey | idk anyone | 19:36 |
| rhallisey | I mean let's talk it out | 19:36 |
| sdake_ | sbezverk i think you can just merge what you got when its working and not in warning mode in the commit message | 19:36 |
| rhallisey | how would the deploy work | 19:36 |
| sdake_ | and we can iterate from there | 19:36 |
| kfox1111 | so, keystone may be a good example of a nontrivial thing that helm by itself may have issues doing. | 19:36 |
| kfox1111 | say you want to have an optional memcached deployment as part of keystone. | 19:37 |
| *** krtaylor has quit IRC | 19:37 | |
| sbezverk | rhallisey: It looks very simple with appraoch I am proposing | 19:37 |
| kfox1111 | it could be just a flag in the 3rdparty resource. "dedicated_memcached=true/false" | 19:37 |
| rhallisey | kfox1111, even simpler, let's just deploy | 19:37 |
| rhallisey | sbezverk, ok so with your approach do you have like: action: deploy | 19:37 |
| kfox1111 | then depending on that state, the operator helm deployes a memcached or not. | 19:37 |
| sbezverk | yes | 19:37 |
| kfox1111 | action is imparitive though. | 19:38 |
| rhallisey | looks like you have state | 19:38 |
| sbezverk | rhallisey: I do.. initial state is standby | 19:38 |
| rhallisey | kfox1111, the user input is imperative | 19:38 |
| rhallisey | thirdpartyresource represents user input | 19:38 |
| sbezverk | of the controller, as soon as ut detect change in state, it will react based on the state which is == command | 19:39 |
| kfox1111 | no other k8s imput is imperitive. | 19:39 |
| sbezverk | rhallisey: yes | 19:39 |
| kfox1111 | you give k8s a declarative input of desired state. | 19:39 |
| rhallisey | kfox1111, we can change the name later let's just use state for now | 19:39 |
| kfox1111 | k8s then performs actions to bring the real state inline with the desired state. | 19:39 |
| rhallisey | ok to deploy you set state = deploy in the thirdpartyresource | 19:39 |
| kfox1111 | it seems like a subtile distinction but really important Ithink. | 19:40 |
| kfox1111 | rhallisey: who sets state=deploy? | 19:40 |
| kfox1111 | the user shouldn't. | 19:40 |
| sbezverk | kfox1111: user or tool higher level | 19:40 |
| rhallisey | I would think user | 19:40 |
| kfox1111 | user should never set that. | 19:40 |
| sbezverk | why??? I want to deploy manually | 19:40 |
| sbezverk | why I cannot do it??? | 19:41 |
| rhallisey | kfox1111, why? | 19:41 |
| kfox1111 | that is an imparative operation. | 19:41 |
| kfox1111 | in a declarative model, you request "what", not "how" | 19:41 |
| kfox1111 | "I want a memcached" | 19:41 |
| rhallisey | operators are an declarative API driven by imperative user input | 19:41 |
| kfox1111 | not, "I want you to deploy a memcached" | 19:41 |
| kfox1111 | no. declarative api driven by a declarative user input. | 19:42 |
| kfox1111 | ralized by a bit of code that usually is imparative. | 19:42 |
| kfox1111 | (unless you implement your operator in make or lisp :) | 19:42 |
| kfox1111 | in no other k8s object type do you set state=deploy. | 19:43 |
| kfox1111 | you just say you want a rc, or a svc. | 19:43 |
| sbezverk | kfox1111: could you exaplain me why it is that important whether it is imperative or declerative??? I might be wrong but what metter is to complete sucessfully action | 19:43 |
| kfox1111 | the operator itself figures out it needs to deploy a new one. | 19:43 |
| *** fguillot has quit IRC | 19:43 | |
| sbezverk | how you call it, should not matter | 19:43 |
| rhallisey | sbezverk, IMO, I think how you call it does matter. I think imperative is crucial here | 19:44 |
| kfox1111 | let me try and find a good document on the subject. | 19:44 |
| kfox1111 | k8s is all about declarative. | 19:44 |
| sbezverk | rhallisey: I will finish memcached and wait to alternatives | 19:44 |
| rhallisey | I know it is, that's why the operator runs as a declarative daemon in the cluster | 19:45 |
| *** salv-orl_ has quit IRC | 19:45 | |
| kfox1111 | the operator is the code that conversts the declarative request into imparative instructions for creating the things. | 19:45 |
| kfox1111 | otherwise, if imparative, why does the operator even need to exist? | 19:46 |
| kfox1111 | the user knows how to drive it themselves then. | 19:46 |
| rhallisey | the operator is declarative. It exists in the cluster moving the cluster to how it should look. But the it activly takes user input | 19:46 |
| rhallisey | the reason it activly takes user input is so that the operator can be altered on the fly handle more or less of the day 2 operations | 19:47 |
| kfox1111 | agree on that point. but the input is also declarative. otherwise, the user doesn't need the operator. | 19:47 |
| kfox1111 | agreed the request needs to be modifyable | 19:48 |
| kfox1111 | we just disagree on the content. | 19:48 |
| kfox1111 | the content should be what you need, not how to make it so. | 19:48 |
| sbezverk | kfox1111: 3rd party operator is declaration, then operator code takes it and react on it by pushing objects to kubernetes api exactly the same way it does RC | 19:48 |
| rhallisey | ok we may be arguing semantics | 19:48 |
| sbezverk | the only difference, we do not monitor etcd branches | 19:48 |
| kfox1111 | sbezverk: I agree with that. | 19:48 |
| sbezverk | but we already have specs in form of chart temapltes | 19:49 |
| sbezverk | I do not see where we have disconnect :( | 19:49 |
| kfox1111 | the user should say "I want a memcached on port 80" not, "I want a memcached on port 80, state=deploy" | 19:49 |
| rhallisey | the only thing I'm still confused on is state=deploy | 19:49 |
| kfox1111 | state=deploy is implied by it being a new resource. | 19:49 |
| sbezverk | kfox1111: that is possible | 19:49 |
| kfox1111 | its an imparative action of how, not what. | 19:50 |
| sbezverk | operator can monitor ANY change in the 3rd party object and react accordingly | 19:50 |
| kfox1111 | yes. thaat. :) | 19:50 |
| kfox1111 | that makes it declarative. :) | 19:50 |
| sbezverk | kfox1111: deploy I consider as one of possible state changes | 19:51 |
| rhallisey | we agree there, but on the thirdpartyresource telling the operator to deploy | 19:51 |
| rhallisey | I think we differ | 19:51 |
| *** jtriley has quit IRC | 19:51 | |
| kfox1111 | the 3rdparty resource doesn't tell the operator to do an action. | 19:52 |
| sbezverk | rhallisey: ok how we can deploy then with third party? just by creating it? | 19:52 |
| kfox1111 | the operator decides based on 3rd party resource info to take an action. | 19:52 |
| kfox1111 | . user createsa a 3rd party resource, stating they want a memcached, there is nothing on the k8s system, therefore, its a deploy operation of memcached. | 19:53 |
| sbezverk | kfox1111: I agree with kfox1111 on this | 19:53 |
| rhallisey | ok how do they upgrade it then | 19:53 |
| kfox1111 | user deletes 3rdparyt resource. this implies the user doesn't want the resource anymore. operator deletes the stuff it launched previously. | 19:54 |
| kfox1111 | changea version var in the request? | 19:54 |
| kfox1111 | version=2.0.2 initially. user uploads newer 3rdparty resource with version=3.0.2. | 19:55 |
| kfox1111 | operator helm upgrades corresponding packages | 19:55 |
| openstackgerrit | sayantani proposed openstack/kolla: Improve Quick Start documentation https://review.openstack.org/404993 | 19:55 |
| sbezverk | kfox1111: no need to upload new you can just change existing | 19:55 |
| kfox1111 | that woudl work too. | 19:55 |
| sdake_ | mgiles - operators are where rubber meets the road - you might be interested in this conversation | 19:55 |
| kfox1111 | its at the operator's descression on what the best logic to realize the request. :) | 19:55 |
| sdake_ | srwilkers portdirect ^^ | 19:55 |
| rhallisey | how would I tell the operator to upgrade only nova | 19:56 |
| kfox1111 | thats the real benifit of operators. | 19:56 |
| Pavo | so...... just a fyi someone might wanna add that root directory has to have most of the storage on compute nodes or they will run into all kinds of trouble when spinning up 100+ instances at onxe | 19:56 |
| Pavo | once | 19:56 |
| Pavo | that needs to added to kolla-ansible docs | 19:56 |
| sbezverk | rhallisey: you would need to communicate that via nova's operator | 19:56 |
| rhallisey | ok but in that case, what if I wanted to do my db by hand | 19:56 |
| sbezverk | by updating nova operator 3rd party object | 19:56 |
| sbezverk | rhallisey: then you would use microservices charts ;) | 19:57 |
| sbezverk | operator imho is higher level orchestration | 19:57 |
| kfox1111 | rhallisey: could add aditional info to the request. or accept that if doing automatic orchestartion looses you some flexability. | 19:57 |
| rhallisey | what I'm trying to get at, is there any scenario in which an human operator will be able to do what they want without something strange happening under the hood | 19:57 |
| mgiles | sdake_ yes, I am - I'll need to read the scrollback as I've not been keeping up so far | 19:58 |
| kfox1111 | rhallisey: I think thats the tradoff I've been wanting as an op. | 19:58 |
| kfox1111 | rhallisey: I want microservices so I get total control over everything. | 19:58 |
| sbezverk | kfox1111: agree | 19:58 |
| kfox1111 | but some ops folks want automation without as much flexability. | 19:58 |
| sbezverk | you want micro manage, you use microservices | 19:58 |
| kfox1111 | thats the tradeoff between helm+kubernetes-entrypoint/operators/microservices. | 19:59 |
| sbezverk | you want orchestrate you use services or operators | 19:59 |
| * kfox1111 nods | 19:59 | |
| *** jtriley has joined #openstack-kolla | 19:59 | |
| rhallisey | I'm confused about the microservice part though because isn't that handled by helm? | 19:59 |
| *** msimonin has joined #openstack-kolla | 19:59 | |
| rhallisey | so in the db case were I would use the microservice part, that would require a workflow | 20:00 |
| kfox1111 | yeah. but something has to drive helm. | 20:00 |
| sbezverk | rhallisey: yes but you can still go at very detailed step of deployment | 20:00 |
| kfox1111 | in microservices case, its the ops person themself. | 20:00 |
| srwilkers | sdake_, back from lunch. sup? | 20:00 |
| rhallisey | gotcha | 20:00 |
| kfox1111 | so, on my cloud, I'll just be helm installing a bunch of microservices. | 20:00 |
| kfox1111 | which means I have total control of the upgrade cycle. | 20:00 |
| sdake_ | srwilkers just a discussion on operators happening - thought you would want to be involved | 20:01 |
| kfox1111 | but lack all the rich creamy goodness of automatic operator based upgrades. | 20:01 |
| sdake_ | i guess portdirect is asleep as he should be :) - he can always read scrollback | 20:01 |
| kfox1111 | same goes for helm+kubernetes-entrypoint. | 20:01 |
| kfox1111 | helm+kubernetes-entrypoint and operators are two different apporaches to automatic orchestartion. | 20:02 |
| sbezverk | kfox1111: can I ask you to add some ideas you mentioned to the review for tracking purposes and I will try to make it happen or not ;-) | 20:02 |
| rhallisey | kfox1111, to fully circle back, the reason I mentioned imperative for thirdpartyresource is so they can be interacted with on the fly. But, you're saying that would be covered with manual intervention at the microservice layer | 20:02 |
| kfox1111 | sbezverk: ok. will do. :) | 20:03 |
| kfox1111 | rhallisey: yeah. I think thats what I'm saying. | 20:03 |
| sbezverk | rhallisey: I think in this case memcached operator acts exactly how human operator would do with helm microservices | 20:03 |
| srwilkers | reading scrollback | 20:04 |
| jemcevoy | Neutron question: I have one 10Gig bonded uplink that I need to share with Ceph and the External Internet on different VLANs. How do I go about setting this up? Can Ceph access the Neutron networks? | 20:04 |
| sdake_ | sbezverk agree | 20:04 |
| openstackgerrit | sayantani proposed openstack/kolla: Improve Quick Start documentation https://review.openstack.org/404993 | 20:04 |
| sdake_ | jemcevoy ceph may be able to access thosse networks - but I am uncertian unfortunately - not a netowrking wiz | 20:04 |
| kfox1111 | sbezverk: agreed. "here, upgrade this" is the kind of requests I usually get. :) | 20:04 |
| sdake_ | jemcevoy you might try asking on ask.openstack.org | 20:04 |
| sdake_ | jemcevoy that way we can centralize all Q&A into one place | 20:05 |
| jemcevoy | OK | 20:05 |
| sdake_ | jemcevoy many of our cores are networking experts and can answer q's there | 20:05 |
| sdake_ | however more then 60% of the team is EMEA and APAC | 20:05 |
| kfox1111 | sbezverk: srwilkers: sdake_: https://review.openstack.org/#/c/418587/ shoudl be ready. | 20:06 |
| sbezverk | rhallisey: even from phsyhological point of view, operator is idle most of the time, boss tells him to do things, he does it and go back to idle state or monitor state | 20:06 |
| sdake_ | damn forgot to start the builds | 20:06 |
| kfox1111 | sbezverk: yeah | 20:06 |
| sdake_ | and have a meeting soon - will try to do the centos builds in a few hours kfox1111 | 20:06 |
| sdake_ | inc0 can you get the 3.0.2 ubuntu builds done today? | 20:06 |
| kfox1111 | sdake_: please start the builds. :) | 20:06 |
| sdake_ | inc0 if not i'll suck it up and install ubuntu on a vm and build it - but can't test it unfortunately | 20:07 |
| inc0 | yeah sorry lots on my mind, starting building now | 20:07 |
| Mech422 | jemcevoy: bridges | 20:07 |
| rhallisey | kfox1111, sbezverk , I think you defined in well re your memcached setence | 20:07 |
| sdake_ | inc0 we are in the same spot ;) | 20:07 |
| jemcevoy | thanks sdake_ | 20:07 |
| *** sdake_ is now known as sdake | 20:07 | |
| Mech422 | jemcevoy: I can share my configs if you use openvswitch | 20:07 |
| jemcevoy | I have vlans, vxlan and ovs | 20:08 |
| openstackgerrit | sayantani proposed openstack/kolla: Update Bifrost documentation https://review.openstack.org/417114 | 20:08 |
| Mech422 | jemcevoy: same here | 20:08 |
| Mech422 | jemcevoy: you basically just create ovs 'internal' ports to use as 'normal' adapters | 20:08 |
| *** l4yerffeJ has quit IRC | 20:08 | |
| *** l4yerffeJ has joined #openstack-kolla | 20:09 | |
| sdake | Jeffrey4l is hiding out :) | 20:09 |
| sbezverk | kfox1111: rhallisey: I have a meeting need to run, but huge thank you for this discussion. Appreciate if you add your thought to the review so they are duly noted.. | 20:09 |
| rhallisey | kfox1111, ah ok | 20:10 |
| rhallisey | kfox1111, another thought | 20:10 |
| kfox1111 | sbezverk: thanks fror talking through it too. :) | 20:10 |
| rhallisey | sbezverk, absolutely! | 20:10 |
| kfox1111 | and rhallisey too. :) | 20:10 |
| jemcevoy | I have a 1Gig bond that I will use to reach the bare metal and a bonded 10Gig uplink that I can play with the VLANs.... I want most of the I/O on the 10Gig side | 20:10 |
| rhallisey | kfox1111, I'm going to go back to the nova upgrade | 20:10 |
| rhallisey | kfox1111, and you wanted to upgrade the db by hand | 20:10 |
| rhallisey | since your process differs from the operator, you would have to do the entire things by hand | 20:11 |
| kfox1111 | rhallisey: k. | 20:11 |
| kfox1111 | yes/no. | 20:11 |
| rhallisey | say you did the db first | 20:11 |
| kfox1111 | I think what's likely to happen, is there will be a bunch of helm microservice jobs. | 20:11 |
| rhallisey | then tirggered a nove upgrade, it would alos upgrade the db right? | 20:11 |
| kfox1111 | for example, nova-manage-db-job. :) | 20:11 |
| kfox1111 | that does the nova db upgrade. | 20:11 |
| kfox1111 | the operator will call that at the right time. | 20:12 |
| kfox1111 | nothings stopping a human from running that themselves though. | 20:12 |
| rhallisey | ok that's new to me | 20:12 |
| kfox1111 | so the code can be shared between the automatic code and the human run code. | 20:12 |
| rhallisey | how would the helm mircoservice job work? Is that a kubernetes job? | 20:12 |
| kfox1111 | yeah. k8s job. | 20:12 |
| kfox1111 | oddly, in openstack, | 20:12 |
| jemcevoy | Cool I will thy it Mech422... I am new to OVS what is the command to create an internal port I can use to access the 10Gig uplink and tag that port for a VLAN | 20:12 |
| kfox1111 | the creation of the initial schema, and the upgrading of a db is done by the same exact code. | 20:13 |
| Mech422 | jemcevoy: http://paste.openstack.org/show/594621/ | 20:13 |
| kfox1111 | so we had to build that building block anyway just to boot the system. | 20:13 |
| rhallisey | kfox1111, just a thought (thought it is imperative), what if you told the operator to do the mariadb upgade | 20:13 |
| rhallisey | only | 20:13 |
| kfox1111 | but we made it a seperate package, so it could be rerun at any time. :) | 20:13 |
| rhallisey | or skip it entirely | 20:14 |
| rhallisey | s/thought/though | 20:14 |
| kfox1111 | a declarative way to do that might be, to set the desired db state as a var. | 20:14 |
| kfox1111 | if desired state == real state, don't run the db upgrade job. | 20:14 |
| rhallisey | let's say this | 20:15 |
| Mech422 | jemcevoy: oh! my configs have 'post-up's to do policy based routing, so packets only go out the interface they came in on... | 20:15 |
| kfox1111 | user in the 3rd party resource then just puts desired db ver. | 20:15 |
| Mech422 | jemcevoy: you may or may not want that :-P | 20:15 |
| rhallisey | state = upgrade nova db_state=skip | 20:15 |
| jemcevoy | Thanks Mech422.... It think that gets me working.... Just need to translate to CentOS | 20:15 |
| kfox1111 | thats a how again though. | 20:15 |
| kfox1111 | it should be a what | 20:15 |
| rhallisey | translate to a what | 20:16 |
| rhallisey | for me | 20:16 |
| kfox1111 | "version=pike" or something like that. | 20:16 |
| kfox1111 | not state=upgrade | 20:16 |
| kfox1111 | state=upgrade is implied by current deployed version being ocata, and | 20:16 |
| rhallisey | what about the db though? | 20:16 |
| kfox1111 | requested version being pike. | 20:16 |
| rhallisey | how can I signal to skip the db upgrade | 20:16 |
| rhallisey | in what terms | 20:17 |
| kfox1111 | that one's kind of fuzzy to me, because the goal is for the desired state to be reachable by the operator. | 20:17 |
| kfox1111 | but maybe a general flag saying "never_upgrade_db" | 20:17 |
| kfox1111 | in the 3rd party resource. | 20:17 |
| rhallisey | but does that become a how | 20:18 |
| rhallisey | and no longer a what | 20:18 |
| kfox1111 | kind of? which is fuzzy to me. its saying what not to do, not what to do. | 20:18 |
| kfox1111 | again though, this is kind of already breaking the rules of walking around the operator. | 20:19 |
| rhallisey | how do you mean? You walk around because you want to customize?? | 20:20 |
| *** Pavo has quit IRC | 20:20 | |
| kfox1111 | an operator is basically an expert system. it implements a computerized human to perform the actions for you. | 20:22 |
| rhallisey | but what if the operator give you 95% of what you want and you need a custom 5% | 20:22 |
| *** satyar has quit IRC | 20:22 | |
| kfox1111 | so putting in a request to the human of "I want a pike nova but I want to do db upgrades myself" may make sense? | 20:22 |
| kfox1111 | at some point thoyugh, it may become unwieldy for there to be logic that covers every 5% case you want. | 20:23 |
| rhallisey | I agree, so in that case would I still get to use the 95% | 20:24 |
| Mech422 | jemcevoy: oh hey - this is about the best 'howto' I've seen for networking - its libery based, but gives you the idea: http://www.opencloudblog.com/?p=614 | 20:25 |
| kfox1111 | I think its probably pretty easy to get the operator to skip a step like, "don't upgrade the db" | 20:25 |
| kfox1111 | as that ends up being in reality, don't launch nova-manage-db | 20:26 |
| kfox1111 | not sure other steps will be so forgiving. | 20:26 |
| rhallisey | ya but that's not a nove upgrade if I don't nova-manage-db | 20:26 |
| kfox1111 | I could be wrong though. | 20:26 |
| kfox1111 | well, it would still upgrade everything else. | 20:26 |
| kfox1111 | all the containers/services/etc. | 20:27 |
| kfox1111 | if config files are managed, configfiles... etc. | 20:27 |
| rhallisey | ya I agree, but it may not give me what I want which would lead me to not using the operator for the upgrade at all | 20:28 |
| kfox1111 | which parts of the upgrade are you concerned about? | 20:30 |
| kfox1111 | (I share your concerns btw) | 20:30 |
| kfox1111 | (I haven't yet seen a foolproof orchestration system for something as complicated as openstack. :/ ) | 20:30 |
| rhallisey | agreed :) | 20:30 |
| kfox1111 | which is why I plan on using microservcies for a while. | 20:31 |
| rhallisey | nice | 20:31 |
| kfox1111 | once the orchestration stuff matures and proves itself, then I'll [D[D[D[D[D[D[D[D[D[D[Dconsider it. | 20:31 |
| rhallisey | hehe | 20:31 |
| kfox1111 | I figure that will be at least 6 - 12 months. | 20:31 |
| kfox1111 | ugrades are hard. :) | 20:31 |
| rhallisey | the part about the upgrade that conerns me is people wanted to do anything outside what we describe as the norm | 20:31 |
| kfox1111 | thats the problem with expert systems though. | 20:32 |
| rhallisey | for declarative to work, you need to have a good description of what a user wants | 20:32 |
| kfox1111 | if you don't code the logic into them, then they can't adapt to the new stuff very well. | 20:32 |
| rhallisey | if you descriptions doesn't cover their use case they have to improvise | 20:32 |
| kfox1111 | declarative is more flexable for that case though. as it can try and fix things that weren't invisioned initially. | 20:32 |
| rhallisey | let me rephrase | 20:32 |
| rhallisey | the operator is declarative | 20:33 |
| rhallisey | 100% agree | 20:33 |
| rhallisey | but the input is where I disagree | 20:33 |
| rhallisey | I want to leave room to improvise | 20:33 |
| kfox1111 | k, so what do you suggest? | 20:33 |
| kfox1111 | and where do youy think you will need to? | 20:34 |
| kfox1111 | (improvise?) | 20:34 |
| rhallisey | that tasks like skip db be added to the declarative api (operator) and the user choosing to use them | 20:34 |
| jemcevoy | Thanks again Mech422, That blog clears up a lot of mystery... | 20:34 |
| *** sdake has quit IRC | 20:35 | |
| kfox1111 | the user can just launch the helm microservice package jobs to do the specific bits? | 20:36 |
| harlowja | if u guys get some time to look at http://lists.openstack.org/pipermail/openstack-dev/2017-January/109917.html that'd be super, klindgren and i would like any feedback/comments/other | 20:36 |
| kfox1111 | or are you worried about managed objects such as deployments? | 20:36 |
| Mech422 | jemcevoy: yeah - it helped me tons - especially since its written for ovs bonded networking from the start | 20:36 |
| rhallisey | the user could use helm, but some of these cases would require workflow | 20:36 |
| kfox1111 | or.. maybe there's a 3rd aproach.... | 20:37 |
| Mech422 | kfox1111: maybe looking at terraform could give you some ideas? it sounds sorta similiar | 20:37 |
| *** sdake has joined #openstack-kolla | 20:37 | |
| kfox1111 | if the consern is the potential of doing partial upgrades... | 20:37 |
| rhallisey | kfox1111, my concern is based on the princple that users do what they want and now what we want | 20:37 |
| kfox1111 | maybe we extend the desired state to be optionally more explicit? | 20:37 |
| rhallisey | the partial upgrade would be an example | 20:38 |
| *** krtaylor has joined #openstack-kolla | 20:38 | |
| kfox1111 | version: allpike nova-api:ocata | 20:38 |
| kfox1111 | version: all: pike nova-api:ocata | 20:38 |
| kfox1111 | then the user could override specific versions as needed? | 20:38 |
| kfox1111 | thats the problem with automatic orchestration though. | 20:38 |
| rhallisey | interesting | 20:38 |
| kfox1111 | it is always about what the coder of the orchestration engine believes the user wants to do. :/ | 20:39 |
| kfox1111 | at somepoint it can't be infinately flexible. :/ | 20:39 |
| rhallisey | agreed | 20:39 |
| kfox1111 | otherwise there would be no reason to have human operators. ever. :) | 20:39 |
| harlowja | we need AI | 20:39 |
| harlowja | lol | 20:39 |
| harlowja | AI will take our jerbs | 20:39 |
| kfox1111 | indeed. :) | 20:40 |
| *** gfidente has joined #openstack-kolla | 20:40 | |
| rhallisey | kfox1111, what about this: version: all: pike nova: <import-custom-workflow> | 20:40 |
| kfox1111 | what does custom workflow looklike? | 20:40 |
| kfox1111 | python? go? aws lambda? | 20:41 |
| rhallisey | custom workflow would be some code handled my custom case | 20:41 |
| rhallisey | python I suppose | 20:41 |
| *** sdake has quit IRC | 20:41 | |
| rhallisey | s/handled/handling | 20:41 |
| kfox1111 | I'd much rather say, don't do X, I'll do X. | 20:41 |
| rhallisey | that's fair | 20:42 |
| *** adrian_otto has joined #openstack-kolla | 20:42 | |
| rhallisey | that's probably better | 20:42 |
| kfox1111 | then try and figure out how to get the operator to do it. | 20:42 |
| rhallisey | less things will go wrong | 20:42 |
| kfox1111 | yeah. | 20:42 |
| kfox1111 | that even gives me the chance to manually poke at the db and fix things when they totally go wrong. | 20:43 |
| kfox1111 | (not that I've ever had to do that...... ;) | 20:43 |
| rhallisey | that's good: don't do X , I'll do it myself | 20:43 |
| kfox1111 | so, request would be something like "version=pike, never_upgrdade_db" | 20:43 |
| kfox1111 | then I'm free to upgrade the db myself. it won't try. | 20:44 |
| kfox1111 | it could even be smart enough then, | 20:44 |
| kfox1111 | to do half the upgrade, | 20:44 |
| kfox1111 | want until the db gets upgraded, then finish the upgrade. | 20:44 |
| rhallisey | that's possible | 20:44 |
| kfox1111 | and the human op can do their piece of the task. | 20:44 |
| rhallisey | kfox1111, good stuff | 20:46 |
| rhallisey | I think that's flexible enough | 20:47 |
| kfox1111 | cool. | 20:47 |
| rhallisey | the only thing I like about the <bring your own code> is the user owns it versus the operator | 20:48 |
| kfox1111 | user would still bring their own code. it just wouldn't run int he operator container. | 20:48 |
| kfox1111 | which would be safer. | 20:48 |
| rhallisey | in that case, there would have to be a list of tasks that can be skipped | 20:49 |
| kfox1111 | yeah. | 20:49 |
| *** sdake has joined #openstack-kolla | 20:49 | |
| rhallisey | which may extend to every task | 20:50 |
| kfox1111 | yeah. could be. | 20:50 |
| kfox1111 | I'm still thinking though, you want so much control over things, you will be hapier using microservices directly. | 20:51 |
| rhallisey | I agree | 20:51 |
| kfox1111 | (I totally get that, cause I'm in the same boat. :) | 20:51 |
| rhallisey | I'm just defending the 95% - 5% case | 20:51 |
| rhallisey | cause you never know | 20:51 |
| kfox1111 | sure. | 20:51 |
| kfox1111 | it would be nice if we couold pin down the 5% to not be 100% coverage though. | 20:52 |
| kfox1111 | I think its likely a user would want to skip the db upgrade, | 20:52 |
| kfox1111 | as I've seen that code actually fail to work. | 20:52 |
| kfox1111 | as for the rest, not sure. | 20:52 |
| kfox1111 | find the 20% that the 5% user base will ever want to disable. | 20:53 |
| kfox1111 | this is one of those maturaty things thouhg. you won't know what those are, until you have a bunch of users doing it. | 20:54 |
| rhallisey | ya | 20:54 |
| kfox1111 | so, 6-12 months out. :) | 20:54 |
| rhallisey | the best way to go I'd say is to name tasks | 20:54 |
| kfox1111 | yeah | 20:54 |
| rhallisey | and leave it at that level of granularity | 20:54 |
| rhallisey | that's probably at the 1-5% level | 20:55 |
| rhallisey | thanks kfox1111. Very useful discussion :) | 20:55 |
| kfox1111 | +1 | 20:56 |
| kfox1111 | yeah, thanks for talking through it. :) | 20:56 |
| rhallisey | that will make for a good read | 20:56 |
| rhallisey | forget specs | 20:56 |
| rhallisey | XD | 20:56 |
| kfox1111 | can I get a second for https://review.openstack.org/#/c/418587/ please? | 20:57 |
| kfox1111 | hehe. yeah. | 20:57 |
| *** dmsimard|afk is now known as dmsimard | 21:00 | |
| *** bjolo has quit IRC | 21:00 | |
| kfox1111 | ok. so, once that patch merges, there's one more once the 3.0.2 containers are ready and we're good to go for release. | 21:02 |
| *** gfidente has quit IRC | 21:04 | |
| *** adrian_otto has quit IRC | 21:11 | |
| *** adrian_otto has joined #openstack-kolla | 21:12 | |
| *** salv-orlando has joined #openstack-kolla | 21:25 | |
| *** l4yerffeJ_ has joined #openstack-kolla | 21:34 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 21:34 | |
| *** l4yerffeJ has quit IRC | 21:35 | |
| *** Jeffrey4l has quit IRC | 21:35 | |
| kfox1111 | inc0: sdake: build status? :) | 21:38 |
| *** DanyC has joined #openstack-kolla | 21:42 | |
| * sbezverk back | 21:45 | |
| *** kproskurin has quit IRC | 21:45 | |
| *** kproskurin has joined #openstack-kolla | 21:46 | |
| rhallisey | sbezverk, yo | 21:46 |
| kfox1111 | hi | 21:46 |
| *** kproskurin has quit IRC | 21:46 | |
| *** kproskurin has joined #openstack-kolla | 21:47 | |
| inc0 | kfox1111, pushing ubuntu-binary | 21:47 |
| inc0 | as we speak | 21:47 |
| sbezverk | rhallisey: kfox1111: anything else came out of that discussion? | 21:47 |
| inc0 | it's few hrs tho | 21:47 |
| rhallisey | sbezverk, ya, check the logs | 21:47 |
| *** kproskurin has quit IRC | 21:47 | |
| kfox1111 | inc0: cool. thanks. | 21:47 |
| sbezverk | rhallisey: cool, will read scroll back | 21:47 |
| sdake | kfox1111 - starting now hopefully | 21:47 |
| *** kproskurin has joined #openstack-kolla | 21:48 | |
| *** kproskurin has quit IRC | 21:48 | |
| kfox1111 | sdake: cool. thanks. I'll start changing the ps to point to 3.0.2. | 21:48 |
| *** msimonin has quit IRC | 21:48 | |
| *** DanyC has quit IRC | 21:48 | |
| *** kproskurin has joined #openstack-kolla | 21:48 | |
| *** kproskurin has quit IRC | 21:49 | |
| *** DanyC has joined #openstack-kolla | 21:49 | |
| *** msimonin has joined #openstack-kolla | 21:49 | |
| *** kproskurin has joined #openstack-kolla | 21:49 | |
| *** kproskurin has quit IRC | 21:49 | |
| *** sayantani01 has quit IRC | 21:51 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Set versions to 0.4.0 and 3.0.2 https://review.openstack.org/419082 | 21:51 |
| inc0 | https://hub.docker.com/u/kolla/ | 21:53 |
| inc0 | ubuntu-source-collectd has 100k pulls! | 21:53 |
| inc0 | how I ask!? | 21:53 |
| portdirect | thast not a good sign :( | 21:53 |
| *** dave-mccowan has joined #openstack-kolla | 21:54 | |
| *** krtaylor has quit IRC | 21:54 | |
| portdirect | I did 30K pulls of one of my own images in a weekend once :/ casue it was bugging out and I had pull on every restart... :D | 21:54 |
| sdake | kfox1111 i may not have enoguh disk space to build in my vm - but giving it a go anyway | 21:54 |
| kfox1111 | automation? | 21:54 |
| sdake | kfox1111 if that fails I have 3 boxes in my network i can use | 21:55 |
| kfox1111 | ok. cool. | 21:55 |
| sdake | inc0 the one i look at is the toolbox container | 21:55 |
| sdake | every kolla deploy requires toolbox | 21:55 |
| sdake | kfox1111 i will have centos binary and source built before i sign off tonight | 21:55 |
| inc0 | yeah, but not every one needs collectd | 21:55 |
| sdake | i have a meeting in 5 minutes for 1 hour - then exclusinvely focused on building the images | 21:55 |
| inc0 | and yet it has 10x more pulls than next one | 21:55 |
| *** schwicht has joined #openstack-kolla | 21:55 | |
| kfox1111 | sdake: ok. cool. maybe we will get lucky and the release team will tag tomorrow. | 21:56 |
| sdake | kfox1111 they may | 21:56 |
| sdake | kfox1111 the release team is really cool like infra | 21:56 |
| sdake | kfox1111 i just try to hit wednesday with them so they have time to do their thing | 21:56 |
| kfox1111 | sdake: maybe you could giv ethem a heads up? | 21:56 |
| sdake | kfox1111 are all other 0.4.0 patches merged? | 21:56 |
| kfox1111 | sdake: that one and one other. | 21:56 |
| sdake | kfox1111 i prefer jeffrey (our build liason) do that | 21:56 |
| kfox1111 | waiting for a second on: | 21:56 |
| sdake | rather our release liason :) | 21:57 |
| kfox1111 | https://review.openstack.org/#/c/418587/ | 21:57 |
| kfox1111 | ok. | 21:57 |
| kfox1111 | Jeffrey4l_: can you do it? :) | 21:57 |
| sdake | zomg want to buy tempalting :) | 21:57 |
| kfox1111 | heh. helm native needs some build system work. ;) | 21:58 |
| *** DanyC has left #openstack-kolla | 21:58 | |
| sdake | kfox1111 nice - looks like went with new names for files too | 21:59 |
| sdake | like the renames :) | 21:59 |
| jrich523 | any of you guys work with the REST API at all? | 21:59 |
| sdake | jrich523 i think sbezverk has written operator code which uses it | 21:59 |
| kfox1111 | sdake: just the subdirs for now. | 21:59 |
| sdake | ok have meeting - last one for th eday | 21:59 |
| sdake | then can get focused on the builds - although they are proceeding now | 21:59 |
| kfox1111 | as those have to match the name in the Chart.yaml or helm breaks. | 21:59 |
| kfox1111 | yeah. | 22:00 |
| *** nihilifer has quit IRC | 22:00 | |
| *** nihilifer has joined #openstack-kolla | 22:00 | |
| *** goldyfruit has quit IRC | 22:03 | |
| *** sdake has quit IRC | 22:03 | |
| *** sdake has joined #openstack-kolla | 22:04 | |
| *** rhallisey has quit IRC | 22:05 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Add -type to all microservices missing them. https://review.openstack.org/418587 | 22:07 |
| *** jtriley has quit IRC | 22:09 | |
| sbezverk | kfox1111: so 0.4.0 is out or not? | 22:10 |
| kfox1111 | sbezverk: not yet. one more patch to land, and it depends on 3.0.2 containers on the hub. | 22:10 |
| kfox1111 | which are building now. | 22:10 |
| kfox1111 | the ps is: https://review.openstack.org/#/c/419082/ | 22:10 |
| *** DanyC has joined #openstack-kolla | 22:14 | |
| *** sdake has quit IRC | 22:16 | |
| *** DanyC has left #openstack-kolla | 22:21 | |
| *** srwilkers has quit IRC | 22:21 | |
| *** salv-orl_ has joined #openstack-kolla | 22:22 | |
| *** sayantani01 has joined #openstack-kolla | 22:26 | |
| *** salv-orlando has quit IRC | 22:26 | |
| kfox1111 | portdirect: you around? | 22:27 |
| *** Pavo has joined #openstack-kolla | 22:28 | |
| sbezverk | kfox1111: what is interesting that your last ps passes fine iscsi gate but fails on ceph gate | 22:28 |
| sbezverk | is there is something special with ceph 3.0.2 images? | 22:28 |
| kfox1111 | it passed? | 22:28 |
| * kfox1111 goes and looks... | 22:28 | |
| Pavo | gota a question all | 22:29 |
| Pavo | what is the recommended way to partition CentOS OS on the target nodes? | 22:30 |
| kfox1111 | are you looking at the previous checkexperimental runs? https://review.openstack.org/#/c/419082/ doesn't show any currently. | 22:30 |
| sbezverk | kfox1111: not sure it is the one currenlty showing by gerrit | 22:30 |
| kfox1111 | could be the previous version. the previous version had 3.0.1 still. | 22:31 |
| portdirect | kfox1111: hey | 22:31 |
| kfox1111 | portdirect: did you push a trunk build to the hub? | 22:31 |
| portdirect | i will be | 22:31 |
| kfox1111 | https://hub.docker.com/r/kolla/ubuntu-binary-neutron-server/tags/ shows one 15 hours old. :) | 22:31 |
| kfox1111 | just curious how it got there. :) | 22:31 |
| portdirect | yeah - thats been running for a week or so | 22:32 |
| kfox1111 | cool. :) | 22:32 |
| sbezverk | kfox1111: you are right it is older PS while it was still WIP | 22:32 |
| kfox1111 | sbezverk: ok. cool. you had me worried there for a second. :) | 22:32 |
| kfox1111 | I may be insane, but I didn't think that insane. :) | 22:33 |
| portdirect | kfox1111: I would have used that machine to push 3.0.2 but my vpn fell over and I cant get tot the box its on easily :( | 22:33 |
| Pavo | kfox1111 what would you recommend the way to partition the target node for the OS? | 22:33 |
| portdirect | (working from home) | 22:33 |
| kfox1111 | portdirect: bummer. | 22:33 |
| sbezverk | portdirect: if you have a script I can push images | 22:34 |
| Pavo | because letting CentOS installer just partition it only gives root 50Gb which does not work when you are spinning up 100+ instances at once | 22:34 |
| sbezverk | I have not very fast but stable connection | 22:34 |
| kfox1111 | Pavo: for a host? probably a fair amount of space for a btrfs partition for docker. | 22:34 |
| portdirect | sbezverk: https://github.com/portdirect/kolla-cron-builder | 22:34 |
| Pavo | btrfs? | 22:34 |
| portdirect | sbezverk: will need a tweak | 22:34 |
| kfox1111 | Pavo: depends on if ceph backed or not too. I'm guessing not? | 22:35 |
| Pavo | kfox1111 I am about to reinstall my CentOS OS on a target node, mind walking me through it and I am going to be using Ceph as a backend for Nova, Cinder, Swift | 22:35 |
| kfox1111 | Pavo: centos defaults to a docker backend that is a loopback device. I've seen that eat itself on my systems more then once. | 22:35 |
| Pavo | ok kfox1111 once I get done reinstalling the OS I will hit you up if you don't mind, never done the btrfs thing before | 22:36 |
| kfox1111 | /boot a copule hundered megs. /var/lib/docker btrfs 100g? rest in / ? | 22:36 |
| kfox1111 | don't know for kolla-ansible though. that would be a general guess. | 22:37 |
| *** jheroux has quit IRC | 22:38 | |
| jemcevoy | Pavo and kfox1111 Redhat used lvm thin pools for docker. I have set that stuff up... Or you can use btrfs which is easier. Just make the root partition btrfs and make var/lib/docker a subvolume... | 22:41 |
| Pavo | well what it seemed like the process for it was instance was spinning up in nova-compute container which I am assuming uses /root partition for the instances, we went to spin up 100+ instances and ran out of space, looked at the parititons using df -h and /root was 100% full | 22:41 |
| Pavo | jemcevoy once I get CentOS installed, you mind walking me through thebtrfs stuff? | 22:41 |
| jemcevoy | Pava: Choose btrfs during the install not after... | 22:42 |
| *** sayantani01 has quit IRC | 22:42 | |
| kfox1111 | yeah, if your not backing nova with ceph, it all goes in /var/lib/nova | 22:42 |
| kfox1111 | jemcevoy: or just make /var/lib/docker btrfs. it works too. | 22:42 |
| *** sayantani01 has joined #openstack-kolla | 22:42 | |
| Pavo | ok about to start the reinstall now then, maybe you can walk me through it jemcevoy | 22:42 |
| jemcevoy | Can we do a google hangout? | 22:43 |
| Pavo | ummm sure give me a few minutes though | 22:43 |
| jemcevoy | Sure | 22:43 |
| *** lrensing has quit IRC | 22:44 | |
| Pavo | ok how do I start a hangout? | 22:45 |
| portdirect | yeah if your not backing nova with ceph getting /var/lib/nova on its own partition is pretty key to long term happyness - though can make for fun with selinux | 22:46 |
| Pavo | yeah I have ceph as a backend for nova, cinder and swift | 22:46 |
| Pavo | have selinux in permissive mode because of issues | 22:46 |
| jemcevoy | Pavo: https://hangouts.google.com/hangouts/_/penguincomputing.com/centosdockerstorage when you're ready | 22:50 |
| kfox1111 | fun... http://status.docker.com/ | 23:00 |
| *** awiddersheim has quit IRC | 23:01 | |
| *** krtaylor has joined #openstack-kolla | 23:02 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator https://review.openstack.org/418073 | 23:02 |
| sbezverk | kfox1111: hmm who they pissed off this time ;) ?? | 23:03 |
| kfox1111 | thats a good question. | 23:04 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator https://review.openstack.org/418073 | 23:09 |
| *** dave-mccowan has quit IRC | 23:11 | |
| *** lamt has quit IRC | 23:11 | |
| inc0 | still uploading binary... | 23:13 |
| inc0 | it takes some crazy amounts of time | 23:13 |
| *** awiddersheim has joined #openstack-kolla | 23:13 | |
| kfox1111 | they are experiencing some issues right now too. :/ | 23:19 |
| *** saneax-_-|AFK is now known as saneax | 23:20 | |
| *** sdake has joined #openstack-kolla | 23:21 | |
| portdirect | inc0: oddly it depends on the docker client you are running (in my experence) | 23:25 |
| *** adrian_otto has quit IRC | 23:25 | |
| portdirect | it takes me 45 mins to build&upload a master from the box I cant get to - but hours on my other machine | 23:26 |
| sbezverk | kfox1111: something strange is happening, all jobs are failing on ceph step.. is it because of the images or it is somehitng else? | 23:30 |
| *** sdake_ has joined #openstack-kolla | 23:30 | |
| openstackgerrit | sayantani proposed openstack/kolla-ansible: Modify config.yml playbook for bifrost role https://review.openstack.org/419196 | 23:30 |
| *** sdake has quit IRC | 23:31 | |
| *** lrensing has joined #openstack-kolla | 23:31 | |
| *** mannamne has joined #openstack-kolla | 23:31 | |
| *** adrian_otto has joined #openstack-kolla | 23:32 | |
| *** schwicht_at_work has joined #openstack-kolla | 23:33 | |
| sbezverk | kfox1111: hmmm, disreagard for a moment ;-) | 23:34 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator https://review.openstack.org/418073 | 23:34 |
| *** JoseMello has quit IRC | 23:35 | |
| openstackgerrit | sayantani proposed openstack/kolla: Update Bifrost documentation https://review.openstack.org/417114 | 23:35 |
| *** schwicht has quit IRC | 23:36 | |
| kfox1111 | sbezverk: I'd guess the docker hub being sketchy at the moment. | 23:37 |
| *** sdake has joined #openstack-kolla | 23:37 | |
| *** sdake_ has quit IRC | 23:38 | |
| *** schwicht_at_work has quit IRC | 23:41 | |
| inc0 | dear kolla community, can I have more eyes on this please? https://review.openstack.org/#/c/404993/37 we want to create kolla install guide based on this in main openstack docs repo | 23:42 |
| inc0 | I'd appreciate keeping this change reviewed | 23:42 |
| kfox1111 | inc0: is that in the wrong repo? | 23:43 |
| inc0 | quickstart is in kolla repo still | 23:43 |
| inc0 | we should sort it out later, for it to point to kolla-ansible | 23:43 |
| inc0 | but as long as ansible is only prod deployer for kolla, I'd keep it in main kolla repo to avoid confusion for newcommers | 23:44 |
| openstackgerrit | sayantani proposed openstack/kolla: Improve Quick Start documentation https://review.openstack.org/404993 | 23:51 |
| *** salv-orl_ has quit IRC | 23:52 | |
| *** sdake_ has joined #openstack-kolla | 23:55 | |
| *** sdake has quit IRC | 23:56 | |
| sdake_ | kfox1111 build going now | 23:58 |
| kfox1111 | sdake_: cool. thanks. | 23:58 |
| sdake_ | btw kolla 3.0.2 seems DOA to me - oslo-config is busted | 23:58 |
| kfox1111 | ugg. | 23:59 |
| sdake_ | its ok - i cn still build - and it may not affect the images | 23:59 |
| sdake_ | kolla-build after pip install kolla doesn't work | 23:59 |
| sdake_ | pip uses oslo.config | 23:59 |
| sdake_ | oslo.config uses oslo.i18n | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!