| *** schwicht has joined #openstack-kolla | 00:01 | |
| jascott1 | there is actually good beer in SA if you have a sherpa to help you locate it... | 00:01 |
|---|---|---|
| *** absubram has quit IRC | 00:02 | |
| srwilkers | yes, agreed | 00:03 |
| *** cnf has quit IRC | 00:03 | |
| srwilkers | texans idea of good beer vastly differs from mine | 00:03 |
| *** krtaylor has joined #openstack-kolla | 00:04 | |
| srwilkers | saint louis has a bunch of micro breweries that are doing really well. one of my favorites here has a russian imperial stout blended with a honey mead almost. perfect for this time of year to keep you warm ;p | 00:05 |
| jascott1 | nice | 00:05 |
| *** Pavo has quit IRC | 00:07 | |
| *** Pavo has joined #openstack-kolla | 00:08 | |
| jascott1 | srwilkers: do you have any teammates in OR? | 00:08 |
| *** chas has quit IRC | 00:09 | |
| *** chas has joined #openstack-kolla | 00:09 | |
| srwilkers | jascott1: not that im aware of, but i volunteer as tribute if our company ever wants to open a development shop up that way | 00:10 |
| jascott1 | :) | 00:10 |
| SamYaple | sean-k-mooney: have you seen dasiy+kolla for using opnfv with kolla? | 00:10 |
| SamYaple | sean-k-mooney: i just stumbled on it while doing my opnfv research | 00:11 |
| jascott1 | love it here. Portland has more than enough big city for me without the traffic and BS of Seattle | 00:11 |
| inc0 | SamYaple, Sean lives in Ireland | 00:11 |
| inc0 | so I'd bet he's deep into drinking by now | 00:11 |
| SamYaple | he has irc logs. i dont expect an immedite response | 00:11 |
| srwilkers | yeah, i want to visit portland. ive always wanted to end up in that corner of the world if im stuck here in the states | 00:12 |
| srwilkers | saint louis is very lackluster. expecting almost 80 degrees F tomorrow, then expecting possible snow this weekend | 00:13 |
| srwilkers | if you dont like the weather here, just wait a day | 00:13 |
| jascott1 | hehe | 00:13 |
| jascott1 | been to a couple concerts there back in "the day" | 00:13 |
| *** chas has quit IRC | 00:14 | |
| *** hfu has joined #openstack-kolla | 00:14 | |
| *** msimonin1 has quit IRC | 00:14 | |
| jascott1 | it was alright.. preferred it to Cinci OH for sure | 00:14 |
| jascott1 | my friend met Sebastian Bach while we were eating.. | 00:15 |
| inc0 | hmm..isn't he dead for like..300 years? | 00:15 |
| jascott1 | haha | 00:15 |
| jascott1 | pretty sure his career has been dead for 15+ years | 00:16 |
| jascott1 | oh wait he was on Trailer Park Boys a few years ago | 00:16 |
| inc0 | I dunno, Mass in B Minor is great | 00:16 |
| jascott1 | lol | 00:16 |
| jascott1 | ( as performed by Skid Row) | 00:17 |
| jascott1 | (now) | 00:17 |
| srwilkers | haha | 00:17 |
| jascott1 | I had a stint in JCC as a music major when was a wee lad and got to participate in an award winning chorus at a competition. got to say, im not a religious person but those mass songs are powerful | 00:18 |
| jascott1 | especially in a room designed for them with a couple hundred people singing | 00:18 |
| *** zhurong has joined #openstack-kolla | 00:19 | |
| inc0 | https://www.youtube.com/watch?v=tdLCcQixNvg | 00:19 |
| jascott1 | ha i got ATT commercial | 00:19 |
| *** dave-mccowan has joined #openstack-kolla | 00:19 | |
| jascott1 | bah proxy is killing it for e | 00:20 |
| jascott1 | me | 00:20 |
| srwilkers | ;) | 00:21 |
| *** tonanhngo has joined #openstack-kolla | 00:25 | |
| *** diogogmt has joined #openstack-kolla | 00:27 | |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-ansible: Modify the letter case with 'True' in ansible yml file. https://review.openstack.org/398133 | 00:27 |
| *** tonanhngo has quit IRC | 00:29 | |
| *** yingjun has joined #openstack-kolla | 00:33 | |
| *** yingjun has quit IRC | 00:33 | |
| *** yingjun has joined #openstack-kolla | 00:33 | |
| *** yingjun has quit IRC | 00:33 | |
| *** yingjun has joined #openstack-kolla | 00:34 | |
| jascott1 | inc0 did you see the Google Earth for Vive? | 00:34 |
| jascott1 | https://vr.google.com/earth/ | 00:34 |
| inc0 | omfg | 00:35 |
| jascott1 | have fun :) | 00:35 |
| *** david-lyle_ is now known as david-lyle | 00:35 | |
| inc0 | jascott1, readiness probes looks good | 00:38 |
| inc0 | should be easy enough:) | 00:39 |
| inc0 | can you guys include it in helm from day 1? | 00:39 |
| wirehead_ | I really wanted to hack on readiness probes really early in the getting-it-working-in-the-first-place stage. | 00:40 |
| *** zhurong has quit IRC | 00:40 | |
| sdake | where do readiness probes go - in containers? | 00:41 |
| wirehead_ | What I should have done was accepted that a readieness probe that's really lame, perhapse even degenerate, will encourage a future contributor to improve it. | 00:41 |
| wirehead_ | It's part of the pod definition. | 00:41 |
| sdake | wirehead_ right that would have been the correct approach | 00:41 |
| inc0 | we need to have them in container somehow, but can be done with volumes | 00:41 |
| srwilkers | +1 wirehead_ | 00:41 |
| inc0 | that being said, I think it'll be useful even if backed into container | 00:41 |
| inc0 | ansible could use it too | 00:42 |
| wirehead_ | I was going to do it for MariaDB, but I got bogged down in making sure the mysql client had the right creds. | 00:42 |
| wirehead_ | Yeah, the longer view suggests that kolla-ansible would also benefit from that infra. | 00:43 |
| *** tonanhngo has joined #openstack-kolla | 00:45 | |
| kfox1111 | inc0: our deployments use readyness probes | 00:45 |
| jascott1 | so far stackenetes mostly used httpGet / and openstack-helm has used socket | 00:45 |
| kfox1111 | see kolla-kubernetes/services/common/common-deployment.yml.j2 | 00:46 |
| jascott1 | ha theres a handful in kk8s | 00:46 |
| jascott1 | ha/ya/s | 00:46 |
| kfox1111 | yup. | 00:46 |
| inc0 | cool | 00:46 |
| inc0 | let's fill up the gaps | 00:46 |
| kfox1111 | I think a fair amount of the remainders are things that should be converted to use more common code but just havent yet. | 00:47 |
| kfox1111 | well... and the apache stuff... | 00:47 |
| kfox1111 | that needs a preStop hook too... | 00:47 |
| kfox1111 | just little things mostly. | 00:47 |
| jascott1 | only ones that ppl seem to use exec for are rabbit and db's | 00:47 |
| kfox1111 | benefits/drawbacks to both apraoaches. | 00:48 |
| kfox1111 | httpGet works over the network, but only works with basic http | 00:48 |
| *** eaguilar has quit IRC | 00:48 | |
| kfox1111 | exec runs a script in the container, so insanely flexible. | 00:48 |
| inc0 | "#programming is being sad for money" | 00:48 |
| kfox1111 | but doen't test the network unless you code it to. | 00:48 |
| kfox1111 | I usually use exec for most of my other projects. | 00:48 |
| *** cnf has joined #openstack-kolla | 00:49 | |
| *** tonanhngo has quit IRC | 00:49 | |
| jascott1 | do we need a minimum standard here or is it whatever works? | 00:49 |
| kfox1111 | inc0: do you want to workflow the spec? | 00:50 |
| kfox1111 | jascott1: good question. | 00:50 |
| jascott1 | socket check just tells you 'something' is listening on that port so seems to lite | 00:50 |
| kfox1111 | right. | 00:50 |
| jascott1 | http could prob get us half way there | 00:50 |
| kfox1111 | and only works for http. | 00:50 |
| kfox1111 | but most openstack stuff is http | 00:50 |
| kfox1111 | (or rabbitmq based) | 00:50 |
| kfox1111 | but most of our stuff is already in deployments which have the check already. | 00:51 |
| kfox1111 | the only ones I think that don't are keystone and horizon, which both need a bit of love in other ways too. | 00:51 |
| kfox1111 | both should be converted to deployments, | 00:51 |
| kfox1111 | but are a little different from the other deployments as they are using apache instad of raw python. | 00:51 |
| kfox1111 | so they don't need the haproxy container and the prestop hook should be to tell apache to shutdown safelyl. | 00:52 |
| kfox1111 | and we need readyness hooks for rabbit/mariadb too. | 00:52 |
| kfox1111 | I don't think we need readyness hooks on the rpc services... | 00:53 |
| inc0 | nah | 00:53 |
| *** adrian_otto has quit IRC | 00:53 | |
| jascott1 | hmm | 00:53 |
| kfox1111 | requests shoudl just queue up in rabbit till they start. | 00:53 |
| inc0 | they will just start working | 00:53 |
| jascott1 | we talked earlier about chart per daemon | 00:55 |
| kfox1111 | inc0: do you want to workflow the spec or should I? and when? | 00:55 |
| jascott1 | meaning we have deployment.yaml per daemon? | 00:55 |
| kfox1111 | I'm thinking chart per microservice. | 00:55 |
| kfox1111 | and then if folks want to orchestrate with a service package, | 00:55 |
| kfox1111 | they can just add the microservice packages as deps. | 00:55 |
| inc0 | kfox1111, do the honors | 00:55 |
| jascott1 | microservice=api for openstack service? | 00:56 |
| kfox1111 | inc0: k. I'll do at COB then. ~30 min. | 00:56 |
| jascott1 | I know what mircoservices are | 00:56 |
| jascott1 | but its kinda overloaded | 00:56 |
| kfox1111 | jascott1: mostly one to one with k8s objects I think. | 00:56 |
| kfox1111 | so its really easy to launch exactly the pices yhou want, if you want to go that route. | 00:56 |
| jascott1 | right | 00:56 |
| kfox1111 | I have a prototype that can do the same thing, where all the service objects are packaged together, | 00:57 |
| kfox1111 | but you luanch only one microservice at a time out of it. | 00:57 |
| jascott1 | so we have N different images for neutron pieces | 00:57 |
| kfox1111 | which is kind of clever/works, but not sure its a good idea. | 00:57 |
| jascott1 | is Chart N or less than N | 00:57 |
| jascott1 | number of charts | 00:57 |
| kfox1111 | very different usage then what helm sems to have been intended for. | 00:57 |
| kfox1111 | a k8s object can be made of multiple containers. | 00:58 |
| kfox1111 | so no 1-1 mapping. | 00:58 |
| inc0 | kfox1111, but we don't want pod-per-microservice right? | 00:58 |
| jascott1 | thats what Im getting at | 00:58 |
| kfox1111 | object per microservice. | 00:58 |
| inc0 | we can bundle couple containers in single pod | 00:58 |
| kfox1111 | one microservice might launch multiple pods. | 00:58 |
| jascott1 | dynamic pods contents? | 00:58 |
| kfox1111 | for example | 00:58 |
| kfox1111 | kolla-kuberntes has a daemonset object for neutron-l3-agent. | 00:59 |
| kfox1111 | that one object can spawn 300 identicle pods, | 00:59 |
| kfox1111 | each with a few containers per pod. | 00:59 |
| jascott1 | right | 00:59 |
| *** zhubingbing has joined #openstack-kolla | 01:02 | |
| kfox1111 | the pod has, the l3-agent container, as well as fluentd or heka, | 01:02 |
| kfox1111 | as well as kubernetes-entrypoint as an init-container. | 01:02 |
| kfox1111 | and a lot of our pods already use kolla-toolbox as an init container too. | 01:02 |
| kfox1111 | I've mostly followed the pattern of, | 01:02 |
| kfox1111 | one k8s object per microservice. so, like, | 01:02 |
| kfox1111 | neutron-l3-agent gets one. neutron-openvwitch-agent gets one, neutron-server is one. etc. | 01:02 |
| jascott1 | so im thinking deployment.yamls in templates, DaemonSets aside basically, would we have one chart and deployment.yaml for glance-api and another for glance-registry and both wrapped in glance chart? | 01:04 |
| openstackgerrit | Joshua Harlow proposed openstack/kolla: Use itertools.chain and show how many files utime modified https://review.openstack.org/398145 | 01:05 |
| kfox1111 | one chart for glance-api, another for glance-registry I think. | 01:05 |
| jascott1 | ok thats what I thought you have been saying | 01:05 |
| jascott1 | makes sense but might be case by case | 01:05 |
| kfox1111 | I think we temlate out the build of charts, so we can share variables/chart.yaml code as much as possible. | 01:05 |
| kfox1111 | yeah. | 01:05 |
| jascott1 | like you said, some neutron needs to be in daemonsets | 01:06 |
| kfox1111 | I've got a wip prototype here: | 01:06 |
| kfox1111 | https://review.openstack.org/#/c/396296/ | 01:06 |
| kfox1111 | it actually works. :) | 01:06 |
| kfox1111 | but has some cleanup to do. and still messing around with a few ideas to see what will fly. | 01:06 |
| jascott1 | im def into this conversation but must run pickup my daughter brb! | 01:07 |
| kfox1111 | kk | 01:07 |
| *** inc0 has quit IRC | 01:10 | |
| *** duonghq has joined #openstack-kolla | 01:10 | |
| duonghq | morning guys | 01:10 |
| openstackgerrit | Li Yingjun proposed openstack/kolla-ansible: Disable notifications for cinder when no consumer https://review.openstack.org/398669 | 01:12 |
| srwilkers | i think this is a solid start kfox1111. been working on keystone/mariadb and think things are starting to become clear | 01:15 |
| *** jtriley has joined #openstack-kolla | 01:16 | |
| *** eaguilar has joined #openstack-kolla | 01:16 | |
| kfox1111 | srwilkers: cool. :) | 01:16 |
| kfox1111 | so if you want, you can base a ps on top of that wip one, and should be able to test out the keystone/mariadb helm packages in the gate pretty easily. | 01:17 |
| *** dave-mccowan has quit IRC | 01:18 | |
| kfox1111 | in tests/bin/ceph-workflow.sh, just replace the kollakube res command for the template with a helm install variant. | 01:18 |
| srwilkers | alright, sounds good. as soon as its ready, ill toss it up | 01:19 |
| kfox1111 | cool. :) | 01:19 |
| jascott1 | back | 01:25 |
| kfox1111 | wb. :) | 01:26 |
| kfox1111 | I gota head out in a few though. :/ | 01:26 |
| jascott1 | time! boo time! | 01:26 |
| kfox1111 | so, it sounds like slack won't work for our normal traffic. :/ | 01:27 |
| kfox1111 | maybe we just use it to ping each other when needed, set the topic to recoommend people visit here? | 01:27 |
| kfox1111 | closing the spec in about 5 min | 01:29 |
| jascott1 | makes sense I guess. like it has been said, should make talking to k8s/helm ppl easier | 01:34 |
| jascott1 | openstack overlords should carve out some budget to run RocketChat! | 01:35 |
| jascott1 | kfox1111 so the point of defining the whole resource as a named template is for re-use? | 01:36 |
| kfox1111 | I hadent heard of rocketchat yet. heard some good things about mattermost. gitlab seems to really like it. | 01:37 |
| kfox1111 | jascott1: was a prototype. | 01:37 |
| jascott1 | mm is written in golang which is cool but rocketchat is much further along last i checked (>6mos) | 01:37 |
| kfox1111 | a way to use a single package to package multiple microservices, but still launch only one microservice at a time. | 01:38 |
| kfox1111 | interesting. I"ll have to give rocketchat a look. :) | 01:38 |
| kfox1111 | it was just an attempt to see what helm could/could not do. | 01:38 |
| jascott1 | i ran several of them internally for my team last year and rocketchat won | 01:38 |
| jascott1 | great work | 01:38 |
| jascott1 | on the helm | 01:39 |
| kfox1111 | thanks. It still needs some love, but helm held up to the abuse I sent its way, so can't be all bad. :) | 01:39 |
| jascott1 | no one seems to be a fan of go tpl but it would be great not to have to jinja everything imo | 01:39 |
| kfox1111 | yeah. go tmpl's if conditionals are weird. | 01:39 |
| kfox1111 | its like a funky forth. | 01:40 |
| jascott1 | ive been messing with go for several years but have never used their tpl | 01:40 |
| kfox1111 | #if and foo bar .... | 01:40 |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible: Specify 'become' to neccesary tasks (general roles) https://review.openstack.org/398682 | 01:40 |
| jascott1 | cool thing you can do w golang, you can create a single page js app and embed it in the single go binary | 01:40 |
| jascott1 | single page = react, angular etc | 01:41 |
| kfox1111 | yeah.... | 01:41 |
| kfox1111 | though, | 01:41 |
| kfox1111 | I'm kind of in the camp now that thinks, containers are the statically linked tool of the future, | 01:42 |
| jascott1 | yup | 01:42 |
| kfox1111 | rather then make language / platoform specific ones. | 01:42 |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible: Specify 'become' for only neccesary tasks (default roles) https://review.openstack.org/398684 | 01:42 |
| kfox1111 | so you can esentially do the same thing with anything, just in containers instead of go. | 01:42 |
| jascott1 | cross platform compile is nice too. used it last night on entrypoint | 01:42 |
| kfox1111 | kind of desolves one of go's main selling points. | 01:42 |
| kfox1111 | ah. yeah. | 01:43 |
| kfox1111 | that toolchain is nice. :) | 01:43 |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible: Specify 'become' for only neccesary tasks (all other roles) https://review.openstack.org/398685 | 01:43 |
| jascott1 | i think my lack of openstack expertise is hurting me here but trying to visualize the whole set of charts | 01:44 |
| jascott1 | but essentially its flat, we dont use requirements.yaml | 01:44 |
| jascott1 | in most cases anyway | 01:44 |
| *** zhangyufei has joined #openstack-kolla | 01:45 | |
| jascott1 | oh wait we do use requirements, because we will have wrapper charts for each service correct? | 01:46 |
| kfox1111 | I really need to head out, but there may be a heiracry to it for those that want it. | 01:46 |
| kfox1111 | that is one option, yes. | 01:46 |
| jascott1 | k have a great evening | 01:46 |
| kfox1111 | for those that want to use helm/kubernetes-entrypoint as their entire orchestration, | 01:46 |
| kfox1111 | wrapping them up in a genreal openstack-package, with service-packages, then with all the microservice packages I think would work. | 01:47 |
| kfox1111 | closing the spec.... | 01:47 |
| duonghq | sdake, ping | 01:47 |
| sdake | sup duonghq | 01:47 |
| pron | soo, how do one set virt_type in kolla? | 01:48 |
| duonghq | do anybody setup linking between gerrit and launchpad of kolla-ansible? | 01:48 |
| pron | http://docs.openstack.org/developer/kolla/quickstart.html there is mentioned some majical nova-compute.conf , wich i couldnt find | 01:48 |
| pron | ;( | 01:48 |
| * pron hides in cornee and cries | 01:48 | |
| duonghq | pron, http://docs.openstack.org/developer/kolla/advanced-configuration.html#openstack-service-configuration-in-kolla | 01:49 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Spec - Kolla-Kubernetes Deployment Architecture https://review.openstack.org/392257 | 01:50 |
| duonghq | sdake, or some migration error?, this bp: https://blueprints.launchpad.net/kolla-ansible/+spec/ansible-specific-task-become | 01:50 |
| sdake | duonghq sorry dont understand the q | 01:51 |
| *** zhurong has joined #openstack-kolla | 01:51 | |
| duonghq | sdake, the bp is moved to kolla-ansible, but the tag in https://review.openstack.org/#/c/398682/ does not work | 01:52 |
| pron | duonghq: propably a reference to advaced config there in docs could help :P , but thx <3 | 01:52 |
| sdake | duonghq you need to resubmit the review | 01:53 |
| sdake | to the kolla-ansible repo | 01:53 |
| sdake | unfortunately for our 12 pages of reviews in queue this is necessary | 01:54 |
| sdake | dont blame me :) | 01:54 |
| duonghq | sdake, the link I posted is on kolla-ansible repo already | 01:54 |
| duonghq | I abandoned the kolla ones | 01:54 |
| duonghq | cherry-pick all 3 ps :) | 01:54 |
| jascott1 | pron did you find the solution? | 02:01 |
| sdake | duonghq can i get back to you - have critical work emergency atm | 02:01 |
| *** f13o has joined #openstack-kolla | 02:02 | |
| duonghq | sdake, thank you | 02:04 |
| *** Pavo has quit IRC | 02:07 | |
| *** Pavo has joined #openstack-kolla | 02:08 | |
| *** f13o has quit IRC | 02:09 | |
| jascott1 | kubecon videos are up: https://www.youtube.com/playlist?list=PLj6h78yzYM2PqgIGU1Qmi8nY7dqn9PCr4 | 02:19 |
| duonghq | thank jascott1 | 02:22 |
| jascott1 | this might be useful.. combined stream of all logs in pod or filter using label https://github.com/dtan4/k8stail | 02:25 |
| jascott1 | thanks reddit! | 02:25 |
| pron | jascott1: yes | 02:25 |
| jascott1 | cool | 02:26 |
| pron | at leadt i think | 02:26 |
| pron | so, but now kolla errored out :P | 02:26 |
| pron | somethign vip related, stops at mysql trying to connect tru vip | 02:27 |
| pron | but what i see that i cant ping that vip address from any host on network except that network node where that vip lives | 02:28 |
| jascott1 | doesnt sound good | 02:30 |
| pron | aye | 02:31 |
| * pron is trying to figure out keepalived logs | 02:31 | |
| pron | where to find em | 02:32 |
| *** duonghq has quit IRC | 02:32 | |
| *** tonanhngo has joined #openstack-kolla | 02:34 | |
| *** tonanhngo has quit IRC | 02:38 | |
| *** g3ek has quit IRC | 02:38 | |
| *** fragatin_ has joined #openstack-kolla | 02:47 | |
| *** g3ek has joined #openstack-kolla | 02:48 | |
| *** dave-mccowan has joined #openstack-kolla | 02:49 | |
| *** duonghq has joined #openstack-kolla | 02:50 | |
| *** fragatina has quit IRC | 02:50 | |
| *** fragatin_ has quit IRC | 02:51 | |
| *** newmember has quit IRC | 02:54 | |
| *** fragatina has joined #openstack-kolla | 02:57 | |
| *** fragatina has quit IRC | 02:58 | |
| *** adrian_otto has joined #openstack-kolla | 02:59 | |
| pron | hmmm i star to suspect that its another openstacks fault where i run this kolla in ;x | 02:59 |
| *** Pavo has quit IRC | 03:01 | |
| *** jtriley has quit IRC | 03:01 | |
| *** Pavo has joined #openstack-kolla | 03:04 | |
| *** adrian_otto has quit IRC | 03:04 | |
| *** adrian_otto1 has joined #openstack-kolla | 03:04 | |
| *** Jeffrey4l has quit IRC | 03:11 | |
| *** srwilkers has quit IRC | 03:12 | |
| *** unicell has quit IRC | 03:14 | |
| *** mgiles has quit IRC | 03:15 | |
| *** coolsvap has joined #openstack-kolla | 03:15 | |
| *** fragatina has joined #openstack-kolla | 03:24 | |
| Pavo | why does murano show up in horizon when I enabled it for deploymnet? | 03:25 |
| Pavo | doesn't I mean | 03:25 |
| *** fragatina has quit IRC | 03:29 | |
| Pavo | anyone? | 03:32 |
| jascott1 | sorry havent made it that far up the stack yet ;) | 03:33 |
| *** dave-mccowan has quit IRC | 03:33 | |
| jascott1 | i would shell into horizon container and and check it out | 03:33 |
| Pavo | well I looked into kolla/ansible/roles/horizon/templates/local_settings and nothing in there about murano at all | 03:34 |
| jascott1 | shell in and see if those bits even made it to the container | 03:35 |
| *** tonanhngo has joined #openstack-kolla | 03:35 | |
| *** tonanhngo has quit IRC | 03:35 | |
| jascott1 | arent some horizon addons manual? | 03:36 |
| Pavo | murano shouldn't be manual | 03:36 |
| *** jtriley has joined #openstack-kolla | 03:36 | |
| Pavo | I know it can be but if you enable it, there should be something in the json file that reads that its enabled in globals and turns that feature on inside horizon | 03:37 |
| Pavo | at least thats the logic I would assume it is | 03:37 |
| jascott1 | is it this? https://github.com/openstack/murano-dashboard | 03:38 |
| Pavo | I would assume yes | 03:38 |
| jascott1 | maybe hack in that symlink and work backwards | 03:39 |
| Pavo | guess I could test something out with it | 03:42 |
| Pavo | but if there is an option currently in kolla to enable murano it should work unless its using murano in a different manner than I would assume it would | 03:43 |
| *** tonanhngo has joined #openstack-kolla | 03:44 | |
| *** sdake_ has joined #openstack-kolla | 03:48 | |
| *** tonanhngo has quit IRC | 03:48 | |
| *** sdake has quit IRC | 03:50 | |
| sdake_ | pavo that is an often requested feature which is unimplented | 03:50 |
| sdake_ | pavo recommend pinging pbourke - i think he has a solution but its not upstream | 03:50 |
| sdake_ | pavo pretty sure pbourke would share, its a "we just rewrote the dockerfiles fo rthe 5th time" thing | 03:51 |
| Pavo | lol ok | 03:52 |
| Pavo | will do | 03:52 |
| Pavo | ping pbourke | 03:52 |
| Pavo | not bad resource usage | 03:54 |
| Pavo | http://pastebin.com/1tWxjJPK | 03:54 |
| Pavo | thats the controller, network and storage node resource usage from a tls deploymnet | 03:54 |
| Pavo | compute, storage node resource usage with tls deployment http://pastebin.com/7qXQFggZ | 03:56 |
| sdake_ | my mortal form grows weary need sustinance | 03:59 |
| sdake_ | later | 03:59 |
| Pavo | disk i/o usage on controller tops out around 1k K/s on average 700 K/s | 03:59 |
| *** sdake_ has quit IRC | 04:00 | |
| Pavo | l8r | 04:00 |
| *** Pavo has quit IRC | 04:07 | |
| *** Pavo has joined #openstack-kolla | 04:08 | |
| *** adrian_otto1 has quit IRC | 04:13 | |
| *** eaguilar has quit IRC | 04:16 | |
| *** mrhillsman has quit IRC | 04:29 | |
| openstackgerrit | Merged openstack/kolla: Changed the start script name in cinder-volume https://review.openstack.org/393858 | 04:34 |
| openstackgerrit | Merged openstack/kolla: Fix cinder volume permission issue https://review.openstack.org/393859 | 04:35 |
| *** tonanhngo has joined #openstack-kolla | 04:55 | |
| *** tonanhngo has quit IRC | 04:56 | |
| *** mottaman85 has joined #openstack-kolla | 04:57 | |
| *** unicell has joined #openstack-kolla | 05:23 | |
| *** sdake has joined #openstack-kolla | 05:33 | |
| *** sdake_ has joined #openstack-kolla | 05:37 | |
| *** liyifeng has joined #openstack-kolla | 05:37 | |
| *** sdake has quit IRC | 05:37 | |
| *** fragatina has joined #openstack-kolla | 05:47 | |
| *** fragatina has quit IRC | 05:52 | |
| *** chas has joined #openstack-kolla | 05:54 | |
| openstackgerrit | Yongfeng Du proposed openstack/kolla-ansible: Fix /etc/kolla permission https://review.openstack.org/398753 | 05:57 |
| *** chas has quit IRC | 06:01 | |
| *** chas has joined #openstack-kolla | 06:06 | |
| *** Pavo has quit IRC | 06:07 | |
| *** msimonin has joined #openstack-kolla | 06:08 | |
| *** tonanhngo has joined #openstack-kolla | 06:08 | |
| *** chas has quit IRC | 06:11 | |
| *** mottaman85 has quit IRC | 06:12 | |
| *** Pavo has joined #openstack-kolla | 06:12 | |
| *** tonanhngo has quit IRC | 06:13 | |
| *** msimonin has quit IRC | 06:14 | |
| *** haplo37_ has quit IRC | 06:20 | |
| sdake_ | pbourke around? | 06:26 |
| sdake_ | any core reviewers about? | 06:27 |
| sdake_ | coolsvap ping | 06:27 |
| coolsvap | sdake_: hey | 06:27 |
| *** haplo37_ has joined #openstack-kolla | 06:29 | |
| sdake_ | Daviey around? | 06:30 |
| *** chas has joined #openstack-kolla | 06:38 | |
| *** chas has quit IRC | 06:43 | |
| *** jtriley has quit IRC | 06:48 | |
| *** fragatina has joined #openstack-kolla | 06:49 | |
| *** fragatina has quit IRC | 06:54 | |
| bjolo | morning | 06:55 |
| bjolo | pbourke, i opened a ticket now https://bugs.launchpad.net/neutron/+bug/1642303 | 06:59 |
| openstack | Launchpad bug 1642303 in neutron "neutron multiple external flat networks fails" [Undecided,New] | 06:59 |
| bjolo | opened as a neutron bug as well since i cant figure out if it is kolla or neutron misbehaving | 07:00 |
| *** msimonin has joined #openstack-kolla | 07:01 | |
| *** DTadrzak has joined #openstack-kolla | 07:07 | |
| *** jtriley has joined #openstack-kolla | 07:26 | |
| *** msimonin has quit IRC | 07:26 | |
| *** jtriley has quit IRC | 07:34 | |
| *** tonanhngo has joined #openstack-kolla | 07:35 | |
| *** imcsk8 has quit IRC | 07:38 | |
| *** tonanhngo has quit IRC | 07:39 | |
| *** imcsk8 has joined #openstack-kolla | 07:42 | |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-ansible: Modify the letter case with 'false' in ansible yml file. https://review.openstack.org/398798 | 07:44 |
| *** f13o has joined #openstack-kolla | 07:45 | |
| *** rmart04 has joined #openstack-kolla | 07:45 | |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-ansible: Modify the letter case in ansible yml file. https://review.openstack.org/398798 | 07:56 |
| *** athomas has joined #openstack-kolla | 07:57 | |
| *** rmart04 has quit IRC | 08:06 | |
| *** Pavo has quit IRC | 08:07 | |
| *** Pavo has joined #openstack-kolla | 08:12 | |
| *** imcsk8 has quit IRC | 08:14 | |
| *** shardy_afk is now known as shardy | 08:15 | |
| *** imcsk8 has joined #openstack-kolla | 08:15 | |
| *** msimonin has joined #openstack-kolla | 08:21 | |
| *** f13o_ has joined #openstack-kolla | 08:27 | |
| *** jroll has quit IRC | 08:27 | |
| *** f13o has quit IRC | 08:29 | |
| *** rmart04 has joined #openstack-kolla | 08:33 | |
| *** matrohon has joined #openstack-kolla | 08:38 | |
| *** jroll has joined #openstack-kolla | 08:41 | |
| *** egonzalez90 has joined #openstack-kolla | 08:47 | |
| egonzalez90 | morning! | 08:48 |
| bjolo | morning egonzalez90 | 08:49 |
| *** duonghq has quit IRC | 08:58 | |
| *** duonghq has joined #openstack-kolla | 08:59 | |
| *** duonghq has quit IRC | 09:02 | |
| zhubingbing | morning! | 09:07 |
| *** gfidente has joined #openstack-kolla | 09:07 | |
| *** gfidente has joined #openstack-kolla | 09:07 | |
| *** portdirect_away_ has quit IRC | 09:08 | |
| magicboiz | morning | 09:11 |
| *** gfidente has quit IRC | 09:11 | |
| magicboiz | I'm facing some strange docker behaviour | 09:11 |
| magicboiz | 10.20.0.10:4000/kolla/ubuntu-binary-ceph-osd:3.0.1 "kolla_start" 4 minutes ago Restarting (1) About a minute ago ceph_osd_2 | 09:11 |
| magicboiz | has anyone seen this before? | 09:12 |
| magicboiz | contanier is always restarting | 09:12 |
| *** fragatina has joined #openstack-kolla | 09:12 | |
| *** DaveTurner has quit IRC | 09:14 | |
| *** gfidente has joined #openstack-kolla | 09:16 | |
| *** gfidente has joined #openstack-kolla | 09:16 | |
| *** fragatina has quit IRC | 09:18 | |
| egonzalez90 | magicboiz: what does docker logs ceph_osd_2 says? | 09:18 |
| *** zhurong has quit IRC | 09:20 | |
| magicboiz | egonzalez90: http://paste.openstack.org/show/589548/ | 09:20 |
| *** zhurong has joined #openstack-kolla | 09:20 | |
| *** Serlex has joined #openstack-kolla | 09:22 | |
| egonzalez90 | magicboiz: osd_1 is in the same machine? Is it working? | 09:22 |
| magicboiz | egonzalez90: no OSD is working in the environment, they are all "restarting" but kolla-ansible deploy didn't detect it..... | 09:23 |
| *** jtriley has joined #openstack-kolla | 09:25 | |
| egonzalez90 | sounds like a permission issue | 09:25 |
| liyifeng | ** ERROR: osd init failed: (1) Operation not permitted | 09:25 |
| magicboiz | yes, but why?? after some (deploy/destroy) I started to get this problem.... | 09:26 |
| liyifeng | perhaps selinux? | 09:27 |
| egonzalez90 | did you run cleanup-containers and cleanup-host? | 09:27 |
| *** YuYangWang has quit IRC | 09:33 | |
| magicboiz | I always run "destroy" | 09:34 |
| egonzalez90 | also, partitions need to be removed before re deploying an osd and make the label for kolla osd | 09:34 |
| magicboiz | Bu I did notice that "destroy" does 2 things I don't like: | 09:34 |
| *** jtriley has quit IRC | 09:35 | |
| magicboiz | 1. It leaves ceph-osd-x dir mounted | 09:35 |
| egonzalez90 | not anymore ;) | 09:35 |
| magicboiz | 2. the partition label is changed, it's KOLLA_CEPH_DATA_1_J + KOLLA_CEPH_DATA_1 and not KOLLA_CEPH_OSD_BOOTSTRAP | 09:36 |
| egonzalez90 | https://review.openstack.org/#/c/380028/ | 09:36 |
| egonzalez90 | that partition label os for kolla know what disk use to bootstrap osd | 09:37 |
| portdirect_away | morning | 09:37 |
| *** portdirect_away is now known as portdirect | 09:37 | |
| egonzalez90 | otherwise, every time you deploy will bootstrap again those disks | 09:37 |
| egonzalez90 | pbourke: around? | 09:38 |
| *** yingjun has quit IRC | 09:38 | |
| magicboiz | problem solved | 09:38 |
| magicboiz | Workaround: umount + restore partition label on every osd node | 09:39 |
| magicboiz | now, I don't have any osd container in restarting state.... | 09:39 |
| magicboiz | 10.20.0.10:4000/kolla/ubuntu-binary-ceph-osd:3.0.1 "kolla_start" 4 minutes ago Up 4 minutes ceph_osd_5 | 09:39 |
| *** f13o_ has quit IRC | 09:40 | |
| *** f13o has joined #openstack-kolla | 09:40 | |
| *** schwicht has quit IRC | 09:41 | |
| egonzalez90 | you created the lables the first time you deployed right? | 09:42 |
| egonzalez90 | *labels | 09:42 |
| egonzalez90 | as said before, KOLLA_CEPH_OSD_BOOTSTRAP label is used to know which devices are going to be used as osd, then are labeled as KOLLA_CEPH_DATA* | 09:43 |
| magicboiz | egonzalez90: yes, as doc http://docs.openstack.org/developer/kolla/ceph-guide.html states, I did tag my disks with parted | 09:46 |
| egonzalez90 | but when re deployed didn't or did you? | 09:47 |
| *** openstackgerrit has quit IRC | 09:48 | |
| *** openstackgerrit has joined #openstack-kolla | 09:48 | |
| pbourke | egonzalez90: morning | 09:51 |
| bjolo | hmm just discovered a regression | 09:51 |
| bjolo | https://bugs.launchpad.net/kolla/+bug/1642521 | 09:51 |
| openstack | Launchpad bug 1642521 in kolla "kolla-ansible fails on dedicated tunnel_interface" [Undecided,New] | 09:51 |
| pbourke | egonzalez90: hope you're surviving the rain :p | 09:51 |
| egonzalez90 | snow ATM | 09:52 |
| pbourke | ah yeah I heard that actually | 09:52 |
| portdirect | just been reading through the logs: Pavo/sdake: I've been using nova-docker quite a bit, but it pretty much stalled at mitaka (unless someones picked it up): I never managed to get it to work with nova beyond about half way through the libery dev cycle. Also re muranodash-board: you pretty much need to be running the db caching backend (and memcached if you want any semblance of performance) in django for it to function: so thats worth noting when | 09:52 |
| portdirect | oking to deploy it. | 09:52 |
| magicboiz | egonzalez90: when re-deploying, I didn't re-tag them | 09:53 |
| bjolo | pbourke, https://bugs.launchpad.net/neutron/+bug/1642303 | 09:54 |
| openstack | Launchpad bug 1642303 in kolla "neutron multiple external flat networks fails" [High,Triaged] | 09:54 |
| magicboiz | egonzalez90: at some point, the deploy/destroy game changed the disks in some way that posterior deployment failedf | 09:54 |
| magicboiz | failed | 09:54 |
| bjolo | pbourke, but i found something odd in the docs http://docs.openstack.org/newton/networking-guide/deploy-ovs-provider.html | 09:54 |
| egonzalez90 | thats the problem, every time an osd is destroyed, need to be re labeled. if not will use old ceph config and cant join cluster | 09:54 |
| portdirect | (need coffe - I meant newton not liberty... | 09:54 |
| magicboiz | egonzalez90: so now, manually, I have umounted and the re-tagged with parted | 09:54 |
| egonzalez90 | as far as i know, deploy baremetals does not label disks | 09:55 |
| bjolo | pbourke, they still specify network_vlan_ranges = provider | 09:55 |
| bjolo | even though they are not using vlans | 09:55 |
| pbourke | bjolo: "The provider value in the network_vlan_ranges option lacks VLAN ID ranges to support use of arbitrary VLAN IDs. | 09:57 |
| pbourke | " | 09:57 |
| pbourke | not sure what that means... | 09:57 |
| bjolo | pbourke, not me either?! | 09:58 |
| bjolo | thats what i mean by odd | 09:58 |
| *** zhurong has quit IRC | 10:02 | |
| *** hfu has quit IRC | 10:03 | |
| *** portdirect has quit IRC | 10:04 | |
| bjolo | pbourke, did you say that they had found some issues internally with this as well? | 10:05 |
| *** ccesario has joined #openstack-kolla | 10:06 | |
| *** harbor has joined #openstack-kolla | 10:06 | |
| *** harbor is now known as portdirect | 10:06 | |
| *** Pavo has quit IRC | 10:07 | |
| pbourke | bjolo: i thought they did but think it was something else... Tell you what, I'll try setting this up today | 10:10 |
| pbourke | bjolo: see if I can reproduce | 10:10 |
| *** Pavo has joined #openstack-kolla | 10:12 | |
| bjolo | pbourke, that would be great to get the bug confirmed or not | 10:12 |
| *** shardy is now known as shardy_mtg | 10:16 | |
| *** harbor has joined #openstack-kolla | 10:20 | |
| *** harbor is now known as portdirect_away_ | 10:20 | |
| *** portdirect has quit IRC | 10:23 | |
| *** zhangyufei has quit IRC | 10:23 | |
| *** zhangyufei has joined #openstack-kolla | 10:25 | |
| *** harbor has joined #openstack-kolla | 10:25 | |
| *** harbor is now known as portdirect | 10:25 | |
| *** portdirect_away_ has quit IRC | 10:34 | |
| *** hieulq has quit IRC | 10:36 | |
| *** fragatina has joined #openstack-kolla | 10:42 | |
| *** strigazi_AFK is now known as strigazi | 10:42 | |
| *** fragatina has quit IRC | 10:47 | |
| *** shardy_mtg is now known as shardy | 11:00 | |
| *** zhubingbing has quit IRC | 11:02 | |
| *** ppalacios has joined #openstack-kolla | 11:03 | |
| *** khamtamtun has joined #openstack-kolla | 11:06 | |
| *** fragatina has joined #openstack-kolla | 11:07 | |
| *** fragatina has quit IRC | 11:12 | |
| *** khamtamtun has quit IRC | 11:12 | |
| *** kamtamtun has joined #openstack-kolla | 11:14 | |
| *** DTadrzak has quit IRC | 11:21 | |
| sean-k-mooney | SamYaple: hi sam yes i did see daisy+kolla in opnfv however i have never seen them engage here so as far as i know its an independet effort by daisycloud. | 11:22 |
| *** ppalacios has quit IRC | 11:25 | |
| *** DTadrzak has joined #openstack-kolla | 11:29 | |
| sdake_ | pavo if you want to see some perf data | 11:30 |
| sdake_ | pavo you about? | 11:30 |
| sdake_ | pavo i did a brownbag on our osic scale testing | 11:30 |
| *** athomas has quit IRC | 11:32 | |
| pbourke | bjolo: are you deploying in libvirt guests or baremeteal? | 11:34 |
| sean-k-mooney | sdake_: oh at the summit? was it recorded? im still going through the videos but that sounds like one i should add to my list | 11:35 |
| sdake_ | sean-k-mooney indeed it was | 11:35 |
| sean-k-mooney | sdake_: cool good to know | 11:35 |
| sdake_ | sean-k-mooney https://www.youtube.com/watch?v=HsMhEjcdb_Q&list=PL2rC-8e38bUWityOA6O8RcqBU-aHvhVdR | 11:36 |
| sdake_ | about 50-70 people in the audience | 11:36 |
| sdake_ | it was a little chaotic | 11:36 |
| sdake_ | it was in open space | 11:37 |
| sdake_ | rather then a room | 11:37 |
| sean-k-mooney | sdake_: cool i was just looking for it on the youtube page | 11:39 |
| sdake_ | ya bar graph | 11:42 |
| sdake_ | fubared that one ;) | 11:42 |
| *** zhubingbing_ has joined #openstack-kolla | 11:44 | |
| *** imcsk8 has quit IRC | 11:45 | |
| *** kamtamtun has quit IRC | 11:45 | |
| *** imcsk8 has joined #openstack-kolla | 11:47 | |
| sdake_ | only had 10 mins to work with | 11:48 |
| sdake_ | jeffrey4l helped produce the slides as well :) | 11:48 |
| sdake_ | I think I may ahve left him out of the credits | 11:48 |
| sdake_ | which is shameful | 11:48 |
| sdake_ | oh well, not perfect ;( | 11:48 |
| sean-k-mooney | 40% of the 37% cpu time spent on the networkstack is an interesting datapoint | 11:48 |
| sdake_ | right | 11:48 |
| sean-k-mooney | the controlers were also the network nodes correct? | 11:49 |
| sdake_ | right | 11:49 |
| *** eaguilar has joined #openstack-kolla | 11:49 | |
| *** hfu has joined #openstack-kolla | 11:49 | |
| sdake_ | controller nodes definately I/O bound | 11:49 |
| sdake_ | ceph data shows either ceph can't perform at 10gig as well | 11:50 |
| sdake_ | not either | 11:50 |
| bjolo | pbourke, bare metal | 11:50 |
| sdake_ | well either ceph can't perform or the network stack is pokey | 11:50 |
| sean-k-mooney | sdake_: i think going to jumboframes would have helped ceph | 11:50 |
| sdake_ | sean-k-mooney keen point - we didn't deploy with jumbo frames although I would have liked to | 11:50 |
| sdake_ | also we had bonding in the mix | 11:50 |
| sdake_ | with vlans | 11:50 |
| sdake_ | all that crap hurts perf | 11:51 |
| sdake_ | the bottom line is the performance was fantastic from my pov | 11:51 |
| sdake_ | and kolla worked well under load | 11:51 |
| sdake_ | and tls isn't too expensive in terms of application performance | 11:51 |
| sean-k-mooney | yep i think it was a very positive test run | 11:51 |
| sdake_ | its not like a cherrypicked data either | 11:52 |
| *** kamtamtun has joined #openstack-kolla | 11:52 | |
| sdake_ | if i had cherrypicked data, i'd probably have left the slide about ES/heka | 11:52 |
| sdake_ | out | 11:52 |
| sean-k-mooney | sdake_: do you know if a repeat is on the cards for pike cycle? | 11:52 |
| sdake_ | sean-k-mooney inc0 is heading that up | 11:52 |
| sdake_ | sean-k-mooney inc0 i think needs to produce a whitepaper or soething for osic | 11:53 |
| sdake_ | i'm not sure what his agreement was with osic | 11:53 |
| *** kamtamtun has quit IRC | 11:53 | |
| sean-k-mooney | cool it would be nice to see 16.04 and centos next time, i know time just was not on our side this time | 11:53 |
| sdake_ | its not the lash they fear, its my divine power ;) | 11:53 |
| sdake_ | sean-k-mooney it took a long time to figure out how to bootstrap the nodes | 11:53 |
| sdake_ | sean-k-mooney i was really keen to see centos comparison | 11:54 |
| sean-k-mooney | well at least nextime it should go more smothly as people can build on the experice for this run. all in all i think it a good indicator of the maturaty of kolla in terms of deployment time and performance | 11:55 |
| sean-k-mooney | sdake_: by the way i agree with you endorcment of ceph. one question though have you ever tried the docker rbd volume dirver? | 11:57 |
| sean-k-mooney | sdake_: http://ceph.com/planet/getting-started-with-the-docker-rbd-volume-plugin/ | 11:58 |
| sean-k-mooney | my last kolla cloud died because i ran out of disk space on 2 of my controllers and bad things happened. | 11:59 |
| sean-k-mooney | im debating about installin ceph nativly on the host and then configuring kolla with external ceph so that i can use the rbd driver to have all my container volumes be backed by ceph too | 12:00 |
| *** zhubingbing_ has quit IRC | 12:01 | |
| *** zhugaoxiao has quit IRC | 12:01 | |
| *** zhubingbing_ has joined #openstack-kolla | 12:01 | |
| *** zhugaoxiao has joined #openstack-kolla | 12:01 | |
| *** ppalacios has joined #openstack-kolla | 12:02 | |
| *** jtriley has joined #openstack-kolla | 12:02 | |
| *** kamtamtun has joined #openstack-kolla | 12:04 | |
| sdake_ | sean-k-mooney agree | 12:05 |
| sdake_ | sean-k-mooney sort of disappointed again about centos, especially since the team worked miracles to get centos deployed | 12:05 |
| *** kamtamtun has quit IRC | 12:05 | |
| sdake_ | and we just basically ran out of time to run the tests | 12:05 |
| sdake_ | not just deployed but configured | 12:06 |
| *** jtriley has quit IRC | 12:06 | |
| sdake_ | the problem is each test case took 8+ hrs to run | 12:06 |
| sdake_ | and it took 3 weeks to get the cloud into a reliablly deployed state | 12:06 |
| sdake_ | and get the test cases all setup | 12:06 |
| sdake_ | if we had proper prepration | 12:06 |
| sdake_ | we would have avoided piss poor performance :) | 12:06 |
| sdake_ | realistically we could do xenail and centos with same data gathering in about a week now | 12:07 |
| sdake_ | possiblye 1.5 weeks | 12:07 |
| *** DTadrzak has quit IRC | 12:07 | |
| *** Pavo has quit IRC | 12:07 | |
| sean-k-mooney | qoute sdake:" we could do .. in a week now" :) | 12:08 |
| sdake_ | not kidding | 12:08 |
| sdake_ | with same team | 12:08 |
| sdake_ | the hard part is deployment | 12:08 |
| *** fragatina has joined #openstack-kolla | 12:08 | |
| sdake_ | getting the bare metal deployed i mean | 12:08 |
| sdake_ | after that the tests are all scripted | 12:08 |
| *** zhangyufei has quit IRC | 12:09 | |
| sdake_ | everything is nearly automated | 12:09 |
| sdake_ | it needs a watcher | 12:09 |
| sdake_ | someone to look after it and run the next test | 12:09 |
| sdake_ | and set it up | 12:09 |
| sdake_ | but that takes about 10-20 minutes | 12:09 |
| sdake_ | the bare metal deployment is a huge pain in the ass | 12:09 |
| sdake_ | bifrost wasn't ready at teh time | 12:10 |
| sean-k-mooney | no unfrotunetly it wasnt | 12:10 |
| sdake_ | same team could use same tools to do the bare metal deploy in about 4-8 hrs | 12:10 |
| sdake_ | sean-k-mooney no criticism | 12:10 |
| sdake_ | sean-k-mooney just a statement of fact ;) | 12:10 |
| sdake_ | sean-k-mooney i'm not sure it would have made things easier | 12:10 |
| sdake_ | it would actually probably have complicated matters | 12:10 |
| sdake_ | since the code was fairly green | 12:11 |
| sdake_ | we did use the host setup | 12:11 |
| sdake_ | that works pretty well | 12:11 |
| sdake_ | worked pretty well | 12:11 |
| sdake_ | haven't tried it lately :) | 12:11 |
| *** Pavo has joined #openstack-kolla | 12:12 | |
| *** papacz has joined #openstack-kolla | 12:13 | |
| *** fragatina has quit IRC | 12:13 | |
| pbourke | bjolo: so I have two external networks, I've created a network on one and booted a VM on it. So far can't ping/ssh but that could be any number of reasons | 12:15 |
| pbourke | bjolo: so dont know if related yet or not | 12:15 |
| *** sp_ has quit IRC | 12:19 | |
| *** awiddersheim has quit IRC | 12:19 | |
| *** mgoddard_ has joined #openstack-kolla | 12:20 | |
| *** mgoddard has quit IRC | 12:23 | |
| *** zhurong has joined #openstack-kolla | 12:24 | |
| *** ppalacios has quit IRC | 12:28 | |
| *** DTadrzak has joined #openstack-kolla | 12:30 | |
| *** dave-mccowan has joined #openstack-kolla | 12:31 | |
| *** ppalacios has joined #openstack-kolla | 12:38 | |
| sean-k-mooney | sdake_: i used it a little while ago. there are som usablity tweek i have been meaning to make and some addtions such as supprot for proxy servers but it more or less did what i wanted. | 12:41 |
| sdake_ | sean-k-mooney referecing which? | 12:42 |
| *** ppalacios has quit IRC | 12:42 | |
| *** schwicht has joined #openstack-kolla | 12:46 | |
| *** mgiles has joined #openstack-kolla | 12:48 | |
| *** liyifeng has quit IRC | 12:50 | |
| *** srwilkers has joined #openstack-kolla | 12:50 | |
| *** chopmann has joined #openstack-kolla | 12:51 | |
| chopmann | hello everyone | 12:52 |
| sean-k-mooney | sdake_: the host playbook that installs the deps for kolla | 12:52 |
| sdake_ | right - that definateldy works | 12:52 |
| sdake_ | I think it needs some additions around bonding | 12:53 |
| sdake_ | and networking setup in general | 12:53 |
| sean-k-mooney | sdake_: yep i would love to add that | 12:53 |
| sean-k-mooney | also i was thinking it might be nice for it to be able to reuse the ceph or ovs deployment scrtips to set those up ahead of the main kolla run | 12:54 |
| sdake_ | sean-k-mooney honeslty too tired to think | 12:54 |
| sdake_ | been up tfor 24 hrs | 12:54 |
| sdake_ | goign to crash for a couple hours | 12:54 |
| *** msimonin has quit IRC | 12:54 | |
| sdake_ | then have all day meeting | 12:54 |
| * sdake_ groans | 12:54 | |
| sean-k-mooney | no worries catch you later | 12:54 |
| srwilkers | rip sdake_ | 12:59 |
| *** msimonin has joined #openstack-kolla | 12:59 | |
| *** duonghq has joined #openstack-kolla | 13:00 | |
| sdake_ | srwilkers no shit | 13:03 |
| *** ppalacios has joined #openstack-kolla | 13:03 | |
| sdake_ | srwilkers hoepfully i wake up tomorrow :) | 13:03 |
| *** msimonin has quit IRC | 13:03 | |
| sdake_ | rather today | 13:03 |
| sdake_ | 3 hrs for powernap | 13:03 |
| srwilkers | im not envious | 13:04 |
| *** msimonin has joined #openstack-kolla | 13:05 | |
| *** schwicht has quit IRC | 13:05 | |
| sdake_ | only have to make it to 2pm today | 13:06 |
| sdake_ | then can ptfo for 12 hrs ;) | 13:06 |
| sdake_ | ok i'm off | 13:06 |
| sdake_ | not gettinga powernap on irc | 13:06 |
| *** ppalacios has quit IRC | 13:10 | |
| chopmann | does anyone got a blog-post tutorial for a mitaka --> newton upgrade? :-) | 13:10 |
| *** lamt has joined #openstack-kolla | 13:14 | |
| duonghq | sbezverk, do you notice that kubeadm master keep writing into disk? | 13:20 |
| duonghq | or it's my setup problem? | 13:20 |
| *** athomas has joined #openstack-kolla | 13:23 | |
| *** zhongshengping has joined #openstack-kolla | 13:25 | |
| srwilkers | silly question. wheres the ansible code live now? cant find it on github | 13:26 |
| zhongshengping | git clone https://git.openstack.org/openstack/kolla-ansible | 13:26 |
| zhongshengping | kolla-ansible is up! All ansible changes goes there now | 13:26 |
| *** zhurong has quit IRC | 13:28 | |
| srwilkers | thanks mate | 13:28 |
| *** athomas has quit IRC | 13:28 | |
| *** zhurong has joined #openstack-kolla | 13:28 | |
| *** sdake_ has quit IRC | 13:29 | |
| *** Jeffrey4l has joined #openstack-kolla | 13:30 | |
| *** eaguilar has quit IRC | 13:31 | |
| *** eaguilar has joined #openstack-kolla | 13:34 | |
| *** hfu has quit IRC | 13:35 | |
| *** rhallisey has joined #openstack-kolla | 13:37 | |
| openstackgerrit | zhongshengping proposed openstack/kolla-ansible: Deprecate scheduler_max_attempts option in nova https://review.openstack.org/399011 | 13:38 |
| *** vagrant` has joined #openstack-kolla | 13:42 | |
| *** zhurong has quit IRC | 13:43 | |
| *** zhurong has joined #openstack-kolla | 13:45 | |
| *** msimonin has quit IRC | 13:46 | |
| *** jheroux has joined #openstack-kolla | 13:46 | |
| *** fguillot has joined #openstack-kolla | 13:46 | |
| *** msimonin has joined #openstack-kolla | 13:46 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 13:47 |
| *** narasimah_SV has joined #openstack-kolla | 13:48 | |
| *** zhongshengpin has joined #openstack-kolla | 13:48 | |
| *** zhongshengping has quit IRC | 13:48 | |
| *** vagrant` has quit IRC | 13:48 | |
| narasimah_SV | http://paste.openstack.org/show/589588/ | 13:49 |
| narasimah_SV | gnocchi containers are unable to read the ceph keyring file | 13:49 |
| narasimah_SV | i placed that file in /etc/kola/config folder and also in gnochi-api folder too | 13:50 |
| narasimah_SV | can anyone help what exactly I am missing here | 13:50 |
| *** zhongshenping has joined #openstack-kolla | 13:50 | |
| *** zhongshengpin has quit IRC | 13:51 | |
| *** schwicht has joined #openstack-kolla | 13:52 | |
| *** zhongshenping has quit IRC | 13:53 | |
| *** RichWellum has joined #openstack-kolla | 13:53 | |
| *** zhongshenping has joined #openstack-kolla | 13:53 | |
| *** zhongshenping has quit IRC | 13:56 | |
| *** RichWellum has quit IRC | 13:56 | |
| *** sdake has joined #openstack-kolla | 13:56 | |
| egonzalez90 | narasimah_SV: the file is not copied to /etc/gnocchi folder in controller nodes | 13:57 |
| *** RichWellum has joined #openstack-kolla | 13:57 | |
| egonzalez90 | there is no task to do that in ansible/roles/gnocchi/task/config.yml | 13:57 |
| *** diogogmt has quit IRC | 13:58 | |
| sdake | well that sucked | 14:02 |
| sdake | no powernap | 14:02 |
| sdake | sup folks | 14:02 |
| sdake | i guess i'm awake for the day until 2pm pst ;) | 14:02 |
| *** schwicht has quit IRC | 14:03 | |
| duonghq | sdake, did you sleep more when you younger? | 14:03 |
| *** sdake_ has joined #openstack-kolla | 14:03 | |
| *** mgoddard_ has quit IRC | 14:06 | |
| *** mgoddard has joined #openstack-kolla | 14:06 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 14:06 |
| *** sdake has quit IRC | 14:07 | |
| *** Pavo has quit IRC | 14:07 | |
| narasimah_SV | egonzalez90: this ius mentioned in external_ceph.yml file | 14:08 |
| *** liyifeng has joined #openstack-kolla | 14:08 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use kolla-ansible for deployment https://review.openstack.org/398506 | 14:09 |
| *** fragatina has joined #openstack-kolla | 14:10 | |
| egonzalez90 | narasimah_SV: sorry, didn't reallize there was an external task file | 14:10 |
| narasimah_SV | ok | 14:10 |
| narasimah_SV | in gnocchi component templates it is mentioned as this | 14:11 |
| narasimah_SV | and in external ceph.yml file it is like this | 14:12 |
| *** Pavo has joined #openstack-kolla | 14:12 | |
| bjolo | pbourke, the weirdness we see is that the openflow rules that gets created are in random. so some things might work, and others wont | 14:13 |
| *** hieulq has joined #openstack-kolla | 14:13 | |
| *** inc0 has joined #openstack-kolla | 14:13 | |
| pbourke | bjolo: is the end result that you can't connect to your instance? | 14:13 |
| bjolo | pbourke, one time we could trace all trafic going out the wrong external bridge. i.e. 802 IP traffic ended up going out the 805 bridge | 14:13 |
| egonzalez90 | narasimah_SV: other roles as glance and nova/ permissions over ceph files are 700 instead of 600 | 14:14 |
| bjolo | yes, no traffic in or out works as it should. Some things might work, but it just random | 14:14 |
| narasimah_SV | for me 644 for those ceph files is working | 14:14 |
| egonzalez90 | but that is not the copy issue, this is execution issue | 14:15 |
| *** msimonin has quit IRC | 14:15 | |
| bjolo | since the ex bridges get disconnected all the time, they dont seem to get the right openflow rules | 14:15 |
| *** fragatina has quit IRC | 14:15 | |
| inc0 | good morning | 14:15 |
| portdirect | 'morning | 14:16 |
| egonzalez90 | narasimah_SV: where you have the file? what path? | 14:16 |
| *** msimonin has joined #openstack-kolla | 14:16 | |
| egonzalez90 | * in kolla-ansible node | 14:17 |
| narasimah_SV | http://paste.openstack.org/show/589592/ | 14:17 |
| narasimah_SV | see the templates and external_ceph file | 14:18 |
| narasimah_SV | the path is different | 14:18 |
| narasimah_SV | will this cause any issue | 14:18 |
| narasimah_SV | ? | 14:18 |
| narasimah_SV | tha path to copy form these ceph file from config directory to /etc/ceph/ | 14:19 |
| bjolo | pbourke, on network node do: docker exec vswitchd bash | 14:20 |
| bjolo | watch -d -n 1 "ovs-vsctl show; ovs-ofctl show br-int; ovs-ofctl dump-flows br-int; ovs-ofctl show br-vlan802; ovs-ofctl dump-flows br-vlan802" | 14:20 |
| pbourke | ok one sec | 14:20 |
| bjolo | replace with your bridge names | 14:20 |
| *** ppalacios has joined #openstack-kolla | 14:20 | |
| bjolo | then on one VM instance, ping the external gw | 14:21 |
| *** zhurong has quit IRC | 14:21 | |
| bjolo | watch -d will show what OF rules are hit by the traffic | 14:21 |
| *** eaguilar_ has joined #openstack-kolla | 14:21 | |
| bjolo | note you need a large screen. I have 4K monitor and the output barely fits :) | 14:22 |
| bjolo | also what do see in your ovs logfiles? do you get the disconnects | 14:22 |
| *** eaguilar has quit IRC | 14:22 | |
| pbourke | no Im not seeing those | 14:23 |
| bjolo | hmmm | 14:23 |
| pbourke | but I have other problems, cant even ssh to my instance atm | 14:23 |
| bjolo | i could not ssh either | 14:23 |
| bjolo | had to use the console in horizon | 14:23 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 14:23 |
| pbourke | ok i'll try that | 14:23 |
| egonzalez90 | narasimah_SV: mean, where your keyrings are in ansible node, /etc/kolla/config/* and if in controller nodes the files are properly copied? | 14:24 |
| *** msimonin has quit IRC | 14:24 | |
| pbourke | bjolo: are you using floating ips or launching directly on one of the external networks | 14:24 |
| narasimah_SV | yes in ansible node i have them in /etc/kolla/config | 14:25 |
| narasimah_SV | and even they got copied to controller nodes too | 14:26 |
| *** msimonin has joined #openstack-kolla | 14:26 | |
| *** msimonin has quit IRC | 14:26 | |
| openstackgerrit | Pete Birley proposed openstack/kolla-ansible: Update Barbican to use cirrect mysql connection string https://review.openstack.org/399033 | 14:26 |
| openstackgerrit | Pete Birley proposed openstack/kolla-ansible: Update Barbican to use correct mysql connection string https://review.openstack.org/399033 | 14:28 |
| *** msimonin has joined #openstack-kolla | 14:28 | |
| *** eaguilar has joined #openstack-kolla | 14:30 | |
| *** eaguilar_ has quit IRC | 14:32 | |
| *** ppalacios1 has joined #openstack-kolla | 14:34 | |
| *** ppalacios1 has quit IRC | 14:34 | |
| bjolo | pbourke, tried both, but mostly floating ips | 14:34 |
| *** ppalacios has quit IRC | 14:34 | |
| pbourke | bjolo: I can't ping/ssh my VM even with one flat network right now | 14:34 |
| *** adrian_otto has joined #openstack-kolla | 14:35 | |
| bjolo | ok you might have some other issue then | 14:35 |
| bjolo | when you say one, how much one? | 14:35 |
| bjolo | only one bridge ex? | 14:35 |
| bjolo | or only one neutron network defined? | 14:36 |
| *** sdake has joined #openstack-kolla | 14:36 | |
| bjolo | my issues start on the bridge level | 14:36 |
| sdake | morning | 14:36 |
| pbourke | one external network | 14:36 |
| bjolo | as soon as i add a second external bridge, things break | 14:36 |
| egonzalez90 | narasimah_SV: keyring should be in /etc/kolla/config/gnocchi/gnocchi-api/ at deployer node | 14:36 |
| *** ppalacios has joined #openstack-kolla | 14:36 | |
| bjolo | independent on how many neutron networks i have | 14:36 |
| srwilkers | sdake, go to bed | 14:37 |
| egonzalez90 | and in /etc/kolla/gnocchi-api at control node | 14:37 |
| sdake | srwilkers offsite today | 14:37 |
| sdake | no can do | 14:37 |
| sdake | srwilkers have to power through it | 14:37 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: Use mariadb.cnf for mariadb custom file name https://review.openstack.org/399038 | 14:37 |
| sdake | Jeffrey4l say i am too tired to look atm, but did you submit releasee for both kolla-ansible and kolla repo? | 14:37 |
| Jeffrey4l | sdake, yep and it merged. | 14:38 |
| Jeffrey4l | kolla-ansible https://review.openstack.org/398434 | 14:38 |
| sdake | cool thx :) | 14:38 |
| sdake | no need to link | 14:38 |
| Jeffrey4l | kolla https://review.openstack.org/398433 | 14:38 |
| Jeffrey4l | ;) | 14:38 |
| sdake | if release team merged it | 14:38 |
| sdake | its gtg | 14:38 |
| *** sdake_ has quit IRC | 14:39 | |
| *** ChanServ sets mode: +o inc0 | 14:39 | |
| *** adrian_otto has quit IRC | 14:40 | |
| inc0 | New to Kolla? Please read the documentation here: http://docs.openstack.org/developer/kolla/; Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA); w00t for Kolla!; Kolla-ansible is up! All ansible changes goes there now; 4.0.0b1 released! | 14:40 |
| *** inc0 changes topic to "New to Kolla? Please read the documentation here: http://docs.openstack.org/developer/kolla/; Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA); w00t for Kolla!; Kolla-ansible is up! All ansible changes goes there now; 4.0.0b1 released!" | 14:40 | |
| kfox1111 | morning. | 14:40 |
| *** inc0 sets mode: -o inc0 | 14:40 | |
| inc0 | thanks:) | 14:40 |
| srwilkers | good morning inc0 | 14:40 |
| srwilkers | good morning kfox1111 | 14:40 |
| *** duonghq has quit IRC | 14:40 | |
| inc0 | and hello kfox1111 srwilkers | 14:40 |
| narasimah_SV | egonzalez90: when I placed it in side /etc/kolla/config/gnocchi/gnocchi-api folder it was not taking it | 14:42 |
| openstackgerrit | Pete Birley proposed openstack/kolla: Remove unneeded PythonMySQL libs from OpenStack Base Image https://review.openstack.org/399042 | 14:42 |
| narasimah_SV | after changing it /etc/kolla/config/gnocchi-api it worked | 14:42 |
| narasimah_SV | according to external_ceph.yml file it is expecting file in /etc/kolla/config/gnocchi-api | 14:42 |
| *** adrian_otto has joined #openstack-kolla | 14:42 | |
| narasimah_SV | folder structure | 14:42 |
| narasimah_SV | but in templates again it is different | 14:43 |
| sdake | portdirect bit concerned with that pythonmysql change | 14:44 |
| sdake | portdirect considering our gates are afu atm | 14:44 |
| *** schwicht has joined #openstack-kolla | 14:45 | |
| portdirect | sdake: k, I'll fix it | 14:45 |
| sdake | portdirect not sure naything wrong with the patch | 14:45 |
| sdake | portdirect did you test it | 14:45 |
| inc0 | pprokop, did you release entrypoint binary by any chance? | 14:45 |
| openstackgerrit | Pete Birley proposed openstack/kolla: DO NOT MERGE!: Remove unneeded PythonMySQL libs from OpenStack Base Image https://review.openstack.org/399042 | 14:45 |
| sdake | portdirect i was thinking something less drastic, like confirming you tested and finding a victim to test it to confirm ;) | 14:46 |
| portdirect | the ansible one yes, but not fully tested all images (only centos binary) - I've marked it as do not merge untill the gates are back | 14:46 |
| sdake | portdirect that may be some time before we have cross repo gating | 14:46 |
| sdake | portdirect i guess if yo udont mind waiting wfm | 14:47 |
| portdirect | sdake: i can test it properly here (just about to run another build/deployment - then I'll remove the scary looking warning) | 14:47 |
| sdake | portdirect we have had alot of nasty experiences with those python mysql libraries | 14:48 |
| sdake | especially as it relates to kolla-toolbox | 14:48 |
| inc0 | Jeffrey4l, I see you're handling kolla-ansible installatin in kolla gates? | 14:48 |
| Jeffrey4l | inc0, yep. | 14:48 |
| sdake | nice :) | 14:48 |
| inc0 | ok, so I won't remove these gates yet | 14:48 |
| inc0 | I'll make build gates voting tho | 14:49 |
| portdirect | sdake: cheers for the heads up :) | 14:49 |
| Jeffrey4l | install kolla-ansible in kolla and install kolla in kolla-ansible. ;) | 14:49 |
| Jeffrey4l | re voting, yep. this will make the code more stable. | 14:49 |
| sdake | portdirect normally i wouldn't care too much about dep changes in that list, but i've spent probably 100+ hours dealing with that particular library | 14:49 |
| inc0 | I'll add source builds to voting for now | 14:49 |
| Jeffrey4l | btw, i removed build related job in kolla-ansible. | 14:49 |
| inc0 | as they're much more reliable | 14:49 |
| sdake | maybe 75 hours | 14:50 |
| sdake | anyway alot of time burned up on it | 14:50 |
| inc0 | Jeffrey4l, did you do it in infra too? | 14:50 |
| sdake | portdirect fyi the problem is we install the python libraries, and then toolbox has problems because of setuptools | 14:50 |
| Jeffrey4l | inc0, yep. what all i did is change the project-config. | 14:50 |
| inc0 | ok so I probably need to rebase mine | 14:51 |
| Jeffrey4l | inc0, https://review.openstack.org/398503 | 14:51 |
| inc0 | https://review.openstack.org/#/c/398534/5 I had this but I need to change it since you're making zuul cloner a thing | 14:51 |
| *** liyifeng has quit IRC | 14:52 | |
| portdirect | sdake: thanks - I've had similar pain when the migration started to PyMySQL - though not your issue, cinder refused to work (reliably) with it for ages... | 14:54 |
| egonzalez90 | narasimah_SV: saw the error. typo in external-ceph.yml | 14:54 |
| portdirect | (at least for me) | 14:54 |
| egonzalez90 | keyring is copied as keryring | 14:55 |
| narasimah_SV | ok thanks for the help | 14:55 |
| narasimah_SV | I didnt check about it | 14:55 |
| *** DTadrzak has quit IRC | 14:55 | |
| egonzalez90 | feel free to push a change | 14:55 |
| *** harbor has joined #openstack-kolla | 14:55 | |
| *** harbor is now known as portdirect_away_ | 14:55 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 14:56 |
| *** lrensing has joined #openstack-kolla | 14:56 | |
| narasimah_SV | sure I will so it by today | 14:56 |
| kfox1111 | so.... | 14:56 |
| kfox1111 | for the kolla-kubernetes gate, | 14:57 |
| kfox1111 | we aonly use genconfig. | 14:57 |
| kfox1111 | do I need the kolla repo then? | 14:57 |
| kfox1111 | or do I just s/kolla/kolla-ansible/' in the gate? | 14:57 |
| egonzalez90 | narasimah_SV: the typo is present in stable/newton open a bug to backport | 14:58 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 14:58 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use kolla-ansible for deployment https://review.openstack.org/398506 | 14:58 |
| narasimah_SV | sure | 14:58 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Fix gate after the repo split https://review.openstack.org/399050 | 14:59 |
| *** yingjun has joined #openstack-kolla | 15:00 | |
| *** mgoddard_ has joined #openstack-kolla | 15:01 | |
| *** portdirect_away_ has quit IRC | 15:04 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: Use mariadb.cnf for mariadb custom file name https://review.openstack.org/399038 | 15:04 |
| *** mgoddard has quit IRC | 15:04 | |
| *** chopmann has quit IRC | 15:07 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: Create empty destroy.yml for all role https://review.openstack.org/399056 | 15:08 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Fix gate after the repo split https://review.openstack.org/399050 | 15:10 |
| *** adrian_otto has quit IRC | 15:11 | |
| *** adrian_otto has joined #openstack-kolla | 15:11 | |
| pprokop | inc0: yes | 15:12 |
| sdake | hey peeps gotta drop off for today | 15:12 |
| sdake | be back later | 15:12 |
| kfox1111 | have a good one | 15:12 |
| pprokop | https://github.com/stackanetes/kubernetes-entrypoint/releases/download/v0.1.0/kubernetes-entrypoint | 15:12 |
| *** jtriley has joined #openstack-kolla | 15:12 | |
| kfox1111 | pprokop: is there a container yet? | 15:12 |
| pprokop | yeh | 15:13 |
| pprokop | quay.io/stackanetes/kubernetes-entrypoint:v0.1.0 | 15:13 |
| pprokop | based on debian | 15:14 |
| kfox1111 | cool. should make testing a lot easier. thanks. :) | 15:14 |
| *** DTadrzak has joined #openstack-kolla | 15:14 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 15:15 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use kolla-ansible for deployment https://review.openstack.org/398506 | 15:16 |
| inc0 | kfox1111, I'll curl it to our kolla-k8s-toolbox container | 15:17 |
| inc0 | and I was thinking of using it as entrypoint | 15:17 |
| *** harbor has joined #openstack-kolla | 15:21 | |
| *** harbor is now known as portdirect_away_ | 15:21 | |
| sdake | inc0 sounds good to me | 15:21 |
| sdake | gotta jet | 15:21 |
| sdake | lets just make sur ewe stick to the optional thign :) | 15:21 |
| kfox1111 | inc0: I think we should just amake a kubernetes-entrypoint container. | 15:23 |
| kfox1111 | the toolboxen are doing too much. | 15:23 |
| *** ppalacios has quit IRC | 15:23 | |
| inc0 | kfox1111, technically toolbox doesn't do anything | 15:23 |
| inc0 | it just holds various binaries:) | 15:23 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 15:24 |
| sbezverk | inc0: I checked doc for entry point and have a question, since you are pushing it so much I guess you know the answer ;-) One required parameter of it is command, which is executed when all dependecy get resolved. I am not sure how it aligns with approach that init container is used just to wait for dependecy and then exit to allow main container to start.. could you elaborate on this | 15:25 |
| sbezverk | a bit? | 15:25 |
| kfox1111 | inc0: yeah. and the k8s way usually is lots of tiny containers. | 15:25 |
| inc0 | sbezverk, command can be exit 1:) | 15:26 |
| inc0 | or exit 0 rather | 15:26 |
| sbezverk | inc0: got it, thank you.. | 15:26 |
| kfox1111 | sbezverk: I filed an issue for that | 15:26 |
| kfox1111 | https://github.com/stackanetes/kubernetes-entrypoint/issues/12 | 15:26 |
| kfox1111 | was fixed really quickly. :) | 15:27 |
| kfox1111 | I'd rather have containers that do what they do well, rather then bundling up binaries together, making it harder to test upgrading one thing without breaking another. | 15:28 |
| inc0 | kfox1111, fair enough, it can be dedicated | 15:28 |
| sbezverk | kfox1111: cool, thanks for the link. makes perfect sense.. | 15:28 |
| inc0 | pprokop, this issue was part of released binary right? | 15:28 |
| sbezverk | kfox1111: I agree about the statement about multiple small containers, but here is a problem which I think is triggered by this approach. If you code version of image directly into the container, it will be very difficult to change it for something else. Example I run etcd for canal and it is 2.2.1, I want to use different version, then I need to change pod definition.. | 15:32 |
| *** ppalacios has joined #openstack-kolla | 15:32 | |
| sbezverk | with centralized storage of this in tool box it might be easier as version controller in a single place.. | 15:32 |
| inc0 | sbezverk, on the other hand if you want to change just one thing and leave another intact | 15:33 |
| inc0 | having single container makes this impossible:) | 15:33 |
| sbezverk | inc0: right but if you need to change it for 100 pods ;-) ?? | 15:34 |
| *** absubram has joined #openstack-kolla | 15:34 | |
| inc0 | sbezverk, thats why we use helm:) | 15:34 |
| *** absubram_ has joined #openstack-kolla | 15:35 | |
| sbezverk | inc0: make sense but in this case we have to make sure we use variables for all images used in pod by containers | 15:36 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 15:37 |
| *** fragatina has joined #openstack-kolla | 15:37 | |
| *** absubram has quit IRC | 15:38 | |
| *** absubram_ is now known as absubram | 15:38 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 15:39 |
| Jeffrey4l | inc0, we have removed all branches in kolla-ansible. should we need remove all the tags? | 15:41 |
| inc0 | Jeffrey4l, besides 4.0.0b1 ofc | 15:41 |
| *** fragatina has quit IRC | 15:41 | |
| Jeffrey4l | inc0, yes. but how? do u have any idea how to remove tag? | 15:42 |
| *** fragatina has joined #openstack-kolla | 15:42 | |
| inc0 | hmm...not sure too | 15:42 |
| *** srwilkers is now known as srwilkers_away | 15:43 | |
| inc0 | sbezverk, we hardcode version in dockerfile a lot | 15:44 |
| inc0 | https://github.com/openstack/kolla/blob/master/docker/heka/Dockerfile.j2#L17 an example | 15:44 |
| *** diogogmt has joined #openstack-kolla | 15:46 | |
| kfox1111 | sbezverk: I thats actually a feature. | 15:47 |
| kfox1111 | you want to know exactly the version you need is running identically on all nodes. | 15:47 |
| kfox1111 | so all versions should be explicit. | 15:47 |
| kfox1111 | in k8s. | 15:47 |
| kfox1111 | so yes, your right, it would require tweaking the packages. | 15:48 |
| kfox1111 | but that lets you test each package and verify it works, and then you know when a user wants to deploy a specific package, that package works. | 15:48 |
| sbezverk | kfox1111: but if you hardcode version in image, no tweaking in package will help I think | 15:48 |
| sbezverk | I mean docker image | 15:49 |
| sbezverk | ot the image name must have the actual software version like entrypoint-0.1.0-4.0.0b1 | 15:49 |
| kfox1111 | so, let me go through a workflow I usually use as an example. | 15:50 |
| *** matrohon has quit IRC | 15:50 | |
| kfox1111 | tag should have the version. | 15:50 |
| kfox1111 | or be date tagged or something. | 15:50 |
| kfox1111 | so, I run a physics system with k8s now. | 15:50 |
| *** mgoddard has joined #openstack-kolla | 15:50 | |
| *** mgoddard_ has quit IRC | 15:50 | |
| kfox1111 | I have a bunch of containers. but lets walk through one. | 15:51 |
| kfox1111 | I have the source for the container here: https://github.com/pnnlhep/osg-fetch-crl | 15:51 |
| kfox1111 | associated with that is a autobuilding repo on the hub. https://hub.docker.com/r/pnnlhep/osg-fetch-crl/ | 15:52 |
| kfox1111 | so I can push a change to the git repo and get a new container, or I can go to the hub and trigger a new build. | 15:52 |
| kfox1111 | I then docker pull pnnlhep/osg-fetch-crl:latest and inspect/test it. | 15:52 |
| kfox1111 | once I'm happy with that image, I | 15:53 |
| kfox1111 | docker tag pnnlhep/osg-fetch-crl:latest pnnlhep/osg-fetch-crl-stable:2016-11-17 | 15:53 |
| kfox1111 | docker push pnnlhep/osg-fetch-crl-stable:2016-11-17 | 15:53 |
| kfox1111 | making a release of that container. | 15:53 |
| kfox1111 | then I go into my deployment in k8s, update the image to point to that specific version, and upgrade the k8s object on the cluster. | 15:54 |
| kfox1111 | this then deploys the change to all the nodes. | 15:54 |
| kfox1111 | so I know if I ever have to reinstall a node, it is consistent with al lthe other nodes. | 15:54 |
| kfox1111 | if I run into a problem, I can just change the image tag back to the previous version. | 15:54 |
| sbezverk | kfox1111: in your example you do not hardcode a version of software you add into the docker image, you rely on yum to get you one. | 15:55 |
| kfox1111 | yeah. | 15:55 |
| kfox1111 | usually I don't pin a particular version unless I run into some problem. | 15:55 |
| sbezverk | but when it is hardcoded, then you might end with the same version of software, but wil different tagging | 15:55 |
| sbezverk | if you build this software at different dates | 15:56 |
| kfox1111 | right. | 15:56 |
| kfox1111 | its kind of rare you don't get a different container though anyway. | 15:56 |
| sbezverk | there is no way to tell what actual software version was used when the image was built | 15:56 |
| kfox1111 | the underlying containers/ppackages usually get updates too. | 15:56 |
| sbezverk | kfox1111: yes, if you yum for it | 15:56 |
| kfox1111 | true. mostly I don't care so long as it passes my tests. | 15:57 |
| sbezverk | but when you curl specific version it does not | 15:57 |
| inc0 | kfox1111, sbezverk one of you has to change length of your nicks;) | 15:57 |
| sbezverk | inc0: sure, could you propose one for me ;-) | 15:57 |
| kfox1111 | inc0: or you could add 0's until your equal length. ;) | 15:57 |
| *** inc0 is now known as inc00000 | 15:58 | |
| inc00000 | hmm | 15:58 |
| inc00000 | make life harder | 15:58 |
| kfox1111 | sbezverk: realy, I think one of our outstanding issues is that we don't have a ci/cd kind of tool to keep our containers up to date. | 15:58 |
| sbezverk | nice, good binary number ;-) | 15:58 |
| kfox1111 | they often need many more upgrades then we can track. | 15:58 |
| *** inc00000 is now known as inc0 | 15:58 | |
| kfox1111 | though that is a general docker/kubernetes problem. | 15:58 |
| kfox1111 | so, thats kind of one thing I want to talk about at some point. | 15:59 |
| kfox1111 | versioning. | 15:59 |
| sbezverk | kfox1111: right I am experiencing it myself now, that is why ibrought it up | 15:59 |
| sbezverk | introducing versions might help us in future | 15:59 |
| sbezverk | but it is optional | 15:59 |
| kfox1111 | so, we version after kolla. in our containers. | 16:00 |
| kfox1111 | so, we have centos-binary-nova-api:2.0.2 or 3.0.1 | 16:00 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:00 |
| sbezverk | after or before, does not matter | 16:00 |
| kfox1111 | that reflects the packaging of the container not the stuff in int. | 16:00 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: Remove non-existent branches in releasenote https://review.openstack.org/399111 | 16:01 |
| openstackgerrit | Javier Castillo AlcÃbar proposed openstack/kolla-ansible: Fix missing libvirt python module in ceilometer-compute (v2) https://review.openstack.org/399113 | 16:01 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: Use mariadb.cnf for mariadb custom file name https://review.openstack.org/399038 | 16:01 |
| kfox1111 | but it doesn't reflect changes in version of the thing people probably care about. nova-api itself. | 16:02 |
| kfox1111 | or none of its dependencies aren't reflected. | 16:02 |
| kfox1111 | in most packaging systems, the version of the base program is used, | 16:02 |
| *** portdirect_away_ has quit IRC | 16:02 | |
| *** absubram has quit IRC | 16:03 | |
| *** ppalacios1 has joined #openstack-kolla | 16:03 | |
| sbezverk | kfox1111: changing what is used now for kolla is impossible, but changing aproach for kolla-kube specific image might be a viable option. | 16:03 |
| kfox1111 | say, nova 14.0.0, 14.0.1, etc. | 16:03 |
| kfox1111 | packaging level changes are done with a revision. | 16:03 |
| *** ppalacios has quit IRC | 16:03 | |
| *** ppalacios1 is now known as ppalacios | 16:03 | |
| sbezverk | since I heard you guis want to keep kube specific images in kolla-kubernetes repo | 16:03 |
| kfox1111 | so, 14.0.1-1, 14.0.1-2, etc. | 16:03 |
| *** zhubingbing_ has quit IRC | 16:04 | |
| kfox1111 | with the way we're doing things tough with packages, we have one more level of stuff that needs versioning. :/ | 16:04 |
| kfox1111 | we have 3 things in flight. | 16:04 |
| kfox1111 | the version of the software, the version of the packaging, and some kind of version for all the other unversioned stuff (dependencies) | 16:05 |
| kfox1111 | so, for example. | 16:05 |
| sbezverk | kfox1111: no need to change it right now, all I asked in is to put FIXME tag to come back later and re-evaluate.. | 16:05 |
| *** rmart04 has quit IRC | 16:05 | |
| kfox1111 | we cut nova-api:14.0.1-1 (specific netruon verson, verson 1 of our build scripts) | 16:06 |
| kfox1111 | then the next openssl volnerability happens and openssl version n+1 comes out. | 16:06 |
| kfox1111 | nova-api doesn't change, and our packaging doesn't change. | 16:06 |
| kfox1111 | so I think we need a build number too. | 16:06 |
| kfox1111 | nova-api:14.0.1-1-1 | 16:06 |
| kfox1111 | so that's for the package builder to increment each time a new container's built or accepted or whatever their procedure is. | 16:07 |
| sbezverk | kfox1111: if we are going towards more kube like images then we should do it | 16:07 |
| kfox1111 | the first one would be nova-api:14.0.1-1-1. when rebuild to handle the openssl security issue, nova-api:14.0.1-1-2 gets cut. | 16:07 |
| *** Pavo has quit IRC | 16:07 | |
| kfox1111 | I think kolla-ansible would benifit from it too. | 16:08 |
| kfox1111 | it has the same issue. | 16:08 |
| kfox1111 | its really a kolla-containers issue | 16:08 |
| kfox1111 | "how do you keep containers up to date and atomic" | 16:09 |
| sbezverk | kfox1111: Oh man, if you try to fix kolla-ansible, it will never be done in our lifetime ;-) let's try to fix for kube for now.. | 16:09 |
| kfox1111 | "and prevent version skews in your system" | 16:09 |
| kfox1111 | yeah. | 16:09 |
| kfox1111 | I think we start with the helm packages. | 16:09 |
| inc0 | -.- | 16:09 |
| *** ayoung has quit IRC | 16:10 | |
| *** ppalacios has quit IRC | 16:10 | |
| sbezverk | kfox1111: right helm should help but there should be support in naming the images too, otherwise it is useless.. | 16:11 |
| *** Pavo has joined #openstack-kolla | 16:12 | |
| kfox1111 | yeah. | 16:12 |
| kfox1111 | I think we can start with versioning the helm packages the way that will be the most flexible. | 16:13 |
| kfox1111 | then work backwards to fix the container tagging. | 16:13 |
| kfox1111 | I think we can make it configurable in the package templates in the mean time too. | 16:13 |
| *** jtriley has quit IRC | 16:14 | |
| *** ppalacios has joined #openstack-kolla | 16:14 | |
| *** jtriley has joined #openstack-kolla | 16:15 | |
| *** adrian_otto has quit IRC | 16:15 | |
| *** fragatina has quit IRC | 16:15 | |
| *** jemcevoy has joined #openstack-kolla | 16:15 | |
| openstackgerrit | Javier Castillo AlcÃbar proposed openstack/kolla-ansible: Fix missing libvirt python module in ceilometer-compute (v2) https://review.openstack.org/399113 | 16:16 |
| *** RichWellum has quit IRC | 16:17 | |
| *** ppalacios1 has joined #openstack-kolla | 16:20 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:20 |
| *** ppalacios1 has quit IRC | 16:20 | |
| inc0 | I have strange bug oO | 16:21 |
| *** ppalacios has quit IRC | 16:21 | |
| inc0 | my build takes very very old registry IP | 16:21 |
| inc0 | and I have no idea from where | 16:21 |
| inc0 | it's in my subnet and there was registry at that address long ago | 16:22 |
| inc0 | but all the configs points to new one | 16:22 |
| inc0 | build is still stupid | 16:22 |
| *** ppalacios has joined #openstack-kolla | 16:22 | |
| openstackgerrit | Merged openstack/kolla-ansible: Remove non-existent branches in releasenote https://review.openstack.org/399111 | 16:25 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible: Squashed commit of the following: https://review.openstack.org/398563 | 16:30 |
| *** fragatina has joined #openstack-kolla | 16:30 | |
| *** ppalacios1 has joined #openstack-kolla | 16:30 | |
| *** eaguilar has quit IRC | 16:30 | |
| egonzalez90 | inc0: container is depenency or dependency? | 16:31 |
| *** ppalacios has quit IRC | 16:31 | |
| *** adrian_otto has joined #openstack-kolla | 16:31 | |
| *** ppalacios1 is now known as ppalacios | 16:31 | |
| inc0 | ahh | 16:32 |
| inc0 | how fun | 16:32 |
| inc0 | made a typo | 16:32 |
| egonzalez90 | ;) | 16:32 |
| inc0 | good catch | 16:32 |
| egonzalez90 | code blocks have the same typo | 16:33 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:33 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Fix gate after the repo split https://review.openstack.org/399050 | 16:33 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:34 |
| inc0 | kfox1111, I refrain of using word "entrypoint" here as it's a container, not entrypoint | 16:35 |
| inc0 | I don't want to confuse people | 16:36 |
| kfox1111 | its the software. that is entrypoint. | 16:36 |
| kfox1111 | where its instantiated, thats deployment-init-container. | 16:37 |
| kfox1111 | thats like renaming nova-api the container, cloud-vm-rest-thingy. | 16:37 |
| kfox1111 | :) | 16:37 |
| inc0 | hmm | 16:37 |
| inc0 | ehh, I used this word in spec, can we roll with it? | 16:38 |
| kfox1111 | I don't think anyone objects to kubernetes-entrypoint the software being a continer. :) | 16:38 |
| inc0 | ok...I can rename it | 16:38 |
| kfox1111 | the contraversy is in how its used. | 16:38 |
| inc0 | I'll call it ensign | 16:38 |
| inc0 | how about that?;) | 16:39 |
| * kfox1111 shrugs. | 16:39 | |
| *** ppalacios has quit IRC | 16:40 | |
| kfox1111 | I like the container name to match up what the daemon running in it is. makes it easier to debug. | 16:40 |
| kfox1111 | at this point, it doens't matter that much though. | 16:40 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:40 |
| inc0 | too late, already changed it | 16:41 |
| kfox1111 | works for me. | 16:41 |
| *** fragatina has quit IRC | 16:41 | |
| *** ppalacios has joined #openstack-kolla | 16:42 | |
| *** ppalacios has quit IRC | 16:42 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:42 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Dependency init container https://review.openstack.org/399082 | 16:44 |
| *** tonanhngo has joined #openstack-kolla | 16:46 | |
| kfox1111 | inc0: so, a work item... either we need a gate job for kolla-kubernetes that builds updated containers from kolla repo, or we need to make an alpha release of that container. (or both) | 16:48 |
| inc0 | kfox1111, we can use zuul-cloner to build images in gates | 16:48 |
| kfox1111 | that woudl work, but would add significant time to the gate jobs. | 16:49 |
| kfox1111 | and make it harder to debug the gate. | 16:49 |
| kfox1111 | as multiple things could be changing at once. | 16:49 |
| inc0 | kfox1111, https://review.openstack.org/#/c/398506/ | 16:50 |
| kfox1111 | right now the kolla-kubernetes job just tests kolla-kubernetes deployment against preexinsting containers. which means they don't randomly break. its a known good ground state. | 16:50 |
| inc0 | long term solution is registry | 16:50 |
| inc0 | if you want to help me work on it | 16:50 |
| inc0 | with infra | 16:50 |
| inc0 | we can jump right to it | 16:50 |
| kfox1111 | inc0: big +1 from me. :) | 16:51 |
| inc0 | I need someone to help me with puppet there:P | 16:51 |
| kfox1111 | that I'm not so good at. :/ | 16:51 |
| kfox1111 | been a few years since I touched puppet, and then it was pretty much just to evalueate it vs other things. | 16:52 |
| inc0 | I take it didn't exactly pass? | 16:52 |
| kfox1111 | we ended up with chef. | 16:52 |
| kfox1111 | but that was partially because at the time the open source community was leaving puppet for chef, because the company behind puppet was doing some bad things. | 16:52 |
| kfox1111 | then the flow reversed once puppet company cleaned up their act. | 16:53 |
| *** DTadrzak has quit IRC | 16:53 | |
| kfox1111 | the chef company kind of did the same thing later and pushed people back twoards puppet. | 16:54 |
| kfox1111 | not sure when the opensource community will really wake up to the problems associated with opensource ruled by one copany. | 16:54 |
| kfox1111 | you'd think the oracle buyout feasco woudl have trained people once and for all. :/ | 16:55 |
| inc0 | well, people feel good when there is someone to call | 16:56 |
| openstackgerrit | Javier Castillo AlcÃbar proposed openstack/kolla-ansible: Fix missing heka/heka-ceilometer.toml in ceilometer_compute https://review.openstack.org/399134 | 16:56 |
| inc0 | look at linux distros | 16:56 |
| kfox1111 | yeah. nothing wrong with that. | 16:56 |
| kfox1111 | its more of a governance issue. | 16:56 |
| inc0 | RHEL, Ubuntu..both run by single company | 16:56 |
| kfox1111 | you need opensource governance with companies for support. | 16:56 |
| kfox1111 | not companies for control. | 16:56 |
| kfox1111 | unless there's no single company controlling it. | 16:57 |
| *** ppalacios has joined #openstack-kolla | 16:57 | |
| *** eaguilar has joined #openstack-kolla | 16:57 | |
| *** Serlex has quit IRC | 16:58 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate https://review.openstack.org/398501 | 17:00 |
| *** eaguilar has quit IRC | 17:02 | |
| *** alteriks has joined #openstack-kolla | 17:02 | |
| *** Satya_ has joined #openstack-kolla | 17:02 | |
| Satya_ | Hi sdake | 17:03 |
| Satya_ | Hi jeffrey | 17:03 |
| *** srwilkers_away is now known as srwilkers | 17:03 | |
| kfox1111 | we need to make a gate job to cache kubernetes stuff too. :/ | 17:04 |
| kfox1111 | they keep breaking the gate. :/ | 17:04 |
| *** egonzalez90 has quit IRC | 17:04 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Fix gate after the repo split https://review.openstack.org/399050 | 17:08 |
| inc0 | kfox1111, I promised sdake to write a bp about moving configs out of ansible | 17:08 |
| inc0 | but I'll write a spec | 17:09 |
| sdake | roger | 17:09 |
| sdake | spec is good | 17:09 |
| sdake | gotta jet | 17:09 |
| sdake | lets just get it done | 17:09 |
| inc0 | yeah, it's too important | 17:09 |
| sdake | and not debate it forever | 17:09 |
| sdake | pls :) | 17:09 |
| inc0 | one spec to solve one well defined problem | 17:09 |
| sdake | gotta jet for the day - so can't particiapte in irc | 17:09 |
| inc0 | have fun | 17:09 |
| sdake | fun isnt' what i'd call it :) | 17:10 |
| sdake | thanks for the support :) | 17:10 |
| kfox1111 | inc0: +1 for a spec. | 17:10 |
| sdake | ttyl see you at 2pmish | 17:10 |
| pbourke | bjolo: finally got single network working | 17:10 |
| *** bmace has quit IRC | 17:10 | |
| *** ayoung has joined #openstack-kolla | 17:10 | |
| pbourke | bjolo: now redeploying with two | 17:10 |
| bjolo | pbourke, nice | 17:10 |
| bjolo | what was the issue? | 17:10 |
| *** bmace has joined #openstack-kolla | 17:10 | |
| pbourke | bjolo: my external interfaces werent up | 17:11 |
| pbourke | bjolo: :/ | 17:11 |
| bjolo | haha classic one | 17:11 |
| *** msimonin has quit IRC | 17:12 | |
| *** adrian_otto has quit IRC | 17:12 | |
| bjolo | open question for the channel. anyone have experience with mellanox spectrum switches and mlnx-os? | 17:13 |
| kfox1111 | xkcd is good today. http://xkcd.com/ | 17:13 |
| kfox1111 | I sware that same thing happend to me 2 weeks ago... :) | 17:13 |
| kfox1111 | why a modern computer still has over/underscanning issues on a 1080p screen is frickin rediculous. | 17:15 |
| kfox1111 | and I have a bios with a gui that won't talk to half the tv's I have. :/ but works fine outside the bios. seriously... | 17:15 |
| kfox1111 | ok. rant over. :) | 17:15 |
| *** strigazi is now known as strigazi_AFK | 17:16 | |
| srwilkers | thats a good one kfox1111 | 17:16 |
| *** Satya_ has quit IRC | 17:17 | |
| pbourke | bjolo: it works! | 17:19 |
| bjolo | no!!! | 17:19 |
| *** fragatina has joined #openstack-kolla | 17:19 | |
| bjolo | dont say that | 17:19 |
| pbourke | bjolo: yeah I know that's not actually what you wanted to hear | 17:19 |
| *** ppalacios1 has joined #openstack-kolla | 17:19 | |
| pbourke | but tbh if it didnt our internal QA would have shouted at us | 17:19 |
| *** ppalacios has quit IRC | 17:20 | |
| bjolo | ok you need to give me some info here | 17:20 |
| pbourke | just to be sure I'm going to boot a VM on the second ext net | 17:20 |
| pbourke | then I'll dump whatever you need | 17:20 |
| bjolo | host os, container os, source/binary | 17:20 |
| bjolo | etc etc | 17:20 |
| pbourke | kolla master | 17:20 |
| kfox1111 | kolla-kubernetes cores: https://review.openstack.org/#/c/399050/ | 17:20 |
| kfox1111 | fixes the gate breakage. | 17:21 |
| bjolo | ok one diff, i run stable/newtwon | 17:21 |
| pbourke | base oraclelinux, type source. Host OS is oraclelinux 7.2, running as kvm guests | 17:21 |
| openstackgerrit | Javier Castillo AlcÃbar proposed openstack/kolla: Fix missing libvirt python module in ceilometer-compute (v2) https://review.openstack.org/399145 | 17:21 |
| bjolo | ok pbourke | 17:21 |
| pbourke | our internal builds use kolla newton with openstack mitaka | 17:21 |
| bjolo | can you do a centos source build on stable/newton and deploy that? | 17:21 |
| pbourke | will do | 17:21 |
| *** ppalacios1 is now known as ppalacios | 17:22 | |
| pbourke | I have to go soon but will kick off a build of stable/newton and deploy in the morning | 17:22 |
| *** yingjun has quit IRC | 17:22 | |
| *** yingjun has joined #openstack-kolla | 17:23 | |
| pbourke | also this is the openvswitch I have in my images: openvswitch-2.5.0-2.el7.x86_64 | 17:23 |
| bjolo | i grab the latest tarballs in my builds | 17:23 |
| bjolo | location = http://tarballs.openstack.org/neutron/neutron-stable-newton.tar.gz | 17:23 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Move config templates to Kolla spec https://review.openstack.org/399147 | 17:23 |
| kfox1111 | can we get that review merged asap? | 17:23 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use kolla-ansible for deployment https://review.openstack.org/398506 | 17:23 |
| inc0 | Jeffrey4l, do you know puppet? | 17:24 |
| Jeffrey4l | inc0, a little. | 17:24 |
| inc0 | we need a module to install docker registry with authentication on ubuntu | 17:24 |
| inc0 | I can try doint parts of it, but I have no diea how puppet works | 17:24 |
| Jeffrey4l | why not use ansible ;) | 17:25 |
| inc0 | because it's infra | 17:25 |
| inc0 | and openstack-infra uses puppet | 17:25 |
| bjolo | pbourke, tnx | 17:25 |
| Jeffrey4l | inc0, hmm. sorry no idea for this ;( | 17:25 |
| inc0 | kk, just asking around;) | 17:26 |
| bjolo | i got to go. svimclass for the big girl | 17:26 |
| inc0 | I'm afraid to ask bjolo | 17:26 |
| inc0 | well...have fun | 17:26 |
| bjolo | haha inc0 | 17:26 |
| bjolo | dont let your imagination run wild | 17:26 |
| inc0 | yeah, vim is a strange fetish | 17:27 |
| *** tonanhngo has quit IRC | 17:27 | |
| inc0 | "change mode to insert now!" | 17:27 |
| *** yingjun has quit IRC | 17:27 | |
| *** fragatina has quit IRC | 17:29 | |
| *** fragatina has joined #openstack-kolla | 17:30 | |
| *** TxGirlGeek has joined #openstack-kolla | 17:31 | |
| srwilkers | lol | 17:32 |
| inc0 | yes, jokes which causes instant silence in channel are...concerning | 17:32 |
| inc0 | sorry about that | 17:32 |
| pbourke | you should hear SamYaple's jokes | 17:33 |
| inc0 | oh I've heard those | 17:33 |
| pbourke | i call them dad jokes | 17:34 |
| pbourke | which is ok because he actually is a father :p | 17:34 |
| rhallisey | kfox1111, roger that | 17:34 |
| kfox1111 | rhallisey: thx. | 17:34 |
| inc0 | rhallisey, https://review.openstack.org/#/c/399147/1 | 17:35 |
| inc0 | just in case you thought you're done with specs for now | 17:35 |
| rhallisey | kk | 17:35 |
| rhallisey | what's the time block for that? | 17:36 |
| rhallisey | let's keep it to like 2 days | 17:36 |
| inc0 | nooooo | 17:36 |
| inc0 | this takes as much time as it needs | 17:36 |
| kfox1111 | yeah. the config bits are important, but uncontravertial I think. | 17:37 |
| kfox1111 | I doubt it will drag on for longer then needed. | 17:37 |
| kfox1111 | but getting it right is important. | 17:37 |
| inc0 | yeah, it's about "how to make it work without breaking stuff" | 17:37 |
| * kfox1111 nods | 17:37 | |
| kfox1111 | and while we're at it, fixing some design issues. since we're in the code already. | 17:37 |
| * inc0 nods | 17:38 | |
| pbourke | inc0: can/should we issue a note to cores to not merge stuff till the gates are fixed up | 17:40 |
| kfox1111 | pbourke: +1 | 17:40 |
| kfox1111 | that provides incentive to fix the gates. | 17:40 |
| kfox1111 | and not futher break stuff. | 17:40 |
| inc0 | I'd hope that we don't have to make note like that | 17:40 |
| kfox1111 | making it harder to fix the gates. :) | 17:40 |
| pbourke | well im seeing +2's go on things | 17:41 |
| pbourke | maybe people know not to merge but I think no harm in making that explicit | 17:41 |
| *** adrian_otto has joined #openstack-kolla | 17:41 | |
| kfox1111 | +1 | 17:41 |
| kfox1111 | too much communication is better then not enough. | 17:41 |
| pbourke | right | 17:41 |
| *** unicell has quit IRC | 17:41 | |
| inc0 | I don't want to send mail to ML with "dont merge if gates are red"... | 17:41 |
| pbourke | i guess we'll just keep an eye on it then | 17:42 |
| kfox1111 | we don't have a way to email cores? | 17:42 |
| pbourke | im looking now and nothing seems to have merged since the split so thats good | 17:42 |
| inc0 | yeah I hope we get out of gate crysis soon enough | 17:43 |
| inc0 | this is short lived crisis I'd hope | 17:43 |
| pbourke | the gates can run crysis? | 17:43 |
| inc0 | yeah | 17:43 |
| inc0 | can't run that is | 17:43 |
| pbourke | that was my bad joke for the day | 17:44 |
| *** goldyfruit has quit IRC | 17:44 | |
| jascott1 | only with all the settings on low | 17:44 |
| inc0 | ahh...well, you're not English so you have no excuse for bad jokes....sorry mate | 17:44 |
| kfox1111 | rhallisey: https://review.openstack.org/#/c/399050/ workflow please? the experimental tests are still running, but they long pass the point of the thing fixed. we're good there. | 17:45 |
| inc0 | now when I think about it, I don't think that we have any Englishman around here | 17:45 |
| rhallisey | kfox1111, ya I waiting for them to come back | 17:46 |
| kfox1111 | looks like it wil finish any second though. | 17:46 |
| rhallisey | kk | 17:46 |
| kfox1111 | all but the multinode already passed. the multinode one takes a couple minutes longer. | 17:47 |
| *** DaveTurner has joined #openstack-kolla | 17:47 | |
| pbourke | we do | 17:47 |
| pbourke | trying to remember his name | 17:47 |
| *** Satya_ has joined #openstack-kolla | 17:48 | |
| *** msimonin has joined #openstack-kolla | 17:49 | |
| kfox1111 | rhallisey: passed. :) | 17:51 |
| rhallisey | kk | 17:51 |
| *** msimonin has quit IRC | 17:51 | |
| inc0 | rhallisey, https://review.openstack.org/#/c/399082/ | 17:51 |
| *** goldyfruit has joined #openstack-kolla | 17:51 | |
| kfox1111 | rhallisey: thanks. :) | 17:51 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Fix gate after the repo split https://review.openstack.org/399050 | 17:53 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 17:54 |
| *** alteriks has quit IRC | 17:54 | |
| *** ayoung has quit IRC | 17:55 | |
| *** msimonin has joined #openstack-kolla | 17:56 | |
| *** alteriks has joined #openstack-kolla | 17:56 | |
| *** narasimah_SV has quit IRC | 17:56 | |
| *** derekjhyang has joined #openstack-kolla | 17:58 | |
| *** castulo has joined #openstack-kolla | 17:58 | |
| *** alteriks has quit IRC | 18:00 | |
| *** schwicht has quit IRC | 18:01 | |
| jascott1 | helm 2.0 is released https://deis.com/blog/2016/helm-is-now-stable/ | 18:01 |
| mliima | guys, need some reviews here https://review.openstack.org/#/c/398516/ | 18:04 |
| *** msimonin has quit IRC | 18:04 | |
| *** fragatin_ has joined #openstack-kolla | 18:04 | |
| kfox1111 | yay! :) | 18:05 |
| *** tonanhngo has joined #openstack-kolla | 18:07 | |
| *** fragatina has quit IRC | 18:07 | |
| *** Pavo has quit IRC | 18:07 | |
| *** harbor has joined #openstack-kolla | 18:09 | |
| *** harbor is now known as portdirect_away_ | 18:09 | |
| *** portdirect_away_ has quit IRC | 18:09 | |
| sbezverk | kfox1111: curious if there a procedure to update helm rc2 to final release | 18:09 |
| *** tonanhngo_ has joined #openstack-kolla | 18:10 | |
| openstackgerrit | Merged openstack/kolla: Add how configure multiple back ends section https://review.openstack.org/398516 | 18:10 |
| *** eaguilar has joined #openstack-kolla | 18:11 | |
| *** tonanhngo_ has quit IRC | 18:11 | |
| *** tonanhngo has quit IRC | 18:11 | |
| *** tonanhngo has joined #openstack-kolla | 18:11 | |
| *** pbourke has quit IRC | 18:12 | |
| *** Pavo has joined #openstack-kolla | 18:12 | |
| *** pbourke has joined #openstack-kolla | 18:12 | |
| *** tonanhngo has quit IRC | 18:13 | |
| *** TxGirlGeek has quit IRC | 18:14 | |
| *** TxGirlGeek has joined #openstack-kolla | 18:14 | |
| *** tonanhngo has joined #openstack-kolla | 18:15 | |
| jascott1 | sbezverk I think you just deploy new binary and upgrade tiller https://github.com/kubernetes/helm/blob/master/docs/install.md#deleting-or-reinstalling-tiller | 18:15 |
| sbezverk | kfox1111: never mind, figured out. you just need to edit tiller-deploy to point to the latest version.. | 18:15 |
| jascott1 | haha | 18:15 |
| *** eaguilar has quit IRC | 18:15 | |
| *** unicell has joined #openstack-kolla | 18:15 | |
| *** tonanhngo has quit IRC | 18:17 | |
| qwang | jascott1: Do you know how to completely remove a release from helm? | 18:17 |
| jascott1 | helm delete <release> ? | 18:17 |
| jascott1 | ^ thats supposed to work | 18:18 |
| *** ppalacios has quit IRC | 18:18 | |
| qwang | jascott1: I used helm delete, but when I try to create another release with the same name, helm won't let me do it | 18:18 |
| jascott1 | it wont | 18:18 |
| jascott1 | it keeps the release history | 18:18 |
| *** ppalacios has joined #openstack-kolla | 18:18 | |
| jascott1 | I never use named releases | 18:18 |
| jascott1 | you can helm list all and see one sec | 18:18 |
| jascott1 | helm list --all | 18:19 |
| qwang | I can see the old releases with status "deleted" | 18:19 |
| *** papacz has quit IRC | 18:21 | |
| jascott1 | ya you either need to bump a version with your same release name or use random names | 18:21 |
| *** ayoung has joined #openstack-kolla | 18:21 | |
| jascott1 | I just let it create names. i install and delete so much I created an alias: hda="helm list -q | xargs helm delete" | 18:22 |
| jascott1 | doesnt help your issue but makes it easy to remove and not care about random names | 18:22 |
| qwang | jascott1: thx | 18:23 |
| kfox1111 | I think we're going to want to use named releases... | 18:24 |
| kfox1111 | do you know if thye fixed the really short release name max? | 18:24 |
| jascott1 | they did | 18:24 |
| jascott1 | its 53 max chars now | 18:25 |
| kfox1111 | whats the limit know? | 18:25 |
| kfox1111 | ah. nice. that should be enough. :) | 18:25 |
| kfox1111 | thanks. | 18:25 |
| kfox1111 | so we have to upgrade the gate to the final release too... | 18:25 |
| *** strigazi_AFK is now known as strigazi | 18:33 | |
| *** Satya_ has quit IRC | 18:34 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible: Tacker NFV Ansible support https://review.openstack.org/399179 | 18:34 |
| *** Satya_ has joined #openstack-kolla | 18:34 | |
| bmace | I think the overview on https://launchpad.net/kolla-ansible needs to be updated since it is actually active now. | 18:35 |
| *** coolsvap has quit IRC | 18:37 | |
| *** krtaylor has quit IRC | 18:38 | |
| qwang | +1 | 18:39 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Tacker global variables https://review.openstack.org/389790 | 18:39 |
| *** newmember has joined #openstack-kolla | 18:39 | |
| *** egonzalez90 has joined #openstack-kolla | 18:40 | |
| *** sdake has quit IRC | 18:44 | |
| *** msimonin has joined #openstack-kolla | 18:48 | |
| *** diogogmt has quit IRC | 18:55 | |
| *** adrian_otto has quit IRC | 18:56 | |
| *** diogogmt has joined #openstack-kolla | 18:57 | |
| *** shardy has quit IRC | 18:57 | |
| *** gfidente has quit IRC | 19:00 | |
| *** fragatin_ has quit IRC | 19:01 | |
| *** fragatina has joined #openstack-kolla | 19:02 | |
| *** diogogmt has quit IRC | 19:02 | |
| *** eaguilar has joined #openstack-kolla | 19:03 | |
| *** eaguilar has quit IRC | 19:07 | |
| *** ppalacios1 has joined #openstack-kolla | 19:13 | |
| *** ppalacios1 has quit IRC | 19:13 | |
| *** msimonin has quit IRC | 19:14 | |
| *** msimonin has joined #openstack-kolla | 19:14 | |
| *** ppalacios has quit IRC | 19:14 | |
| *** ppalacios has joined #openstack-kolla | 19:15 | |
| *** msimonin has quit IRC | 19:17 | |
| *** schwicht has joined #openstack-kolla | 19:18 | |
| *** tonanhngo has joined #openstack-kolla | 19:19 | |
| *** eaguilar has joined #openstack-kolla | 19:25 | |
| *** strigazi is now known as strigazi_AFK | 19:25 | |
| *** Satya_ has quit IRC | 19:29 | |
| *** eaguilar has quit IRC | 19:30 | |
| kfox1111 | arg... | 19:31 |
| *** v1k0d3n has joined #openstack-kolla | 19:31 | |
| kfox1111 | the world really needs a shell with better variable manipulation. | 19:31 |
| kfox1111 | python's painful for automating shell like tasks but good at data manipulation. bash, the reverse. :/ | 19:31 |
| *** severion has joined #openstack-kolla | 19:31 | |
| *** msimonin has joined #openstack-kolla | 19:32 | |
| *** v1k0d3n has quit IRC | 19:35 | |
| *** awiddersheim has joined #openstack-kolla | 19:36 | |
| *** severion has quit IRC | 19:36 | |
| *** v1k0d3n has joined #openstack-kolla | 19:36 | |
| v1k0d3n | hey all | 19:37 |
| srwilkers | hey v1k0d3n | 19:37 |
| v1k0d3n | yo | 19:37 |
| v1k0d3n | finally. a "real" internet connection at the office. | 19:37 |
| v1k0d3n | bout darn time! | 19:37 |
| *** mgiles has quit IRC | 19:38 | |
| srwilkers | time to leave the matrix | 19:39 |
| *** castulo has left #openstack-kolla | 19:39 | |
| sean-k-mooney | hi just a 2 second dumb question. with the repo split for https://review.openstack.org/#/c/342354/ the anible change go to kolla-ansible, docker chages to kolla. where do the docs chages go? | 19:40 |
| sean-k-mooney | kolla-ansible? | 19:40 |
| openstackgerrit | BorneMace proposed openstack/kolla-ansible: Ignore dom0 qemu processes during destroy https://review.openstack.org/399203 | 19:40 |
| *** krtaylor has joined #openstack-kolla | 19:42 | |
| inc0 | sean-k-mooney, not a dumb question, docs goes to kolla | 19:42 |
| inc0 | for now | 19:42 |
| kfox1111 | v1k0d3n: hi | 19:42 |
| sean-k-mooney | ok cool i was going to break this up into different patches anyway so this is as good a time as any to do it. | 19:43 |
| sean-k-mooney | inc0: regarding your extensible ansible change. i think we might want to look at prototyping something with this review also https://review.openstack.org/#/c/387854/ | 19:44 |
| *** openstackgerrit has quit IRC | 19:48 | |
| *** openstackgerrit has joined #openstack-kolla | 19:48 | |
| *** ppalacios has quit IRC | 19:51 | |
| *** ppalacios has joined #openstack-kolla | 19:57 | |
| *** sbezverk_ has joined #openstack-kolla | 20:01 | |
| egonzalez90 | sean-k-mooney inc0 thats a woot for kolla? | 20:02 |
| *** sbezverk has quit IRC | 20:02 | |
| inc0 | interesting | 20:03 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 20:04 |
| sean-k-mooney | egonzalez90: that being? | 20:04 |
| inc0 | v1k0d3n, did you switch internet from ATnT to Comcast? | 20:04 |
| kfox1111 | lets see how this flies... different way of doing helm... | 20:04 |
| v1k0d3n | inc0: ha...ha. so funny. mr. fun guy. | 20:04 |
| v1k0d3n | :) | 20:04 |
| inc0 | so I have inet from ATT now, but since I'm moving, will have it no longer | 20:05 |
| inc0 | no ATT coverage where I'm going | 20:05 |
| egonzalez90 | sean-k-mooney for collectd people using kolla and showing docs how to integrate | 20:05 |
| jascott1 | kfox1111 what did you change with approach? | 20:05 |
| sean-k-mooney | inc0: can you get good broadband like google fiber? | 20:05 |
| kfox1111 | yeah. I think att still only offers 1.5 mb here. | 20:05 |
| egonzalez90 | think is a good step | 20:05 |
| kfox1111 | jascott1: more in line with what we were talking about yesterday. building a package per k8s object. | 20:06 |
| sean-k-mooney | egonzalez90: ah well they work on my team and i like kolla so asked them to add support | 20:06 |
| jascott1 | cool will check it out | 20:06 |
| egonzalez90 | cool :) | 20:06 |
| inc0 | sean-k-mooney, not yet | 20:06 |
| inc0 | they arrive to San Antonio but Im moving out | 20:07 |
| *** Pavo has quit IRC | 20:07 | |
| kfox1111 | jascott1: thanks. plz let me know what you think. | 20:08 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 20:09 |
| inc0 | https://github.com/openstack-infra/puppet-docker_registry | 20:10 |
| *** Pavo has joined #openstack-kolla | 20:12 | |
| kfox1111 | inc0: nice. | 20:14 |
| kfox1111 | there a pr going to follow/contribute to? | 20:14 |
| inc0 | we need that think filled with goodness | 20:14 |
| inc0 | https://review.openstack.org/#/c/399221/ kfox1111 it's normal openstack project | 20:14 |
| inc0 | when we prep this module | 20:15 |
| kfox1111 | and another question... I know its not exactly how they normally do it. but should we deploy it with docker? | 20:15 |
| inc0 | no, they want it to be standalone | 20:15 |
| kfox1111 | have the puppet module load docker / kubelet standalone, then just run it as a container? | 20:15 |
| inc0 | and use their existing apache for auth | 20:15 |
| kfox1111 | ah. | 20:15 |
| inc0 | that being said | 20:15 |
| v1k0d3n | inc0: makes total sense. we moved down the street and don't have att. | 20:16 |
| inc0 | standalone registry is literally apt-get install docker-registry | 20:16 |
| v1k0d3n | when we had it was aweosme though | 20:16 |
| kfox1111 | could still container it and tehn use the existing apache to auth/proxy. | 20:16 |
| inc0 | v1k0d3n, yeah, I'm actually quite happy | 20:16 |
| inc0 | a bit overpriced here, but meh, wild west | 20:16 |
| kfox1111 | ah. ok. didn't think they had dpkg packages for the registry | 20:16 |
| inc0 | kfox1111, issue is nobody wants to run docker deamon there;) | 20:16 |
| inc0 | also registry is golang blob, we can just curl and run it | 20:17 |
| inc0 | easier than setting up full docker infra | 20:17 |
| pprokop | qwang: try helm delete --purge | 20:17 |
| pprokop | it's like docker rm -f | 20:18 |
| qwang | pprokop: thanks. Just tried, it works like a charm :) | 20:20 |
| kfox1111 | ~. | 20:20 |
| kfox1111 | ~> | 20:20 |
| qwang | pprokop: I have another question. Are livenessprobe and readinessprobe usually the same? I mean the code part, I know they are functional differently | 20:21 |
| *** newmember has quit IRC | 20:22 | |
| *** reek269 has joined #openstack-kolla | 20:22 | |
| *** ppalacios has quit IRC | 20:22 | |
| rhallisey | qwang, liveness is to check the pod is behaving. Readiness is to check the pod is ready to start | 20:27 |
| *** adrian_otto has joined #openstack-kolla | 20:27 | |
| *** egonzalez90 has quit IRC | 20:28 | |
| *** ppalacios has joined #openstack-kolla | 20:28 | |
| *** eaguilar has joined #openstack-kolla | 20:29 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 20:29 |
| kfox1111 | rhallisey: I think readyness checks continuously run? | 20:33 |
| kfox1111 | and liveness probe is a readyness probe that shoots the pod on failure. | 20:33 |
| *** eaguilar has quit IRC | 20:33 | |
| rhallisey | http://kubernetes.io/docs/user-guide/production-pods/#liveness-and-readiness-probes-aka-health-checks | 20:35 |
| kfox1111 | yeah, the docs are less then clear last I read them. | 20:36 |
| kfox1111 | I was under the impression, | 20:36 |
| kfox1111 | a pod could go unready if it passed a check later on. | 20:37 |
| kfox1111 | it pulls it back out of the lb but leaves the pod alive. | 20:37 |
| rhallisey | they changed the docs | 20:37 |
| rhallisey | :/ | 20:37 |
| rhallisey | a bunch | 20:37 |
| rhallisey | Kubernetes supports the (optional) specification of a pre-stop lifecycle hook, which will execute prior to sending SIGTERM. | 20:37 |
| *** schwicht has quit IRC | 20:37 | |
| *** awiddersheim has quit IRC | 20:37 | |
| kfox1111 | rhallisey: yeah. I use prestop with my compute clusters. | 20:37 |
| rhallisey | nice | 20:38 |
| kfox1111 | lets the compute container drain the job safely before termination. | 20:38 |
| kfox1111 | you gota change the default destroy timeout too. | 20:38 |
| rhallisey | the name operator gets another strike.... operator.py is a base module... | 20:38 |
| kfox1111 | it defaults to 30 seconds. wich is way to small. | 20:38 |
| *** ppalacios has quit IRC | 20:38 | |
| rhallisey | ya | 20:38 |
| kfox1111 | the ugly in it is its per container, not per pod. | 20:38 |
| kfox1111 | so you have to build some orchestration logic between containers in a pod if they need to work together during shutdown. | 20:39 |
| kfox1111 | see our common-deployment yaml. | 20:39 |
| kfox1111 | has some of that in it. | 20:39 |
| rhallisey | ya all pieces to the puzzle | 20:40 |
| kfox1111 | I put in a request of a way to make the prestop hook blocking for all containers rather then just the one its attached to. | 20:40 |
| kfox1111 | we'l see if t hat goes through. | 20:40 |
| kfox1111 | doh.. | 20:41 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 20:42 |
| *** eaguilar has joined #openstack-kolla | 20:45 | |
| *** sdake has joined #openstack-kolla | 20:46 | |
| sdake | sup cats | 20:46 |
| rhallisey | sup | 20:47 |
| inc0 | need someone with puppet knowledge | 20:47 |
| inc0 | for profit of kolla | 20:47 |
| *** kbyrne has quit IRC | 20:49 | |
| *** kbyrne has joined #openstack-kolla | 20:52 | |
| *** severion has joined #openstack-kolla | 20:57 | |
| severion | hey sdake | 20:57 |
| severion | crap one sec. | 20:57 |
| *** severion has quit IRC | 20:57 | |
| *** severion has joined #openstack-kolla | 20:57 | |
| severion | btw...severion is v1k0d3n away :) | 20:57 |
| *** v1k0d3n has quit IRC | 20:58 | |
| *** ayoung has quit IRC | 20:59 | |
| *** v1k0d3n has joined #openstack-kolla | 21:05 | |
| kfox1111 | there we go.... | 21:05 |
| kfox1111 | helm install kolla/neutron-l3-agent --version 2.0.2-1 --set enable_kube_logger=false,type=network --namespace kolla --name neutron-l3-agent-network | 21:05 |
| *** severion has quit IRC | 21:06 | |
| jemcevoy | sean-k-mooney, Are you at Intel in Hillsboro? There is a OpenStack PDX meetup at Puppet tonight at 6pm... | 21:09 |
| kfox1111 | k. now to try a second microservice and a service package. | 21:09 |
| inc0 | jemcevoy, he's pretty far away, Ireland | 21:11 |
| inc0 | I'm moving to Hillsboro in a month tho:) | 21:12 |
| inc0 | so I'll be around these meetups | 21:12 |
| jemcevoy | Excellent... | 21:12 |
| jemcevoy | inc0, where are you moving from? | 21:13 |
| inc0 | san antonio | 21:13 |
| jemcevoy | My inlaws were in Boerne but have passed on... | 21:14 |
| *** sdake_ has joined #openstack-kolla | 21:23 | |
| inc0 | afk, need to go to store | 21:24 |
| *** inc0 has quit IRC | 21:24 | |
| *** severion has joined #openstack-kolla | 21:25 | |
| *** msimonin has quit IRC | 21:26 | |
| *** schwicht has joined #openstack-kolla | 21:26 | |
| *** sdake has quit IRC | 21:27 | |
| kfox1111 | adding a second tempalte wan't too hard... | 21:27 |
| *** jtriley_ has joined #openstack-kolla | 21:27 | |
| *** v1k0d3n has quit IRC | 21:27 | |
| *** goldyfruit has quit IRC | 21:27 | |
| *** goldyfruit has joined #openstack-kolla | 21:28 | |
| *** jtriley has quit IRC | 21:30 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 21:32 |
| *** severion has quit IRC | 21:32 | |
| *** jtriley_ has quit IRC | 21:32 | |
| *** fguillot has quit IRC | 21:33 | |
| *** v1k0d3n has joined #openstack-kolla | 21:33 | |
| *** Jeffrey4l has quit IRC | 21:33 | |
| *** msimonin has joined #openstack-kolla | 21:45 | |
| *** msimonin has quit IRC | 21:45 | |
| *** msimonin1 has joined #openstack-kolla | 21:45 | |
| *** v1k0d3n has quit IRC | 21:46 | |
| *** schwicht has quit IRC | 21:49 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Kubernetes Operator base https://review.openstack.org/399262 | 21:56 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Mariadb Kubernetes Operator https://review.openstack.org/399263 | 21:56 |
| *** egonzalez90 has joined #openstack-kolla | 21:59 | |
| *** TxGirlGeek has quit IRC | 22:00 | |
| kfox1111 | wow.. that kind of seemed to work... | 22:05 |
| *** lrensing has quit IRC | 22:05 | |
| *** Pavo has quit IRC | 22:07 | |
| *** srwilkers has quit IRC | 22:07 | |
| *** Pavo has joined #openstack-kolla | 22:08 | |
| *** jheroux has quit IRC | 22:08 | |
| kfox1111 | so, building a meta package mostly worked. | 22:09 |
| kfox1111 | the vars stuff isn't quite working though... weird. | 22:09 |
| *** rhallisey has quit IRC | 22:10 | |
| *** rhallisey has joined #openstack-kolla | 22:10 | |
| kfox1111 | hmm... | 22:10 |
| kfox1111 | though I can see a way this might never work without operators... :/ | 22:11 |
| kfox1111 | you'd want an instance that had one neutron-server but maybe multiple l3 agents subinstances. | 22:12 |
| *** TxGirlGeek has joined #openstack-kolla | 22:16 | |
| *** rhallisey has quit IRC | 22:17 | |
| sdake_ | kfox1111 now we have to prove that operators actually work :) | 22:17 |
| kfox1111 | yeah. | 22:18 |
| kfox1111 | you may want different sets of instances too, depending on if you want dvr enabled or not. and how many nova compute types you have... | 22:19 |
| *** inc0 has joined #openstack-kolla | 22:23 | |
| *** srwilker has joined #openstack-kolla | 22:27 | |
| kfox1111 | hmm.. and this time it rendered totally wrong. :/ | 22:27 |
| kfox1111 | seems like this is racy. :/ | 22:27 |
| inc0 | kfox1111, helm? | 22:27 |
| inc0 | how template engine can be racy? | 22:27 |
| *** krtaylor has quit IRC | 22:30 | |
| kfox1111 | helm. | 22:31 |
| kfox1111 | the way I'm doing macros... | 22:32 |
| kfox1111 | it orders them in a random order. | 22:32 |
| kfox1111 | and I had some macros with the same name. | 22:32 |
| kfox1111 | which works for independent packages, | 22:32 |
| kfox1111 | but when you merge in the working packages into a nested package, it breaks. | 22:32 |
| inc0 | I blame goroutines | 22:34 |
| sdake_ | inc0 good thing to blame | 22:34 |
| sdake_ | inc0 threads re an abomination | 22:34 |
| kfox1111 | python might have the same issue. | 22:34 |
| sdake_ | python is single thread with eventlet | 22:34 |
| kfox1111 | just throwing it all in a hash, then processing htem. | 22:34 |
| inc0 | kfox1111, python isn't pushing multithreading as much | 22:34 |
| kfox1111 | thats pretty random too. | 22:34 |
| kfox1111 | I'm guessing its a hashing issue. | 22:34 |
| sdake_ | kfox1111 posissble | 22:35 |
| inc0 | then every time it would be the same | 22:35 |
| kfox1111 | it outputs linearlyk, so I doubt it uses a goroutine. | 22:35 |
| inc0 | strange, but same strange | 22:35 |
| kfox1111 | inc0: no. python does the same. load a json document in python and print it out. | 22:35 |
| kfox1111 | it comes out random ordered. | 22:35 |
| inc0 | but order will always be the same | 22:35 |
| inc0 | not random | 22:36 |
| kfox1111 | not nessisarily. | 22:36 |
| inc0 | not same as input, but 100 times you run it, it'll always came out same | 22:36 |
| kfox1111 | I don't belive thats garanteed. | 22:36 |
| kfox1111 | likely, but not garanteed. | 22:36 |
| inc0 | hash doesn't change no matter how much you run it | 22:36 |
| sdake_ | depends on what else gets hashed with it | 22:37 |
| kfox1111 | the contents don't but the memory struture has no garantee. | 22:37 |
| sdake_ | if the ordering is inconsistent sounds like a bug to me | 22:37 |
| inc0 | ok, if you use timestamp as key then meh | 22:37 |
| kfox1111 | anyway. doesn't matter. it has no well defined behavior here, so I have to do something differnt.. | 22:38 |
| kfox1111 | should be able to just extend things a little bit and just ensure I don't use the same macroname twice. | 22:38 |
| *** reek269 has quit IRC | 22:38 | |
| kfox1111 | ok... got it to work. | 22:41 |
| kfox1111 | so, this concept might work for things that don't need multiple subinstances. | 22:41 |
| kfox1111 | I'll contribute the code, and those wanting to play with entrypoint only can run with it. | 22:41 |
| kfox1111 | and the operator folks can use the base packages. | 22:42 |
| *** TxGirlGeek has quit IRC | 22:43 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 22:43 |
| inc0 | one more +2 and we have voting gates https://review.openstack.org/#/c/398534/ << runs away >> | 22:46 |
| sdake_ | running is appropriate | 22:46 |
| sdake_ | inc0 which repo btw | 22:47 |
| sdake_ | kolla repo? | 22:47 |
| inc0 | yes | 22:47 |
| inc0 | source builds for major distros | 22:47 |
| sdake_ | cool | 22:47 |
| sdake_ | i'd leave binary nonvoting for now ;) | 22:47 |
| kfox1111 | inc0: nice. | 22:47 |
| inc0 | yeah | 22:47 |
| kfox1111 | we may want to do the same for kolla-kubernetes gates soon. | 22:47 |
| kfox1111 | they seem pretty stable. | 22:48 |
| sdake_ | maybe dmsimard can pull a rabbit out of his hat to make sure we have cross repo gating | 22:48 |
| inc0 | source builds aren't that volitale | 22:48 |
| sdake_ | then voting binary could be a thing | 22:48 |
| inc0 | speaking of dmsimard, 3rd party CI for RHEL would be awesome;) | 22:48 |
| *** f13o has quit IRC | 22:52 | |
| *** v1k0d3n has joined #openstack-kolla | 22:54 | |
| kfox1111 | ok.... I think this will fly. | 23:04 |
| kfox1111 | its passing all the tests. | 23:04 |
| kfox1111 | should get more eyes on it. | 23:04 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP helm https://review.openstack.org/396296 | 23:05 |
| openstackgerrit | BorneMace proposed openstack/kolla-ansible: Added support for a stop playbook in kolla https://review.openstack.org/399289 | 23:06 |
| *** goldyfruit has quit IRC | 23:10 | |
| *** ayoung has joined #openstack-kolla | 23:10 | |
| *** egonzalez90 has quit IRC | 23:11 | |
| *** liyifeng has joined #openstack-kolla | 23:14 | |
| kfox1111 | his is ready for a workflow: https://review.openstack.org/#/c/396812/ | 23:15 |
| inc0 | I'm going off, have a good one guys | 23:17 |
| alanmeadows | we're piloting a ceph chart which may be of interest: https://github.com/att-comdev/aic-helm | 23:17 |
| *** inc0 has quit IRC | 23:17 | |
| kfox1111 | v1k0d3n: have a gerrit account? | 23:19 |
| kfox1111 | alanmeadows: is this another project to openstack on k8s? | 23:21 |
| v1k0d3n | yup | 23:22 |
| v1k0d3n | oh i mean yes to kfox1111 your questions. | 23:23 |
| v1k0d3n | question...man i can't type or even think today. | 23:23 |
| v1k0d3n | so kfox1111 this isn't meant to compete at all with kolla... | 23:23 |
| v1k0d3n | we're using this to test out, make sure it's solid...and plan to contribute and push this all into kolla. | 23:23 |
| *** sbezverk has joined #openstack-kolla | 23:24 | |
| v1k0d3n | at the end of the day for us...kolla is the be-all...that's where it's at. | 23:24 |
| kfox1111 | nice. k. | 23:24 |
| kfox1111 | though the traditional workflow for openstack projects has been to develop in tree in regular reviews. | 23:24 |
| v1k0d3n | we will obviously have our own deployment, but if kolla is an agnostic view of what we need....this is desired for us. | 23:25 |
| kfox1111 | makes it much easier to review/merge the stuff. | 23:25 |
| v1k0d3n | i get it totally. | 23:25 |
| v1k0d3n | we're engaging other resources though too, who are not openstack. | 23:25 |
| v1k0d3n | so they can look at github a little easier and quicker than gerrit | 23:25 |
| kfox1111 | ah. sure. | 23:25 |
| v1k0d3n | i'll be honest...not a gerrit whiz. github or a gitflow workflow...i feel right at home. | 23:26 |
| kfox1111 | so, I'd recomend ensuring reviews are going as soon as pssible on the kolla side. its been really hard to deal with large code drops. :/ | 23:26 |
| *** sbezverk_ has quit IRC | 23:27 | |
| kfox1111 | yeah. its a bit different between the two. | 23:27 |
| kfox1111 | I kind of prefer gerrit after having used both. | 23:27 |
| v1k0d3n | i plan on doing reviews of your charts though. have just been caught up with setting up our development lab. after next week should be much, much better. | 23:27 |
| v1k0d3n | well... gerrit is amazing for review process. | 23:27 |
| v1k0d3n | this part....is awesome. anyone can jump in and help. | 23:27 |
| v1k0d3n | and voting system is great too. | 23:28 |
| kfox1111 | v1k0d3n: cool. if you can review https://review.openstack.org/#/c/396296/ and have some other folks look at it, that woudl be great. | 23:28 |
| v1k0d3n | bottom line, it's perfect for openstack. | 23:28 |
| v1k0d3n | was going to pull tonight and have a look. | 23:28 |
| kfox1111 | I think its an ok attempt, and deploys, but not sure if there will be other issues with it. | 23:28 |
| kfox1111 | cool. thanks. :) | 23:29 |
| v1k0d3n | kfox1111: remember we were talking about a "seed host" concept using ironic/bifrost? | 23:29 |
| kfox1111 | I'm a little surprised k8s doesn't use gerrit. | 23:29 |
| kfox1111 | considering google wrote it. | 23:29 |
| kfox1111 | v1k0d3n: yeah. | 23:29 |
| kfox1111 | I see maas mentioned there. which is kind of a compeditor to ironic as far as I understand it. | 23:30 |
| v1k0d3n | we're not a huge ironic shop. we ended up writing this ( alanmeadows did ) https://github.com/att-comdev/dockerfiles/tree/master/maas | 23:30 |
| kfox1111 | and ubuntu specific. | 23:30 |
| v1k0d3n | now this ^^ is really cool. been using it today. | 23:30 |
| kfox1111 | yeah. its interesting. but I probably wont/cant use it. :/ | 23:31 |
| *** ntpttr has quit IRC | 23:31 | |
| *** Jeffrey4l has joined #openstack-kolla | 23:31 | |
| kfox1111 | my goals to be able to be able to have an undercloud that can be used to deploy vm's on, | 23:31 |
| kfox1111 | or let the users deploy bare metal stuff. | 23:31 |
| *** ntpttr has joined #openstack-kolla | 23:31 | |
| kfox1111 | the only tool I know of that can do that is ironic. :/ | 23:31 |
| kfox1111 | where ironic has a tenant for the overcloud. | 23:32 |
| kfox1111 | and then the rest of the normal tenants can use it too. | 23:32 |
| *** krtaylor has joined #openstack-kolla | 23:32 | |
| kfox1111 | if there was a nova maas driver though, that might work. | 23:33 |
| v1k0d3n | i get it. | 23:35 |
| alanmeadows | we have complex networking needs that ironic doesn't really fulfill right now | 23:36 |
| alanmeadows | but thought the ceph chart could be of some value | 23:36 |
| *** lamt has quit IRC | 23:37 | |
| kfox1111 | yeah. I'll ahve a look. thanks. :) | 23:37 |
| kfox1111 | there's another ceph on kubernetes project I saw somewhere.... where was that... | 23:38 |
| alanmeadows | there are some instructions baked into ceph-docker for k8s | 23:38 |
| alanmeadows | however, this is the first ceph chart I am aware of | 23:38 |
| kfox1111 | https://github.com/ceph/ceph-docker/tree/master/examples/kubernetes | 23:38 |
| kfox1111 | yeah. | 23:38 |
| kfox1111 | ah. helm. yeah. | 23:39 |
| kfox1111 | true. I don't think I've seen any charts. | 23:39 |
| kfox1111 | cool. :) | 23:39 |
| *** sdake_ has quit IRC | 23:39 | |
| v1k0d3n | i actually think that we can get rid of some of the prep too (possibly) by launching a container that can do some of the prep locally on the host. | 23:40 |
| v1k0d3n | have to really think about this though. | 23:40 |
| alanmeadows | two fold, part of it is the need for a "sidecar" container, to lessen requirements on the host | 23:41 |
| kfox1111 | alanmeadows: have a look at the ceph stuff in kolla-kubernetes. Its mostly just for testing, | 23:41 |
| kfox1111 | but there is one thing I think is pretty clever. | 23:41 |
| kfox1111 | the ceph-rbd object. | 23:41 |
| kfox1111 | https://github.com/openstack/kolla-kubernetes/blob/master/services/ceph/ceph-rbd-pod.yml.j2 | 23:42 |
| alanmeadows | and the second is the helm lack of any kind dynamic run-time configuration and its inability to gobble up whole directories for configmaps and especially secrets | 23:42 |
| kfox1111 | makes it so you don't have to install anything on the nodes. | 23:42 |
| *** dave-mccowan has quit IRC | 23:42 | |
| alanmeadows | which is why our ceph chart has some Makefile magic to sidestep | 23:42 |
| alanmeadows | but ultimately that would go away | 23:42 |
| kfox1111 | yeah. I think as helm starts getting used, they will quickly see the gaps. | 23:42 |
| alanmeadows | exactly | 23:43 |
| kfox1111 | now that its released, it should go faster. :) | 23:43 |
| alanmeadows | we want to help with some of that - as soon you step outside an nginx example | 23:43 |
| alanmeadows | you really start to see them | 23:43 |
| * kfox1111 nods. | 23:43 | |
| alanmeadows | luckily they are reasonably small gaps | 23:43 |
| v1k0d3n | ^^ yup just all of that | 23:43 |
| v1k0d3n | exactly | 23:43 |
| alanmeadows | easy features to close | 23:43 |
| kfox1111 | alanmeadows: when you get a chance can you weigh in on https://review.openstack.org/#/c/396296/ too? | 23:43 |
| alanmeadows | I can't include a file and base64 encode it, really? | 23:43 |
| alanmeadows | you dont clean up secrets, hmm... | 23:43 |
| kfox1111 | would love some feedback on just how crazy I am. :) | 23:43 |
| alanmeadows | checking | 23:44 |
| kfox1111 | yeah. I had the same issue with the jinja2 stuff we were doing. | 23:44 |
| kfox1111 | had to add some pretty common stuff to do the same. base64 wasn't built in. :/ | 23:44 |
| alanmeadows | the rbd-pod is interesting | 23:46 |
| kfox1111 | I did it for minikube support. it didn't supprot ceph tools on the host. | 23:46 |
| kfox1111 | the other advantage is the ceph client then exactly matches whats in the rest of the containers. | 23:50 |
| *** v1k0d3n has quit IRC | 23:50 | |
| *** v1k0d3n has joined #openstack-kolla | 23:51 | |
| *** rhallisey has joined #openstack-kolla | 23:51 | |
| alanmeadows | yup thats how it should be | 23:52 |
| alanmeadows | i like some of the ideas in 3the WIP | 23:52 |
| alanmeadows | I will try and comment this evening | 23:53 |
| alanmeadows | i like how you're tackling neutron first ;-) | 23:53 |
| kfox1111 | alanmeadows: cool. thanks for looking at it/reviewing. :) the help's appreciated. :) | 23:53 |
| kfox1111 | heh. hard stuff to prove out the concept. | 23:53 |
| kfox1111 | down hill from there. :) | 23:53 |
| alanmeadows | thats whats necessary | 23:53 |
| kfox1111 | I hate spending a bounch of time on something that works for easy stuff, then have to rewrite it all when something tough proves the thing was a bad idea. :/ | 23:54 |
| kfox1111 | so I try and prototype the really hard stuff. :) | 23:55 |
| alanmeadows | hehe, see it break quickly to realize your path was flawed all along | 23:55 |
| * kfox1111 nods | 23:55 | |
| kfox1111 | and thats also why I'm seeking feedback, as I think it will fly, and I think I got most of the big gotcha's. but there may be some I didn't see yet and others have. | 23:55 |
| *** v1k0d3n has quit IRC | 23:56 | |
| kfox1111 | I think it will be pretty easy to add kubernetes-entrypoint to in this arangement too. | 23:56 |
| kfox1111 | can add it to common-lib. | 23:56 |
| kfox1111 | then just include it where needed. | 23:56 |
| alanmeadows | thats what I'm trying to pioneer as well (wrt entrypoint) in aic-helm | 23:56 |
| alanmeadows | also one thing I haven't really played with | 23:56 |
| kfox1111 | cool. :) | 23:56 |
| alanmeadows | is just how well upgrade works ;-) | 23:57 |
| alanmeadows | for the complex scenarios | 23:57 |
| kfox1111 | yeah. I'm a bit skeptical of helm being able to handle that piece. :/ | 23:57 |
| alanmeadows | i don't know enough yet to say | 23:57 |
| alanmeadows | just that I share the same skepticism | 23:57 |
| kfox1111 | I've done many upgrades of openstack systems over the years. | 23:57 |
| kfox1111 | haven't lost a vm yet. :) | 23:57 |
| kfox1111 | but it took a whole lot of kergiggering. | 23:57 |
| kfox1111 | not sure helm's concept of upgrades aligns perfectly with openstacks view of them. :/ | 23:58 |
| kfox1111 | but we're trying to tool ourselves so that we're really flexible, | 23:59 |
| alanmeadows | well I think it mostly relies on k8s functionality | 23:59 |
| kfox1111 | and able to use whichever worklfow tools are best for each site. | 23:59 |
| alanmeadows | and where I sort of seeing it sitting | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!