| kfox1111 | I even managed to get rid of all of the, install ceph on the compute nodes, copy config file, copy ceph keyring, etc. | 00:00 |
|---|---|---|
| kfox1111 | have a daemonset for rbd. :) | 00:00 |
| *** severion has joined #openstack-kolla | 00:01 | |
| kfox1111 | and a configmap/secret for the rest. k8s can then deploy it where needed rather then having to do it manually. | 00:01 |
| *** mtaylor22 has quit IRC | 00:03 | |
| *** cj has joined #openstack-kolla | 00:03 | |
| *** v1k0d3n has quit IRC | 00:04 | |
| severion | well, one thing to check out... | 00:05 |
| severion | this just got released, and it really fits well for this project actually... | 00:05 |
| severion | (one sec...link incoming). | 00:05 |
| *** harlowja has joined #openstack-kolla | 00:06 | |
| severion | i see this being adopted into kubernetes proper quickly....in fact, i'm in complete support of the effort | 00:06 |
| severion | kfox1111: https://github.com/deis/steward | 00:06 |
| severion | very cool stuff (sorry name flipped over when i changed wifi networks) | 00:07 |
| severion | one sec... | 00:07 |
| *** severion has quit IRC | 00:07 | |
| *** v1k0d3n has joined #openstack-kolla | 00:08 | |
| kfox1111 | hmmm. interesting. I'll have to look closer at this. :) | 00:08 |
| kfox1111 | been keeping a close eye on helm too. | 00:11 |
| v1k0d3n | i'm a massive fan of helm. | 00:12 |
| v1k0d3n | so.... | 00:12 |
| v1k0d3n | this might help | 00:12 |
| v1k0d3n | helm is native, and charts are currently in incubation (as i'm sure you know). | 00:12 |
| v1k0d3n | anway...talked with a good friend of mine at dies. i think we're going to do containerize parts of our infra in charts... | 00:13 |
| v1k0d3n | this way, people can deploy via helm. that's a huge win and makes things pretty easy. | 00:13 |
| kfox1111 | kolla? | 00:13 |
| v1k0d3n | well...see....i'm trying to figure out how that would all fit. | 00:13 |
| v1k0d3n | what if kolla built, created charts, and then kolla-kube was deployed via the charts? | 00:14 |
| kfox1111 | yeah... we should all talk... :) | 00:14 |
| v1k0d3n | that adds a lot of CI/CD capabilities that kolla alone doesn't solve. | 00:14 |
| kfox1111 | from what I've seen, I'm not sure helm charts by themselves are totally flexilbe to handle everything kolla-kubernetes needs. but | 00:14 |
| kfox1111 | the package management part and the multiinstance thing is awesome. | 00:14 |
| v1k0d3n | well, if there's something missing...let's work on that, and fix it upstream in kubernetes/helm. | 00:15 |
| v1k0d3n | we're totally on board with fixing this. | 00:15 |
| kfox1111 | so I was kind of thinking I might try and prototype having kolla-kubernetes "build/compile" a helm chart repository. | 00:15 |
| v1k0d3n | ok, i'm very interested in that. | 00:16 |
| v1k0d3n | are you going to be at the summit? | 00:16 |
| v1k0d3n | can we bring that up as a discussion point for kolla-kube? | 00:16 |
| kfox1111 | so you checkout kolla-kubernetes, run build, and it pops out a chart repo. | 00:16 |
| kfox1111 | I'm missing this one. :( | 00:16 |
| v1k0d3n | ^^ yes, that's good | 00:16 |
| v1k0d3n | not the missing part :) | 00:16 |
| kfox1111 | yeah. | 00:16 |
| v1k0d3n | yeah, i remember you saying that. man, that sucks! | 00:16 |
| v1k0d3n | wish you were | 00:16 |
| v1k0d3n | want to sync up on a call with our team and ryan before teh summit then? | 00:17 |
| v1k0d3n | so we can present this idea? | 00:17 |
| kfox1111 | yeah. would love to. | 00:17 |
| kfox1111 | one of the things kolla-kubernetes really lacks is multinstance support. | 00:17 |
| v1k0d3n | maybe you/ryan/i can sync up as a smaller group, and discuss...and then bring up to a larger audience. | 00:17 |
| v1k0d3n | agreed. | 00:17 |
| kfox1111 | for my production system, I need multiple rabbits and multiple mariadb clusters. | 00:17 |
| kfox1111 | helm I think would handle that awesomely. :) | 00:18 |
| v1k0d3n | yeah, we need to talk about this. | 00:18 |
| v1k0d3n | helm is now part of kuberetes... | 00:18 |
| v1k0d3n | so the discussion should happen. | 00:18 |
| kfox1111 | add sbezverk_ too. | 00:18 |
| kfox1111 | the other thing, | 00:18 |
| v1k0d3n | word. i'll reach out to ryan tomorrow about this. | 00:18 |
| kfox1111 | I have that kolla-kubernetes-toolbox to make it easy to start going. I'm going to work on getting that into kolla-kubernetes repo proper. | 00:18 |
| kfox1111 | but... | 00:18 |
| kfox1111 | I was also thinking it should go in the main helm repo. | 00:19 |
| v1k0d3n | so we can make this happen. needs to enter incubation period though. | 00:19 |
| kfox1111 | you could then easily launch all the kolla-kubernetes tools all at once, and then use that to bootstrap all the rest. | 00:19 |
| * kfox1111 nods | 00:19 | |
| v1k0d3n | i can talk to the folks at deis about this though. see what we can do to make this effort successful. | 00:19 |
| kfox1111 | that woudl be cool. :) | 00:20 |
| v1k0d3n | we just containerized MaaS today too...we want to use that as our pre-infra BMaaS for kubernetes. | 00:20 |
| *** huikang has joined #openstack-kolla | 00:20 | |
| v1k0d3n | i want this as part of a seed host, an aio kubernetes node that starts the kick-off of the entire BM infra... | 00:20 |
| kfox1111 | nice. | 00:20 |
| kfox1111 | this ubuntu's maas? | 00:20 |
| v1k0d3n | yup | 00:21 |
| v1k0d3n | https://quay.io/repository/attcomdev/maas | 00:21 |
| kfox1111 | ah. | 00:21 |
| v1k0d3n | it's rough...working on the ENV items tomorrow and through next week... | 00:21 |
| v1k0d3n | so MaaS would be part of the initial seed host...to start everything else. | 00:21 |
| v1k0d3n | just boot up, OS and Kubernetes is installed...labeled, and added to the cluster. | 00:22 |
| v1k0d3n | then daemonsets follow, etc. | 00:22 |
| kfox1111 | I've kind of avoided maas so far, as I do a lot of non ubuntu stuff and haven't heard that it works well outside of ubuntu. | 00:22 |
| kfox1111 | what I was thinking for long term: | 00:22 |
| v1k0d3n | we're looking at other things too...just have MaaS exp right now. i'm suggesting that we look at RackHD too...along with CoreOS Ignition. | 00:23 |
| kfox1111 | launch minikube, helm install kolla-kubernetes toolbox, and enter it. | 00:23 |
| kfox1111 | run script that stands up an all in one ironic cloud in the minikube. | 00:23 |
| awiddersheim | where do things log now? | 00:23 |
| kfox1111 | add a few bare metal nodes to that ironic. use it to launch them. they join the ironic cloud. | 00:23 |
| awiddersheim | trying to look at keepalived logs | 00:23 |
| awiddersheim | but not seeing a volume mapped | 00:24 |
| awiddersheim | anymore | 00:24 |
| *** frzn_ has joined #openstack-kolla | 00:24 | |
| *** frzn_ is now known as mtaylor22 | 00:24 | |
| kfox1111 | should be able to remove the seed at that point, as the bare metal is then self hosting the cluster. | 00:24 |
| v1k0d3n | would you loose your ironic cluster management if you blow away the seed host? | 00:24 |
| kfox1111 | the bare metal would be running k8s, and when the seed goes away, it would just relaunch the services on the bare metal. | 00:24 |
| *** haplo37_ has quit IRC | 00:25 | |
| v1k0d3n | i'd be ok with that, we just need a way to keep the bare metal management after removing the seed host/container | 00:25 |
| jascott1 | awiddersheim on the node you might try /var/log/upstart | 00:25 |
| kfox1111 | yeah. :) | 00:25 |
| v1k0d3n | good stuff! | 00:25 |
| v1k0d3n | yeah we sould definitely sync up. | 00:25 |
| kfox1111 | last I tried the ironic python agent, it would let you keep a bare metal node working after a reboot. | 00:25 |
| v1k0d3n | i'll reach out to ryan tomorrow. this is good stuff! | 00:25 |
| kfox1111 | so if the intalled image had a k8s cluster working in it, it should work fine after reboot. | 00:25 |
| awiddersheim | jascott1 I'm on centos | 00:26 |
| awiddersheim | not seeing that | 00:26 |
| v1k0d3n | kfox1111: let me know if we can help with some of that effort. | 00:26 |
| kfox1111 | yeah. that would be cool. :) | 00:26 |
| v1k0d3n | we have a lab we're currently working on these sorts of ideas on. | 00:26 |
| v1k0d3n | could possible collaborate. | 00:26 |
| v1k0d3n | *possibly | 00:26 |
| kfox1111 | I have not had much chacne to play with kolla's ironic containers yet. | 00:26 |
| kfox1111 | one of the things on my todo list. | 00:26 |
| *** haplo37_ has joined #openstack-kolla | 00:27 | |
| v1k0d3n | anyway...gotta catch dinner with the wife. :) she's sitting next to me, reading facebook, and politics started to come up. | 00:27 |
| v1k0d3n | time to get out of here! lol | 00:27 |
| kfox1111 | if we can get a prototype of that working, the rest I think will be easy. | 00:27 |
| v1k0d3n | :D | 00:27 |
| kfox1111 | k. have a good one. :) | 00:27 |
| v1k0d3n | sounds good. great convo kfox1111. have a good night man! | 00:27 |
| kfox1111 | yup. we'll talk more later. :) | 00:27 |
| kfox1111 | good night. :) | 00:27 |
| *** v1k0d3n has quit IRC | 00:28 | |
| *** v1k0d3n has joined #openstack-kolla | 00:28 | |
| *** mtaylor22 has quit IRC | 00:28 | |
| *** mark-casey1 has quit IRC | 00:30 | |
| *** duonghq has joined #openstack-kolla | 00:30 | |
| duonghq | morning guys | 00:30 |
| duonghq | sdake_ can you reproduce rabbitmq bug easily? | 00:32 |
| *** frzn_ has joined #openstack-kolla | 00:32 | |
| sdake_ | duonghq it was user error | 00:33 |
| sdake_ | that is why rc2 is tagged | 00:33 |
| *** v1k0d3n has quit IRC | 00:33 | |
| jascott1 | awiddersheim can you do docker logs <containerid> ? | 00:33 |
| awiddersheim | yeah that gives me nothing | 00:33 |
| awiddersheim | for the keepalived container | 00:33 |
| awiddersheim | and no logs anywhere that I can find | 00:33 |
| *** Pavo has quit IRC | 00:34 | |
| duonghq | sdake_ understood | 00:34 |
| *** Pavo has joined #openstack-kolla | 00:34 | |
| duonghq | wonder why rc2 is tagged when read the morning news | 00:34 |
| jascott1 | awiddersheim then nothing in k8s dashboard logs then I guess | 00:35 |
| *** robcresswell has quit IRC | 00:35 | |
| awiddersheim | not using k8 | 00:35 |
| awiddersheim | just docker on hw | 00:35 |
| jascott1 | ah | 00:35 |
| kfox1111 | /var/log/kolla/*/* | 00:35 |
| awiddersheim | in the container | 00:36 |
| awiddersheim | or the host? | 00:36 |
| kfox1111 | container. | 00:36 |
| awiddersheim | that is empty in keepalived | 00:36 |
| sdake_ | duonghq we have 10 rc3 bugs | 00:37 |
| sdake_ | thats our lst rc | 00:37 |
| duonghq | yeah | 00:37 |
| sdake_ | if you want to fix a bug there - huge :) | 00:37 |
| *** robcresswell has joined #openstack-kolla | 00:37 | |
| awiddersheim | I don't know if keepalived is even making logs | 00:39 |
| awiddersheim | at all | 00:39 |
| awiddersheim | anywhare | 00:39 |
| kfox1111 | possible. | 00:41 |
| kfox1111 | you might have to tweak the config file to add more logging. | 00:41 |
| *** david-lyle_ has joined #openstack-kolla | 00:42 | |
| *** zhubingbing has quit IRC | 00:42 | |
| kfox1111 | at least on my box, it doesn't look like its got any explicit logging stuff in it. | 00:42 |
| kfox1111 | hmm.. just a quick google search says it goes through syslog. | 00:43 |
| kfox1111 | which I'm not sure works very well in containers. | 00:43 |
| kfox1111 | might try --log-console ? | 00:44 |
| kfox1111 | oh. looks like its aready specified. same as -l | 00:45 |
| kfox1111 | so nothing in docker logs for the container? | 00:45 |
| *** david-lyle has quit IRC | 00:45 | |
| *** yingjun has joined #openstack-kolla | 00:46 | |
| awiddersheim | nothing | 00:46 |
| *** inc0 has joined #openstack-kolla | 00:47 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Revert "Build source from Newton rather then Master" https://review.openstack.org/386286 | 00:48 |
| awiddersheim | it's almost like my container | 00:50 |
| awiddersheim | isn't actually running keepalive | 00:50 |
| awiddersheim | at all | 00:50 |
| awiddersheim | I'm not seeing the pid | 00:50 |
| awiddersheim | or anything in the process list | 00:50 |
| *** huikang has quit IRC | 00:51 | |
| *** huikang has joined #openstack-kolla | 00:52 | |
| awiddersheim | but if I log into the container | 00:52 |
| inc0 | awiddersheim, I don't have context | 00:52 |
| awiddersheim | I can start keepalived manually | 00:52 |
| inc0 | mind bringing me up to speed? | 00:52 |
| *** ssurana has quit IRC | 00:52 | |
| awiddersheim | sure | 00:52 |
| awiddersheim | let me back up | 00:52 |
| awiddersheim | just tried deploying a fresh rc2 setup | 00:52 |
| awiddersheim | it got to brinigng up keepalived | 00:53 |
| awiddersheim | and wait for the vip but the vip never showed up | 00:53 |
| awiddersheim | I noticed in chat you recommended to someone else that this might need changing | 00:53 |
| awiddersheim | https://github.com/openstack/kolla/blob/master/etc/kolla/globals.yml#L91 | 00:53 |
| awiddersheim | I have a feeling I might need to do the same but as I started to try to verify | 00:53 |
| awiddersheim | like.... I'm not getting any logs back from my container | 00:54 |
| awiddersheim | for keepalived | 00:54 |
| awiddersheim | doesn't even really seem like keepalived is running | 00:54 |
| awiddersheim | when I log into the container | 00:54 |
| awiddersheim | the /run/keepalived.pid isn't even there | 00:54 |
| inc0 | awiddersheim, so my solution applies if you have multiple keepalived running | 00:54 |
| awiddersheim | and /var/log/kolla is empty | 00:54 |
| inc0 | but docker logs should show that | 00:54 |
| awiddersheim | right | 00:54 |
| inc0 | docker logs keepalived is empty? | 00:55 |
| awiddersheim | yes | 00:55 |
| *** huikang_ has joined #openstack-kolla | 00:55 | |
| awiddersheim | I don't know if like... heka or something else is eating them up | 00:55 |
| awiddersheim | we have another cluster running but that is kind of like | 00:55 |
| awiddersheim | a red herring or whatever for the fact that I can't even see logs | 00:55 |
| awiddersheim | one thing lead to another I guess and I'm more concerned about not seeing logs | 00:56 |
| awiddersheim | and will figure out the other thing later | 00:56 |
| awiddersheim | but yeah it's almost like keepalived isn't even running | 00:56 |
| *** huikang has quit IRC | 00:56 | |
| awiddersheim | in the container | 00:56 |
| inc0 | no, docker logs are there | 00:56 |
| awiddersheim | I have same issue with haproxy container | 00:57 |
| inc0 | check out syslog if docker daemon itself isn't crying for life | 00:57 |
| awiddersheim | docker logs <foo> | 00:57 |
| inc0 | seems like docker having issues | 00:57 |
| awiddersheim | shows nothing | 00:57 |
| awiddersheim | ok | 00:57 |
| inc0 | syslog might have something | 00:57 |
| jascott1 | where do logs that would be in /var/log/upstart go on centos? | 00:57 |
| jascott1 | oh wait you arent using k8s sorry | 00:58 |
| awiddersheim | alright | 00:58 |
| awiddersheim | the /var/log/messages didn't have much | 00:59 |
| *** phuongnh has joined #openstack-kolla | 00:59 | |
| awiddersheim | but I restarted docker | 00:59 |
| awiddersheim | and can see things now | 00:59 |
| awiddersheim | I didn't comb through the syslog stuff though | 00:59 |
| inc0 | jascott1, centos doesn't use upstart:) | 00:59 |
| inc0 | systemd | 00:59 |
| awiddersheim | so might be something in there I"m not seeing | 00:59 |
| jascott1 | correct but where would those logs be? | 00:59 |
| jascott1 | sorry doesnt matter as awiddersheim said hes not using k8s | 01:00 |
| jascott1 | i noticed k8s logs to /v/l/upstart on ubuntu 14 | 01:00 |
| inc0 | jascott1, /var/log/messages on centos | 01:00 |
| inc0 | or /var/log/syslog on buntu 16 | 01:01 |
| jascott1 | thx good to know | 01:01 |
| *** yuanying has quit IRC | 01:03 | |
| sbezverk_ | inc0: could you please remove your -2 from fluentd PS :-) since master is opened for business | 01:03 |
| *** liyifeng has quit IRC | 01:04 | |
| inc0 | sbezverk_, would you be so kind to provide me a link?:) | 01:05 |
| *** yuanying has joined #openstack-kolla | 01:07 | |
| *** zhubingbing has joined #openstack-kolla | 01:07 | |
| *** huikang has joined #openstack-kolla | 01:10 | |
| sbezverk_ | inc0: sure thing: https://review.openstack.org/#/c/375615/13 | 01:11 |
| inc0 | here you go | 01:11 |
| inc0 | I'll review it later on | 01:11 |
| inc0 | hows the fencing going? | 01:11 |
| sbezverk_ | inc0: well it is working. but Kevin sent tons of comments, so going through them.. | 01:12 |
| sbezverk_ | my cluster died after so many reboots | 01:13 |
| sbezverk_ | so I am rebuilding it | 01:13 |
| *** huikang_ has quit IRC | 01:13 | |
| *** zhugaoxiao has joined #openstack-kolla | 01:16 | |
| *** neilus has joined #openstack-kolla | 01:20 | |
| awiddersheim | inc0: this is still.. like super wierd | 01:22 |
| awiddersheim | it's like... | 01:22 |
| awiddersheim | the keepalived container | 01:22 |
| awiddersheim | doesn't actually run keepalived | 01:22 |
| awiddersheim | when I start it | 01:22 |
| *** zhurong has joined #openstack-kolla | 01:22 | |
| awiddersheim | does ansible need to do anything to kick that off | 01:22 |
| inc0 | oO | 01:22 |
| awiddersheim | seems like the container has all the things it would need | 01:22 |
| inc0 | not really | 01:22 |
| *** yuanying_ has joined #openstack-kolla | 01:23 | |
| awiddersheim | should I see the keepalived process? | 01:24 |
| *** yingjun_ has joined #openstack-kolla | 01:24 | |
| awiddersheim | in ps? | 01:24 |
| inc0 | well, from inside docker | 01:24 |
| inc0 | docker exec -it keepalived ps aux | 01:24 |
| *** yingjun has quit IRC | 01:25 | |
| *** neilus has quit IRC | 01:25 | |
| awiddersheim | I have nothing :( | 01:25 |
| awiddersheim | except kolla_start | 01:25 |
| awiddersheim | and sleep | 01:25 |
| *** MarMat has quit IRC | 01:25 | |
| openstackgerrit | zhongshengping proposed openstack/kolla: Deprecate auth_plugin option https://review.openstack.org/385805 | 01:26 |
| inc0 | something seems to be seriously wrong | 01:26 |
| inc0 | try to do full rebuild | 01:26 |
| *** yuanying has quit IRC | 01:26 | |
| *** dupengfei has joined #openstack-kolla | 01:26 | |
| awiddersheim | ah ok | 01:27 |
| awiddersheim | I see it | 01:27 |
| awiddersheim | it's waiting | 01:27 |
| awiddersheim | for heka socket :( | 01:27 |
| inc0 | and heka? | 01:27 |
| inc0 | what about heka> | 01:27 |
| awiddersheim | it's a new install ummm | 01:28 |
| awiddersheim | I only have haproxy | 01:28 |
| awiddersheim | and keepalived containers | 01:28 |
| awiddersheim | and it bombed after that | 01:28 |
| dupengfei | Hi, I found a K8s problem：The sky-dns service will failure for a period, and automatic recovery after 20 seconds, who knows it? | 01:29 |
| awiddersheim | https://github.com/openstack/kolla/blob/d75847772d0e880329eef5867caf4b87035d2ae0/docker/base/start.sh#L5-L9 | 01:29 |
| awiddersheim | that is the part I"m talking about but | 01:29 |
| awiddersheim | I guess on initial startup | 01:29 |
| awiddersheim | I dont' know.... I never got a heka container | 01:30 |
| awiddersheim | I'll need to dig into that | 01:30 |
| inc0 | awiddersheim, so docker ps doesnt show heka? | 01:32 |
| inc0 | how about docker ps -a | 01:33 |
| inc0 | ? | 01:33 |
| awiddersheim | no but | 01:35 |
| awiddersheim | I tracked it down | 01:35 |
| awiddersheim | something dumb specific to me | 01:35 |
| awiddersheim | where the common kolla role wasn't running | 01:35 |
| awiddersheim | that would put heka there | 01:35 |
| inc0 | ah yeah | 01:35 |
| inc0 | common rolle has to be there | 01:35 |
| *** zhurong has quit IRC | 01:40 | |
| *** zhurong has joined #openstack-kolla | 01:41 | |
| *** huikang has quit IRC | 01:43 | |
| *** huikang has joined #openstack-kolla | 01:44 | |
| *** huikang has quit IRC | 01:48 | |
| *** derekjhyang has joined #openstack-kolla | 01:49 | |
| *** sdake_ is now known as sdake | 02:01 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: add kibana to precheck https://review.openstack.org/378410 | 02:02 |
| *** liyifeng has joined #openstack-kolla | 02:07 | |
| duonghq | sdake: about the heka replacement, the snap is snap-telemetry from intelsdi? | 02:10 |
| Jeffrey4l_ | inc0, what PTG is? | 02:10 |
| inc0 | Jeffrey4l_, this is this new mini summit idea | 02:11 |
| inc0 | Project Team Gathering | 02:11 |
| Jeffrey4l_ | got. | 02:11 |
| inc0 | smaller than summit | 02:11 |
| inc0 | I'm not super happy with what it's turing out to be | 02:11 |
| *** haplo37 has quit IRC | 02:11 | |
| inc0 | but we'll see, first one happends in Feb in Atlanta | 02:11 |
| Jeffrey4l_ | we have 4 summit now per year ;( | 02:12 |
| *** g3ek has quit IRC | 02:12 | |
| inc0 | yeah, and I don't think that's gonna be good | 02:12 |
| inc0 | but everyone discusses that | 02:13 |
| *** MarMat has joined #openstack-kolla | 02:14 | |
| *** haplo37 has joined #openstack-kolla | 02:16 | |
| openstackgerrit | Merged openstack/kolla: Remove default values from conf file https://review.openstack.org/382936 | 02:17 |
| *** g3ek has joined #openstack-kolla | 02:17 | |
| *** ayoung_ has joined #openstack-kolla | 02:21 | |
| *** ayoung has quit IRC | 02:21 | |
| *** inc0 has quit IRC | 02:22 | |
| *** unicell1 has quit IRC | 02:23 | |
| *** ayoung_ is now known as ayoung | 02:25 | |
| *** huhaoran has joined #openstack-kolla | 02:26 | |
| sdake | Jeffrey4l_ wee wont have midcycles anymore i thin is the theory | 02:27 |
| sdake | Jeffrey4l_ so 3 summits a year | 02:27 |
| Jeffrey4l_ | got. | 02:28 |
| duonghq | sdake: the snap you mentioned as Heka replacement is snap telemetry? | 02:31 |
| sdake | duonghq inc0 mentioned snap - i know nothingabout snap | 02:32 |
| *** huikang has joined #openstack-kolla | 02:32 | |
| duonghq | sdake: sorry, I'll ask him later | 02:32 |
| sdake | Jeffrey4l_ the part of ptgs that sucks for us is alot of our contribs are international | 02:32 |
| sdake | and two events a year is easier then three | 02:33 |
| Jeffrey4l_ | yes. it will take lots of energy. | 02:33 |
| sdake | Jeffrey4l_ ptg is where we will get the work done | 02:33 |
| sdake | Jeffrey4l_ whereas there will be ops sessions and cross sesssiosn workshops at the main summit | 02:33 |
| sdake | cross proejct sessions that is | 02:34 |
| zhubingbing | I think we should have a good survey of how to replace the Heka | 02:34 |
| sdake | Jeffrey4l_ it looks like almost every container has a named volume | 02:43 |
| Jeffrey4l_ | yes. they are not stateless | 02:44 |
| sdake | i think for new containers we dont have to do naything | 02:44 |
| sdake | for old ocntianers we hve to do something | 02:45 |
| Jeffrey4l_ | uid/gid issue? | 02:45 |
| sdake | yup | 02:45 |
| Jeffrey4l_ | hmm tbh only upgrade from mitaka to newton need change owner/group | 02:46 |
| Jeffrey4l_ | but my fix change owner always | 02:46 |
| sdake | Jeffrey4l_ your fix was for one container | 02:48 |
| sdake | we need for many | 02:48 |
| *** MarMat has quit IRC | 02:48 | |
| Jeffrey4l_ | sdake, yes. i need fix others before rc3 | 02:48 |
| Jeffrey4l_ | i/we | 02:48 |
| sdake | Jeffrey4l_ are you going to do that or we splitting it up | 02:48 |
| sdake | whatever is most effecient | 02:49 |
| sdake | to get the bugs we have outstnading fixed | 02:49 |
| Jeffrey4l_ | sdake, https://etherpad.openstack.org/p/kolla-bug-1631503 split it up please. it is hard for me to fix all containers. | 02:49 |
| Jeffrey4l_ | feel free to pick one to fix. | 02:49 |
| Jeffrey4l_ | since we have newton branch now. we need fix it in master and backport it to newton branch. | 02:50 |
| sdake | right | 02:50 |
| Jeffrey4l_ | so how can we track this? using bug? or simple etherpad? sdake | 02:50 |
| sdake | Jeffrey4l_ etherpad - i'll record the full work itmes - i think there are mor e then those 4 | 02:51 |
| sdake | Jeffrey4l_ can you confirm https://bugs.launchpad.net/bugs/1632905 | 02:51 |
| openstack | Launchpad bug 1632905 in kolla "heat bootstrap failed" [Critical,Triaged] | 02:51 |
| sdake | Jeffrey4l_ i can replicate it multiple times | 02:51 |
| *** yuanying_ has quit IRC | 02:51 | |
| sdake | heat is enabled by default | 02:51 |
| sdake | Jeffrey4l_ by confirm i mean replicate | 02:52 |
| *** yuanying has joined #openstack-kolla | 02:52 | |
| Jeffrey4l_ | sdake, i deployed heat yesterday. do not found such issue. | 02:52 |
| sdake | i attempted to confirm https://bugs.launchpad.net/bugs/1625648 today | 02:53 |
| openstack | Launchpad bug 1625648 in kolla "horizon login fails with TLS enabled" [Critical,Triaged] | 02:53 |
| sdake | but fell asleep at the keyboard | 02:53 |
| sdake | super tired | 02:53 |
| sdake | Jeffrey4l_ perhaps it was an image pull problem | 02:53 |
| Jeffrey4l_ | got some sleep first ;) | 02:53 |
| sdake | i did for about 4 hours today | 02:53 |
| sdake | now i'm awake | 02:53 |
| *** MarMat has joined #openstack-kolla | 02:53 | |
| Jeffrey4l_ | sdake, maybe. | 02:53 |
| sdake | and 4-6 hrs last night | 02:54 |
| Jeffrey4l_ | i can confirm this on your node. it u do not using them now. | 02:54 |
| Jeffrey4l_ | it/if | 02:54 |
| sdake | go ahead | 02:54 |
| sdake | i wa s using destroy | 02:55 |
| sdake | but was not aware of the destroy images flag | 02:55 |
| sdake | this is a new workflow | 02:55 |
| Jeffrey4l_ | sdake, yep. it is a feature undocumented. | 02:55 |
| sdake | which is why i got tripped up in this | 02:55 |
| sdake | i thinkd eploy should pull | 02:55 |
| sdake | pull is documented but the flag is not | 02:55 |
| sdake | and the deploy behavior is not | 02:56 |
| sdake | it used to not behave in that way | 02:56 |
| sdake | it being kolla | 02:56 |
| sdake | it would alwaysp ull on deploy | 02:56 |
| sdake | there was at one point a pull strategy variable | 02:57 |
| sdake | or something like that | 02:57 |
| sdake | i feature removed for what I think is not good reason | 02:57 |
| sdake | but it is what it is | 02:57 |
| sdake | lets fix the problems we have at present | 02:57 |
| Jeffrey4l_ | hmm. pull on deploy may case other issue. 1. takes more time 2. it may trigger upgrade when image in registry and on node are not the same | 02:58 |
| sdake | Jeffrey4l_ i see | 02:58 |
| Jeffrey4l_ | pull strategy variable during deploy is a better solution | 02:59 |
| Jeffrey4l_ | usually, i run pull before deploy. | 03:00 |
| Jeffrey4l_ | fyi, destroy openstack and try to re-deploy on your nodes sdake | 03:00 |
| sdake | Jeffrey4l_ do whatevero u need | 03:02 |
| Jeffrey4l_ | sdake, so the heat issue is tiggerred by simple deploy right? | 03:02 |
| *** coolsvap has quit IRC | 03:02 | |
| sdake | yup | 03:03 |
| sdake | although i may have pulled already new images | 03:03 |
| sdake | i dont recall | 03:03 |
| sdake | if its an old image problem then we can mark it invalid | 03:03 |
| Jeffrey4l_ | ok. we will get the answer in few minutes. | 03:03 |
| sdake | one less problem to worry about :) | 03:03 |
| *** huikang has quit IRC | 03:05 | |
| *** huikang has joined #openstack-kolla | 03:05 | |
| *** neilus has joined #openstack-kolla | 03:09 | |
| *** tovin07 has joined #openstack-kolla | 03:10 | |
| *** unicell has joined #openstack-kolla | 03:10 | |
| *** huikang has quit IRC | 03:10 | |
| duonghq | sdake: does the doc fix need trivialfix tag? | 03:11 |
| sdake | duonghq nope | 03:11 |
| sdake | duonghq if it hs one its ok | 03:11 |
| sdake | duonghq but not required | 03:11 |
| sdake | the only things really requireing a bug id are thigns tobe backported | 03:12 |
| duonghq | sdake: ok | 03:12 |
| duonghq | sdake: I mean trivialfix tag, not bug id | 03:12 |
| sdake | its nice to have a bug record as well if its a long lived bug | 03:12 |
| sdake | ya docs are not backported | 03:12 |
| sdake | so no need for bug id | 03:12 |
| sdake | that could change in future | 03:13 |
| sdake | (the backporting part of things) | 03:13 |
| sdake | so tired | 03:14 |
| sdake | need more minerals | 03:14 |
| *** neilus has quit IRC | 03:14 | |
| duonghq | sdake: you are mentioning bug id, I mention TrivialFix, iirc 2 things do not presents together | 03:14 |
| duonghq | sdake: oh or a nap? | 03:14 |
| sdake | duonghq nah i woke up from nap an hour or two ago | 03:14 |
| *** unicell has quit IRC | 03:14 | |
| *** unicell has joined #openstack-kolla | 03:14 | |
| sdake | duonghq I don't know that we really need trivialfix anymore tbh | 03:15 |
| sdake | duonghq these are things to be sorted out in barcelona | 03:15 |
| *** yuanying has quit IRC | 03:15 | |
| *** yuanying has joined #openstack-kolla | 03:15 | |
| duonghq | ya, I don't go to Barcelona, :( | 03:15 |
| sdake | right but we will have decision from there going forward | 03:16 |
| sdake | (I hope) | 03:16 |
| duonghq | okay | 03:16 |
| duonghq | many decision need to make about coding/committing convention? | 03:16 |
| sdake | for now use existing process | 03:17 |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Fix docs formatting error https://review.openstack.org/380161 | 03:17 |
| sdake | duonghq we have good process in place already | 03:17 |
| sdake | but i think it can be finetuned | 03:17 |
| *** yuanying_ has joined #openstack-kolla | 03:17 | |
| sdake | i'd like us to use a spec repo for example | 03:17 |
| sdake | with only 2 +2 votes required | 03:17 |
| sdake | where a core reviewer can ask for a rollcall vote on tehe spec if necessary | 03:17 |
| sdake | i'd like us to stop using trivialfix for changes | 03:18 |
| *** yuanying has quit IRC | 03:18 | |
| sdake | and instead either ue bugs where it makes sense or use nothing | 03:18 |
| *** huikang has joined #openstack-kolla | 03:18 | |
| sdake | (or launchpad blueprint ids) | 03:18 |
| duonghq | yup, | 03:18 |
| sdake | basically optimizing the process | 03:19 |
| sdake | the reason we need bug ids is only for backports | 03:19 |
| sdake | at a different time in the project we needed bug ids for a different reason | 03:19 |
| sdake | (to show we were doing real work and had real problems to attract devs) | 03:19 |
| sdake | again thoe long lived issues, still like to see those tracked in bugs | 03:20 |
| sdake | so people cn fix them over time | 03:20 |
| sdake | sometimes it takes us months to fix a bug | 03:20 |
| sdake | and the bug may be serious | 03:20 |
| duonghq | ok | 03:21 |
| sdake | Jeffrey4l_ you will nee dto reeanble heat in my env btw :) | 03:21 |
| sdake | duonghq but its really up to what the core team decides :) | 03:21 |
| duonghq | sure, | 03:21 |
| openstackgerrit | zhubingbing proposed openstack/kolla: add magnum to precheck https://review.openstack.org/378394 | 03:21 |
| duonghq | still cannot understand docs gate passed but when I do tox -e docs, it failed | 03:22 |
| *** yuanying_ has quit IRC | 03:22 | |
| Jeffrey4l_ | sdake, already done. heat works fine. | 03:22 |
| sdake | duonghq not sure on that one | 03:23 |
| sdake | Jeffrey4l_ cool | 03:23 |
| sdake | Jeffrey4l_ one less bug :) | 03:23 |
| sdake | can you mark it invalid in launchpad | 03:23 |
| sdake | and explain the old images - need a kolla-ansible pull | 03:23 |
| Jeffrey4l_ | np. | 03:23 |
| sdake | we will probably want to document that deploy doesn't pull images at some point | 03:24 |
| sdake | pull new images that is | 03:24 |
| Jeffrey4l_ | yes. the point need be feature out is: deploy won't pull latest image. | 03:24 |
| Jeffrey4l_ | and upgrade don't, too. | 03:24 |
| sdake | upgrade doesn't either? | 03:25 |
| Jeffrey4l_ | yes. | 03:25 |
| sdake | if you put 3.0.1 in it should | 03:25 |
| sdake | and i htink it does s;) | 03:25 |
| Jeffrey4l_ | correct. | 03:25 |
| Jeffrey4l_ | when using the different image tag, deploy/upgrade will pull image because it not exist locally. | 03:25 |
| *** rstarmer has joined #openstack-kolla | 03:30 | |
| sdake | su prs | 03:39 |
| sdake | sup rstarmer | 03:39 |
| rstarmer | hey sdake :) | 03:40 |
| *** v1k0d3n has joined #openstack-kolla | 03:44 | |
| *** huhaoran has quit IRC | 03:48 | |
| *** huhaoran has joined #openstack-kolla | 03:49 | |
| *** yuanying has joined #openstack-kolla | 03:52 | |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes: Add workflow for cinder https://review.openstack.org/386144 | 03:57 |
| *** confisurya has joined #openstack-kolla | 03:57 | |
| *** salv-orlando has joined #openstack-kolla | 04:05 | |
| *** rstarmer has quit IRC | 04:07 | |
| sdake | Jeffrey4l_ re https://bugs.launchpad.net/bugs/1626958 | 04:09 |
| openstack | Launchpad bug 1626958 in kolla "neutron_server not working when lbaas enabled" [High,Confirmed] - Assigned to Swapnil Kulkarni (coolsvap) | 04:09 |
| *** salv-orlando has quit IRC | 04:09 | |
| *** huikang has quit IRC | 04:09 | |
| *** huhaoran has quit IRC | 04:10 | |
| *** huhaoran has joined #openstack-kolla | 04:11 | |
| Jeffrey4l_ | sdake, yep | 04:11 |
| *** absubram has joined #openstack-kolla | 04:14 | |
| *** ssurana has joined #openstack-kolla | 04:23 | |
| sdake | Jeffrey4l_ th eissue there is that lbaas has an enable line in the config file | 04:25 |
| sdake | yet if you enable it - bad things happen | 04:25 |
| sdake | i almost brought it into rc3 | 04:25 |
| sdake | bjolo wants iti in rc3 | 04:26 |
| sdake | i think the customization thing is a hacky workaround | 04:26 |
| sdake | for this particular problem | 04:26 |
| sdake | what we wwant to be careful of though is that we don't create a spiral effect here | 04:26 |
| sdake | of having to includ every plugin in every container | 04:26 |
| Jeffrey4l_ | sdake, but lots of thing work like that. like horizon plugins, horizon-lbaas-ui | 04:27 |
| sdake | horizon plguins have enable_plugin_z? | 04:27 |
| sdake | in the config file | 04:27 |
| sdake | or horizon-lbaas-ui? | 04:27 |
| Jeffrey4l_ | sdake, no. | 04:27 |
| sdake | right, that is wher the issue is | 04:28 |
| sdake | customizations are great, but if somethign doens't work out of the box without reubilding - seems like a problem | 04:28 |
| sdake | and enable_lbass = crater | 04:28 |
| Jeffrey4l_ | vpnaas have the same issue with lbaas. | 04:29 |
| sdake | these two casess are the onl ones i tcan think of | 04:29 |
| sdake | right i know | 04:29 |
| Jeffrey4l_ | another solution is using legacy plugin mech in kolla. and neutron-server-plugin-xxxx in config.py file. | 04:30 |
| sdake | that still requires rebuilding | 04:30 |
| Jeffrey4l_ | we can add it in default. | 04:30 |
| sdake | i see | 04:31 |
| sdake | that sounds pretty good | 04:31 |
| sdake | although the containers would work standalone | 04:31 |
| sdake | without the config.py magic | 04:31 |
| Jeffrey4l_ | yes. | 04:31 |
| *** frzn_ has quit IRC | 04:34 | |
| *** david-lyle_ has quit IRC | 04:39 | |
| *** david-lyle has joined #openstack-kolla | 04:39 | |
| *** salv-orlando has joined #openstack-kolla | 05:06 | |
| *** salv-orlando has quit IRC | 05:10 | |
| *** sdake has quit IRC | 05:23 | |
| *** dupengfei has quit IRC | 05:25 | |
| *** msimonin has joined #openstack-kolla | 05:28 | |
| *** msimonin has quit IRC | 05:31 | |
| *** msimonin has joined #openstack-kolla | 05:38 | |
| *** ssurana has quit IRC | 05:41 | |
| *** v1k0d3n has quit IRC | 05:46 | |
| *** msimonin has quit IRC | 05:48 | |
| *** neilus has joined #openstack-kolla | 05:50 | |
| *** zhurong has quit IRC | 05:51 | |
| *** huhaoran has quit IRC | 05:52 | |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla: Prechecks fail if using separate api interface https://review.openstack.org/385375 | 05:52 |
| *** huhaoran has joined #openstack-kolla | 05:53 | |
| *** neilus has quit IRC | 05:55 | |
| *** egonzalez90 has joined #openstack-kolla | 06:00 | |
| *** salv-orlando has joined #openstack-kolla | 06:07 | |
| *** salv-orlando has quit IRC | 06:11 | |
| *** msimonin has joined #openstack-kolla | 06:19 | |
| *** msimonin has quit IRC | 06:20 | |
| *** msimonin has joined #openstack-kolla | 06:25 | |
| *** phuongnh has quit IRC | 06:29 | |
| *** phuongnh has joined #openstack-kolla | 06:30 | |
| *** msimonin has quit IRC | 06:31 | |
| *** salv-orlando has joined #openstack-kolla | 06:34 | |
| *** ankush has joined #openstack-kolla | 06:34 | |
| *** msimonin has joined #openstack-kolla | 06:39 | |
| *** msimonin has quit IRC | 06:39 | |
| *** mnasiadka has joined #openstack-kolla | 06:50 | |
| *** janem has joined #openstack-kolla | 06:53 | |
| *** phuongnh has quit IRC | 07:00 | |
| bjolo | morning all | 07:04 |
| openstackgerrit | caoyuan proposed openstack/kolla: Add watcher checking to port_checks.yml https://review.openstack.org/378233 | 07:05 |
| *** neilus has joined #openstack-kolla | 07:05 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix keystone upgrade permission issue https://review.openstack.org/386421 | 07:09 |
| *** neilus has quit IRC | 07:10 | |
| *** msimonin has joined #openstack-kolla | 07:10 | |
| openstackgerrit | caoyuan proposed openstack/kolla: Add watcher checking to port_checks.yml https://review.openstack.org/378233 | 07:15 |
| *** bjolo has quit IRC | 07:17 | |
| *** bjolo has joined #openstack-kolla | 07:18 | |
| *** shardy has joined #openstack-kolla | 07:19 | |
| Jeffrey4l_ | bjolo, morning. when you testing the reconfigure, did u enable ssh pipline feature? | 07:21 |
| bjolo | believe i did | 07:22 |
| Jeffrey4l_ | ok.. | 07:22 |
| openstackgerrit | caoyuan proposed openstack/kolla: Add watcher checking to port_checks.yml https://review.openstack.org/378233 | 07:23 |
| bjolo | just checked, i have pipelining = True in ansible.cfg | 07:24 |
| Jeffrey4l_ | roger. | 07:25 |
| bjolo | are you going to the summit Jeffrey4l_ ? | 07:25 |
| Jeffrey4l_ | bjolo, yep. | 07:25 |
| bjolo | nice | 07:25 |
| Jeffrey4l_ | what about u? will be there right? | 07:25 |
| bjolo | im hoping i will go, still trying to piece it together the schedule with my family. I have small kids and a career wife so it is challenging :) | 07:26 |
| bjolo | need a lot of babysitter help :) | 07:26 |
| bjolo | if i go, i will only be there wednesday lunch to friday afternoon | 07:27 |
| bjolo | looking forward to meeting you in that case :) | 07:27 |
| Jeffrey4l_ | roger ;) | 07:27 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix glance upgrade permission issue https://review.openstack.org/386431 | 07:30 |
| bjolo | Jeffrey4l_, what are all these permission issues all about? | 07:31 |
| Jeffrey4l_ | bjolo, check the commit message here https://review.openstack.org/#/c/384598/3 | 07:31 |
| Jeffrey4l_ | the uid/gid in image from mitaka to newton may different. | 07:32 |
| bjolo | ah ok gotcha | 07:32 |
| Jeffrey4l_ | so we have lots of service to be fixed now. https://etherpad.openstack.org/p/kolla-bug-1631503 | 07:33 |
| bjolo | i bet | 07:33 |
| Jeffrey4l_ | feel free to take one ;) | 07:34 |
| bjolo | just need to add permissions part to the templates right? | 07:36 |
| *** rmart04 has joined #openstack-kolla | 07:36 | |
| Jeffrey4l_ | yes. simple easy. | 07:36 |
| bjolo | i have a question how to search gerrit. | 07:36 |
| bjolo | https://bugs.launchpad.net/kolla/+bug/1631944 | 07:36 |
| openstack | Launchpad bug 1631944 in kolla "cron container fails to start. missing sahara.conf" [Undecided,New] | 07:36 |
| openstackgerrit | caoyuan proposed openstack/kolla: Add watcher checking to port_checks.yml https://review.openstack.org/378233 | 07:37 |
| Jeffrey4l_ | bjolo, what do u want to search? | 07:37 |
| bjolo | i believe i say a PS fly by the other day to fix this bugg, but i cannot find it in gerrit | 07:37 |
| Jeffrey4l_ | ok. | 07:37 |
| Jeffrey4l_ | you can search the commit message or subject. | 07:37 |
| bjolo | linking back to the discussion about lbaas, i think this should be fixed in newton as well | 07:38 |
| Jeffrey4l_ | bjolo, re search: check the search methods https://review.openstack.org/Documentation/user-search.html | 07:38 |
| Jeffrey4l_ | bjolo, yep. should be fixed. | 07:38 |
| bjolo | ah thanks | 07:39 |
| bjolo | i.e. the link to the documentation | 07:39 |
| Jeffrey4l_ | np | 07:39 |
| bjolo | trying to troubleshoot my TLS issue | 07:41 |
| Jeffrey4l_ | cool. tbh, i never deployed tls env | 07:41 |
| bjolo | seems like people can not reproduce it, and for me it is consistent but i cannot find the error. drives me nuts :) | 07:41 |
| Jeffrey4l_ | hmm. i can try it. will give u some feedback. | 07:42 |
| Jeffrey4l_ | did u have bug id now? | 07:42 |
| bjolo | one sec | 07:42 |
| bjolo | https://bugs.launchpad.net/kolla/+bug/1625648 | 07:42 |
| openstack | Launchpad bug 1625648 in kolla "horizon login fails with TLS enabled" [Critical,Triaged] | 07:42 |
| bjolo | so to refrase | 07:43 |
| bjolo | deploy w/o TLS works just fine | 07:43 |
| Jeffrey4l_ | i will try tls now. | 07:44 |
| bjolo | then i do a destroy, enable TLS, and deploy | 07:44 |
| bjolo | CLI works, but not horizon | 07:44 |
| Jeffrey4l_ | OK. | 07:44 |
| bjolo | i know my certificate is fine | 07:45 |
| bjolo | i have also tried with a selfsigned, but that does not work either | 07:45 |
| Jeffrey4l_ | seems like some configuration issue. not certificate. | 07:45 |
| bjolo | unfortunately, logging in HA proxy is not working too well | 07:45 |
| bjolo | https://bugs.launchpad.net/kolla/+bug/1626938 | 07:46 |
| openstack | Launchpad bug 1626938 in kolla "haproxy logging not working" [Undecided,New] | 07:46 |
| bjolo | but nevermind haproxy loggin, since we are removing heka for ocata that wont be fixed | 07:46 |
| Jeffrey4l_ | OK. | 07:47 |
| *** salv-orlando has quit IRC | 07:49 | |
| bjolo | Jeffrey4l_, i saw that the newton branch is created now | 07:53 |
| Jeffrey4l_ | bjolo, yes. rc3 is also tagged. | 07:53 |
| Jeffrey4l_ | sorry rc2 | 07:53 |
| bjolo | i suppose that is what i should be using when testing now right? | 07:54 |
| *** ankush has quit IRC | 07:54 | |
| Jeffrey4l_ | yes. stop using master, it is O now. lots of thing is merged into master. | 07:55 |
| bjolo | gotcha :) | 07:55 |
| *** MarMat has quit IRC | 07:57 | |
| duonghq | anybody got Unable to connect to the server: Forbidden when run kolla-k8s toolbox using minikube | 07:57 |
| bjolo | i guess we should change the tag on master to 4.0.0 in globals.yml then :) | 07:59 |
| *** tonanhngo has quit IRC | 08:08 | |
| *** tonanhngo has joined #openstack-kolla | 08:09 | |
| Jeffrey4l_ | yes | 08:10 |
| *** tonanhngo has quit IRC | 08:10 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix keystone upgrade permission issue https://review.openstack.org/386421 | 08:11 |
| *** athomas has joined #openstack-kolla | 08:14 | |
| *** Jeffrey4l_ has quit IRC | 08:16 | |
| *** zhugaoxiao has quit IRC | 08:16 | |
| *** zhurong has joined #openstack-kolla | 08:16 | |
| *** zhugaoxiao has joined #openstack-kolla | 08:16 | |
| *** Serlex has joined #openstack-kolla | 08:27 | |
| openstackgerrit | Merged openstack/kolla: Add watcher checking to port_checks.yml https://review.openstack.org/378233 | 08:27 |
| openstackgerrit | Merged openstack/kolla: Remove unneeded blank lines https://review.openstack.org/383045 | 08:28 |
| openstackgerrit | Merged openstack/kolla: add magnum to precheck https://review.openstack.org/378394 | 08:28 |
| *** janem has quit IRC | 08:35 | |
| *** timss has quit IRC | 08:36 | |
| *** timss has joined #openstack-kolla | 08:36 | |
| *** ssurana has joined #openstack-kolla | 08:41 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix glance upgrade permission issue https://review.openstack.org/386431 | 08:43 |
| *** Jeffrey4l_ has joined #openstack-kolla | 08:43 | |
| *** salv-orlando has joined #openstack-kolla | 08:44 | |
| *** ssurana has quit IRC | 08:46 | |
| *** phuongnh has joined #openstack-kolla | 08:49 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: move sys.exit out of try finally block in set_config.py file https://review.openstack.org/386478 | 08:51 |
| *** janem has joined #openstack-kolla | 09:03 | |
| openstackgerrit | James McCarthy proposed openstack/kolla: Reorder the interface/network checks. https://review.openstack.org/386485 | 09:04 |
| openstackgerrit | zhubingbing proposed openstack/kolla: 11 https://review.openstack.org/386486 | 09:06 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix horizon upgrade permission issue https://review.openstack.org/386493 | 09:18 |
| *** berendt has joined #openstack-kolla | 09:18 | |
| *** ankush has joined #openstack-kolla | 09:27 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix haproxy upgrade permission issue https://review.openstack.org/386500 | 09:28 |
| *** salv-orlando has quit IRC | 09:31 | |
| *** salv-orlando has joined #openstack-kolla | 09:31 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix keepalived upgrade permission issue https://review.openstack.org/386431 | 09:32 |
| berendt | newton branch is available, this means we can move forward with patches for ocata, right? | 09:34 |
| wznoinsk | berendt: whatever is merged to master now is ocata already as of 1pm CET yesterday | 09:35 |
| berendt | wznoinsk thanks, then i can remove some of my -2 | 09:35 |
| *** openstackstatus has quit IRC | 09:44 | |
| *** openstack has joined #openstack-kolla | 09:45 | |
| *** openstackstatus has joined #openstack-kolla | 09:46 | |
| *** ChanServ sets mode: +v openstackstatus | 09:46 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix keepalived upgrade permission issue https://review.openstack.org/386431 | 09:46 |
| *** v1k0d3n has joined #openstack-kolla | 09:48 | |
| *** liyifeng has quit IRC | 09:51 | |
| *** v1k0d3n has quit IRC | 09:53 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix heat upgrade permission issue https://review.openstack.org/386512 | 09:54 |
| *** tonanhngo has joined #openstack-kolla | 09:57 | |
| *** yingjun_ has quit IRC | 09:58 | |
| *** tonanhngo has quit IRC | 09:58 | |
| *** tovin07 has quit IRC | 10:00 | |
| *** zhurong has quit IRC | 10:02 | |
| *** ankush has quit IRC | 10:03 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix mariadb upgrade permission issue https://review.openstack.org/386515 | 10:04 |
| *** duonghq has quit IRC | 10:10 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Fix elasticsearch upgrade permission issue https://review.openstack.org/386523 | 10:12 |
| *** openstackstatus has quit IRC | 10:13 | |
| *** openstack has joined #openstack-kolla | 10:13 | |
| *** openstackstatus has joined #openstack-kolla | 10:14 | |
| *** ChanServ sets mode: +v openstackstatus | 10:14 | |
| *** huhaoran has quit IRC | 10:16 | |
| *** ankush has joined #openstack-kolla | 10:17 | |
| *** matrohon has quit IRC | 10:19 | |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 10:26 |
| *** phuongnh has quit IRC | 10:27 | |
| *** bjolo has quit IRC | 10:28 | |
| *** bjolo has joined #openstack-kolla | 10:28 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Move demos and dev directory into a new contrib directory https://review.openstack.org/379500 | 10:57 |
| *** dave-mccowan has joined #openstack-kolla | 10:58 | |
| *** rhallisey has joined #openstack-kolla | 11:05 | |
| *** dwalsh has joined #openstack-kolla | 11:06 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Support HiPE (High-Performance Erlang) in RabbitMQ image https://review.openstack.org/367262 | 11:08 |
| *** dwalsh_ has joined #openstack-kolla | 11:12 | |
| *** dwalsh has quit IRC | 11:13 | |
| openstackgerrit | Merged openstack/kolla: Reorder the interface/network checks. https://review.openstack.org/386485 | 11:14 |
| *** mliima has joined #openstack-kolla | 11:16 | |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla: Prechecks fail if using separate api interface https://review.openstack.org/385375 | 11:17 |
| *** dwalsh_ has quit IRC | 11:17 | |
| *** dwalsh_ has joined #openstack-kolla | 11:19 | |
| openstackgerrit | Merged openstack/kolla: add kibana to precheck https://review.openstack.org/378410 | 11:24 |
| *** dwalsh_ has quit IRC | 11:24 | |
| janem | could anyone explain when to use 'kolla-ansible upgrade' and 'kolla-ansible reconfigure' ? | 11:25 |
| berendt | janem reconfigure when you changed something in your configuration files, upgrade when you want to deployer newer docker images | 11:26 |
| janem | like, I forgot to enable cinder, and did enable ceph.. so could I just edit that in the globals.yml and run reconfigure? | 11:26 |
| berendt | janem when you want to deploy cinder you want to use the deploy command | 11:26 |
| mliima | morning guys | 11:26 |
| *** dgonzalez has quit IRC | 11:26 | |
| janem | berendt: ok, so how does that work with a already deployed stack? | 11:27 |
| janem | berendt: it will just leave what is running alone? | 11:27 |
| berendt | deploy followed by reconfigure | 11:27 |
| janem | berendt: 10-4 | 11:27 |
| berendt | reconfigure only needed when the new service needs to be integrated into an already deployed service | 11:27 |
| janem | berendt: got it, thanks! | 11:28 |
| janem | berendt: so upgrade would be if i changed openstack version in the global.yml ? | 11:29 |
| berendt | you do not have to change the openstack version, when you want to deploy newer docker images you have to bump the version in globals.yaml and kolla-build.conf | 11:30 |
| berendt | then rebuild all image, then run upgrade | 11:30 |
| janem | berendt: I see. | 11:30 |
| berendt | this way the specified version of images will be deployed | 11:30 |
| berendt | i do this from time to time to rollout newer ubuntu packages but without changing the used openstack packages | 11:31 |
| janem | berendt: ok, but now i tried to deploy cinder, changed globals.yml , ran deploy and then reconfigure.. but Volumes does not show in Horizon.. | 11:35 |
| janem | am I missing something? | 11:35 |
| berendt | maybe you have to manually restart horizon because of some caching | 11:36 |
| berendt | is cinder working at all? | 11:36 |
| janem | berendt: no, when I create a volume with cli I get error | 11:38 |
| janem | status | 11:38 |
| janem | on the volume | 11:38 |
| berendt | check the logs... | 11:38 |
| janem | berendt: cinder-scheduler says 'No valid host was found' | 11:41 |
| *** dgonzalez has joined #openstack-kolla | 11:43 | |
| berendt | that can be caused by many.. | 11:43 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Update Manila Guide https://review.openstack.org/386550 | 11:44 |
| janem | berendt: it's no biggie, its a new deployment, can just try and redeploy, but thought it would be nice to figure it out | 11:45 |
| *** dwalsh has joined #openstack-kolla | 11:46 | |
| janem | berendt: I did enable ceph initially .. that might cause some confusement? | 11:46 |
| janem | berendt: enabeling ceph before cinder I mean | 11:46 |
| berendt | play around with it and document it, last time i deployed cinder after the initial deployment it just worked for me | 11:46 |
| berendt | this should not be a probem, cinder only uses ceph as storage backend | 11:46 |
| *** strigazi is now known as strigazi_AFK | 11:48 | |
| *** dwalsh has quit IRC | 11:51 | |
| *** neilus has joined #openstack-kolla | 11:52 | |
| *** salv-orlando has quit IRC | 11:52 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Add guide for HNAS backend for Manila https://review.openstack.org/386552 | 11:52 |
| *** neilus has quit IRC | 11:56 | |
| *** akscram1 is now known as akscram | 11:58 | |
| *** liyifeng has joined #openstack-kolla | 11:58 | |
| *** ayoung has quit IRC | 12:00 | |
| *** dwalsh has joined #openstack-kolla | 12:04 | |
| *** liyifeng has quit IRC | 12:09 | |
| *** awiddersheim has quit IRC | 12:13 | |
| *** awiddersheim has joined #openstack-kolla | 12:13 | |
| *** asalkeld has quit IRC | 12:14 | |
| *** egonzalez90 has quit IRC | 12:18 | |
| *** egonzalez90 has joined #openstack-kolla | 12:19 | |
| *** srwilkers has joined #openstack-kolla | 12:27 | |
| *** lamt has quit IRC | 12:29 | |
| *** tonanhngo has joined #openstack-kolla | 12:31 | |
| *** tonanhngo has quit IRC | 12:32 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix keepalived upgrade permission issue https://review.openstack.org/386431 | 12:33 |
| sbezverk_ | morning, anybody could run "openstack catalog list" and paste it for me? | 12:36 |
| *** msimonin has quit IRC | 12:40 | |
| *** openstackstatus has quit IRC | 12:43 | |
| *** openstack has joined #openstack-kolla | 12:44 | |
| Jeffrey4l_ | np | 12:44 |
| *** openstackstatus has joined #openstack-kolla | 12:44 | |
| *** ChanServ sets mode: +v openstackstatus | 12:44 | |
| *** cristicalin has joined #openstack-kolla | 12:48 | |
| *** fguillot has joined #openstack-kolla | 12:50 | |
| *** salv-orlando has joined #openstack-kolla | 12:53 | |
| *** ayoung has joined #openstack-kolla | 12:54 | |
| *** goldyfruit has quit IRC | 12:54 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix ceilometer idempotent https://review.openstack.org/386576 | 12:55 |
| *** karlamrhein has quit IRC | 12:55 | |
| *** karlamrhein has joined #openstack-kolla | 12:56 | |
| *** salv-orlando has quit IRC | 12:58 | |
| *** egonzalez90 has quit IRC | 13:00 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Update Manila Guide https://review.openstack.org/386550 | 13:03 |
| *** sdake_ has joined #openstack-kolla | 13:03 | |
| *** cristicalin has quit IRC | 13:10 | |
| *** jheroux has joined #openstack-kolla | 13:11 | |
| *** unicell1 has joined #openstack-kolla | 13:12 | |
| *** unicell has quit IRC | 13:13 | |
| *** v1k0d3n has joined #openstack-kolla | 13:14 | |
| openstackgerrit | sean mooney proposed openstack/kolla: updates sources.list and apt prefereses filename to match dockerfile. https://review.openstack.org/386592 | 13:14 |
| *** tonanhngo has joined #openstack-kolla | 13:14 | |
| *** tonanhngo has quit IRC | 13:14 | |
| *** duonghq has joined #openstack-kolla | 13:15 | |
| duonghq | evening guys | 13:15 |
| *** goldyfruit has joined #openstack-kolla | 13:16 | |
| *** ayoung has quit IRC | 13:17 | |
| *** Guest10825 is now known as zeus` | 13:19 | |
| *** zeus` is now known as zeus | 13:19 | |
| *** zeus has quit IRC | 13:20 | |
| *** zeus has joined #openstack-kolla | 13:20 | |
| *** portdirect has quit IRC | 13:24 | |
| *** portdirect has joined #openstack-kolla | 13:24 | |
| sdake_ | morning peeps | 13:28 |
| goldyfruit | morning | 13:29 |
| *** jroll is now known as jrollinhatin | 13:29 | |
| *** jtriley has joined #openstack-kolla | 13:30 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix glance upgrade permission issue https://review.openstack.org/386431 | 13:31 |
| *** inc0 has joined #openstack-kolla | 13:33 | |
| inc0 | good morning | 13:33 |
| inc0 | sdake_, pbourke other cores please take a look https://review.openstack.org/#/q/topic:bug/1631503 | 13:38 |
| sdake_ | i see jeffrey4l was busy lsat night | 13:38 |
| inc0 | we need to ack these patches, fill the gaps (nova, mariadb any other) and also make backports | 13:38 |
| inc0 | all before 18th | 13:38 |
| inc0 | so let's get busy:) | 13:39 |
| *** Marcellin__ has joined #openstack-kolla | 13:39 | |
| Jeffrey4l_ | feel free to take one in https://etherpad.openstack.org/p/kolla-bug-1631503 | 13:39 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Add guide for HNAS backend for Manila https://review.openstack.org/386552 | 13:40 |
| inc0 | Jeffrey4l_, logs are issue too right? | 13:40 |
| inc0 | kolla_logs volume | 13:41 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Fixing cinder v1 and v2 endpoints https://review.openstack.org/386619 | 13:41 |
| Jeffrey4l_ | inc0, yep. all named volume matter | 13:41 |
| inc0 | fuck. | 13:41 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Add guide for HNAS backend for Manila https://review.openstack.org/386552 | 13:43 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix Stale namespace removal issue https://review.openstack.org/385559 | 13:44 |
| Jeffrey4l_ | inc0 sdake_ pbourke sbezverk_ , above ^^ one need more eyes and test too. | 13:45 |
| inc0 | yeah this change is scary | 13:45 |
| sdake_ | Jeffrey4l_ 6am dude, give me an hour to get rolling | 13:45 |
| sdake_ | ya scray because laurge | 13:45 |
| inc0 | no, scary because we don't know what might happen | 13:45 |
| sdake_ | and scary because we dont know what we need in run | 13:45 |
| *** jmccarthy has joined #openstack-kolla | 13:45 | |
| sdake_ | but it must be done | 13:46 |
| inc0 | it's not that large, it's just scary | 13:46 |
| Jeffrey4l_ | we dont know what we need in run <-- yes. | 13:46 |
| sdake_ | so lets answer what we need in run | 13:46 |
| sdake_ | that makess the patch less scary | 13:46 |
| sdake_ | and esay to confirm | 13:46 |
| inc0 | Jeffrey4l_, can we create some prothesis for Newton tho | 13:46 |
| inc0 | ? | 13:46 |
| inc0 | that 100% won't break anything | 13:46 |
| sdake_ | inc0 i don't understand your proposal | 13:47 |
| inc0 | and fix it correctly in Ocata | 13:47 |
| Jeffrey4l_ | what prothesis is? | 13:47 |
| sdake_ | inc0 you mean like some kind of conditional logic? | 13:47 |
| Jeffrey4l_ | workaround? | 13:47 |
| sdake_ | Jeffrey4l_ it means a fake body part | 13:47 |
| inc0 | yeah, workaround | 13:47 |
| Jeffrey4l_ | np. this is the only way to fix. | 13:47 |
| Jeffrey4l_ | no workaround. | 13:47 |
| inc0 | duh ok | 13:47 |
| jmccarthy | Hi all o/ | 13:47 |
| sdake_ | ya no workaround | 13:47 |
| sdake_ | so we need to make damn sure we map what is needed in /run | 13:48 |
| Jeffrey4l_ | i saw this bug more than half year. | 13:48 |
| sdake_ | and make that available | 13:48 |
| sdake_ | Jeffrey4l_ ack - it is crucial too | 13:48 |
| Jeffrey4l_ | it is critical and no other workaround. | 13:48 |
| sdake_ | not being able to delete routers is a huge problem | 13:48 |
| Jeffrey4l_ | yes. | 13:48 |
| inc0 | yeah | 13:48 |
| inc0 | but neutron is most affected right? | 13:48 |
| Jeffrey4l_ | i also hope this can be backport to mitaka branch. ;( | 13:48 |
| sdake_ | Jeffrey4l_ yes on backport | 13:49 |
| inc0 | can we separate neutron to another patchset and prioritize it? | 13:49 |
| Jeffrey4l_ | one the namespace matter. so neutron, yes. | 13:49 |
| Jeffrey4l_ | inc0, no. | 13:49 |
| wznoinsk | is https://etherpad.openstack.org/p/kolla-bug-1631503 only for 'fix' patches or general candidates for backporting? | 13:49 |
| inc0 | duh | 13:49 |
| Jeffrey4l_ | the root cause is: you mount both /run and /run/netns in docker. ( no matter which container mount it) | 13:49 |
| sdake_ | Jeffrey4l_ inc0 makes a good point | 13:49 |
| sdake_ | Jeffrey4l_ what other containers beyond neutron are affected at present | 13:50 |
| Jeffrey4l_ | if you mount /run in keystone and mount /run/netns in neutron, it will trigger this issue too. | 13:50 |
| inc0 | ok | 13:50 |
| inc0 | so we need to remove all mount /run | 13:50 |
| Jeffrey4l_ | correct. | 13:50 |
| inc0 | it would be good to somehow figure out which projects use /run | 13:51 |
| Jeffrey4l_ | https://review.openstack.org/#/c/385559/ removed all /run ( at least i grep /run:/run got nothing after the PS ) | 13:51 |
| Jeffrey4l_ | grep | 13:51 |
| inc0 | let's do git blame on every /run mount to see context in which it was added | 13:51 |
| inc0 | and test case this patchset enabled | 13:52 |
| inc0 | brb need to grab food and coffiee, then I'll get to it | 13:52 |
| *** mgoddard has joined #openstack-kolla | 13:52 | |
| Jeffrey4l_ | tbh, git blame should got nothing. it won't tell us what really is the service use in /run. | 13:52 |
| sdake_ | right | 13:53 |
| sdake_ | i have some deep expertise in why things use run and why they dont | 13:53 |
| sdake_ | get me a list of what uses run in the ehterpad and i'll add some commentary | 13:53 |
| sdake_ | lets map it in etherpad | 13:53 |
| Jeffrey4l_ | ok. i will prepare the list. | 13:54 |
| Jeffrey4l_ | wait for a moment. | 13:54 |
| *** salv-orlando has joined #openstack-kolla | 13:54 | |
| Jeffrey4l_ | sdake_, check this https://etherpad.openstack.org/p/kolla-bug-1616268 | 13:56 |
| sdake_ | Jeffrey4l_ i will do after i warm up and get done with morning emergencies internally | 13:57 |
| Jeffrey4l_ | ok. cool | 13:58 |
| *** salv-orlando has quit IRC | 13:59 | |
| *** mgoddard has quit IRC | 13:59 | |
| *** mgoddard_ has joined #openstack-kolla | 13:59 | |
| *** MarMat has joined #openstack-kolla | 13:59 | |
| *** sdake has joined #openstack-kolla | 14:00 | |
| *** tonanhngo has joined #openstack-kolla | 14:01 | |
| *** yingjun has joined #openstack-kolla | 14:02 | |
| janem | Im getting a Ceph Health warning: too many PGs per OSD (576 > max 300) | 14:02 |
| *** tonanhngo has quit IRC | 14:03 | |
| janem | something I should worry about? | 14:03 |
| *** sdake_ has quit IRC | 14:04 | |
| Jeffrey4l_ | janem, no. it won't cause big issue. and if you have a large cluster, this warning will disappear. | 14:04 |
| *** mgoddard_ has quit IRC | 14:04 | |
| janem | Jeffrey4l_: its an all-in-one | 14:05 |
| sdake | janem no that is just a warning | 14:05 |
| Jeffrey4l_ | janem, why u have three, this warning will disappear. | 14:05 |
| openstackgerrit | James McCarthy proposed openstack/kolla: Remove indentation from l3_agent.ini.j2 https://review.openstack.org/386628 | 14:05 |
| janem | Jeffrey4l_: why I have three? | 14:06 |
| *** mgoddard has joined #openstack-kolla | 14:06 | |
| Jeffrey4l_ | hmm. you need more osds. | 14:06 |
| *** msimonin has quit IRC | 14:07 | |
| Jeffrey4l_ | the PGs / number of osd will < 300 | 14:07 |
| Jeffrey4l_ | this is how ceph works. | 14:07 |
| janem | well, I'm not experienced with Ceph, and this is an all-in-one.. and the disk used for ceph is a 4*1TB raid 10 drive... | 14:08 |
| sbezverk_ | Jeffrey4l_ about that run PS, I think remving /run will break iscsi | 14:09 |
| sdake | janem afor aio ceph specific steps must be followed | 14:09 |
| janem | I guess raid under ceph is generally not recomended(?) | 14:09 |
| sdake | janem di dyou follow those? | 14:09 |
| Jeffrey4l_ | what exact /run/xxx folder iscsi need? | 14:09 |
| sdake | ceph-guide.rst | 14:09 |
| sdake | iscsi needs a mulligan sbezverk_ | 14:09 |
| jmccarthy | Is there a test for neutron and dvr ? I could not get this to work .. | 14:09 |
| Jeffrey4l_ | sbezverk_, we have https://etherpad.openstack.org/p/kolla-bug-1616268 we can make comment on it. | 14:09 |
| janem | sdake: yes... added config for ceph with default and max set to 1 | 14:10 |
| janem | sdake: if thats what you mean.. | 14:10 |
| janem | sdake: everything is working as expected.. just saw this warning I mentioned.. | 14:11 |
| sbezverk_ | Jeffrey4l_: iscsid creates socker and pid file right in /run | 14:14 |
| sbezverk_ | oops, I meant socket and pid files. | 14:14 |
| jmccarthy | If anyone has a minute to check out https://review.openstack.org/386628 that'd be great :) | 14:14 |
| *** tonanhngo has joined #openstack-kolla | 14:15 | |
| Jeffrey4l_ | sbezverk_, does pid file matter? | 14:15 |
| *** tonanhngo has quit IRC | 14:15 | |
| Jeffrey4l_ | sbezverk_, and i do not find the socket file in /run . what the full path of the socket file? | 14:16 |
| sbezverk_ | Jeffrey4l_: iscsiadm which is called by nova-compute check is iscsid is up and running, I do not know all its internals, but it uses /run for figuring it out.. | 14:16 |
| *** mgoddard has quit IRC | 14:17 | |
| Jeffrey4l_ | hmm. if it really use like that. we need figure out some workaround. we must drop the use of /run. | 14:19 |
| sdake | sbezverk_ it uses a filel somewhere in /run | 14:21 |
| sdake | we need to find out which file | 14:21 |
| Jeffrey4l_ | hmm i think iscsiadm talk with iscsid through /sys file. | 14:22 |
| *** VRspace4 has joined #openstack-kolla | 14:22 | |
| sbezverk_ | Jeffrey4l_: I do not have kolla running, but the best way would be to give it a spin without /run and see.. I doubt it will work.. but I might be wrong.. | 14:23 |
| Jeffrey4l_ | ok. i may be try that later. | 14:23 |
| *** berendt has quit IRC | 14:26 | |
| *** jtriley has quit IRC | 14:29 | |
| *** haplo37_ has quit IRC | 14:34 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix glance upgrade permission issue https://review.openstack.org/386431 | 14:34 |
| *** goldyfruit1 has joined #openstack-kolla | 14:34 | |
| *** goldyfruit has quit IRC | 14:34 | |
| *** lamt has joined #openstack-kolla | 14:35 | |
| *** mgoddard has joined #openstack-kolla | 14:36 | |
| *** haplo37_ has joined #openstack-kolla | 14:36 | |
| *** huhaoran has joined #openstack-kolla | 14:36 | |
| *** mnasiadka has quit IRC | 14:37 | |
| *** sdake_ has joined #openstack-kolla | 14:38 | |
| *** msimonin has joined #openstack-kolla | 14:39 | |
| *** salv-orlando has joined #openstack-kolla | 14:40 | |
| *** ayoung has joined #openstack-kolla | 14:40 | |
| *** jtriley has joined #openstack-kolla | 14:40 | |
| *** sdake_ has quit IRC | 14:40 | |
| *** sdake has quit IRC | 14:41 | |
| *** tonanhngo has joined #openstack-kolla | 14:43 | |
| *** salv-orl_ has joined #openstack-kolla | 14:44 | |
| *** michauds has joined #openstack-kolla | 14:44 | |
| *** neilus has joined #openstack-kolla | 14:46 | |
| *** salv-orlando has quit IRC | 14:46 | |
| *** neilus has quit IRC | 14:50 | |
| *** huikang has joined #openstack-kolla | 14:51 | |
| *** sdake has joined #openstack-kolla | 14:53 | |
| *** tonanhngo has quit IRC | 14:53 | |
| *** tonanhngo has joined #openstack-kolla | 14:54 | |
| sdake | Jeffrey4l_ before we get too far down this path: https://review.openstack.org/#/c/386471/1 | 14:54 |
| *** papacz has joined #openstack-kolla | 14:55 | |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla: Check api_interface instead of network_interface https://review.openstack.org/386654 | 14:55 |
| Jeffrey4l_ | sdake, yes it is wrong. | 14:56 |
| Jeffrey4l_ | actually, horizon is no need to touch. all the logs are own by root. | 14:56 |
| *** portdirect has quit IRC | 14:57 | |
| *** portdirect_ has joined #openstack-kolla | 14:58 | |
| *** portdirect_ is now known as portdirect | 14:59 | |
| *** mgoddard has quit IRC | 15:01 | |
| openstackgerrit | Martin André proposed openstack/kolla: Consistently set sys path when using local tools https://review.openstack.org/364735 | 15:02 |
| *** mgoddard has joined #openstack-kolla | 15:03 | |
| *** sdake_ has joined #openstack-kolla | 15:03 | |
| *** strigazi_AFK is now known as strigazi | 15:04 | |
| *** VRspace4 has quit IRC | 15:05 | |
| *** sdake has quit IRC | 15:05 | |
| *** dave-mccowan has quit IRC | 15:09 | |
| *** huhaoran has quit IRC | 15:12 | |
| *** dave-mccowan has joined #openstack-kolla | 15:24 | |
| *** rmart04 has quit IRC | 15:25 | |
| openstackgerrit | Merged openstack/kolla: Fix ceilometer idempotent https://review.openstack.org/386576 | 15:26 |
| *** Serlex has quit IRC | 15:26 | |
| *** yingjun has quit IRC | 15:26 | |
| *** yingjun has joined #openstack-kolla | 15:27 | |
| openstackgerrit | Merged openstack/kolla: Fix keystone upgrade permission issue https://review.openstack.org/386421 | 15:28 |
| *** srwilkers has quit IRC | 15:30 | |
| pbourke | bjolo: long shot, any chance this would help? https://review.openstack.org/#/c/382356/ | 15:31 |
| sdake_ | Jeffrey4l_ this patch looks worng too https://review.openstack.org/#/c/386421/ | 15:31 |
| sdake_ | folks - stop merging the upgrading patches until we are sure they are right :) | 15:31 |
| *** huikang has quit IRC | 15:31 | |
| *** yingjun has quit IRC | 15:31 | |
| inc0 | sdake_, whats wrong about it? | 15:32 |
| inc0 | ahh, ks being root? | 15:32 |
| sdake_ | /var/log/kolla should not have permissions changed | 15:32 |
| inc0 | you sure tho? | 15:32 |
| sdake_ | or it needs special handling | 15:32 |
| sdake_ | i am not sure | 15:32 |
| inc0 | I was thinking about it, and it might actually need it | 15:32 |
| sdake_ | Jeffrey4l_ indicated earlier horizon patch was wrong | 15:32 |
| sdake_ | the log file needs changing | 15:33 |
| inc0 | horizon doesn't need changing as it's running root | 15:33 |
| inc0 | that's different | 15:33 |
| sdake_ | not sure on the entire directory | 15:33 |
| sdake_ | well i didn't think of horizon as running as root and that was not my objection :) | 15:33 |
| sdake_ | here is the deal - if every upgrade goes around chowning /var/log/kolla to its own id | 15:33 |
| sdake_ | does that really make any sense? | 15:34 |
| sdake_ | it needs to obviously chown /var/log/kolla/service | 15:34 |
| sdake_ | and the files therein | 15:34 |
| inc0 | or do GID writable and create kolla group in base | 15:34 |
| inc0 | and we do add group kolla in base | 15:36 |
| sdake_ | kolla group already created in base | 15:36 |
| sdake_ | we use this all over the place | 15:36 |
| sdake_ | i think gid should chane on this file not owner | 15:36 |
| inc0 | so what we need to ensure is that chown is there for gid sync | 15:36 |
| sdake_ | i think Jeffrey4l_ agreed | 15:36 |
| inc0 | yeah ownership is different,. but we don't really have good owner there | 15:36 |
| sdake_ | on this file/on this directory (/var/log/kolla) | 15:36 |
| pbourke | please put -workflow on them if they're not tested | 15:37 |
| pbourke | otherwise I generally give benefit of the doubt | 15:37 |
| sdake_ | pbourke understood | 15:37 |
| *** lamt has quit IRC | 15:37 | |
| inc0 | https://github.com/openstack/kolla/blob/17d5c32f0b0b87dfacfa08879883ad6a22a866db/docker/base/Dockerfile.j2#L288 | 15:37 |
| sdake_ | i didn't submit patches :) | 15:37 |
| inc0 | ok, I guess it doesn't really matter, we don't need to chown anything in /var/log/kolla | 15:38 |
| inc0 | although | 15:38 |
| inc0 | we might | 15:38 |
| inc0 | as files exists | 15:38 |
| sdake_ | /var/log/kolla/service | 15:38 |
| inc0 | so we will change owner but as long as files are gid-writable it's actually good change | 15:38 |
| sdake_ | if gids move around (which they do) v/ar/log/kolla gid needs change at runtime | 15:39 |
| sdake_ | build time is not good enough | 15:39 |
| inc0 | so the change is good | 15:39 |
| sdake_ | [07:54:40] <sdake>Jeffrey4l_ before we get too far down this path: https://review.openstack.org/#/c/386471/1 | 15:40 |
| sdake_ | [07:55:18] papacz (~papacz@stpaul-nat.cray.com) joined the channel | 15:40 |
| sdake_ | [07:55:52] <openstackgerrit>Vladislav Belogrudov proposed openstack/kolla: Check api_interface instead of network_interface https://review.openstack.org/386654 | 15:40 |
| sdake_ | [07:56:02] <Jeffrey4l_>sdake, yes it is wrong. | 15:40 |
| sdake_ | [07:56:29] <Jeffrey4l_>actually, horizon is no need to touch. all the logs are own by root. | 15:40 |
| sdake_ | i guess Jeffrey4l_ wass talking about the fact that horizon was owned by root | 15:40 |
| sdake_ | rather then the real point that /var/log/kolla is unnecessary or inaccurate in that change | 15:40 |
| inc0 | sooo | 15:42 |
| inc0 | interesting | 15:42 |
| inc0 | ls -la of /var/log/kolla in horizon shows something like that: | 15:42 |
| inc0 | http://paste.openstack.org/show/585727/ | 15:43 |
| inc0 | from inside nova it's mostly nova user | 15:43 |
| sdake_ | ya thats because the uids dont match up | 15:43 |
| inc0 | what I'm saying is horizon user owns nova dir | 15:43 |
| inc0 | in nova dir nova user owns nova dir | 15:44 |
| sdake_ | (in horizon container) | 15:44 |
| *** janem has quit IRC | 15:44 | |
| sdake_ | yes this is because of inconsistent uids | 15:44 |
| sdake_ | this is why chowning /var/lib/kolla is not an ideal solution :) | 15:44 |
| inc0 | docker will create uset with same uid/gid | 15:44 |
| sdake_ | /var/log/kolla/nova - sure | 15:44 |
| inc0 | nova uid == horizon uid | 15:44 |
| inc0 | which means this problem becomes much easier | 15:45 |
| *** srwilkers has joined #openstack-kolla | 15:45 | |
| inc0 | we don't need to care about containers that doesn't create users as part of package install | 15:45 |
| inc0 | that means this issue is in rabbitmq and mariadb | 15:45 |
| sdake_ | dont look at /var/log - think about /var/lib instead | 15:45 |
| inc0 | and I think that's it | 15:45 |
| sdake_ | /var/log i a wierd case | 15:45 |
| inc0 | so here's thing | 15:46 |
| sdake_ | but the patches as is are wrong | 15:46 |
| inc0 | if you do useradd nova today and same thing tomorrow | 15:46 |
| inc0 | it will have same uid I think | 15:46 |
| sdake_ | that is no tt he case | 15:46 |
| sdake_ | unless uids are hardcoded | 15:46 |
| inc0 | you sure? | 15:46 |
| sdake_ | a 1 day lapse - ya same thing | 15:46 |
| sdake_ | a 6 month lapse - maybe not same thing | 15:46 |
| sdake_ | the problems are in /var/lib/* not /var/log | 15:47 |
| inc0 | horizon gets uid 1000 | 15:47 |
| *** jrich523 has quit IRC | 15:47 | |
| sdake_ | but /var/log needs addressing | 15:47 |
| inc0 | which seems awfully specific to me | 15:47 |
| sdake_ | ya - centos counts from 1k down | 15:47 |
| sdake_ | on *random* assignments | 15:47 |
| inc0 | I'm running ubuntu | 15:47 |
| sdake_ | ubuntu counts from 100 up on random assignments | 15:47 |
| inc0 | and yet horizon gets 1000 | 15:48 |
| sdake_ | ok - so perhaps horizon has a hardcode uid | 15:48 |
| sdake_ | or are you looking at from source build? | 15:48 |
| inc0 | nova has uid 1000 | 15:48 |
| sdake_ | ya your looking at from source | 15:49 |
| inc0 | this is source, I'll build binary and we'll see | 15:49 |
| sdake_ | binary may have hardcodes | 15:49 |
| sdake_ | its typical of distros to hardcode uids for network facing services | 15:49 |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla: Check api_interface instead of network_interface https://review.openstack.org/386654 | 15:49 |
| sdake_ | the problem is - if you look at the 2.0.3 images, nova is not 1000 | 15:49 |
| *** mgoddard has quit IRC | 15:49 | |
| sdake_ | 14.04 counts from 100 up | 15:49 |
| sdake_ | 16.04 counts from 1k to something else :) | 15:50 |
| sdake_ | inconsistent uids | 15:50 |
| sdake_ | upgrade crater | 15:50 |
| sdake_ | same problem | 15:50 |
| sdake_ | build 2.0.3 and inspect that :) | 15:50 |
| *** dave-mccowan has quit IRC | 15:51 | |
| inc0 | same thing | 15:51 |
| inc0 | 1000 | 15:51 |
| sdake_ | moment switchign networks | 15:52 |
| *** sdake has joined #openstack-kolla | 15:54 | |
| sdake | inc0: http://paste.fedoraproject.org/450208/47646049/ -> http://paste.fedoraproject.org/450208/47646049 | 15:55 |
| sdake | has anyone tried ubuntu upgrade? | 15:55 |
| sdake | that should probably be step 1 here | 15:55 |
| *** sdake_ has quit IRC | 15:56 | |
| sdake | anything with state needs fixing | 15:57 |
| sdake | but /var/log/kolla is not something that needs changing imo | 15:57 |
| sdake | yet is changed in all of Jeffrey4l_ 's patchsets | 15:57 |
| sdake | /var/log/kolla/service needs changing | 15:57 |
| openstackgerrit | Merged openstack/kolla: Remove indentation from l3_agent.ini.j2 https://review.openstack.org/386628 | 15:57 |
| *** srwilkers has quit IRC | 15:59 | |
| kfox1111 | morning | 15:59 |
| *** neilus has joined #openstack-kolla | 16:01 | |
| *** dave-mccowan has joined #openstack-kolla | 16:01 | |
| inc0 | sdake, it needs changig | 16:03 |
| inc0 | to ensure kolla gid | 16:03 |
| sdake | inc0 fair enough | 16:03 |
| sdake | inc0 but not the UID | 16:03 |
| inc0 | we need an uid | 16:03 |
| inc0 | regardless which one | 16:03 |
| sdake | chgrp? | 16:03 |
| inc0 | I don't think changing whole dir will matter | 16:04 |
| openstackgerrit | Merged openstack/kolla: Move demos and dev directory into a new contrib directory https://review.openstack.org/379500 | 16:04 |
| inc0 | dir will be gid writable and service is changed too | 16:04 |
| *** neilus has quit IRC | 16:05 | |
| *** spsingh has joined #openstack-kolla | 16:06 | |
| *** jtriley has quit IRC | 16:07 | |
| *** awebber has joined #openstack-kolla | 16:07 | |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla: Check api_interface instead of network_interface https://review.openstack.org/386654 | 16:11 |
| openstackgerrit | Merged openstack/kolla: Use sudo to create loop device on gate nodes https://review.openstack.org/380127 | 16:14 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Ansible config for nova-serialproxy console https://review.openstack.org/358839 | 16:14 |
| *** jtriley has joined #openstack-kolla | 16:14 | |
| openstackgerrit | Merged openstack/kolla: Fix glance upgrade permission issue https://review.openstack.org/386431 | 16:18 |
| *** pbourke has quit IRC | 16:18 | |
| *** jmccarthy has quit IRC | 16:18 | |
| *** shardy is now known as shardy_afk | 16:22 | |
| *** haplo37 has quit IRC | 16:24 | |
| *** g3ek has quit IRC | 16:24 | |
| *** dwalsh has quit IRC | 16:24 | |
| *** ankush has quit IRC | 16:30 | |
| v1k0d3n | hey rhallisey, kfox1111 and i were talking a bit last night about that repo i worked on for vagrant kubernetes. wanted to get your thoughts and have you try it out. | 16:30 |
| v1k0d3n | https://github.com/att-comdev/halcyon-kubernetes | 16:30 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Port prechecks only for enabled services https://review.openstack.org/377962 | 16:30 |
| v1k0d3n | it could be useful for multi-node kube development for kolla-kube. | 16:30 |
| v1k0d3n | could be used as a submodule | 16:30 |
| kfox1111 | yeah. I think so. :) Would like to see vagrent be optional. and see centos support too. | 16:31 |
| *** huikang has joined #openstack-kolla | 16:32 | |
| v1k0d3n | yeah, kfox1111 i have a request out here to this repo...https://github.com/skippbox/kubeadm-centos/issues/2 | 16:32 |
| v1k0d3n | submodule works really well for "optional" repos like a vagrant environment. | 16:33 |
| kfox1111 | upvoted. :) | 16:33 |
| v1k0d3n | :) | 16:33 |
| *** g3ek has joined #openstack-kolla | 16:33 | |
| rhallisey | neat | 16:33 |
| *** haplo37 has joined #openstack-kolla | 16:34 | |
| rhallisey | v1k0d3n, +1 | 16:34 |
| kfox1111 | v1k0d3n: do you know if the helm mariadb charts are planned to go clustered any time soon? | 16:37 |
| *** ssurana has joined #openstack-kolla | 16:37 | |
| *** huikang has quit IRC | 16:37 | |
| sbezverk_ | kfox1111: need you check this https://review.openstack.org/#/c/386619/1 | 16:38 |
| *** sdake_ has joined #openstack-kolla | 16:38 | |
| kfox1111 | looking | 16:38 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Add guide for HNAS backend for Manila https://review.openstack.org/386552 | 16:39 |
| kfox1111 | http://logs.openstack.org/19/386619/1/experimental/gate-kolla-kubernetes-deploy-ubuntu-binary-ceph-nv/9aebb4e/console.html looks like there is a slight race in the gate... | 16:40 |
| kfox1111 | detaching != in-use.... | 16:40 |
| kfox1111 | so it transitions from in-use -> detaching -> available... | 16:40 |
| *** sdake has quit IRC | 16:40 | |
| kfox1111 | sbezverk_: this is starting to highlight we really need a defaults.yml or something. as anyone who's coppied kolla-kubernetes.yaml before this change will be broken after it. | 16:42 |
| kfox1111 | looks good to me though. | 16:42 |
| *** papacz has quit IRC | 16:43 | |
| openstackgerrit | Merged openstack/kolla: Update Manila Guide https://review.openstack.org/386550 | 16:43 |
| *** rmart04 has joined #openstack-kolla | 16:46 | |
| *** unicell1 has quit IRC | 16:46 | |
| jascott1 | sbezverk_ thanks for the corrections on the cinder and neutron workflows. I bet you can tell Im shooting in the dark :) | 16:47 |
| kfox1111 | v1k0d3n: know of a good tool to docker pull a set of images, do yum upgrade /apt-get upgrade in them, and trigger something if they have packages to update? | 16:49 |
| kfox1111 | I've been tempated to write something several times now but haven't gotten around to it yet. | 16:49 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Fix mariadb upgrade permission issue https://review.openstack.org/386515 | 16:51 |
| *** janem has joined #openstack-kolla | 16:52 | |
| sbezverk_ | jascott1: no worries, more you play with it more you learn :-) | 16:53 |
| *** rmart04_ has joined #openstack-kolla | 16:54 | |
| *** mgoddard has joined #openstack-kolla | 16:56 | |
| *** rmart04 has quit IRC | 16:57 | |
| *** rmart04_ is now known as rmart04 | 16:57 | |
| otavio | Jeffrey4l_: the https://review.openstack.org/#/c/383865/ needs to be merged on master and newton | 16:59 |
| otavio | sdake_: o/ | 16:59 |
| *** mgoddard has quit IRC | 17:00 | |
| *** zhubingbing has joined #openstack-kolla | 17:00 | |
| *** athomas has quit IRC | 17:00 | |
| *** srwilkers has joined #openstack-kolla | 17:01 | |
| *** zhubingbing has quit IRC | 17:02 | |
| *** msimonin has quit IRC | 17:08 | |
| *** zhubingbing has joined #openstack-kolla | 17:10 | |
| *** michauds has quit IRC | 17:12 | |
| *** unicell has joined #openstack-kolla | 17:14 | |
| *** rmart04 has quit IRC | 17:15 | |
| *** duonghq has quit IRC | 17:16 | |
| *** huikang has joined #openstack-kolla | 17:17 | |
| *** sean-k-m1 has joined #openstack-kolla | 17:18 | |
| *** sean-k-mooney has quit IRC | 17:21 | |
| *** athomas has joined #openstack-kolla | 17:21 | |
| *** pbourke has joined #openstack-kolla | 17:28 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Add guide for HNAS backend for Manila https://review.openstack.org/386552 | 17:28 |
| *** huikang has quit IRC | 17:32 | |
| *** neilus__ has joined #openstack-kolla | 17:32 | |
| *** huikang has joined #openstack-kolla | 17:32 | |
| sdake_ | otavio that bug has been triaged | 17:33 |
| *** huikang has quit IRC | 17:36 | |
| *** huikang has joined #openstack-kolla | 17:36 | |
| v1k0d3n | kfox1111: not that i can think of off the top of my head...(thinking) | 17:38 |
| *** huikang has quit IRC | 17:38 | |
| v1k0d3n | kfox1111: not sure about the mariadb charts either. | 17:38 |
| v1k0d3n | man, i'm useless today :) lol | 17:38 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Fix nova upgrade permission issue https://review.openstack.org/386709 | 17:39 |
| *** sean-k-mooney has joined #openstack-kolla | 17:42 | |
| kfox1111 | v1k0d3n: no worries. I think those are the top two issues on my k8s list. been working twoards solving them for a while. | 17:42 |
| kfox1111 | I think k8s will have something really impressive on their hands if they can make those work out of the box trivally. | 17:42 |
| *** absubram has quit IRC | 17:43 | |
| *** mnasiadka has joined #openstack-kolla | 17:43 | |
| *** jtriley has quit IRC | 17:43 | |
| *** mliima_ has joined #openstack-kolla | 17:43 | |
| *** sean-k-m1 has quit IRC | 17:44 | |
| *** mliima has quit IRC | 17:46 | |
| *** athomas has quit IRC | 17:47 | |
| *** mliima__ has joined #openstack-kolla | 17:48 | |
| *** tonanhngo has quit IRC | 17:49 | |
| kfox1111 | uh... how is the gate check working with ceph... the admin key upload portion isn't there anymore.... | 17:50 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix stale namespace removal issue https://review.openstack.org/385559 | 17:50 |
| *** mliima_ has quit IRC | 17:52 | |
| *** Jeffrey4l_ has quit IRC | 17:52 | |
| *** spsingh has quit IRC | 17:53 | |
| sbezverk_ | kfox1111: so is it a PS issue or something related just to gate? | 17:55 |
| *** lamt has joined #openstack-kolla | 17:56 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add guide for HNAS backend for Manila https://review.openstack.org/386552 | 17:58 |
| otavio | sdake_: but the issue is still happening on mitaka and newton | 18:01 |
| *** portdirect has quit IRC | 18:02 | |
| kfox1111 | sbezverk_: a gate issue. there is a slight race in that there's an extra state a cinder volume could go in, and it fails if it ever sees that state. its a quick state though, so I never cought it in my testing. | 18:02 |
| kfox1111 | usually I just saw in-use -> available. but it goes in-use -> detaching -> available. but mostly its so quick only you notice in-use -> available. | 18:03 |
| *** portdirect has joined #openstack-kolla | 18:03 | |
| otavio | Is there someone working on Trove support? | 18:05 |
| mnasiadka | blueprints say yes - https://blueprints.launchpad.net/kolla/+spec/ansible-trove | 18:06 |
| *** huikang has joined #openstack-kolla | 18:06 | |
| openstackgerrit | Merged openstack/kolla: ansible: swift: Fix swift-object-expirer restart loop https://review.openstack.org/383865 | 18:08 |
| otavio | zhubingbing: I noticed there is a minor issue still pending for the trove bp to be merged. Are you looking at it? Is there something we can help with? | 18:12 |
| *** huikang has quit IRC | 18:12 | |
| *** huikang has joined #openstack-kolla | 18:13 | |
| *** jtriley has joined #openstack-kolla | 18:14 | |
| *** dwalsh has joined #openstack-kolla | 18:17 | |
| *** huikang has quit IRC | 18:17 | |
| *** zhubingbing has quit IRC | 18:19 | |
| *** jtriley has quit IRC | 18:19 | |
| openstackgerrit | Otavio Salvador proposed openstack/kolla: ansible: swift: Fix swift-object-expirer restart loop https://review.openstack.org/386730 | 18:19 |
| *** celo_r has joined #openstack-kolla | 18:21 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Fix minor ceph transition state issue with the gate https://review.openstack.org/386731 | 18:21 |
| *** fguillot has quit IRC | 18:22 | |
| kfox1111 | sbezverk_: I think that patch might fix the gate random failure. if so, we should merge it real quick and rebase your one on it so we can test. | 18:23 |
| *** salv-orlando has joined #openstack-kolla | 18:23 | |
| *** salv-orl_ has quit IRC | 18:25 | |
| *** michauds has joined #openstack-kolla | 18:25 | |
| *** jtriley has joined #openstack-kolla | 18:30 | |
| *** MarMat_ has joined #openstack-kolla | 18:33 | |
| *** tonanhngo has joined #openstack-kolla | 18:33 | |
| *** tonanhngo has quit IRC | 18:33 | |
| *** tonanhngo has joined #openstack-kolla | 18:34 | |
| *** inc0 has quit IRC | 18:34 | |
| *** MarMat has quit IRC | 18:35 | |
| *** huikang has joined #openstack-kolla | 18:36 | |
| *** goldyfruit1 has quit IRC | 18:38 | |
| sbezverk_ | kfox1111: done | 18:39 |
| sbezverk_ | will rebase when it gets merged | 18:39 |
| *** severion has joined #openstack-kolla | 18:42 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Fix minor ceph transition state issue with the gate https://review.openstack.org/386731 | 18:42 |
| *** peterjenkins__ has joined #openstack-kolla | 18:43 | |
| *** derekjhyang_ has joined #openstack-kolla | 18:43 | |
| *** nick-ma_ has joined #openstack-kolla | 18:44 | |
| *** pbourke_ has joined #openstack-kolla | 18:45 | |
| *** dotplus_ has joined #openstack-kolla | 18:45 | |
| *** dotplus_ has joined #openstack-kolla | 18:45 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Fixing cinder v1 and v2 endpoints https://review.openstack.org/386619 | 18:45 |
| *** cmceniry has joined #openstack-kolla | 18:46 | |
| *** Aju has joined #openstack-kolla | 18:49 | |
| *** irtermit- has joined #openstack-kolla | 18:50 | |
| *** mark-casey1 has joined #openstack-kolla | 18:51 | |
| *** ntpttr_ has joined #openstack-kolla | 18:51 | |
| *** jtriley has quit IRC | 18:51 | |
| *** lamt has quit IRC | 18:51 | |
| *** pbourke has quit IRC | 18:51 | |
| *** v1k0d3n has quit IRC | 18:51 | |
| *** rhallisey has quit IRC | 18:51 | |
| *** timss has quit IRC | 18:51 | |
| *** derekjhyang has quit IRC | 18:51 | |
| *** ntpttr has quit IRC | 18:51 | |
| *** peterjenkins_ has quit IRC | 18:51 | |
| *** Marx314 has quit IRC | 18:51 | |
| *** NobodyCam has quit IRC | 18:51 | |
| *** irtermite has quit IRC | 18:51 | |
| *** kklimonda has quit IRC | 18:51 | |
| *** jistr has quit IRC | 18:51 | |
| *** qwang has quit IRC | 18:51 | |
| *** afranc has quit IRC | 18:51 | |
| *** cmceniry_ has quit IRC | 18:51 | |
| *** dotplus has quit IRC | 18:51 | |
| *** nick-ma has quit IRC | 18:51 | |
| *** peterjenkins__ is now known as peterjenkins_ | 18:52 | |
| *** derekjhyang_ is now known as derekjhyang | 18:52 | |
| *** nick-ma_ is now known as nick-ma | 18:53 | |
| *** kklimonda has joined #openstack-kolla | 18:53 | |
| *** jistr has joined #openstack-kolla | 18:54 | |
| *** goldyfruit has joined #openstack-kolla | 18:56 | |
| *** timss has joined #openstack-kolla | 18:56 | |
| *** jmccarthy has joined #openstack-kolla | 18:57 | |
| *** jtriley has joined #openstack-kolla | 18:59 | |
| *** akwasnie has quit IRC | 18:59 | |
| *** NobodyCam has joined #openstack-kolla | 19:00 | |
| *** akwasnie has joined #openstack-kolla | 19:01 | |
| *** salv-orlando has quit IRC | 19:02 | |
| *** qwang has joined #openstack-kolla | 19:03 | |
| *** mliima__ has quit IRC | 19:04 | |
| *** mliima has joined #openstack-kolla | 19:05 | |
| kfox1111 | arg... | 19:05 |
| kfox1111 | the cloud app echosystem will never progress until "apps" stop doing stuff like this: https://github.com/bitnami/charts/blob/master/owncloud/charts/mariadb/templates/deployment.yaml | 19:06 |
| kfox1111 | cool, a really easy to deploy owncloud. | 19:06 |
| kfox1111 | oh, but if you dig in, no data is ever saved. completely ephemeral. :/ | 19:06 |
| *** 7YUAAGGIF has joined #openstack-kolla | 19:07 | |
| *** manjeets has quit IRC | 19:08 | |
| *** karlamrhein has quit IRC | 19:09 | |
| *** karlamrhein has joined #openstack-kolla | 19:09 | |
| kfox1111 | ah. in the docs it says to start hacking up the yaml scripts to make it persistant. Not easy to use for newbies. :/ | 19:09 |
| kfox1111 | that really needs to be templated. | 19:09 |
| *** mliima has quit IRC | 19:10 | |
| severion | kfox1111: you're working on a lot of the gating for kolla-kube, right? | 19:12 |
| kfox1111 | yeah. | 19:13 |
| severion | crap...just realized my name failed over again...sorry..one sec (brb) | 19:13 |
| *** severion has quit IRC | 19:13 | |
| *** v1k0d3n has joined #openstack-kolla | 19:13 | |
| kfox1111 | wb. :) | 19:13 |
| v1k0d3n | ok, better... | 19:13 |
| v1k0d3n | :) | 19:13 |
| v1k0d3n | so, soon...i would probably like to get our CI/CD owner involved...his team works on our current CI/CD pipeline, etc. he may be able to contribute/help and offer some ideas on what we experience today; what we'd like to fix "tomorrow". | 19:14 |
| kfox1111 | awesome. :) | 19:15 |
| *** inc0 has joined #openstack-kolla | 19:19 | |
| inc0 | hey, I'm back | 19:19 |
| kfox1111 | wb | 19:19 |
| *** ayoung has quit IRC | 19:21 | |
| *** manjeets has joined #openstack-kolla | 19:22 | |
| *** dave-mccowan has quit IRC | 19:23 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add guide for Hitachi NAS Platform Driver for Manila https://review.openstack.org/386552 | 19:24 |
| mark-casey1 | kfox1111: ping | 19:25 |
| kfox1111 | mark-casey1: hi | 19:26 |
| mark-casey1 | Hi there. Do you know if K8s resource restrictions lie to containers regarding for example how much RAM is in the system, or at some point just start denying processes' allocations even though something like 'free' would show free RAM? | 19:27 |
| *** ayoung has joined #openstack-kolla | 19:31 | |
| *** JoseMello has joined #openstack-kolla | 19:39 | |
| mark-casey1 | kfox1111: ah ok found it fwiw. For RAM, resource limit gets passed to 'docker run' as --memory but 'free' doesn't consider cgroups and would show host memory anyway | 19:41 |
| *** mnasiadka has quit IRC | 19:43 | |
| *** neilus__ has quit IRC | 19:44 | |
| *** dave-mccowan has joined #openstack-kolla | 19:49 | |
| *** neilus has joined #openstack-kolla | 19:55 | |
| *** srwilkers has quit IRC | 19:56 | |
| *** msimonin has joined #openstack-kolla | 19:56 | |
| *** neilus has quit IRC | 20:00 | |
| *** lamt has joined #openstack-kolla | 20:01 | |
| *** huikang has quit IRC | 20:03 | |
| kfox1111 | mark-casey1: I think the scheduler avoids scheduling to nodes that are too full. | 20:03 |
| kfox1111 | I don't know if there's a way to set an overcommit though. I'd assume there would be. | 20:03 |
| *** huikang has joined #openstack-kolla | 20:03 | |
| *** Aju has quit IRC | 20:05 | |
| *** rbbratta has quit IRC | 20:05 | |
| *** rbbratta has joined #openstack-kolla | 20:05 | |
| sdake_ | otavio if it is a problem for newton and mitaka it hould go in teh backport queue too | 20:05 |
| mark-casey1 | kfox1111: cool thanks! | 20:07 |
| kfox1111 | uh.. the gate looks to have failed on a nova bug? http://logs.openstack.org/19/386619/2/experimental/gate-kolla-kubernetes-deploy-ubuntu-binary-ceph-nv/e0d5cba/logs/openstack/kolla-nova-compute-0260l.txt | 20:07 |
| *** huikang has quit IRC | 20:08 | |
| *** huikang has joined #openstack-kolla | 20:14 | |
| *** jtriley has quit IRC | 20:16 | |
| *** jtriley has joined #openstack-kolla | 20:20 | |
| *** strigazi is now known as strigazi_AFK | 20:22 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Revert "Build source from Newton rather then Master" https://review.openstack.org/386763 | 20:24 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Revert "Move RDO to Newton branches" https://review.openstack.org/386764 | 20:24 |
| otavio | sdake_: I made the cherry-pick requests | 20:25 |
| sdake_ | otavio needs to be in the bug tracker as well | 20:25 |
| sdake_ | otavio link to the bug pls? | 20:25 |
| otavio | sdake_: I think it updated the bug | 20:26 |
| otavio | sdake_: it seems so | 20:26 |
| sdake_ | yar - still needs to be in bugtracker ;) | 20:26 |
| sdake_ | link and i'll show you what i mean | 20:26 |
| otavio | sdake_: https://bugs.launchpad.net/kolla/+bug/1630425/comments/4 | 20:28 |
| openstack | Launchpad bug 1630425 in kolla "swift_object_expirer container keeps restarting" [High,Fix released] - Assigned to Otavio Salvador (otavio-ossystems) | 20:28 |
| sdake_ | otavio look at this page: https://bugs.launchpad.net/kolla/+bug/1630425 | 20:29 |
| openstack | Launchpad bug 1630425 in kolla newton "swift_object_expirer container keeps restarting" [High,In progress] | 20:29 |
| otavio | sdake_: yes | 20:30 |
| sdake_ | refresh it - this is the part i was tlaking about | 20:30 |
| otavio | sdake_: ok but how I add other branches? | 20:31 |
| sdake_ | are you in the kolla-drivers group? | 20:31 |
| sdake_ | if so you cn click "target to series" | 20:31 |
| sdake_ | if not i can add you | 20:31 |
| *** mgoddard has joined #openstack-kolla | 20:32 | |
| sdake_ | kolla-drivers group does bug triage/backport planning etc | 20:32 |
| otavio | sdake_: I am not | 20:32 |
| sdake_ | otavio launchpad id is? | 20:32 |
| otavio | sdake_: otavio-ossystems | 20:32 |
| otavio | sdake_: btw I am working on trove bp. Most items are fixed now | 20:33 |
| *** neilus has joined #openstack-kolla | 20:33 | |
| *** eaguilar has joined #openstack-kolla | 20:33 | |
| *** celo_r has quit IRC | 20:34 | |
| *** jtriley has quit IRC | 20:35 | |
| sdake_ | otavio note we only backport high and critical bugs - not features | 20:36 |
| *** huikang has quit IRC | 20:36 | |
| *** MarMat__ has joined #openstack-kolla | 20:37 | |
| *** neilus has quit IRC | 20:38 | |
| *** MarMat_ has quit IRC | 20:40 | |
| *** mgoddard has quit IRC | 20:40 | |
| *** salv-orlando has joined #openstack-kolla | 20:44 | |
| otavio | sdake_: I see | 20:45 |
| *** huikang has joined #openstack-kolla | 20:45 | |
| otavio | sdake_: I noticed you sent PS to move to O branches | 20:45 |
| sdake_ | right | 20:46 |
| sdake_ | you mean move to master branch | 20:46 |
| otavio | sdake_: yes | 20:46 |
| sdake_ | right | 20:46 |
| sdake_ | our end of releease looks like this | 20:46 |
| sdake_ | submit two patches one for source one for binary to move to newton -> branch newton -> revert patches submitted to previous master (that is now newton) | 20:47 |
| *** 7YUAAGGIF has quit IRC | 20:47 | |
| *** spsingh has joined #openstack-kolla | 20:50 | |
| *** spsingh has quit IRC | 20:54 | |
| *** eaguilar has quit IRC | 20:55 | |
| *** msimonin has quit IRC | 20:59 | |
| *** eaguilar has joined #openstack-kolla | 21:00 | |
| *** dave-mccowan has quit IRC | 21:03 | |
| *** michauds has quit IRC | 21:06 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: Trivial change to test gate https://review.openstack.org/386777 | 21:09 |
| *** mgoddard has joined #openstack-kolla | 21:13 | |
| *** MarMat_ has joined #openstack-kolla | 21:13 | |
| *** MarMat__ has quit IRC | 21:15 | |
| *** eaguilar has quit IRC | 21:19 | |
| sbezverk_ | kfox1111: are you going to ack cinder fix? | 21:22 |
| *** mark-casey1 has left #openstack-kolla | 21:22 | |
| *** jheroux has quit IRC | 21:23 | |
| *** huikang has quit IRC | 21:25 | |
| kfox1111 | sbezverk_: I'm still testing. I think it may have actually failed the test. but not sure... | 21:25 |
| kfox1111 | I fired off another vanilla gate check to compare it against. | 21:26 |
| *** huikang has joined #openstack-kolla | 21:26 | |
| *** huikang has quit IRC | 21:26 | |
| *** huikang has joined #openstack-kolla | 21:26 | |
| sbezverk_ | kfox1111: I do not think so, if PS would cause any damage then services would not be working since endpoints were not created properly | 21:26 |
| kfox1111 | sbezverk_: it may be more subtile then that... it may be some misconfiguration where nova is expecting a v1 or a v2 endpoint or something, and it worked before but not enough changed in the ps, and it actually gets nova to do the wrong thing breaking the tests. | 21:27 |
| kfox1111 | we'll see in a couple more minutes. | 21:28 |
| *** jrich523 has joined #openstack-kolla | 21:29 | |
| *** portdirect_ has joined #openstack-kolla | 21:31 | |
| *** portdirect has quit IRC | 21:31 | |
| *** portdirect_ is now known as portdirect | 21:31 | |
| *** ssurana has quit IRC | 21:36 | |
| *** ssurana has joined #openstack-kolla | 21:37 | |
| *** dwalsh has quit IRC | 21:41 | |
| *** huikang has quit IRC | 21:42 | |
| *** huikang has joined #openstack-kolla | 21:42 | |
| *** huikang has quit IRC | 21:47 | |
| *** ssurana has quit IRC | 21:49 | |
| *** inc0 has quit IRC | 21:49 | |
| *** unicell has quit IRC | 22:03 | |
| *** huikang has joined #openstack-kolla | 22:05 | |
| *** salv-orl_ has joined #openstack-kolla | 22:05 | |
| *** sbezverk has joined #openstack-kolla | 22:07 | |
| *** salv-orlando has quit IRC | 22:08 | |
| *** huikang has quit IRC | 22:09 | |
| *** sbezverk_ has quit IRC | 22:10 | |
| *** JoseMello has quit IRC | 22:11 | |
| imcsk8 | flaper87: are you around? | 22:14 |
| *** sdake_ has quit IRC | 22:14 | |
| *** lamt has quit IRC | 22:28 | |
| *** awebber has quit IRC | 22:35 | |
| *** awebber has joined #openstack-kolla | 22:36 | |
| *** papacz has joined #openstack-kolla | 22:41 | |
| *** sdake has joined #openstack-kolla | 22:44 | |
| *** sdake_ has joined #openstack-kolla | 22:45 | |
| *** sdake_ has quit IRC | 22:48 | |
| *** sdake_ has joined #openstack-kolla | 22:49 | |
| *** huikang has joined #openstack-kolla | 22:51 | |
| *** sdake has quit IRC | 22:57 | |
| *** sdake_ has quit IRC | 22:57 | |
| *** Marcellin__ has quit IRC | 22:57 | |
| *** awebber has quit IRC | 22:58 | |
| *** sdake has joined #openstack-kolla | 22:58 | |
| *** sdake has quit IRC | 22:58 | |
| kfox1111 | hmm.... does quemu place any sockets under /var/log/libvirt? | 23:00 |
| *** mgoddard has quit IRC | 23:00 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP: Trivial change to test gate https://review.openstack.org/386777 | 23:02 |
| *** papacz has quit IRC | 23:05 | |
| *** sdake has joined #openstack-kolla | 23:06 | |
| *** sbezverk has quit IRC | 23:20 | |
| *** sdake has quit IRC | 23:21 | |
| *** sbezverk has joined #openstack-kolla | 23:22 | |
| *** afranc has joined #openstack-kolla | 23:28 | |
| *** eaguilar has joined #openstack-kolla | 23:31 | |
| *** huikang has quit IRC | 23:31 | |
| *** huikang has joined #openstack-kolla | 23:32 | |
| *** salv-orl_ has quit IRC | 23:32 | |
| *** sbezverk has quit IRC | 23:34 | |
| *** janem has quit IRC | 23:34 | |
| *** sbezverk has joined #openstack-kolla | 23:36 | |
| *** huikang has quit IRC | 23:37 | |
| *** MarMat_ has quit IRC | 23:45 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!