| jascott1 | ok I was going with 'local registry is required for multinode setup' | 00:00 |
|---|---|---|
| sdake | ok multinode is not ready for prime time | 00:00 |
| jascott1 | sounds good will try it | 00:00 |
| sdake | jascott1 have you deployed AIO yet using minikube? | 00:00 |
| jascott1 | I did the kk8s aio yes | 00:00 |
| *** liyifeng has quit IRC | 00:01 | |
| jascott1 | my k8s cluster seems to be working pretty well so far.. can "do stuff", hit the dashboard etc | 00:01 |
| *** HyperJohnGraham_ has quit IRC | 00:01 | |
| sdake | jascott1 ok - nwo is not a super good time to try multinode | 00:01 |
| sdake | jascott1 until the docs are in good enough shape that sbezverk_ and kfox1111 and rhallisey feel comfortable walking people through them | 00:02 |
| sdake | we worked on that with hyperjohngraham for about 6 hours with no positive outcome | 00:02 |
| sdake | its a docs issue entirely | 00:02 |
| sdake | and i dont know how to do multinode deploy of that deliverable | 00:02 |
| jascott1 | ok good to know. maybe I can help out w it later | 00:02 |
| jascott1 | just knowing non TLS registries are supposed to work is helpful | 00:03 |
| sdake | jascott1 that said, ping sbezverk_ or kfox1111 or rhallisey and see what shape the docs are in | 00:03 |
| sdake | jascott1 tls/nontls both work | 00:03 |
| jascott1 | I will probably keep playing w it on the side but do another AIO to get make some progress | 00:03 |
| sdake | if configured properly | 00:03 |
| sbezverk_ | jascott1: sdake with John's setup the issue was a misconfiguration. atm when we were installing kube, he had selinux turned on and it screwed up configuration of networking | 00:03 |
| sdake | jascott1 i'd try my recommendation of commenting out the registry | 00:03 |
| sdake | sbezverk_ cool thanks for getting to the root cause of that | 00:04 |
| jascott1 | sdake ok will do | 00:04 |
| jascott1 | just a thought but if your home is your 'workplace' then you might need to just get a keggorator like all the hip SF workplaces ;) | 00:05 |
| sdake | sbezverk_ did john get it going | 00:05 |
| sbezverk_ | sdake: do not know. I was busy with fencing pod last few days, did not have opportunity to follow up with him.. | 00:06 |
| sdake | ok time to tag | 00:09 |
| sdake | going to do one lsaat build and deploy | 00:09 |
| sdake | wish me well - bbiaf | 00:09 |
| *** sdake_ has joined #openstack-kolla | 00:11 | |
| *** sdake has quit IRC | 00:14 | |
| sdake_ | fan tastic one of my machines is not operational | 00:15 |
| sdake_ | bjolo ping | 00:17 |
| jascott1 | sdake commenting out registry got me far. I also had to change image version in globals.yml. Now I am seeing sysctl errors but will work on that | 00:24 |
| sdake_ | jascott1 cool | 00:25 |
| sdake_ | jascott1 what is happening there is your using the docker registry | 00:25 |
| sdake_ | it maybe that 12.0.1 or whateverversion you linked earlier doesn't work with that --insecure-registry flag | 00:25 |
| sdake_ | docker was talking of removing that | 00:26 |
| sdake_ | maybe they finally did so | 00:26 |
| jascott1 | hmm these errors are 'setting sysctl values' like this error 'cannot stat /proc/sys/net/ipv4/tcp_max_sync_backlog: No such file or directory' | 00:28 |
| *** tonanhngo has quit IRC | 00:32 | |
| *** yingjun has joined #openstack-kolla | 00:33 | |
| sdake_ | why on earth is xorg installed in kolla containers | 00:33 |
| * sdake_ shakes head | 00:33 | |
| sdake_ | jascott1 your on an ubuntu host? | 00:33 |
| jascott1 | yes | 00:33 |
| jascott1 | 14 | 00:33 |
| sdake_ | jascott1 did you set the distro and install type in globals.yml? | 00:33 |
| jascott1 | no sir | 00:33 |
| * jascott1 looks at feet | 00:33 | |
| sdake_ | try that | 00:33 |
| jascott1 | thanks | 00:33 |
| sdake_ | the defaults are centos bianry | 00:33 |
| sdake_ | you probably want ubuntu source | 00:33 |
| *** tonanhngo has joined #openstack-kolla | 00:34 | |
| *** duonghq has joined #openstack-kolla | 00:37 | |
| duonghq | morning | 00:37 |
| duonghq | it's true autumn here (quite late than the past, but it come finally) | 00:38 |
| *** tonanhngo has quit IRC | 00:39 | |
| jascott1 | woot. progress. made it all the way to 'haproxy : Waiting for virtual IP to appear' | 00:40 |
| *** mtaylor22 has quit IRC | 00:40 | |
| duonghq | jascott1: I'm sorry but which bug do you mention? | 00:41 |
| jascott1 | duonghq sdake and other nice ppls in here have been helping me get a multinode kk8s going | 00:41 |
| jascott1 | im running kolla-ansible deploy but having issues | 00:42 |
| duonghq | jascott1: you are using kolla-ansible or k8s? | 00:42 |
| jascott1 | both | 00:42 |
| jascott1 | I prob dont get your question | 00:43 |
| duonghq | I'm not sure you got trouble with which project | 00:43 |
| jascott1 | I have k8s cluster and am trying to deploy kolla-kubernetes on it | 00:43 |
| *** HyperJohnGraham_ has joined #openstack-kolla | 00:43 | |
| jascott1 | now that I am not using local registry I have gotten all the way to the install step 'haproxy : Waiting for virtual IP to appear' | 00:44 |
| jascott1 | im not sure what IP that should be set to | 00:44 |
| duonghq | let me check that, | 00:45 |
| duonghq | moment | 00:45 |
| jascott1 | thanks so much | 00:45 |
| jascott1 | its in globals.yml but I just have it set to the IP of the host the deploy is running from | 00:46 |
| duonghq | do you have your globals.yml and inventory in some pastebin? | 00:47 |
| jascott1 | one sec | 00:47 |
| jascott1 | http://paste.openstack.org/show/585524/ | 00:48 |
| jascott1 | ^globals.yml | 00:48 |
| jascott1 | http://paste.openstack.org/show/585525/ | 00:50 |
| jascott1 | ^ inventory | 00:50 |
| jascott1 | inventory is stock except for the three IPs I have used at the top | 00:50 |
| *** tonanhngo has joined #openstack-kolla | 00:51 | |
| jascott1 | I will take a look at the role | 00:51 |
| duonghq | can you "docker ps" | 00:52 |
| duonghq | guess that haproxy container and/or keepalived container got trouble | 00:52 |
| *** tonanhngo_ has joined #openstack-kolla | 00:53 | |
| jascott1 | ya on the ps, will look at the containers too thx | 00:53 |
| *** mtaylor22 has joined #openstack-kolla | 00:55 | |
| *** tonanhngo has quit IRC | 00:56 | |
| *** tonanhngo_ has quit IRC | 00:57 | |
| jascott1 | duonghq that ansible task is just waits for a response on <vip> <database_port> | 00:58 |
| duonghq | jascott1, right | 00:59 |
| duonghq | it use database port as signature for VIP | 00:59 |
| *** phuongnh has joined #openstack-kolla | 00:59 | |
| jascott1 | hmm something up with the node it tried to land it on i think | 01:00 |
| duonghq | jascott1, what does keepalived log tell? | 01:08 |
| *** eaguilar has quit IRC | 01:12 | |
| *** MarMat_ has quit IRC | 01:18 | |
| *** ssurana has quit IRC | 01:21 | |
| jascott1 | duonghq haproxy has error [ALERT] 285/173647 (9) : sendmsg logger #1 failed: Connection refused (errno=111) | 01:22 |
| jascott1 | checking config | 01:22 |
| jascott1 | oh | 01:24 |
| duonghq | iirc there is at least a bug about container cannot send log to heka | 01:24 |
| jascott1 | i have wrong IP | 01:25 |
| duonghq | oh | 01:26 |
| *** tonanhngo has joined #openstack-kolla | 01:26 | |
| jascott1 | unfortunately my kubectl command is not working atm | 01:26 |
| jascott1 | but cluster is still trucking... so sayeth the dashboard | 01:27 |
| sdake_ | Jeffrey4l__ you alive yet | 01:27 |
| sdake_ | Jeffrey4l__ i know its been a rough slog | 01:27 |
| sdake_ | curious if you have seen TASK [heat : Running Heat bootstrap container] ********************************* | 01:27 |
| sdake_ | fatal: [minime-01 -> minime-01]: FAILED! => {"changed": true, "failed": true, "msg": "Container exited with non-zero return code"} | 01:27 |
| sdake_ | to retry, use: --limit @/home/sdake/kfinal/kolla/ansible/site.retry | 01:27 |
| *** yingjun has quit IRC | 01:28 | |
| sdake_ | if someone could take a look at this: https://bugs.launchpad.net/kolla/+bug/1632905 | 01:32 |
| openstack | Launchpad bug 1632905 in kolla "heat bootstrap failed" [Critical,Triaged] | 01:32 |
| sdake_ | most appreciated | 01:32 |
| *** yingjun has joined #openstack-kolla | 01:33 | |
| *** liyifeng has joined #openstack-kolla | 01:35 | |
| jascott1 | sdake user error? 'Conflict occurred attempting to store role - Duplicate Entry (HTTP 409) ' | 01:37 |
| sdake_ | huh | 01:37 |
| jascott1 | from that issue | 01:37 |
| jascott1 | you just linked | 01:37 |
| sdake_ | oh yo uthink its user error | 01:37 |
| sdake_ | i disagree | 01:37 |
| sdake_ | i am pretty sure i know how to deploy kolla-ansible ;-) | 01:38 |
| jascott1 | i meant more like dupe line in config or something | 01:38 |
| sdake_ | the config comes from defaults | 01:38 |
| Pavo | seems like a issue with the http service to me | 01:40 |
| Pavo | like a hickup | 01:40 |
| *** zhubingbing has joined #openstack-kolla | 01:42 | |
| jascott1 | there are duplice 'auth_uri' entries in my heat.conf but I guess thats legit? I didnt know we could do that in one section | 01:49 |
| *** Mech422 has quit IRC | 01:49 | |
| sdake_ | jascott1 auth_uri and auth_url | 01:49 |
| sdake_ | two different things | 01:50 |
| sdake_ | the bootstrrap rpoblem is about duplicata roles I think | 01:50 |
| jascott1 | ya | 01:50 |
| jascott1 | wow I am blind | 01:51 |
| sdake_ | i can't see too well either jascott1 | 01:51 |
| jascott1 | 13"mbp | 01:51 |
| sdake_ | occupational hazard ;) | 01:51 |
| jascott1 | no doubt | 01:52 |
| jascott1 | treating myself to 12pt fontsize. livin the dream | 01:52 |
| jascott1 | wow life is much better now. I think that was from some tmux foolery trying to get a happy medium with the resolutions | 01:53 |
| jascott1 | maybe heat_stack_owner_role and heat_stack_user_role set to same name? | 01:58 |
| jascott1 | sake maybe check keystone log? | 02:01 |
| jascott1 | ^sdake_ | 02:01 |
| *** phuongnh has quit IRC | 02:02 | |
| *** phuongnh has joined #openstack-kolla | 02:02 | |
| *** Mech422 has joined #openstack-kolla | 02:03 | |
| *** dave-mccowan has joined #openstack-kolla | 02:09 | |
| *** zhurong has joined #openstack-kolla | 02:19 | |
| sdake_ | jascott1 i am busy tagging at presnet | 02:21 |
| sdake_ | rather testing if tagging is even possible | 02:22 |
| sdake_ | any cores around - critical issue blocking rc2 | 02:26 |
| * sdake_ wants to beat erlang with a shovel | 02:27 | |
| jascott1 | its erlang so you might need a list of shovels :) | 02:29 |
| *** MarMat_ has joined #openstack-kolla | 02:33 | |
| *** huhaoran has joined #openstack-kolla | 02:43 | |
| *** mtaylor22 has quit IRC | 02:47 | |
| *** mtaylor22 has joined #openstack-kolla | 02:47 | |
| *** yuanying_ has joined #openstack-kolla | 02:49 | |
| *** yuanying has quit IRC | 02:49 | |
| sdake_ | Jeffrey4l__ https://bugs.launchpad.net/kolla/+bug/1564773 | 02:52 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 02:52 |
| sdake_ | this is blocking release of rc2 | 02:53 |
| *** huikang has joined #openstack-kolla | 02:53 | |
| sdake_ | anyone else out there available to help - help is appreciatedo n this one ;) | 02:53 |
| *** MarMat_ has quit IRC | 02:59 | |
| *** yuanying_ has quit IRC | 02:59 | |
| *** MarMat_ has joined #openstack-kolla | 03:00 | |
| *** liyifeng has quit IRC | 03:03 | |
| *** yuanying has joined #openstack-kolla | 03:06 | |
| *** yuanying has quit IRC | 03:08 | |
| *** eaguilar has joined #openstack-kolla | 03:09 | |
| sdake_ | MarMat_ around? | 03:10 |
| *** dave-mccowan has quit IRC | 03:10 | |
| MarMat_ | sdake whatsup | 03:10 |
| sdake_ | https://bugs.launchpad.net/kolla/+bug/1564773 | 03:10 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 03:10 |
| sdake_ | blocking release of rc2 | 03:10 |
| sdake_ | can you have a look at the last comment and provide any thoughts | 03:10 |
| sdake_ | i'm really at a loss as to how to proceed | 03:11 |
| sdake_ | so any technical advice anyone can offer is appreciated | 03:11 |
| sdake_ | other then we are not tagging somethign doa | 03:11 |
| MarMat_ | sdake sure, looking into it | 03:11 |
| *** huikang has quit IRC | 03:13 | |
| sdake_ | i think that shell script is busted | 03:17 |
| sdake_ | i think it needss to ignore sigusr2 as well | 03:17 |
| MarMat_ | sdake is that last issue came with rabbitmq update? Any idea when that was updated? | 03:20 |
| sdake_ | it came when we itnrroduced rabbitmq from the cloud sig | 03:20 |
| sdake_ | this was working for me multinode in the past | 03:21 |
| sdake_ | i have no idea why clusterer is throwing a sigusr2 | 03:21 |
| *** phuongnh has quit IRC | 03:23 | |
| *** phuongnh has joined #openstack-kolla | 03:24 | |
| *** absubram has quit IRC | 03:27 | |
| MarMat_ | sdake what node is impacted in your multinode setup? | 03:29 |
| *** huhaoran has quit IRC | 03:30 | |
| *** huhaoran has joined #openstack-kolla | 03:30 | |
| *** huikang has joined #openstack-kolla | 03:31 | |
| sdake_ | Reconfigure ................: kill -SIGUSR2 `cat sneakers.pid` | 03:34 |
| *** g3ek has quit IRC | 03:34 | |
| *** haplo37 has quit IRC | 03:35 | |
| sdake_ | file:///Users/sdake/Downloads/Mastering%20RabbitMQ%20-%202015.pdf | 03:35 |
| sdake_ | looks like sigusr2 should reconfigure rabbitmq | 03:35 |
| *** haplo37 has joined #openstack-kolla | 03:36 | |
| sdake_ | rather https://bitbucket.org/xurongzhong/python-chinese-library/downloads/Mastering%20RabbitMQ%20-%202015.pdf | 03:36 |
| *** g3ek has joined #openstack-kolla | 03:36 | |
| *** absubram has joined #openstack-kolla | 03:38 | |
| *** jidar has quit IRC | 03:39 | |
| *** coolsvap has quit IRC | 03:40 | |
| sdake_ | well need to eat bbiaf | 03:43 |
| sdake_ | MarMat_ if you see anyone else on - ping em | 03:43 |
| Jeffrey4l__ | sdake_, sup | 03:43 |
| sdake_ | https://bitbucket.org/xurongzhong/python-chinese-library/downloads/Mastering%20RabbitMQ%20-%202015.pdf | 03:43 |
| sdake_ | https://bugs.launchpad.net/kolla/+bug/1564773 | 03:44 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 03:44 |
| * Jeffrey4l__ is checking the comments. | 03:44 | |
| sdake_ | lst comment | 03:44 |
| sdake_ | only needs to be read | 03:45 |
| sdake_ | last | 03:45 |
| Jeffrey4l__ | roger | 03:45 |
| Jeffrey4l__ | sdake_, i haven't got the same error. i will re-build my image and try to re-deploy. | 03:46 |
| sdake_ | with 3 nodes? | 03:46 |
| MarMat_ | sdake good time to start learning rabbitmq internals :-) but first would like to see that on my systems... not seen it so far. What to do to reproduce? | 03:46 |
| Jeffrey4l__ | yes. | 03:46 |
| sdake_ | Jeffrey4l__ i built like a couple hours ago | 03:47 |
| sdake_ | but yousee the sigusr2 | 03:47 |
| sdake_ | i'm not sure if thats just a normal thing or not | 03:47 |
| MarMat_ | sdake ok, i'll try to rebuild freshly | 03:47 |
| sdake_ | i think that means "reconfgiure" to rabbitmq | 03:47 |
| *** yuanying has joined #openstack-kolla | 03:47 | |
| sdake_ | but the shell script for rabbitmq-server is wrong and not reconfiguring it | 03:48 |
| sdake_ | MarMat_ how many nodes do you have? | 03:48 |
| MarMat_ | normaly deploying on 2, can have also 3 | 03:48 |
| Jeffrey4l__ | i think it should be OK. dumb-init should handle this properly. | 03:48 |
| sdake_ | read the hsell script | 03:48 |
| sdake_ | it has signal handlers in it | 03:48 |
| sdake_ | sigusr2 is definately cuaght by it | 03:49 |
| sdake_ | its not ignored as it should be | 03:49 |
| *** huikang has quit IRC | 03:49 | |
| sdake_ | but i'm just guessing based upon that book above | 03:49 |
| *** coolsvap has joined #openstack-kolla | 03:52 | |
| *** huikang has joined #openstack-kolla | 03:53 | |
| MarMat_ | sdake you have 2 control nodes? Or what is your node setup? | 03:53 |
| sdake_ | Jeffrey4l__ you can log into my boxe if ou like | 03:54 |
| Jeffrey4l__ | ok. | 03:54 |
| Jeffrey4l__ | which node? | 03:54 |
| sdake_ | MarMat_ 3 control | 03:54 |
| sdake_ | Jeffrey4l__ logint ot103 but i think the continaer donw is on 101 or 102 | 03:54 |
| sdake_ | not sure if its helpful | 03:54 |
| sdake_ | i deubged it in the log a bit | 03:55 |
| Jeffrey4l__ | 192.168.1.103 minime-03 this ? | 03:55 |
| sdake_ | but wouldn't hurt fory ou to take a seocnd look | 03:55 |
| sdake_ | right | 03:55 |
| sdake_ | minime-02 had own container | 03:55 |
| sdake_ | which seemed to crater my deployment | 03:56 |
| Jeffrey4l__ | got. i am check the script. | 03:56 |
| *** neilus has joined #openstack-kolla | 04:00 | |
| *** mewald has joined #openstack-kolla | 04:01 | |
| *** eaguilar has quit IRC | 04:02 | |
| Jeffrey4l__ | sdake_, fyi find the commit message about this https://github.com/rabbitmq/rabbitmq-server/commit/e56bec64777c277833552303caa2d0c09b702831 | 04:02 |
| sdake_ | Jeffrey4l__ that may be same file | 04:04 |
| sdake_ | but I dont thin that is theissue | 04:04 |
| sdake_ | there is some trapstuff in that shell script | 04:04 |
| *** salv-orlando has joined #openstack-kolla | 04:05 | |
| Jeffrey4l__ | sdake_, the trap existed long time ago | 04:05 |
| Jeffrey4l__ | https://github.com/rabbitmq/rabbitmq-server/blame/master/scripts/rabbitmq-server | 04:05 |
| Jeffrey4l__ | https://github.com/rabbitmq/rabbitmq-server/blame/master/scripts/rabbitmq-server#L277 | 04:05 |
| *** neilus has quit IRC | 04:06 | |
| *** mewald has quit IRC | 04:06 | |
| sdake_ | 2o2 | 04:06 |
| sdake_ | wow | 04:06 |
| sdake_ | haven't seen that before | 04:06 |
| sdake_ | i use gitblame on cli | 04:06 |
| Jeffrey4l__ | github is cool ;) | 04:08 |
| Jeffrey4l__ | so this bug can be re-produced in your server? if re-deploy the whole kolla? | 04:08 |
| sdake_ | i dont know i havn't tried | 04:09 |
| sdake_ | feel free to try - i really need to eat | 04:09 |
| *** salv-orlando has quit IRC | 04:09 | |
| Jeffrey4l__ | ok. i will try re-deploy | 04:10 |
| Jeffrey4l__ | now? | 04:10 |
| sdake_ | sure | 04:11 |
| sdake_ | so what is wierd is hat script exits with sigusr2 | 04:11 |
| sdake_ | but i think we should be ignoring it | 04:11 |
| sdake_ | i just dont understand how to ignore it | 04:11 |
| sdake_ | i think this ia a 3.6 problem in rabbitmq upstream | 04:12 |
| Jeffrey4l__ | ok. | 04:12 |
| Jeffrey4l__ | since we use dumb-init now. i can we can add some environment variable to avoid that logical. | 04:13 |
| Jeffrey4l__ | i can/i think. | 04:13 |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla: Visit target nodes without password. https://review.openstack.org/368524 | 04:16 |
| *** mewald has joined #openstack-kolla | 04:16 | |
| *** ssurana1 has joined #openstack-kolla | 04:19 | |
| *** jidar has joined #openstack-kolla | 04:19 | |
| *** zhubingbing has quit IRC | 04:20 | |
| *** jidar has left #openstack-kolla | 04:20 | |
| *** mewald has quit IRC | 04:21 | |
| *** ssurana1 has quit IRC | 04:24 | |
| *** mewald has joined #openstack-kolla | 04:31 | |
| *** unicell has joined #openstack-kolla | 04:32 | |
| *** zhurong has quit IRC | 04:32 | |
| *** unicell has quit IRC | 04:33 | |
| *** mewald has quit IRC | 04:36 | |
| *** unicell has joined #openstack-kolla | 04:37 | |
| Jeffrey4l__ | sdake_, you are using old image on node2 http://pix.toile-libre.org/upload/original/1476333859.png | 04:44 |
| Jeffrey4l__ | sdake_, rabbitmq works find now. ( i didn't re-build the image, just pull latest images in all nodes ) | 04:49 |
| *** mewald has joined #openstack-kolla | 04:51 | |
| *** mewald has quit IRC | 05:00 | |
| *** salv-orlando has joined #openstack-kolla | 05:00 | |
| *** yuanying has quit IRC | 05:00 | |
| *** jmccarthy1 has left #openstack-kolla | 05:06 | |
| *** huhaoran has quit IRC | 05:08 | |
| *** huhaoran has joined #openstack-kolla | 05:08 | |
| *** huikang has quit IRC | 05:19 | |
| *** mewald has joined #openstack-kolla | 05:21 | |
| *** msimonin has joined #openstack-kolla | 05:24 | |
| *** msimonin has quit IRC | 05:24 | |
| *** haplo37_ has quit IRC | 05:37 | |
| *** huikang has joined #openstack-kolla | 05:38 | |
| *** haplo37_ has joined #openstack-kolla | 05:40 | |
| *** neilus has joined #openstack-kolla | 05:42 | |
| *** huikang has quit IRC | 05:43 | |
| *** neilus has quit IRC | 05:47 | |
| *** HyperJohnGraham_ has quit IRC | 05:50 | |
| *** HyperJohnGraham_ has joined #openstack-kolla | 05:54 | |
| MarMat_ | sdake_ freshly build using most recent kolla, deployed to 3 control nodes, rabbitmq seems to be ok | 06:03 |
| *** zhurong has joined #openstack-kolla | 06:04 | |
| *** mtaylor22 has quit IRC | 06:05 | |
| Jeffrey4l__ | MarMat_, i re-deployed openstack on sdake_ 's node. it works. the issue is node2 is not the latest images. | 06:06 |
| MarMat_ | Jeffrey4l__ yep, saw your message before | 06:07 |
| *** zhubingbing has joined #openstack-kolla | 06:08 | |
| *** yuanying has joined #openstack-kolla | 06:10 | |
| *** MarMat_ has quit IRC | 06:15 | |
| duonghq | MarMat_ me too | 06:19 |
| duonghq | 3 control nodes is run properly | 06:19 |
| *** yuanying has quit IRC | 06:23 | |
| *** msimonin1 has joined #openstack-kolla | 06:23 | |
| duonghq | sdake_ which service do you enable for storage node? | 06:23 |
| *** mnasiadka has joined #openstack-kolla | 06:25 | |
| *** MarMat_ has joined #openstack-kolla | 06:26 | |
| *** yuanying has joined #openstack-kolla | 06:29 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix Stale namespace removal issue https://review.openstack.org/385559 | 06:31 |
| *** tovin07 has joined #openstack-kolla | 06:33 | |
| *** yuanying_ has joined #openstack-kolla | 06:37 | |
| *** egonzalez90 has joined #openstack-kolla | 06:38 | |
| *** yuanying has quit IRC | 06:39 | |
| *** mnasiadka has quit IRC | 06:40 | |
| *** mnasiadka has joined #openstack-kolla | 06:40 | |
| *** mnasiadka has quit IRC | 06:40 | |
| *** mewald has quit IRC | 06:41 | |
| *** mnasiadka has joined #openstack-kolla | 06:41 | |
| *** mewald has joined #openstack-kolla | 06:50 | |
| *** mkoderer has joined #openstack-kolla | 06:59 | |
| *** msimonin1 has quit IRC | 06:59 | |
| *** MarMat_ has quit IRC | 06:59 | |
| *** absubram has quit IRC | 07:03 | |
| *** absubram has joined #openstack-kolla | 07:03 | |
| *** b_bezak has joined #openstack-kolla | 07:04 | |
| *** ankush has joined #openstack-kolla | 07:04 | |
| *** zhubingbing has quit IRC | 07:05 | |
| *** zhubingbing has joined #openstack-kolla | 07:08 | |
| *** shardy has joined #openstack-kolla | 07:09 | |
| *** matrohon has joined #openstack-kolla | 07:12 | |
| berendt | have we not yet tagged rc2? | 07:13 |
| berendt | shouldn't this happen yesterday night (my night)? | 07:14 |
| *** mewald has quit IRC | 07:22 | |
| duonghq | what is real reconfigure task mean? for apply new configuration file to existing service, or include change the node role? (i.e add/remove compute node) | 07:29 |
| *** steve-noyes has joined #openstack-kolla | 07:30 | |
| *** mewald has joined #openstack-kolla | 07:30 | |
| *** g3ek has quit IRC | 07:31 | |
| openstackgerrit | zhongshengping proposed openstack/kolla: Deprecate api_paste_config option https://review.openstack.org/385804 | 07:32 |
| *** haplo37 has quit IRC | 07:34 | |
| *** yuanying_ has quit IRC | 07:36 | |
| *** berendt has quit IRC | 07:36 | |
| *** haplo37 has joined #openstack-kolla | 07:36 | |
| openstackgerrit | zhongshengping proposed openstack/kolla: Deprecate auth_plugin option https://review.openstack.org/385805 | 07:36 |
| *** yuanying has joined #openstack-kolla | 07:36 | |
| *** msimonin has joined #openstack-kolla | 07:39 | |
| *** Serlex has joined #openstack-kolla | 07:39 | |
| *** g3ek has joined #openstack-kolla | 07:40 | |
| *** rmart04 has joined #openstack-kolla | 07:48 | |
| *** ankush has quit IRC | 07:59 | |
| *** psanchez has joined #openstack-kolla | 08:06 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix Stale namespace removal issue https://review.openstack.org/385559 | 08:06 |
| *** janem has joined #openstack-kolla | 08:10 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix Stale namespace removal issue https://review.openstack.org/385559 | 08:12 |
| *** mewald has quit IRC | 08:13 | |
| *** kproskurin has joined #openstack-kolla | 08:24 | |
| *** neilus has joined #openstack-kolla | 08:24 | |
| *** mewald has joined #openstack-kolla | 08:25 | |
| *** neilus has quit IRC | 08:29 | |
| *** b_bezak has quit IRC | 08:33 | |
| *** b_bezak has joined #openstack-kolla | 08:33 | |
| *** mewald has quit IRC | 08:34 | |
| openstackgerrit | zhongshengping proposed openstack/kolla: Fix api_paste_config option https://review.openstack.org/385804 | 08:35 |
| *** athomas has joined #openstack-kolla | 08:38 | |
| *** salv-orl_ has joined #openstack-kolla | 08:44 | |
| *** salv-orlando has quit IRC | 08:47 | |
| *** tonanhngo has quit IRC | 08:48 | |
| *** ankush has joined #openstack-kolla | 08:54 | |
| *** yuanying has quit IRC | 08:57 | |
| *** berendt has joined #openstack-kolla | 09:01 | |
| *** zhurong has quit IRC | 09:06 | |
| *** zhurong has joined #openstack-kolla | 09:08 | |
| *** ChanServ has quit IRC | 09:14 | |
| *** egonzalez90 has quit IRC | 09:16 | |
| *** psanchez has quit IRC | 09:16 | |
| *** hyakuhei- has quit IRC | 09:17 | |
| *** wirehead_ has quit IRC | 09:17 | |
| *** berendt has quit IRC | 09:17 | |
| *** Mech422 has quit IRC | 09:17 | |
| *** portdirect has quit IRC | 09:17 | |
| *** ajafo has quit IRC | 09:17 | |
| *** salv-orl_ has quit IRC | 09:17 | |
| *** janem has quit IRC | 09:17 | |
| *** rmart04 has quit IRC | 09:17 | |
| *** msimonin has quit IRC | 09:17 | |
| *** steve-noyes has quit IRC | 09:17 | |
| *** duonghq has quit IRC | 09:17 | |
| *** sdake_ has quit IRC | 09:17 | |
| *** hieulq__ has quit IRC | 09:17 | |
| *** pbourke has quit IRC | 09:17 | |
| *** openstackgerrit has quit IRC | 09:17 | |
| *** ntpttr has quit IRC | 09:17 | |
| *** jascott1 has quit IRC | 09:17 | |
| *** cinerama has quit IRC | 09:17 | |
| *** b_bezak has quit IRC | 09:17 | |
| *** mkoderer has quit IRC | 09:17 | |
| *** tovin07 has quit IRC | 09:17 | |
| *** huhaoran has quit IRC | 09:17 | |
| *** macsz_ has quit IRC | 09:17 | |
| *** zigo has quit IRC | 09:17 | |
| *** rbbratta has quit IRC | 09:17 | |
| *** NachoDuck has quit IRC | 09:17 | |
| *** amaged__ has quit IRC | 09:17 | |
| *** skramaja has quit IRC | 09:17 | |
| *** ediardo has quit IRC | 09:17 | |
| *** kbyrne has quit IRC | 09:17 | |
| *** ccesario has quit IRC | 09:17 | |
| *** manjeets_ has quit IRC | 09:17 | |
| *** robcresswell has quit IRC | 09:17 | |
| *** harlowja has quit IRC | 09:17 | |
| *** strigazi has quit IRC | 09:17 | |
| *** zhenguo has quit IRC | 09:17 | |
| *** DuncanT has quit IRC | 09:17 | |
| *** dmsimard has quit IRC | 09:17 | |
| *** dasm has quit IRC | 09:17 | |
| *** Kimmo__ has quit IRC | 09:17 | |
| *** matrohon has quit IRC | 09:17 | |
| *** zhubingbing has quit IRC | 09:17 | |
| *** coolsvap has quit IRC | 09:17 | |
| *** irtermit- has quit IRC | 09:17 | |
| *** kklimonda has quit IRC | 09:17 | |
| *** karlamrhein has quit IRC | 09:17 | |
| *** Pavo has quit IRC | 09:17 | |
| *** qwang has quit IRC | 09:17 | |
| *** flaper87 has quit IRC | 09:17 | |
| *** afranc has quit IRC | 09:17 | |
| *** cmceniry_ has quit IRC | 09:17 | |
| *** dotplus has quit IRC | 09:17 | |
| *** quasar has quit IRC | 09:17 | |
| *** elemoine_ has quit IRC | 09:17 | |
| *** nick-ma has quit IRC | 09:17 | |
| *** serverascode has quit IRC | 09:17 | |
| *** ankush has quit IRC | 09:17 | |
| *** Serlex has quit IRC | 09:17 | |
| *** shardy has quit IRC | 09:17 | |
| *** HyperJohnGraham_ has quit IRC | 09:17 | |
| *** qbjolof1 has quit IRC | 09:17 | |
| *** jgriffith has quit IRC | 09:17 | |
| *** Guest66666 has quit IRC | 09:17 | |
| *** esmiurium has quit IRC | 09:17 | |
| *** peterjenkins_ has quit IRC | 09:17 | |
| *** EmilienM has quit IRC | 09:17 | |
| *** cu5 has quit IRC | 09:17 | |
| *** wznoinsk has quit IRC | 09:17 | |
| *** NobodyCam has quit IRC | 09:17 | |
| *** denaitre has quit IRC | 09:17 | |
| *** mgkwill has quit IRC | 09:17 | |
| *** SaMnCo has quit IRC | 09:17 | |
| *** kong has quit IRC | 09:17 | |
| *** nihilifer has quit IRC | 09:17 | |
| *** dims has quit IRC | 09:17 | |
| *** harbie has quit IRC | 09:17 | |
| *** harmw has quit IRC | 09:17 | |
| *** cj has quit IRC | 09:17 | |
| *** smekel_ has quit IRC | 09:17 | |
| *** jroll has quit IRC | 09:17 | |
| *** tonyb has quit IRC | 09:17 | |
| *** cloudnull has quit IRC | 09:17 | |
| *** absubram has quit IRC | 09:20 | |
| *** g3ek has quit IRC | 09:20 | |
| *** zeus has quit IRC | 09:21 | |
| *** zeus has joined #openstack-kolla | 09:21 | |
| *** g3ek has joined #openstack-kolla | 09:22 | |
| *** yuanying has joined #openstack-kolla | 09:22 | |
| *** timss has quit IRC | 09:22 | |
| *** igordcard has quit IRC | 09:22 | |
| *** igordcard has joined #openstack-kolla | 09:22 | |
| *** salv-orlando has joined #openstack-kolla | 09:30 | |
| *** karlamrhein has joined #openstack-kolla | 09:32 | |
| *** yingjun has quit IRC | 09:34 | |
| *** yingjun has joined #openstack-kolla | 09:34 | |
| *** salv-orlando has quit IRC | 09:35 | |
| *** yingjun has quit IRC | 09:39 | |
| *** janem has joined #openstack-kolla | 09:42 | |
| *** janem has quit IRC | 09:49 | |
| *** yuanying has quit IRC | 09:49 | |
| *** matrohon has joined #openstack-kolla | 10:06 | |
| *** ajafo has joined #openstack-kolla | 10:06 | |
| *** Mech422 has joined #openstack-kolla | 10:06 | |
| *** portdirect has joined #openstack-kolla | 10:06 | |
| *** serverascode has joined #openstack-kolla | 10:07 | |
| *** DuncanT has joined #openstack-kolla | 10:07 | |
| *** amaged__ has joined #openstack-kolla | 10:07 | |
| *** timss has joined #openstack-kolla | 10:07 | |
| *** Serlex has joined #openstack-kolla | 10:07 | |
| *** shardy has joined #openstack-kolla | 10:07 | |
| *** Guest66666 has joined #openstack-kolla | 10:07 | |
| *** qbjolof1 has joined #openstack-kolla | 10:07 | |
| *** jgriffith has joined #openstack-kolla | 10:07 | |
| *** EmilienM has joined #openstack-kolla | 10:07 | |
| *** esmiurium has joined #openstack-kolla | 10:07 | |
| *** peterjenkins_ has joined #openstack-kolla | 10:07 | |
| *** wznoinsk has joined #openstack-kolla | 10:07 | |
| *** cu5 has joined #openstack-kolla | 10:07 | |
| *** NobodyCam has joined #openstack-kolla | 10:07 | |
| *** denaitre has joined #openstack-kolla | 10:07 | |
| *** mgkwill has joined #openstack-kolla | 10:07 | |
| *** SaMnCo has joined #openstack-kolla | 10:07 | |
| *** harbie has joined #openstack-kolla | 10:07 | |
| *** nihilifer has joined #openstack-kolla | 10:07 | |
| *** harmw has joined #openstack-kolla | 10:07 | |
| *** dims has joined #openstack-kolla | 10:07 | |
| *** cj has joined #openstack-kolla | 10:07 | |
| *** smekel_ has joined #openstack-kolla | 10:07 | |
| *** jroll has joined #openstack-kolla | 10:07 | |
| *** tonyb has joined #openstack-kolla | 10:07 | |
| *** cloudnull has joined #openstack-kolla | 10:07 | |
| *** psanchez has joined #openstack-kolla | 10:09 | |
| *** salv-orlando has joined #openstack-kolla | 10:09 | |
| *** ankush has joined #openstack-kolla | 10:09 | |
| *** kong has joined #openstack-kolla | 10:09 | |
| *** zhubingbing has joined #openstack-kolla | 10:09 | |
| *** coolsvap has joined #openstack-kolla | 10:09 | |
| *** irtermit- has joined #openstack-kolla | 10:09 | |
| *** kklimonda has joined #openstack-kolla | 10:09 | |
| *** Pavo has joined #openstack-kolla | 10:09 | |
| *** qwang has joined #openstack-kolla | 10:09 | |
| *** flaper87 has joined #openstack-kolla | 10:09 | |
| *** afranc has joined #openstack-kolla | 10:09 | |
| *** cmceniry_ has joined #openstack-kolla | 10:09 | |
| *** dotplus has joined #openstack-kolla | 10:09 | |
| *** quasar has joined #openstack-kolla | 10:09 | |
| *** elemoine_ has joined #openstack-kolla | 10:09 | |
| *** nick-ma has joined #openstack-kolla | 10:09 | |
| *** zhurong has quit IRC | 10:10 | |
| *** hyakuhei- has joined #openstack-kolla | 10:10 | |
| *** wirehead_ has joined #openstack-kolla | 10:10 | |
| *** hyakuhei- has quit IRC | 10:10 | |
| *** wirehead_ has quit IRC | 10:10 | |
| *** salv-orlando has quit IRC | 10:10 | |
| *** ankush has quit IRC | 10:10 | |
| *** kong has quit IRC | 10:10 | |
| *** zhubingbing has quit IRC | 10:10 | |
| *** coolsvap has quit IRC | 10:10 | |
| *** irtermit- has quit IRC | 10:10 | |
| *** kklimonda has quit IRC | 10:10 | |
| *** Pavo has quit IRC | 10:10 | |
| *** qwang has quit IRC | 10:10 | |
| *** flaper87 has quit IRC | 10:10 | |
| *** afranc has quit IRC | 10:10 | |
| *** cmceniry_ has quit IRC | 10:10 | |
| *** dotplus has quit IRC | 10:10 | |
| *** quasar has quit IRC | 10:10 | |
| *** elemoine_ has quit IRC | 10:10 | |
| *** nick-ma has quit IRC | 10:10 | |
| *** psanchez has quit IRC | 10:10 | |
| *** serverascode has quit IRC | 10:10 | |
| *** DuncanT has quit IRC | 10:10 | |
| *** amaged__ has quit IRC | 10:10 | |
| *** timss has quit IRC | 10:10 | |
| *** Serlex has quit IRC | 10:10 | |
| *** shardy has quit IRC | 10:10 | |
| *** qbjolof1 has quit IRC | 10:10 | |
| *** Guest66666 has quit IRC | 10:10 | |
| *** jgriffith has quit IRC | 10:10 | |
| *** esmiurium has quit IRC | 10:10 | |
| *** peterjenkins_ has quit IRC | 10:10 | |
| *** EmilienM has quit IRC | 10:10 | |
| *** cu5 has quit IRC | 10:10 | |
| *** wznoinsk has quit IRC | 10:10 | |
| *** NobodyCam has quit IRC | 10:10 | |
| *** denaitre has quit IRC | 10:10 | |
| *** mgkwill has quit IRC | 10:10 | |
| *** SaMnCo has quit IRC | 10:10 | |
| *** harmw has quit IRC | 10:10 | |
| *** nihilifer has quit IRC | 10:10 | |
| *** harbie has quit IRC | 10:10 | |
| *** dims has quit IRC | 10:10 | |
| *** cj has quit IRC | 10:10 | |
| *** smekel_ has quit IRC | 10:10 | |
| *** jroll has quit IRC | 10:10 | |
| *** tonyb has quit IRC | 10:10 | |
| *** cloudnull has quit IRC | 10:10 | |
| *** Mech422 has quit IRC | 10:10 | |
| *** portdirect has quit IRC | 10:10 | |
| *** matrohon has quit IRC | 10:10 | |
| *** ajafo has quit IRC | 10:10 | |
| *** psanchez has joined #openstack-kolla | 10:26 | |
| *** matrohon has joined #openstack-kolla | 10:26 | |
| *** ajafo has joined #openstack-kolla | 10:26 | |
| *** serverascode has joined #openstack-kolla | 10:26 | |
| *** amaged__ has joined #openstack-kolla | 10:26 | |
| *** timss has joined #openstack-kolla | 10:26 | |
| *** Serlex has joined #openstack-kolla | 10:26 | |
| *** shardy has joined #openstack-kolla | 10:26 | |
| *** Guest66666 has joined #openstack-kolla | 10:26 | |
| *** qbjolof1 has joined #openstack-kolla | 10:26 | |
| *** jgriffith has joined #openstack-kolla | 10:26 | |
| *** EmilienM has joined #openstack-kolla | 10:26 | |
| *** esmiurium has joined #openstack-kolla | 10:26 | |
| *** peterjenkins_ has joined #openstack-kolla | 10:26 | |
| *** wznoinsk has joined #openstack-kolla | 10:26 | |
| *** cu5 has joined #openstack-kolla | 10:26 | |
| *** NobodyCam has joined #openstack-kolla | 10:26 | |
| *** denaitre has joined #openstack-kolla | 10:26 | |
| *** mgkwill has joined #openstack-kolla | 10:26 | |
| *** SaMnCo has joined #openstack-kolla | 10:26 | |
| *** harbie has joined #openstack-kolla | 10:26 | |
| *** nihilifer has joined #openstack-kolla | 10:26 | |
| *** harmw has joined #openstack-kolla | 10:26 | |
| *** dims has joined #openstack-kolla | 10:26 | |
| *** cj has joined #openstack-kolla | 10:26 | |
| *** smekel_ has joined #openstack-kolla | 10:26 | |
| *** jroll has joined #openstack-kolla | 10:26 | |
| *** tonyb has joined #openstack-kolla | 10:26 | |
| *** cloudnull has joined #openstack-kolla | 10:26 | |
| *** hyakuhei- has joined #openstack-kolla | 10:27 | |
| *** wirehead_ has joined #openstack-kolla | 10:27 | |
| *** Mech422 has joined #openstack-kolla | 10:27 | |
| *** portdirect has joined #openstack-kolla | 10:27 | |
| *** domi007 has joined #openstack-kolla | 10:27 | |
| *** domi007 has left #openstack-kolla | 10:27 | |
| *** elemoine_ has joined #openstack-kolla | 10:27 | |
| *** DuncanT has joined #openstack-kolla | 10:27 | |
| *** salv-orlando has joined #openstack-kolla | 10:27 | |
| *** ankush has joined #openstack-kolla | 10:27 | |
| *** zhubingbing has joined #openstack-kolla | 10:27 | |
| *** coolsvap has joined #openstack-kolla | 10:27 | |
| *** irtermit- has joined #openstack-kolla | 10:27 | |
| *** kklimonda has joined #openstack-kolla | 10:27 | |
| *** Pavo has joined #openstack-kolla | 10:27 | |
| *** qwang has joined #openstack-kolla | 10:27 | |
| *** flaper87 has joined #openstack-kolla | 10:27 | |
| *** afranc has joined #openstack-kolla | 10:27 | |
| *** cmceniry_ has joined #openstack-kolla | 10:27 | |
| *** dotplus has joined #openstack-kolla | 10:27 | |
| *** quasar has joined #openstack-kolla | 10:27 | |
| *** nick-ma has joined #openstack-kolla | 10:27 | |
| *** janem has joined #openstack-kolla | 10:28 | |
| *** rmart04 has joined #openstack-kolla | 10:28 | |
| *** msimonin has joined #openstack-kolla | 10:28 | |
| *** steve-noyes has joined #openstack-kolla | 10:28 | |
| *** sdake_ has joined #openstack-kolla | 10:28 | |
| *** hieulq__ has joined #openstack-kolla | 10:28 | |
| *** pbourke has joined #openstack-kolla | 10:28 | |
| *** openstackgerrit has joined #openstack-kolla | 10:28 | |
| *** ntpttr has joined #openstack-kolla | 10:28 | |
| *** jascott1 has joined #openstack-kolla | 10:28 | |
| *** cinerama has joined #openstack-kolla | 10:28 | |
| *** kong has joined #openstack-kolla | 10:28 | |
| *** ediardo has joined #openstack-kolla | 10:28 | |
| *** gfidente has joined #openstack-kolla | 10:28 | |
| *** b_bezak has joined #openstack-kolla | 10:28 | |
| *** mkoderer has joined #openstack-kolla | 10:28 | |
| *** macsz_ has joined #openstack-kolla | 10:28 | |
| *** zigo has joined #openstack-kolla | 10:28 | |
| *** rbbratta has joined #openstack-kolla | 10:28 | |
| *** NachoDuck has joined #openstack-kolla | 10:28 | |
| *** skramaja has joined #openstack-kolla | 10:28 | |
| *** kbyrne has joined #openstack-kolla | 10:28 | |
| *** ccesario has joined #openstack-kolla | 10:28 | |
| *** manjeets_ has joined #openstack-kolla | 10:28 | |
| *** robcresswell has joined #openstack-kolla | 10:28 | |
| *** strigazi has joined #openstack-kolla | 10:28 | |
| *** zhenguo has joined #openstack-kolla | 10:28 | |
| *** dmsimard has joined #openstack-kolla | 10:28 | |
| *** dasm has joined #openstack-kolla | 10:28 | |
| *** Kimmo__ has joined #openstack-kolla | 10:28 | |
| *** DuncanT has quit IRC | 10:28 | |
| *** NobodyCam has quit IRC | 10:29 | |
| *** serverascode has quit IRC | 10:29 | |
| *** ChanServ has joined #openstack-kolla | 10:30 | |
| *** tepper.freenode.net sets mode: +o ChanServ | 10:30 | |
| *** elemoine_ has quit IRC | 10:31 | |
| *** coolsvap has quit IRC | 10:32 | |
| *** rmart04 has left #openstack-kolla | 10:33 | |
| *** DuncanT has joined #openstack-kolla | 10:33 | |
| *** rmart04 has joined #openstack-kolla | 10:33 | |
| *** sdake_ has quit IRC | 10:35 | |
| *** karlamrhein has quit IRC | 10:35 | |
| *** karlamrhein has joined #openstack-kolla | 10:35 | |
| openstackgerrit | Mark Duggan proposed openstack/kolla: Parameterising iscsi_helper and setting default value https://review.openstack.org/385553 | 10:35 |
| *** amaged__ has quit IRC | 10:35 | |
| *** amaged__ has joined #openstack-kolla | 10:35 | |
| *** ankush has quit IRC | 10:36 | |
| *** ankush has joined #openstack-kolla | 10:36 | |
| *** DuncanT has quit IRC | 10:37 | |
| *** DuncanT has joined #openstack-kolla | 10:37 | |
| *** NobodyCam has joined #openstack-kolla | 10:38 | |
| *** elemoine_ has joined #openstack-kolla | 10:40 | |
| *** zeus is now known as Guest10825 | 10:42 | |
| *** coolsvap has joined #openstack-kolla | 10:45 | |
| *** yuanying has joined #openstack-kolla | 10:45 | |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: set basic kolla structures for iscsid https://review.openstack.org/381942 | 10:49 |
| *** salv-orlando has quit IRC | 10:51 | |
| *** serverascode has joined #openstack-kolla | 10:51 | |
| openstackgerrit | Mark Duggan proposed openstack/kolla: Parameterising iscsi_helper and setting default value https://review.openstack.org/385553 | 10:51 |
| *** mewald has joined #openstack-kolla | 10:52 | |
| *** awidders_ has quit IRC | 10:59 | |
| *** salv-orlando has joined #openstack-kolla | 11:03 | |
| *** sdake has joined #openstack-kolla | 11:03 | |
| sdake | morning peeps | 11:04 |
| sdake | Jeffrey4l__ any luck looking at that problem? | 11:04 |
| sdake | Jeffrey4l__ I think I feel asleep at the keyboard | 11:04 |
| *** awebber_ has joined #openstack-kolla | 11:06 | |
| sdake | anyone worked on https://bugs.launchpad.net/kolla/+bug/1564773 | 11:07 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 11:07 |
| sdake | blocking tag of rc2 | 11:07 |
| sdake | pbourke ^^ | 11:08 |
| sdake | coolsvap ^^ | 11:08 |
| pbourke | sdake: will look now | 11:08 |
| sdake | just last comment | 11:08 |
| openstackgerrit | Merged openstack/kolla: Change keystone.log's permit to keystone:keystone https://review.openstack.org/384146 | 11:08 |
| *** sdake_ has joined #openstack-kolla | 11:11 | |
| *** sdake has quit IRC | 11:13 | |
| sdake_ | pbourke i guess problem on my end http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2016-10-13.log.html#t2016-10-13T04:44:51 | 11:15 |
| sdake_ | although I'd expect deploy to you know, actually deploy new images | 11:15 |
| *** tonanhngo has joined #openstack-kolla | 11:15 | |
| pbourke | sdake_: yeah it doesn't :/ | 11:16 |
| *** tonanhngo has quit IRC | 11:16 | |
| pbourke | sdake_: you have to explicitly pull or remove them | 11:16 |
| pbourke | sdake_: unsure why that decision was made | 11:16 |
| pbourke | sdake_: so does it work for you now? | 11:16 |
| sdake_ | not sure - need to test the deploy | 11:17 |
| pbourke | ok | 11:17 |
| sdake_ | likei i said i feel asleep att he keyboard | 11:17 |
| awebber_ | sdake i have tested external ceph with cinder however it does not deploy. there is a different approach between external ceph and normal ceph deployment. a 'merge config' function is used, only in the external ceph.yml, that fails. i had to revert to a simple copy approach, forgoing merging config approach, so that the deployment was successful. cinder with external ceph now works. | 11:18 |
| Jeffrey4l__ | sdake_, you are using old image on node2 http://pix.toile-libre.org/upload/original/1476333859.png | 11:20 |
| Jeffrey4l__ | sdake_, rabbitmq works find now. ( i didn't re-build the image, just pull latest images in all nodes ) | 11:20 |
| sdake_ | Jeffrey4l__ any idea why deploy doesn't deploy latest image | 11:22 |
| sdake_ | behavior seems inconsistent to me | 11:22 |
| sdake_ | deploy fresh environet -> pulls images | 11:22 |
| sdake_ | deploy on existing environemnt -> dont pull fresh images | 11:22 |
| Jeffrey4l__ | sdake_, you didn't pull the latest image. and docker-engine doesn't add insecure-registry 103:4000 | 11:22 |
| sdake_ | --insecure-registery 103:4000 wasn't set on 102? | 11:23 |
| Jeffrey4l__ | sdake_, yes. i added it today. | 11:23 |
| sdake_ | was it set at all? | 11:23 |
| Jeffrey4l__ | sorry? | 11:24 |
| Jeffrey4l__ | pbourke, please review this https://review.openstack.org/378762 | 11:27 |
| Jeffrey4l__ | add this https://review.openstack.org/384598 | 11:27 |
| *** sdake has joined #openstack-kolla | 11:27 | |
| pbourke | Jeffrey4l__: I think we need a bug id if we're to merge for Newton | 11:27 |
| *** haplo37_ has quit IRC | 11:27 | |
| pbourke | Jeffrey4l__: Im not seeing any high profile breakages without this | 11:28 |
| Jeffrey4l__ | pbourke, ok. np. i will file a bug. | 11:28 |
| *** sdake_ has quit IRC | 11:28 | |
| Jeffrey4l__ | pbourke, it may cause issue. and when you changed the timezone, the time in the container will be jump from one time to another ( he doesn't know the timezone info ) like from 5am to 8am | 11:29 |
| *** domi007_ has joined #openstack-kolla | 11:29 | |
| pbourke | ok | 11:30 |
| *** haplo37_ has joined #openstack-kolla | 11:30 | |
| domi007_ | hi all, I'd like to get some help on an issue I'm facing: I'm trying to deploy using kolla to a coreos host, and while trying to deploy neutron-dhcp-agent the following error pops up in the log: | 11:30 |
| domi007_ | level=error msg="Handler for POST /v1.21/containers/create returned error: invalid mode for volumes-from: shared" | 11:31 |
| *** mliima has joined #openstack-kolla | 11:31 | |
| pbourke | domi007_: you need to set the mountflags on your docker daemon | 11:31 |
| Jeffrey4l__ | domi007_, check this doc http://docs.openstack.org/developer/kolla/quickstart.html#install-dependencies | 11:31 |
| pbourke | domi007_: check the quickstart docs for this | 11:31 |
| domi007_ | pbourke: already set them | 11:31 |
| Jeffrey4l__ | dmsimard, docker version? | 11:32 |
| Jeffrey4l__ | sorry pinged wrong person. domi007_ ^^ | 11:32 |
| domi007_ | pbourke: MountFlags=1048576 just checked via systemctl | 11:32 |
| pbourke | it needs to be MountFlags=shared | 11:32 |
| domi007_ | Jeffrey4l__: 1.9.1 | 11:32 |
| mliima | morning | 11:32 |
| Jeffrey4l__ | domi007_, we need 1.10 at min | 11:33 |
| domi007_ | Jeffrey4l__: ah okay, let me try to update it | 11:33 |
| domi007_ | and see if it work | 11:33 |
| domi007_ | thank you | 11:33 |
| Jeffrey4l__ | cool. | 11:33 |
| pbourke | Jeffrey4l__: dont suppose you've ever tried setting a difference value for network_interface on one host? | 11:33 |
| Jeffrey4l__ | pbourke, yes. i tried with success. | 11:34 |
| Jeffrey4l__ | but you need remove the network_interface from globals.yml and set it in inventory file. | 11:34 |
| pbourke | Jeffrey4l__: ahhhhh! | 11:34 |
| pbourke | Jeffrey4l__: so obvious! thank you | 11:34 |
| pbourke | wasted hours on this o_0 | 11:35 |
| Jeffrey4l__ | ;) np. | 11:35 |
| openstackgerrit | pawnesh kumar proposed openstack/kolla: Use upper constraints for all jobs in tox.ini https://review.openstack.org/384221 | 11:35 |
| Jeffrey4l__ | this is how ansible works. globals.yml overwrite all values. | 11:35 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Mount host localtime for containers https://review.openstack.org/378762 | 11:36 |
| *** salv-orlando has quit IRC | 11:36 | |
| pbourke | I know just somehow forgot | 11:36 |
| pbourke | globals.yml suck | 11:36 |
| Jeffrey4l__ | pbourke, re localtime add a bug id ^^ | 11:36 |
| pbourke | there are better override mechanisms | 11:36 |
| pbourke | Jeffrey4l__: cool thanks +2 | 11:36 |
| Jeffrey4l__ | ya. | 11:36 |
| Jeffrey4l__ | thanks. | 11:36 |
| *** tonanhngo has joined #openstack-kolla | 11:39 | |
| *** tonanhngo has quit IRC | 11:40 | |
| sdake | Jeffrey4l__ what is your thinking on why deploy doesn't pull if images exist? | 11:42 |
| sdake | i dont think that makes a whole lot of sense | 11:43 |
| *** zhubingbing has quit IRC | 11:43 | |
| sdake | also I have tagged at this hash: 40e768ec2a370dc010be773af37e2ce417adda80 | 11:43 |
| sdake | anything aftre thta needs a backport | 11:43 |
| *** ppalacios has joined #openstack-kolla | 11:44 | |
| openstackgerrit | Praveen N proposed openstack/kolla: [ovs] bridge_mappings new line fix https://review.openstack.org/385924 | 11:48 |
| *** athomas_ has joined #openstack-kolla | 11:53 | |
| *** rmart04_ has joined #openstack-kolla | 11:55 | |
| *** flaper87 has quit IRC | 11:56 | |
| *** athomas has quit IRC | 11:56 | |
| *** rmart04 has quit IRC | 11:57 | |
| *** rmart04_ is now known as rmart04 | 11:57 | |
| *** dwalsh_ has joined #openstack-kolla | 12:02 | |
| *** phuongnh has quit IRC | 12:03 | |
| *** sdake_ has joined #openstack-kolla | 12:05 | |
| *** tonanhngo has joined #openstack-kolla | 12:06 | |
| *** sdake has quit IRC | 12:06 | |
| *** tonanhngo has quit IRC | 12:06 | |
| sdake_ | here is rc3: https://launchpad.net/kolla/+milestone/newton-rc3 | 12:11 |
| openstackgerrit | Praveen N proposed openstack/kolla: Fix template for multiple physical networks https://review.openstack.org/378276 | 12:11 |
| *** flaper87 has joined #openstack-kolla | 12:12 | |
| *** flaper87 has quit IRC | 12:12 | |
| *** flaper87 has joined #openstack-kolla | 12:12 | |
| *** williamcaban has joined #openstack-kolla | 12:14 | |
| sbezverk_ | pbourke: ping | 12:15 |
| pbourke | sbezverk_: hi | 12:15 |
| sbezverk_ | pbourke: good morning, or no afternoon for you :-) I replied to your comment | 12:16 |
| pbourke | thanks :) will look now | 12:16 |
| sbezverk_ | about drop root | 12:16 |
| pbourke | can you remind me the link | 12:16 |
| sbezverk_ | sure | 12:16 |
| *** JoseMello has joined #openstack-kolla | 12:16 | |
| sbezverk_ | https://review.openstack.org/#/c/384988/3 | 12:16 |
| pbourke | ceph will be dropping root very soon | 12:17 |
| pbourke | but it seems you've looked at it so it's good for now | 12:17 |
| *** rhallisey has joined #openstack-kolla | 12:17 | |
| sbezverk_ | pbourke: thanks once it is done on kolla side we will do it too and re-use your sudoers file | 12:17 |
| pbourke | sounds good! | 12:18 |
| sbezverk_ | second question is about fluentd repo location. | 12:18 |
| sbezverk_ | I mean since kolla does not use it do you still wants it to be in the base? | 12:18 |
| sbezverk_ | https://review.openstack.org/#/c/375615/13 | 12:19 |
| sbezverk_ | pbourke: personnaly I do not care, I was just concern to hit more resistance from kolla cores ;-) | 12:20 |
| pbourke | sbezverk_: did you see inc0 also did a k8s toolbox? | 12:20 |
| wznoinsk | sdake_: would we make https://bugs.launchpad.net/kolla/+bug/1632850 targetting both milestones or it's worth a short discussion with the rest of the guys? | 12:21 |
| openstack | Launchpad bug 1632850 in kolla "lack of basic kolla structures for iscsid " [High,In progress] - Assigned to Waldemar Znoinski (wznoinsk) | 12:21 |
| sbezverk_ | really?? when ?? he did not say anything .. if it is the case then we need only one.. | 12:21 |
| pbourke | I think his patch went up first actually but yours seems ready to go so I dont think he'll mind | 12:21 |
| pbourke | I asked him to check yours anyway | 12:21 |
| sbezverk_ | pbourke: I will ping him later for PS | 12:22 |
| *** srwilkers has joined #openstack-kolla | 12:33 | |
| *** tonanhngo has joined #openstack-kolla | 12:35 | |
| *** tonanhngo has quit IRC | 12:36 | |
| *** egonzalez90 has joined #openstack-kolla | 12:38 | |
| sbezverk_ | pbourke: sorry but what about fluentd repo, do you still want me to move to base? | 12:42 |
| bjolo-afk | good afternoon | 12:52 |
| *** prithiv has joined #openstack-kolla | 12:58 | |
| *** zhubingbing has joined #openstack-kolla | 12:58 | |
| *** lamt has joined #openstack-kolla | 12:59 | |
| *** vks100 has joined #openstack-kolla | 13:00 | |
| bjolo-afk | sdake_, around? can i please challenge your verdict on this one :) https://bugs.launchpad.net/kolla/+bug/1626958 | 13:01 |
| openstack | Launchpad bug 1626958 in kolla "neutron_server not working when lbaas enabled" [Medium,Confirmed] - Assigned to Swapnil Kulkarni (coolsvap) | 13:01 |
| *** bjolo-afk is now known as bjolo | 13:01 | |
| sdake_ | fell asleep at thek keyboard again groan | 13:02 |
| sdake_ | bjolo to be brutally frank, I didn't take a hard look at medium bugs for rc3 | 13:02 |
| sdake_ | only high and critical | 13:02 |
| sdake_ | bjolo so if something got missed its probably my fault | 13:02 |
| sdake_ | i did inspect all high and critical bugs for about 8 hours oro so | 13:02 |
| sdake_ | to make sure they were triaged propely | 13:02 |
| sdake_ | and to determine what made rc3 | 13:02 |
| sdake_ | that said, could have mde mistakes there | 13:02 |
| sdake_ | we work at this as a community bjolo no verdict | 13:02 |
| sdake_ | bjolo so if there is something you need for rc3 and its critical lets see if we can get it solved | 13:03 |
| bjolo | :) im new here so i might choose the wrong words sometimes | 13:03 |
| sdake_ | bjolo its good we know english is not native for you | 13:04 |
| sdake_ | bjolo i even choosse wrong words and I am english native speaker | 13:04 |
| bjolo | i just feel that enable lbaas breaking neutron completely is higher than medium. lbaas is the foundation to run cloud native applications so it is a very common feature. about to say standard to run on a cloud | 13:04 |
| sdake_ | ok so looking at bug | 13:04 |
| sdake_ | bjolo give me a moment to process the bug | 13:04 |
| vks100 | Hi, running into some problems with kolla/magnum | 13:05 |
| vks100 | status_reason | Failed to create trustee or trust for Cluster: 34f9066e-c3f4-438f-bfb3-29cff772174c | 13:05 |
| mnasiadka | vks100: stable/mitaka or master? | 13:06 |
| vks100 | pip show kolla --- Metadata-Version: 2.0 Name: kolla Version: 2.0.2 Summary: Kolla OpenStack Deployment Home-page: http://www.openstack.org/ Author: OpenStack Author-email: openstack-dev@lists.openstack.org Installer: pip License: UNKNOWN Location: /usr/lib/python2.7/site-packages Requires: pbr, graphviz, Jinja2, six, oslo.config, pycrypto, gitdb, beautifulsoup4, GitPython, docker-py, setuptools Classifiers: Environment | 13:06 |
| *** awebber_ has quit IRC | 13:06 | |
| vks100 | stable/mitaka.. | 13:06 |
| sdake_ | bjolo i think course of action here is to document how to enable lbaas/vpnaas | 13:06 |
| sdake_ | bjolo as for including fwaas an dvpnaas directly in the contianer themselves, I am not sure that makes alot of sense | 13:07 |
| sdake_ | bjolo I'd have to think about that for 5 minutes after waking up :) | 13:07 |
| sdake_ | bjolo these are plugins -and neutron is going to more plugins not less | 13:08 |
| sdake_ | bjolo what we want to avoid is having 60 plugins installed by default because they are needed for some specific use cse | 13:08 |
| sdake_ | bjolo thats why image customization exists | 13:08 |
| mnasiadka | vks100: I remember stable/mitaka did not had barbican enabled and there were a lot of problems with creating trust/trustees - but wait for some expert advice from the developers ;) | 13:08 |
| sdake_ | bjolo the policy decision about inclluding by dedfault vpnaas and fwaas - i think we need to have that discussion | 13:08 |
| bjolo | sdake_, i can agree with that reasoning to some extent, but here we are talking about kolla supported services enabled in globals.yml | 13:09 |
| sdake_ | bjolo yes i jus tthought of that condition | 13:09 |
| sdake_ | bjolo again awake for 5 minutes :) | 13:10 |
| bjolo | :) | 13:10 |
| bjolo | take your time and have a coffee first :) | 13:10 |
| sdake_ | mexican coke | 13:10 |
| bjolo | or that | 13:10 |
| sdake_ | i dont drink coffee | 13:10 |
| sdake_ | often atleast | 13:10 |
| zhubingbing | hello guys | 13:13 |
| *** salv-orlando has joined #openstack-kolla | 13:14 | |
| *** lamt has quit IRC | 13:14 | |
| *** jheroux has joined #openstack-kolla | 13:14 | |
| mnasiadka | vks100: https://bugs.launchpad.net/kolla/+bug/1551992 - this may come in handy, but I see it was only merged in master | 13:14 |
| openstack | Launchpad bug 1551992 in kolla mitaka "Magnum failed to create trustee" [Critical,Confirmed] - Assigned to Vikram Hosakote (vhosakot) | 13:14 |
| vks100 | mnasiadka thank you. I will try that. | 13:16 |
| mnasiadka | vks100: maybe just wait a week or more for stable/newton kolla release | 13:17 |
| *** awebber_ has joined #openstack-kolla | 13:17 | |
| sdake_ | bjolo theoretically, high/critical bugs in ocata will be backported to 3.0.1 | 13:18 |
| sdake_ | bjolo its too late to give the proper thought to that problem for 3.0.0 | 13:18 |
| sdake_ | bjolo as it affects more then vpnaas and fwaas, your asking a policy be made | 13:18 |
| sdake_ | and pbourke is requesting that policy decision be punted to post newton | 13:18 |
| sdake_ | bjolo fwiw I agree the policy decision should be punted to post newton, and agree with you bjolo that the tricky part is that we have enables for these things yet they dont *enable* | 13:19 |
| bjolo | ok i respect that and i know how to build my own images to get around it | 13:20 |
| *** haplo37 has quit IRC | 13:20 | |
| sdake_ | bjolo we do have an easy way to customize the images | 13:20 |
| sdake_ | bjolo i'm not sure if that is goodd enough for this csaes | 13:20 |
| sdake_ | bjolo i think wht we need to make the policy decision is the full list of things affected now and into the future | 13:21 |
| *** g3ek has quit IRC | 13:21 | |
| *** srwilkers has quit IRC | 13:21 | |
| bjolo | but as you said, they are official kolla services. | 13:21 |
| sdake_ | bjolo right | 13:21 |
| *** haplo37 has joined #openstack-kolla | 13:21 | |
| sdake_ | bjolo so is cinder | 13:21 |
| bjolo | i just fear that new users to kolla will do a pip install and deploy with the official images on docker hub and things dont work | 13:22 |
| sdake_ | cinder has 50+ plugins | 13:22 |
| sdake_ | do we install all 50 plugins for cinder | 13:22 |
| bjolo | yes, but we dont have 50 cinder plugins in globals.yml | 13:22 |
| sdake_ | or do we install only a subset | 13:22 |
| *** g3ek has joined #openstack-kolla | 13:22 | |
| sdake_ | bjolo right - i understand we dont list those plugisn out | 13:22 |
| sdake_ | bjolo maybe where we went wrong is having an enable_vpnanss or enable_fwwas in the first place | 13:22 |
| *** jtriley has joined #openstack-kolla | 13:22 | |
| bjolo | kollas contract to users ought to be that all that is enabled in globals.yml should wokr | 13:22 |
| sdake_ | or maybe they aren't actually plugins | 13:23 |
| sdake_ | bjolo yup tend to agree | 13:23 |
| *** portdirect has quit IRC | 13:23 | |
| *** portdirect_ has joined #openstack-kolla | 13:23 | |
| bjolo | sdake_, agree as well. remove vpn and lbaas from globals.yml | 13:23 |
| *** portdirect_ is now known as portdirect | 13:23 | |
| sdake_ | bjolo - i'm not sure what that would break or fix :) | 13:23 |
| bjolo | if it is in there, it should work with the official binary images on docker hub | 13:24 |
| sdake_ | bjolo ok so thats what it would fix | 13:24 |
| sdake_ | i'm not sure what it would break | 13:24 |
| * sdake_ never wanted to ship stuff on docker hub | 13:24 | |
| * sdake_ sniffles | 13:24 | |
| wznoinsk | sdake_: could you look at my last comment on https://bugs.launchpad.net/kolla/+bug/1632840 | 13:25 |
| openstack | Launchpad bug 1632840 in kolla "ldlinux.c32 File not found in ironic_pxe" [High,In progress] - Assigned to Waldemar Znoinski (wznoinsk) | 13:25 |
| bjolo | its about being 100% clear on what to expect as a user. Only ship what works. things that are halv broken tend to tick people off | 13:26 |
| bjolo | either support and fix vpn and lbaas all the way, including the official docker images or remove it from globals.yml | 13:26 |
| sdake_ | bjolo ya i get that | 13:27 |
| *** inc0 has joined #openstack-kolla | 13:27 | |
| *** ayoung has joined #openstack-kolla | 13:28 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/kolla: Update reno for stable/newton https://review.openstack.org/385985 | 13:28 |
| sdake_ | bjolo i guess we could improve on setting expectations aroudn what works and what doesn't - we have a summit session on fixing that problem in the future | 13:28 |
| sdake_ | bjolo although that session doesn't specficially include this issue | 13:29 |
| inc0 | good morning | 13:29 |
| sdake_ | bjolo here is the deal - if someone fixes it prior to 18th, I dont particularly care if it goes in - as long as its not considered a policy deicision | 13:29 |
| inc0 | dims, I see you're in process of creating stable branch? great! | 13:29 |
| dims | inc0 : done, please cross check | 13:30 |
| inc0 | cool:) thanks! | 13:30 |
| sbezverk_ | sdake_: remember last night one guy had problem with https/http access to registry? I know see this problem with build on the lastest master | 13:30 |
| bjolo | can we have a policy that everything in globals.yml is supported and if it does not work it is critical? :) | 13:30 |
| sbezverk_ | what was the solution for him? | 13:30 |
| *** williamcaban has quit IRC | 13:31 | |
| *** fguillot has joined #openstack-kolla | 13:31 | |
| *** tonanhngo has joined #openstack-kolla | 13:31 | |
| inc0 | bjolo, so you can override every variable in whole ansible in globals;) | 13:32 |
| *** tonanhngo has quit IRC | 13:33 | |
| bjolo | inc0, yes i know and love that part of kolla but :) | 13:33 |
| bjolo | this is more about the contract to the users of what should be guaranteed and tested to work | 13:34 |
| inc0 | bjolo, what are we talking about anyway? what's not working? | 13:34 |
| bjolo | as it is right now, we have kolla services in globals that we know are broke | 13:35 |
| inc0 | bjolo, we'll always have sth like this | 13:35 |
| bjolo | https://bugs.launchpad.net/kolla/+bug/1626958 | 13:35 |
| openstack | Launchpad bug 1626958 in kolla "neutron_server not working when lbaas enabled" [High,Confirmed] - Assigned to Swapnil Kulkarni (coolsvap) | 13:35 |
| inc0 | however what we need to do is to make it clear what is well tested and what isn't | 13:36 |
| inc0 | we probably have biggest portfolio of services deployable by Kolla | 13:36 |
| sdake_ | probably/do | 13:36 |
| inc0 | out of all deployment tools | 13:36 |
| bjolo | yes | 13:37 |
| bjolo | dont get me wrong. kolla fricking rocks! | 13:37 |
| inc0 | it's impossible to keep everything on same level of testing | 13:37 |
| sdake_ | bjolo gets that inc0 | 13:37 |
| sdake_ | this isn't a testing issue | 13:37 |
| sdake_ | this is a policy issue | 13:37 |
| bjolo | spent all day pushing it internally in my company :) | 13:37 |
| sdake_ | the issue is | 13:37 |
| inc0 | bjolo, I hear ya, what I was thinking is to make it crystal clear somewhere that "this service doesnt have CI" | 13:37 |
| inc0 | like a matrix | 13:37 |
| sdake_ | 1) if a service can be enabled in globals.yml, it should work out of the box without customizations in the docker images." | 13:38 |
| inc0 | so operators will know what to expect and new contributors will know what needs fixing | 13:38 |
| inc0 | so ofc, if it doesn't work ootb - that is a bug | 13:39 |
| sdake_ | inc0 bjolo isn't really talking about that | 13:39 |
| inc0 | and it needs fixing | 13:39 |
| sdake_ | right - thats why its not marked invalid | 13:39 |
| sdake_ | bjolo was more concerned with the fact that its not marked critical I think :) | 13:40 |
| *** prithiv has quit IRC | 13:40 | |
| sbezverk_ | sdake: I check kolla log which is working for me and it does not have commit commit e8e1ad70893496f8956559775391500149a1205d | 13:40 |
| bjolo | sdake_, yes pushed to ocata-1 | 13:40 |
| sbezverk_ | sdake: I think that might bethe PS broke build.by | 13:41 |
| bjolo | my view is simply since it is a official kolla service, it should be critical and fixed | 13:41 |
| sdake_ | bjolo anyone in the drivers group can bring stuff into rc3 | 13:41 |
| sdake_ | bjolo are you in the drivers group? | 13:41 |
| bjolo | dont think so | 13:41 |
| sdake_ | launchpad id pls | 13:41 |
| bjolo | bjolo | 13:41 |
| bjolo | bjorn-lofdahl | 13:41 |
| sdake_ | christ | 13:42 |
| sdake_ | bjolo your lauchpad id isn't bjolo? | 13:42 |
| bjolo | yes it is | 13:42 |
| *** dave-mccowan has joined #openstack-kolla | 13:42 | |
| sdake_ | bjolo can ou please change the maintainer of kolla to kolla-drivers | 13:43 |
| bjolo | sorry nope | 13:43 |
| bjolo | bjorn-lofdahl | 13:43 |
| sdake_ | i fat fingered because i'm beat | 13:43 |
| bjolo | thats my launchpad ID | 13:43 |
| *** kproskurin has quit IRC | 13:43 | |
| bjolo | sdake_, how do i do that? | 13:44 |
| inc0 | bjolo, basically you are allowed now to change priorities and everything of bugs | 13:44 |
| inc0 | if you feel something is critical, just change it to critical | 13:44 |
| sdake_ | bjolo ok - first lets fix my mistake | 13:44 |
| sdake_ | bjolo open up https://launchpad.net/kolla | 13:45 |
| sdake_ | see where it says "maintainer: bjolo" | 13:45 |
| sdake_ | click the yllow button and set it to "kolla-drivers" pls | 13:45 |
| sdake_ | bjolo - the drivers team is our extended team of bug triagers | 13:46 |
| sdake_ | bjolo the triage process is described in the documentation | 13:46 |
| sdake_ | bjolo you can help as much or as little as you want :) | 13:46 |
| coolsvap | sdake_: bjolo when I tested it last week I think, I did not get the error trace as mentioned in the defect | 13:46 |
| bjolo | coolsvap, lets take a look at that. i can reproduce consistently | 13:47 |
| sdake_ | bjolo are you able to chane the maintainer | 13:47 |
| sdake_ | bjolo need maintainer chnaged asap :) | 13:47 |
| bjolo | maintainer says bjolo | 13:47 |
| bjolo | driver says kolla-drivers | 13:47 |
| sdake_ | ya do you see a yellow button next to maintainer | 13:48 |
| sdake_ | next to bjolo | 13:48 |
| *** athomas_ has quit IRC | 13:48 | |
| bjolo | ok done | 13:48 |
| sdake_ | thanks | 13:48 |
| *** mgoddard has joined #openstack-kolla | 13:49 | |
| bjolo | coolsvap, how did you do your test? | 13:49 |
| *** Marx314 has joined #openstack-kolla | 13:49 | |
| bjolo | AIO? | 13:49 |
| bjolo | i do multinode, ubuntu/centos source from kolla master | 13:50 |
| bjolo | fails every time | 13:50 |
| openstackgerrit | Merged openstack/kolla: Fix rabbitmq upgrade permission issue https://review.openstack.org/384598 | 13:50 |
| openstackgerrit | Merged openstack/kolla: Fix keystone fernet file exchange via ssh https://review.openstack.org/369418 | 13:50 |
| bjolo | afk for a while. time to pick up the little devils :) | 13:52 |
| *** MarMat has joined #openstack-kolla | 13:53 | |
| *** srwilkers has joined #openstack-kolla | 13:53 | |
| sdake_ | ok folks -> we need action for 1 week on these bugs: https://launchpad.net/kolla/+milestone/newton-rc3 | 13:55 |
| *** huikang has joined #openstack-kolla | 13:57 | |
| *** huikang has quit IRC | 13:57 | |
| *** huikang has joined #openstack-kolla | 13:57 | |
| *** vks100 has quit IRC | 13:58 | |
| *** huikang has quit IRC | 13:58 | |
| *** huikang has joined #openstack-kolla | 13:59 | |
| *** papacz has joined #openstack-kolla | 14:00 | |
| *** huikang has quit IRC | 14:03 | |
| *** papacz has quit IRC | 14:05 | |
| *** papacz has joined #openstack-kolla | 14:05 | |
| openstackgerrit | Merged openstack/kolla: Mount host localtime for containers https://review.openstack.org/378762 | 14:06 |
| openstackgerrit | Merged openstack/kolla: set basic kolla structures for iscsid https://review.openstack.org/381942 | 14:06 |
| openstackgerrit | Merged openstack/kolla: Update reno for stable/newton https://review.openstack.org/385985 | 14:08 |
| *** sdake_ has quit IRC | 14:09 | |
| *** sdake has joined #openstack-kolla | 14:10 | |
| *** yingjun has joined #openstack-kolla | 14:11 | |
| *** mewald has quit IRC | 14:12 | |
| *** ankush has quit IRC | 14:12 | |
| *** mnasiadka has quit IRC | 14:14 | |
| *** absubram has joined #openstack-kolla | 14:14 | |
| *** huikang has joined #openstack-kolla | 14:14 | |
| *** tonanhngo has joined #openstack-kolla | 14:14 | |
| *** tonanhngo has quit IRC | 14:15 | |
| *** liyifeng has joined #openstack-kolla | 14:17 | |
| *** ChanServ sets mode: +o sdake | 14:17 | |
| *** openstackgerrit has quit IRC | 14:18 | |
| *** openstackgerrit has joined #openstack-kolla | 14:18 | |
| *** sdake changes topic to "FINAL NEWTON DEADLINE - OCT 18, 2016; please work on these bugs https://launchpad.net/kolla/+milestone/newton-rc3; Please read the documentation here: http://docs.openstack.org/developer/kolla/; Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA); Summit schedule ht" | 14:19 | |
| *** ChanServ sets mode: -o sdake | 14:19 | |
| inc0 | sbezverk_, you fencing works?:) | 14:23 |
| sdake | inc0 have a look at comment #9 https://bugs.launchpad.net/kolla/+bug/1564773 | 14:23 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 14:23 |
| inc0 | hmm | 14:24 |
| inc0 | I know why:) | 14:24 |
| sdake | good lets hear it | 14:24 |
| *** mewald has joined #openstack-kolla | 14:24 | |
| sdake | this bug may be fixed in rc2 if the pull mechanics are logical | 14:24 |
| sdake | this is wht all the erlang rigamarole was about | 14:25 |
| inc0 | hmm..we had this var | 14:25 |
| inc0 | docker pull strategy or something which was either always or missing | 14:25 |
| sdake | i remember | 14:25 |
| inc0 | can't see it now tho | 14:25 |
| *** yingjun has quit IRC | 14:26 | |
| *** yingjun has joined #openstack-kolla | 14:26 | |
| inc0 | I guess we need to force upgrade tasks to pull always | 14:26 |
| inc0 | no...we can't | 14:26 |
| *** mark-casey1 has joined #openstack-kolla | 14:27 | |
| inc0 | because if somebody did pre-pull we need to make sure to use it | 14:27 |
| sdake | if someone pre-pulls that is the same thing as deploy pullling again (or ugprade) | 14:27 |
| sdake | pull operation is cheap | 14:27 |
| sdake | if the eimage is already there | 14:28 |
| inc0 | it's not | 14:28 |
| inc0 | well pull image will do docker pull | 14:28 |
| sdake | if someone prepulls image is already there no? | 14:28 |
| inc0 | well I guess it might be cheap | 14:28 |
| inc0 | as docker will detect that layers are ok | 14:28 |
| sdake | right | 14:29 |
| inc0 | but I'd be careful with doing this in code | 14:29 |
| *** tonanhngo has joined #openstack-kolla | 14:29 | |
| sdake | lets sort out why the pull strategy was removed | 14:29 |
| inc0 | I guess because we added pull task | 14:29 |
| sdake | thats a piss poor reason | 14:30 |
| inc0 | and it does make it kinda redundant | 14:30 |
| inc0 | still, nothing we can do about it right now | 14:30 |
| sdake | deploy doesn't work | 14:30 |
| sdake | if old images are on system | 14:30 |
| inc0 | what we can do is to ensure that upgrade docs are correct | 14:30 |
| sdake | DEPLOY not upgrade | 14:30 |
| inc0 | although that wont matter because we tag | 14:30 |
| sdake | god i'd hope upgrade always pulls | 14:31 |
| *** yingjun has quit IRC | 14:31 | |
| inc0 | upgrade should not always pull, but on the other hand if you do actual upgrade there won't be images:3.0.0 in system | 14:31 |
| inc0 | technically upgrade should be always between tags | 14:31 |
| inc0 | so this is only development env issue | 14:32 |
| inc0 | tbh | 14:32 |
| inc0 | because this will happen only if you run 3.0.0 and you redeploy 3.0.0 with different tags | 14:32 |
| inc0 | different code* | 14:32 |
| inc0 | which is master branch | 14:32 |
| *** papacz has quit IRC | 14:32 | |
| sdake | i see | 14:32 |
| inc0 | and for development you should do kolla-ansible destroy with --remove-images | 14:33 |
| sdake | can you document that then | 14:33 |
| inc0 | or whatever the flag is called | 14:33 |
| *** papacz has joined #openstack-kolla | 14:33 | |
| sdake | that is solution to a different bug i think | 14:33 |
| inc0 | are we positive that that's the root cause of this bug? | 14:33 |
| sdake | but my comment in #8 was more about why deploy behaves the way it does | 14:33 |
| sdake | inc0 the root cause of THIS bug is the erlang and rabbitmq changes we went through | 14:34 |
| sdake | there are about 4 or 5 erlang rabbitmq backtraces | 14:34 |
| sdake | my hope is by moving forward, we have fixed all those crashes | 14:34 |
| inc0 | ok, so destroying images and redeploying should be all good | 14:34 |
| *** yingjun has joined #openstack-kolla | 14:34 | |
| sdake | yup that should work | 14:34 |
| sdake | i wasn't aware a --remove-images flag was added | 14:34 |
| sdake | so didn't remove images obviously :) | 14:35 |
| inc0 | pbourke, ping | 14:36 |
| pbourke | inc0: hi | 14:36 |
| inc0 | https://bugs.launchpad.net/kolla/+bug/1564773 | 14:36 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 14:36 |
| inc0 | look at my comment | 14:36 |
| inc0 | if that's the case, I'd bump priority of bug to medium if not even close it alltogether | 14:37 |
| *** HyperJohnGraham_ has joined #openstack-kolla | 14:38 | |
| *** lamt has joined #openstack-kolla | 14:39 | |
| pbourke | inc0: the problem with this bug is every time rabbit fails for someone it's getting bundled in there | 14:39 |
| mewald | sdake: You marked https://bugs.launchpad.net/kolla/+bug/1632811 as no longer affects stable/mitaka. The bug is actually valid for mitaka, too, or did I get it wrong what that means on launchpad? | 14:39 |
| openstack | Launchpad bug 1632811 in kolla "Keystone token table filling up" [Critical,In progress] - Assigned to Mathias Ewald (mewald) | 14:39 |
| pbourke | inc0: and because rabbit is hard to debug its very hard to tell if they're all the same issue | 14:39 |
| sdake | mewald possible i made a mistake - feel free to correct | 14:39 |
| pbourke | inc0: if its working for us Id just close it | 14:39 |
| inc0 | well we do check it every single time in gate;) | 14:40 |
| *** yingjun has quit IRC | 14:40 | |
| sdake | see comment #11 | 14:40 |
| pbourke | inc0: well yeah, not upgrade though | 14:40 |
| *** yingjun has joined #openstack-kolla | 14:40 | |
| pbourke | though this is not an upgrade bug | 14:40 |
| sdake | upgraade includes pull.yml | 14:40 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1564773 - comment #11 | 14:41 |
| openstack | Launchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [Critical,Confirmed] | 14:41 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Fencing pod https://review.openstack.org/383922 | 14:41 |
| *** yingjun has quit IRC | 14:41 | |
| *** yingjun has joined #openstack-kolla | 14:41 | |
| pbourke | sdake: id rather just close it out | 14:42 |
| sdake | pbourke i want to collect any rabbitmq crashes | 14:42 |
| sdake | and it needs to be backported | 14:42 |
| pbourke | there's no bug there | 14:42 |
| pbourke | crashes can be reported as a new bug | 14:42 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Fencing pod https://review.openstack.org/383922 | 14:43 |
| sdake | ok fiar enough - there is a bug here pbourke | 14:43 |
| pbourke | what is it? | 14:43 |
| *** yingjun_ has joined #openstack-kolla | 14:43 | |
| sdake | pbourke the changess to move to rabbitmq 3.6 in centos and ubuntu fixed them | 14:43 |
| sdake | there are multiple bugs in this report | 14:43 |
| *** salv-orl_ has joined #openstack-kolla | 14:43 | |
| sdake | that set of changes related to rabbitmq 3.6 needs backporting | 14:43 |
| pbourke | ceph jewel is so flaky for me | 14:44 |
| pbourke | i have to wonder if our configs are still good | 14:45 |
| *** yingjun has quit IRC | 14:46 | |
| *** yingjun_ is now known as yingjun | 14:46 | |
| *** salv-orlando has quit IRC | 14:46 | |
| sdake | pbourke sounds frieghtening | 14:47 |
| sdake | pbourke multinode? | 14:47 |
| pbourke | yeah | 14:47 |
| pbourke | more frustrating | 14:47 |
| sdake | pbourke how mnay nodes | 14:47 |
| pbourke | I put ceph in the rabbitmq category of difficulty to debug | 14:48 |
| pbourke | 3 mons, 3 osds. The issue is the mons will not talk to each other | 14:48 |
| sdake | is that missing in the rc3 list? | 14:48 |
| *** HyperJohnGraham_ has quit IRC | 14:49 | |
| pbourke | i haven't logged it because I dont know if its a kolla bug or just my setup | 14:49 |
| pbourke | seems to be ok for others | 14:50 |
| sdake | pbourke i wouldn't ssume that unless you have verbal confirmation | 14:50 |
| sdake | pbourke although AIO works in the kube gate | 14:50 |
| sdake | you are doing multinode | 14:50 |
| pbourke | inc0: Jeffrey4l__: sean-k-mooneyAFK: ceph working ok for you guys? | 14:51 |
| inc0 | pbourke, yeah | 14:51 |
| sdake | multinode? | 14:51 |
| inc0 | always | 14:51 |
| pbourke | i dont know what Im doing wrong :/ | 14:51 |
| inc0 | 3 mons 2 osds | 14:51 |
| *** sean-k-mooneyAFK is now known as sean-k-mooney | 14:51 | |
| sdake | pbourke i noticed ole isn't using jewel some time ago | 14:51 |
| sean-k-mooney | pbourke: i will be setting up a ceph cluster i a while | 14:51 |
| sdake | a couple days back | 14:51 |
| sean-k-mooney | pbourke: are you still haveing the same ceph mon issue i was haveing a few weeks ago where the mons timeout? | 14:53 |
| sdake | pbourke nm i see that problem has been fixed | 14:53 |
| pbourke | sean-k-mooney: yeah | 14:53 |
| *** huikang_ has joined #openstack-kolla | 14:54 | |
| sean-k-mooney | pbourke: ok as i said above i will be deploy ceph multinode soon. im just building the images again. ill see if i hit it again or not | 14:54 |
| pbourke | sean-k-mooney: thanks a mill | 14:54 |
| inc0 | I'll do full upgrade ceph included in a few | 14:55 |
| pbourke | in summary last time I tried the master build it was happening, did not see it in our internal builds. Now Im seeing it there too which leads me to think something is flaky | 14:55 |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: copy iscsid logrotate config https://review.openstack.org/386027 | 14:55 |
| inc0 | pbourke, and osd's can't access mons? | 14:55 |
| sdake | pbourke the last time you tried master perhaps you were using hammer version of ceph... | 14:55 |
| pbourke | sdake: maybe but dont think so. I'll try confirm | 14:56 |
| pbourke | inc0: when the mons dont form a quorum 'ceph status' just hangs | 14:56 |
| *** huikang has quit IRC | 14:56 | |
| pbourke | inc0: playbook doesn't notice this until it tries to start the osds though | 14:56 |
| wznoinsk | inc0: sdake my bad, I missed one line, here's fix https://review.openstack.org/#/c/386027/ , could you look? | 14:57 |
| sdake | 0a50cb88 docker/base/Dockerfile.j2 (Michal (inc0) Jastrzebski 2016-09-06 15:52:24 +0000 129) http://mirror.centos.org/centos-7/7/extras/x86_64/Packages/centos-release-ceph-jewel-1.0-1.el7.centos.noarch.rpm \ | 14:57 |
| sdake | pbourke does that 9/6 date line up with when problems started occuring? | 14:57 |
| inc0 | sdake, yeah...I did ceph upgrade:P | 14:58 |
| sdake | inc0 - was pointing out date not blamee :) | 14:58 |
| pbourke | maybe but it worked initially | 14:58 |
| pbourke | sdake: dont worry it honestly may just be a problem on my end | 14:58 |
| pbourke | what sucks it the lack of useful debugging info from ceph but c'est la vie | 14:59 |
| inc0 | well, or oracle linux for whatever reason | 14:59 |
| *** sdake has quit IRC | 15:00 | |
| *** srwilkers has quit IRC | 15:02 | |
| sean-k-mooney | pbourke: my only concern there is i hit the same issue as you | 15:02 |
| pbourke | sean-k-mooney: i think you said you had 6 mons which would make sense though | 15:03 |
| pbourke | sean-k-mooney: has to be an odd number | 15:03 |
| sean-k-mooney | pbourke: you would think that but it works with 2 and 4 | 15:03 |
| inc0 | well we never confirmed that even number is an issue | 15:03 |
| inc0 | maybe it's just mon scallability problem | 15:04 |
| sean-k-mooney | pbourke: in my case though i stopped haveing the isue when i went to 3 but it could be a timeing ting. eg it take longer to have a quourm with more mons | 15:04 |
| sean-k-mooney | if ceph could not work with an even number of mons the if you lost a single mon everything would break | 15:04 |
| *** sdake has joined #openstack-kolla | 15:05 | |
| *** egonzalez90 has quit IRC | 15:05 | |
| kfox1111 | sean-k-mooney: it works until it doesn't. | 15:05 |
| sean-k-mooney | kfox1111: you tested 4 mons on your setup correct and that worked for you? | 15:05 |
| kfox1111 | 4 will work, so long as only one dies at a time. if it ever goes split brained evenly badness ensues. | 15:06 |
| inc0 | sean-k-mooney, I wouldn't be surprised if ceph builds cluster witn n**2 complexity | 15:06 |
| kfox1111 | so, not a good idea. | 15:06 |
| inc0 | as in every mon talks to every mon | 15:06 |
| kfox1111 | everything must be in sync, | 15:07 |
| inc0 | so scallability can quicky becomes an issue | 15:07 |
| kfox1111 | so there isn't much benifit in not having every mon talk to every other. | 15:07 |
| sean-k-mooney | perhps if we started the mons serially it may resolve the qurom issue? | 15:07 |
| *** zhugx has joined #openstack-kolla | 15:07 | |
| inc0 | yeah I agree, what I'm saying is that is plausable that 6 mons simply is too much | 15:07 |
| sdake | sean-k-mooney we tested 120 mons... | 15:07 |
| kfox1111 | we've run 7. worlked fine. | 15:08 |
| sdake | sean-k-mooney oh nah, that was osds | 15:08 |
| inc0 | 120 osds sdake | 15:08 |
| kfox1111 | it just has to be odd for safety. | 15:08 |
| sean-k-mooney | sdake: i think only 3 mons were used in osic | 15:08 |
| sdake | yes | 15:08 |
| sdake | been up all ngiht sorting out this tagging | 15:08 |
| pbourke | i think bringing them up serially is well worth trying | 15:08 |
| sdake | not at top of game ;) | 15:08 |
| kfox1111 | bootstrapping or restarting? | 15:09 |
| *** jistr has joined #openstack-kolla | 15:09 | |
| kfox1111 | bootstrapping, its a good idea to do them all serial. | 15:09 |
| kfox1111 | restarting should be fine in parallel if you have an odd number. | 15:09 |
| kfox1111 | otherwise you risk splitbraining your cluster. | 15:09 |
| sdake | here is the bug: https://bugs.launchpad.net/kolla/+bug/1629237 | 15:11 |
| openstack | Launchpad bug 1629237 in kolla "Ceph monitors not responding (Error connecting to cluster: TimedOut)" [Critical,Incomplete] - Assigned to Paul Bourke (pauldbourke) | 15:11 |
| kfox1111 | ah. yeah, I'm guessing it is probably splitbraining itself. | 15:12 |
| kfox1111 | serial bootstrapping should help, as is using a proper number of mons. | 15:12 |
| sdake | terrible that software designed with ha built lin doesn't use virtual synchrony | 15:13 |
| sdake | there would be no split brain if true replication were used properly | 15:13 |
| kfox1111 | I haven't seen a ha cluster that handles an even number of nodes well. not really a thing. | 15:13 |
| kfox1111 | and bootstrapping is a special case. | 15:13 |
| sean-k-mooney | one modifciation to ceph i would like to do for ocata is preset the lables so that we can restore them when we do a destroy. it would make debuging ceph a lot simpler. | 15:14 |
| kfox1111 | it happens so infrequently that they probably didn't put much effort into ensuring it properly assembles itself in parallel. | 15:14 |
| sbezverk_ | kfox1111: hey!! any news for me? | 15:14 |
| *** vks100 has joined #openstack-kolla | 15:14 | |
| sdake | kfox1111 they do exist - but require virtual synchrony | 15:14 |
| kfox1111 | sbezverk_: been really swamped :/ had a crazy outage thrust on me. | 15:15 |
| kfox1111 | they forced us to turn on cadf across every service in the ha cloud. :/ | 15:15 |
| *** vhosakot has joined #openstack-kolla | 15:15 | |
| kfox1111 | sdake: yeah but still not seen a virtual synchrony system handle failure well in a non even number of node situation. | 15:16 |
| sdake | note, I asked the reporter of this bug: https://bugs.launchpad.net/kolla/+bug/1617334 | 15:16 |
| openstack | Launchpad bug 1617334 in kolla "reconfigure action fails on [neutron | Restart the neutron_openvswitch_agent container] " [Critical,Incomplete] | 15:16 |
| sdake | to visit us in irc so we could have a debug session of his system | 15:17 |
| sdake | if you come acorss him, pleasse help him :) | 15:17 |
| sdake | so we can either close this bug out as invalid or fixx the problem | 15:17 |
| sbezverk_ | kfox1111: no prob.. | 15:17 |
| sdake | most likely user error | 15:17 |
| *** salv-orl_ has quit IRC | 15:17 | |
| *** ssurana has joined #openstack-kolla | 15:18 | |
| sdake | that whole deploy doesn't pull is a new can of worm we wil lbe dealing with constantly | 15:18 |
| * sdake groans | 15:18 | |
| sdake | still dealing with pbr nonsense | 15:18 |
| sdake | especially since kolla-kubernetes tells folks to pip install kolla.... | 15:18 |
| sdake | confusing :) | 15:18 |
| kfox1111 | hmm... is pip install bad? | 15:19 |
| sdake | kfox1111 yes | 15:20 |
| sdake | kfox1111 for developers | 15:20 |
| kfox1111 | but for ops? | 15:20 |
| sdake | kfox1111 that wnat to build master images - it doesn't work correctly | 15:20 |
| sdake | for ops its the right approach | 15:20 |
| kfox1111 | ah. k. | 15:21 |
| sdake | kfox1111 reference kolla docs | 15:21 |
| *** huikang_ has quit IRC | 15:21 | |
| sdake | there are two workflows | 15:21 |
| *** zhugx has quit IRC | 15:22 | |
| *** huikang has joined #openstack-kolla | 15:22 | |
| *** b_bezak has quit IRC | 15:23 | |
| kfox1111 | ah. so its the git tagging thing? | 15:23 |
| inc0 | god our ceph deployment became better since mitaka | 15:26 |
| openstackgerrit | Merged openstack/kolla: Cleanup container script improvement https://review.openstack.org/385860 | 15:30 |
| openstackgerrit | Merged openstack/kolla: copy ldlinux.c32 to /tftpboot in ironic_pxe https://review.openstack.org/385612 | 15:31 |
| sdake | kfox1111 sort of | 15:32 |
| sdake | kfox1111 if you read the docs it explains the issue in detail | 15:32 |
| *** rmart04 has quit IRC | 15:36 | |
| openstackgerrit | Merged openstack/kolla: Move loc tool into tools directory https://review.openstack.org/379497 | 15:37 |
| *** fguillot has quit IRC | 15:38 | |
| openstackgerrit | Merged openstack/kolla: Update the location of Glance Registry HAProxy checks https://review.openstack.org/378183 | 15:38 |
| *** irtermit- is now known as irtermite | 15:39 | |
| *** tonanhngo has quit IRC | 15:41 | |
| sdake | coolsvap ping re requirements | 15:47 |
| sdake | coolsvap will stable/newton get an automatic update from the global requirements proposal bot? | 15:47 |
| openstackgerrit | caoyuan proposed openstack/kolla: Add watcher checking to port_checks.yml https://review.openstack.org/378233 | 15:47 |
| openstackgerrit | Waldemar Znoinski proposed openstack/kolla: copy iscsid logrotate config https://review.openstack.org/386027 | 15:52 |
| *** papacz has quit IRC | 15:55 | |
| openstackgerrit | Mathias proposed openstack/kolla: Use memcache for keystone token persistence https://review.openstack.org/385586 | 15:56 |
| *** huikang has quit IRC | 16:00 | |
| sdake | inc0 was that problem that was discussed on irc with the inventory file fixed | 16:03 |
| sdake | inc0 or was a bug filed | 16:03 |
| sdake | bjolo i believe you may have raised the issue | 16:04 |
| sbezverk_ | kfox1111: when have you tried to deploy kube last time? | 16:07 |
| sbezverk_ | I see some issues with keystone bootstarpping | 16:07 |
| *** mark-casey1 has quit IRC | 16:08 | |
| *** papacz has joined #openstack-kolla | 16:08 | |
| sdake | can someone take on confirming this bug: https://bugs.launchpad.net/bugs/1632905 | 16:09 |
| openstack | Launchpad bug 1632905 in kolla "heat bootstrap failed" [Critical,Triaged] | 16:09 |
| sdake | i reproduced it back to back last night | 16:09 |
| *** vhosakot has quit IRC | 16:09 | |
| *** vhosakot_ has joined #openstack-kolla | 16:09 | |
| sdake | but i am the reporter, it needs soemone else to reproduce it | 16:09 |
| sdake | to confirm it | 16:10 |
| *** domi007_ has quit IRC | 16:10 | |
| *** papacz has quit IRC | 16:13 | |
| sdake | bjolo you about | 16:13 |
| sdake | bjolo preparing to reproduce this https://bugs.launchpad.net/kolla/+bug/1625648 | 16:14 |
| openstack | Launchpad bug 1625648 in kolla "horizon login fails with TLS enabled" [Critical,Triaged] | 16:14 |
| *** vhosakot has joined #openstack-kolla | 16:14 | |
| *** vhosakot_ has quit IRC | 16:15 | |
| kfox1111 | sbezverk_: the last gate test I did. :) | 16:15 |
| kfox1111 | what are you seeing? | 16:16 |
| *** awebber_ has quit IRC | 16:16 | |
| openstackgerrit | Merged openstack/kolla: Remove do_reconfigure.yml file for all roles https://review.openstack.org/375878 | 16:20 |
| *** steve-noyes has quit IRC | 16:20 | |
| *** rmart04 has joined #openstack-kolla | 16:22 | |
| pbourke | sdake: fyi ceph master for oraclelinux is up to date (jewel) | 16:22 |
| pbourke | sdake: going to try deploy now | 16:22 |
| openstackgerrit | Paul Bourke (pbourke) proposed openstack/kolla: Allow service_checks to run independently of kolla-ansible https://review.openstack.org/381161 | 16:23 |
| sdake | pbourke yes i know - i showed you the git blame for the change | 16:25 |
| *** huikang has joined #openstack-kolla | 16:25 | |
| sdake | which ws 9/9 iirc | 16:25 |
| pbourke | sdake: oh yeah just thought maybe there an issue with ol | 16:25 |
| sdake | or soething like that | 16:25 |
| *** rmart04_ has joined #openstack-kolla | 16:25 | |
| openstackgerrit | Merged openstack/kolla: Fix a few typos in documentation https://review.openstack.org/380266 | 16:25 |
| sdake | so my theory is you were using hammer | 16:25 |
| sdake | an that was working for you | 16:25 |
| sdake | now your using jewel | 16:25 |
| sdake | and that isn't | 16:25 |
| sdake | not sure if that is correct or not | 16:25 |
| pbourke | lets see | 16:27 |
| sdake | wow gliffy just totally changed around their sharing options | 16:27 |
| *** rmart04 has quit IRC | 16:27 | |
| *** rmart04_ is now known as rmart04 | 16:27 | |
| sdake | to share you ahve to buy the top package | 16:27 |
| sdake | whic his 49$ a month | 16:28 |
| sdake | rediculous | 16:28 |
| sdake | too good to be true - probably is | 16:29 |
| * sdake sadfaces | 16:29 | |
| sdake | google just aquire gliffy already | 16:29 |
| *** harlowja_at_home has joined #openstack-kolla | 16:30 | |
| pbourke | ok ceph master came up first go! | 16:31 |
| pbourke | so something wrong with our internal build | 16:31 |
| pbourke | forks ftl | 16:31 |
| wznoinsk | inc0: mind looking again at https://review.openstack.org/#/c/386027/ ? | 16:32 |
| *** tonanhngo has joined #openstack-kolla | 16:32 | |
| *** msimonin has quit IRC | 16:33 | |
| *** ajpower has quit IRC | 16:34 | |
| *** inc0_ has joined #openstack-kolla | 16:38 | |
| openstackgerrit | Merged openstack/kolla: copy iscsid logrotate config https://review.openstack.org/386027 | 16:38 |
| *** inc0 has quit IRC | 16:40 | |
| *** sdake_ has joined #openstack-kolla | 16:42 | |
| openstackgerrit | Merged openstack/kolla: add ceilometer congress to precheck.yml https://review.openstack.org/378150 | 16:43 |
| *** rmart04_ has joined #openstack-kolla | 16:43 | |
| openstackgerrit | Merged openstack/kolla: Readability Improvements to Advanced Config Doc https://review.openstack.org/347102 | 16:43 |
| *** fguillot has joined #openstack-kolla | 16:45 | |
| *** sdake has quit IRC | 16:46 | |
| *** rmart04 has quit IRC | 16:46 | |
| *** rmart04_ is now known as rmart04 | 16:46 | |
| *** huikang has quit IRC | 16:47 | |
| openstackgerrit | Merged openstack/kolla: Updated from global requirements https://review.openstack.org/385127 | 16:49 |
| openstackgerrit | Merged openstack/kolla: Hyperlink is expired https://review.openstack.org/385246 | 16:50 |
| wznoinsk | what are would be chances to push 3 changes (iscsi+ironic) back to newton? so people can actually use ironic with kolla in newton? | 16:51 |
| openstackgerrit | Merged openstack/kolla: Modify QSG to install dependencies needed to build code with pip https://review.openstack.org/384073 | 16:51 |
| *** rmart04 has quit IRC | 16:52 | |
| wznoinsk | I guess if I find enough cores to back up the idea it's virtually possible? | 16:53 |
| pbourke | bjolo: just wondering if in your TLS horizon bug, is it a self signed cert? | 16:54 |
| *** dwalsh_ has quit IRC | 16:56 | |
| *** huikang has joined #openstack-kolla | 16:57 | |
| *** dwalsh has joined #openstack-kolla | 16:58 | |
| openstackgerrit | Merged openstack/kolla: Fix wrong mongodb address variable in ceilomter bootstrap https://review.openstack.org/385533 | 16:59 |
| *** hyakuhei- has quit IRC | 17:00 | |
| *** hyakuhei- has joined #openstack-kolla | 17:00 | |
| *** hyakuhei- has quit IRC | 17:00 | |
| *** hyakuhei- has joined #openstack-kolla | 17:00 | |
| *** hyakuhei- is now known as hyakuhei | 17:00 | |
| sdake_ | well gliffy was nice while it lasted | 17:00 |
| *** sdake_ is now known as sdake | 17:01 | |
| sdake | i just fired off one of two emails to them | 17:01 |
| openstackgerrit | Merged openstack/kolla: Fix template for multiple physical networks https://review.openstack.org/378276 | 17:01 |
| sdake | first = complaint | 17:01 |
| sdake | second = closing account | 17:01 |
| openstackgerrit | Merged openstack/kolla: precheck fails due to unknown cinder_volume_group https://review.openstack.org/385873 | 17:01 |
| *** srwilkers has joined #openstack-kolla | 17:02 | |
| sdake | rbergeron ping | 17:03 |
| *** msimonin has joined #openstack-kolla | 17:03 | |
| *** msimonin has quit IRC | 17:05 | |
| *** msimonin has joined #openstack-kolla | 17:05 | |
| *** ajpower has joined #openstack-kolla | 17:05 | |
| *** dwalsh has quit IRC | 17:06 | |
| *** msimonin has quit IRC | 17:06 | |
| *** dwalsh has joined #openstack-kolla | 17:06 | |
| *** vks100 has quit IRC | 17:06 | |
| openstackgerrit | Martin Matyas proposed openstack/kolla: Make iscsi config.yml consistent with start.yml https://review.openstack.org/381914 | 17:12 |
| *** lamt has quit IRC | 17:18 | |
| *** huikang has quit IRC | 17:18 | |
| *** mgoddard has quit IRC | 17:20 | |
| *** unicell has quit IRC | 17:24 | |
| *** neilus has joined #openstack-kolla | 17:29 | |
| *** huikang has joined #openstack-kolla | 17:33 | |
| *** neilus has quit IRC | 17:34 | |
| *** dwalsh has quit IRC | 17:34 | |
| *** dwalsh has joined #openstack-kolla | 17:36 | |
| openstackgerrit | Mathias proposed openstack/kolla: Set memcache connection limit to 5000 https://review.openstack.org/386105 | 17:39 |
| *** msimonin has joined #openstack-kolla | 17:40 | |
| *** mkoderer has quit IRC | 17:42 | |
| *** dwalsh has quit IRC | 17:44 | |
| *** dwalsh has joined #openstack-kolla | 17:44 | |
| *** unicell has joined #openstack-kolla | 17:47 | |
| *** yingjun has quit IRC | 17:48 | |
| openstackgerrit | Mathias proposed openstack/kolla: Monitor docker with telegraf https://review.openstack.org/386110 | 17:51 |
| *** rhallisey has quit IRC | 17:51 | |
| *** huikang has quit IRC | 17:56 | |
| *** mgoddard has joined #openstack-kolla | 17:59 | |
| *** rhallisey has joined #openstack-kolla | 18:00 | |
| openstackgerrit | Mathias proposed openstack/kolla: Monitor mariadb with telegraf https://review.openstack.org/386119 | 18:02 |
| *** huikang has joined #openstack-kolla | 18:11 | |
| *** huikang has quit IRC | 18:12 | |
| *** huikang has joined #openstack-kolla | 18:12 | |
| mewald | Is performance tuning related stuff wanted in the code? | 18:16 |
| *** lamt has joined #openstack-kolla | 18:17 | |
| mewald | We've made some experience lately regarding mariadb in this context which I would share if wanted | 18:17 |
| *** huikang has quit IRC | 18:17 | |
| *** sdake_ has joined #openstack-kolla | 18:19 | |
| *** harlowja_still_a has joined #openstack-kolla | 18:20 | |
| *** sdake has quit IRC | 18:20 | |
| *** Serlex has quit IRC | 18:21 | |
| inc0_ | mewald, depends on change I guess | 18:23 |
| inc0_ | although I'd rather keep conf changes to minimum | 18:23 |
| *** harlowja_at_home has quit IRC | 18:23 | |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 18:24 |
| mewald | inc0_: I just submitted patch. Let's see what people say about it. All I can say is: we had a lot of api timeouts and that stuff solved it and gave us better insight into maridb performance | 18:24 |
| inc0_ | mewald, will that break upgrade? | 18:25 |
| inc0_ | I mean if mariadb was running without these confs, will it keep running? | 18:25 |
| mewald | inc0_: I don't think I understand you. Mariadb never crashed for us but responded very very slowly so that api requests timed out | 18:26 |
| *** shardy has quit IRC | 18:26 | |
| inc0_ | mewald, what I mean, if someone had running mitaka | 18:27 |
| inc0_ | and add these confs | 18:27 |
| inc0_ | their data is safe right? | 18:27 |
| mewald | I am not a mariadb specialist but I see nothing bad about increasing buffers :) | 18:28 |
| inc0_ | yeah I don't suppose that is the case too | 18:28 |
| inc0_ | just wanted to be sure | 18:28 |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 18:28 |
| mewald | as I said: I am not 100% sure and I dont want to be charged guilty when something breaks :D But I've talked to some mariadb ops who suggested this stuff | 18:28 |
| *** harlowja_still_a has quit IRC | 18:30 | |
| mewald | inc0_: I also noticed that query_cache_size should be set to 0 when running Galera but the default is 1048576 | 18:31 |
| *** dwalsh has quit IRC | 18:31 | |
| SamYaple | mewald: nah the default is disabled when using galera | 18:33 |
| SamYaple | otherwise youd be seeing some deadlocks | 18:33 |
| mewald | SamYaple: Ah does it do that automatically? fine then :) | 18:33 |
| mewald | so that line is not necessary, will remove it from the patch | 18:33 |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 18:35 |
| SamYaple | mewald: its really strange that you want to enable the logging of every query.... | 18:35 |
| SamYaple | or if you dont the last 4 options are not needed | 18:35 |
| SamYaple | i dont think general-log takes params and the fact that you have it there may be enabling it | 18:35 |
| mewald | SamYaple: it have us great insight into what's happening from our dashboards. does it draw a lot of performance you think? | 18:36 |
| mewald | s/have/gave/ | 18:37 |
| SamYaple | mewald: it shouldnt be default is my point | 18:37 |
| SamYaple | its not on default for a reason | 18:37 |
| mewald | ok makes sense, if people want it they can use ini merge to get it in | 18:38 |
| mewald | wannt get rid of performance_schema = on as well? | 18:38 |
| mewald | as I am on it I can do it right away :) | 18:38 |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 18:41 |
| SamYaple | mewald: i posted a reply to PS3 | 18:42 |
| SamYaple | the first 4 options are good, the rest really shouldn't be touched | 18:43 |
| SamYaple | or in the patchset that is | 18:43 |
| SamYaple | the only really questionable one is skip-name-resolve | 18:43 |
| SamYaple | technically you can point to that being a bad environment configuration, but if lots of people have problems with that then maybe it should be the default for kolla | 18:44 |
| SamYaple | i dont think thats the case though | 18:44 |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes: Add workflow for cinder https://review.openstack.org/386144 | 18:44 |
| *** rhallisey has quit IRC | 18:45 | |
| SamYaple | inc0_: mewald: do note, that on AIO type systems that share compute, you are giving 40% of the ram to mariadb with this configuration. so maybe think through if that needs to be a default, or make it more intelgent | 18:46 |
| SamYaple | like 40% or 8GB, which ever is lower | 18:46 |
| *** rhallisey has joined #openstack-kolla | 18:46 | |
| SamYaple | ini can still override it (I have set these settings since the beginning with Kolla if that matters) | 18:47 |
| *** salv-orlando has joined #openstack-kolla | 18:47 | |
| *** michauds has joined #openstack-kolla | 18:49 | |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 18:49 |
| mewald | SamYaple: AIO is indeed something I didn't think about | 18:50 |
| openstackgerrit | Mathias proposed openstack/kolla: Mariadb performance tuning https://review.openstack.org/386130 | 18:57 |
| mewald | SamYaple: Please check it that works for you | 18:57 |
| mewald | if | 18:57 |
| openstackgerrit | Mathias proposed openstack/kolla: Monitor mariadb with telegraf https://review.openstack.org/386119 | 18:58 |
| SamYaple | mewald: reviewed. lgtm | 18:59 |
| *** sdake has joined #openstack-kolla | 19:01 | |
| mewald | cool! | 19:01 |
| *** dwalsh has joined #openstack-kolla | 19:02 | |
| *** sdake_ has quit IRC | 19:04 | |
| *** dwalsh has quit IRC | 19:08 | |
| *** dwalsh has joined #openstack-kolla | 19:09 | |
| openstackgerrit | Mathias proposed openstack/kolla: Use memcache for keystone token persistence https://review.openstack.org/385586 | 19:10 |
| janem | Im getting a timeout for haproxy - waiting for vip .. aio 2.0.2 ubuntu host.. prechecks are fine. | 19:10 |
| janem | deploying, that is.. | 19:11 |
| mewald | janem: which task exactly? can you paste the output somewhere? | 19:11 |
| *** pbourke has quit IRC | 19:11 | |
| *** pbourke has joined #openstack-kolla | 19:11 | |
| janem | mewald: yeah, just waiting for it to timeout again :) | 19:12 |
| janem | but the task is TASK: [haproxy | Waiting for virtual IP to appear] | 19:13 |
| mewald | janem: it's all-in-one, right? Which environment are you using? VirtualBox? | 19:13 |
| *** rhallisey has quit IRC | 19:13 | |
| janem | Ubuntu - Metal | 19:13 |
| janem | yes aio | 19:14 |
| mewald | is the keepalived container running? | 19:14 |
| mewald | in both cases: see any errors? | 19:15 |
| janem | mewald: http://pastebin.com/WRqEAakJ | 19:15 |
| janem | mewald: keepalived is running | 19:15 |
| janem | at least now after failed deploy.. | 19:16 |
| mewald | "ip addr show" shows the VIP? | 19:16 |
| janem | mewald: no, actually not | 19:17 |
| janem | mewald: shouldnt that have been taken care of during the deploy? | 19:18 |
| mewald | yeah when keepalived starts that should be done | 19:19 |
| janem | hmz | 19:19 |
| mewald | check your keepalived config and output when running "docker start -a keepalived" | 19:19 |
| janem | mewald: http://pastebin.com/QvL8MtaU | 19:21 |
| *** ssurana has quit IRC | 19:21 | |
| *** papacz has joined #openstack-kolla | 19:24 | |
| mewald | janem: never seen that before | 19:25 |
| janem | me neighter ..searching.. | 19:25 |
| mewald | me too ^^ | 19:26 |
| mewald | can you give me your keepalived.conf and output if "ip addr show"? | 19:27 |
| janem | mewald: http://pastebin.com/TM7wG3LL | 19:29 |
| inc0_ | janem, do you have other keepalived runinng in same network? | 19:30 |
| janem | inc0_: yes | 19:31 |
| inc0_ | like for example another kolla? | 19:31 |
| inc0_ | that's the issue. | 19:31 |
| inc0_ | you need to change this variable in your globals.yml: | 19:31 |
| inc0_ | https://github.com/openstack/kolla/blob/master/etc/kolla/globals.yml#L91 | 19:31 |
| inc0_ | you need to put a different number there | 19:32 |
| inc0_ | each keepalived cluster has to have unique router id | 19:32 |
| *** papacz has quit IRC | 19:32 | |
| mewald | haha nice inc0_! I would never have thought of this :D | 19:33 |
| inc0_ | mewald, I've seen that one before | 19:33 |
| janem | inc0_: ahh.. makes sense.. | 19:33 |
| janem | inc0_: thanks, trying again now. | 19:33 |
| mliima | guys, I need some reviews here or just +Workflow. https://review.openstack.org/#/c/382054/ | 19:33 |
| inc0_ | here you go mliima | 19:34 |
| janem | inc0_: mewald shabammm! works. Thanks a million. | 19:34 |
| janem | :) | 19:34 |
| inc0_ | glad to hear that | 19:34 |
| mliima | thanks inc0_ \o/ | 19:35 |
| inc0_ | people ask about that from time to time here | 19:36 |
| inc0_ | if you see "bogus packet" something something in keepalived logs | 19:36 |
| inc0_ | that's probably it | 19:36 |
| openstackgerrit | Merged openstack/kolla: Add HNAS as Manila backend https://review.openstack.org/382054 | 19:37 |
| *** narasimha_SV_ has joined #openstack-kolla | 19:40 | |
| narasimha_SV_ | http://paste.openstack.org/show/585650/ anyone working on ironic in kolla is getting this issue ? | 19:40 |
| janem | inc0_: hopefully I will be able to help the next guy/gal ;) | 19:43 |
| inc0_ | janem, that's how it works, one time you need help another time you help somebody;) | 19:44 |
| janem | inc0_: yup! | 19:45 |
| *** goldyfruit has joined #openstack-kolla | 19:48 | |
| goldyfruit | haplo37 https://blueprints.launchpad.net/kolla/+spec/heka-deprecation | 19:50 |
| *** jtriley has quit IRC | 19:57 | |
| goldyfruit | yop EmilienM | 20:00 |
| goldyfruit | <- Gaetan | 20:00 |
| EmilienM | goldyfruit: what's up? | 20:00 |
| goldyfruit | EmilienM: Start ny new job at Omruco | 20:01 |
| goldyfruit | Ormuco | 20:01 |
| goldyfruit | what about you ? | 20:01 |
| *** janem has quit IRC | 20:02 | |
| bjolo | pbourke, about the TLS cert. i have tried both selfsigned and a real one. same result | 20:13 |
| *** ppalacios has quit IRC | 20:15 | |
| *** rhallisey has joined #openstack-kolla | 20:16 | |
| *** openstackgerrit has quit IRC | 20:19 | |
| *** openstackgerrit has joined #openstack-kolla | 20:19 | |
| *** unicell has quit IRC | 20:20 | |
| *** peterjenkins_ has quit IRC | 20:20 | |
| *** peterjenkins_ has joined #openstack-kolla | 20:23 | |
| *** unicell has joined #openstack-kolla | 20:23 | |
| *** unicell1 has joined #openstack-kolla | 20:24 | |
| *** unicell has quit IRC | 20:24 | |
| *** janem has joined #openstack-kolla | 20:32 | |
| *** salv-orlando has quit IRC | 20:36 | |
| *** narasimha_SV_ has quit IRC | 20:40 | |
| *** mliima has quit IRC | 20:41 | |
| *** mark-casey1 has joined #openstack-kolla | 20:41 | |
| *** sdake has quit IRC | 20:44 | |
| *** ssurana has joined #openstack-kolla | 20:47 | |
| *** salv-orlando has joined #openstack-kolla | 20:48 | |
| *** gfidente has quit IRC | 20:51 | |
| *** huikang has joined #openstack-kolla | 20:52 | |
| *** mewald has quit IRC | 20:56 | |
| *** sdake has joined #openstack-kolla | 20:59 | |
| *** mewald has joined #openstack-kolla | 20:59 | |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes: Ansible workflow for Neutron https://review.openstack.org/386226 | 21:00 |
| *** janem has quit IRC | 21:01 | |
| *** ayoung has quit IRC | 21:02 | |
| *** srwilkers has left #openstack-kolla | 21:02 | |
| *** dave-mccowan has quit IRC | 21:04 | |
| *** dwalsh has quit IRC | 21:08 | |
| *** steve-noyes has joined #openstack-kolla | 21:08 | |
| *** papacz has joined #openstack-kolla | 21:13 | |
| *** absubram has quit IRC | 21:18 | |
| *** steve-noyes has quit IRC | 21:23 | |
| *** fguillot has quit IRC | 21:29 | |
| *** michauds has quit IRC | 21:30 | |
| *** jheroux has quit IRC | 21:34 | |
| *** Jeffrey4l__ has quit IRC | 21:35 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 21:35 | |
| *** huikang has quit IRC | 21:38 | |
| *** huikang has joined #openstack-kolla | 21:38 | |
| *** Pavo has quit IRC | 21:40 | |
| *** JoseMello has quit IRC | 21:41 | |
| *** huikang has quit IRC | 21:42 | |
| *** Pavo has joined #openstack-kolla | 21:44 | |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes: Add workflow for cinder https://review.openstack.org/386144 | 21:46 |
| *** Pavo has quit IRC | 21:51 | |
| *** Pavo has joined #openstack-kolla | 21:56 | |
| *** ssurana has quit IRC | 21:57 | |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes: Ansible workflow for Neutron https://review.openstack.org/386226 | 22:01 |
| *** tonanhngo has quit IRC | 22:04 | |
| *** papacz has quit IRC | 22:08 | |
| *** tonanhngo has joined #openstack-kolla | 22:09 | |
| *** huikang has joined #openstack-kolla | 22:12 | |
| *** tonanhngo has quit IRC | 22:14 | |
| *** tonanhngo has joined #openstack-kolla | 22:15 | |
| *** huikang has quit IRC | 22:17 | |
| *** mgoddard has quit IRC | 22:18 | |
| *** inc0_ has quit IRC | 22:19 | |
| *** tonanhngo has quit IRC | 22:19 | |
| *** Pavo has quit IRC | 22:22 | |
| *** rhallisey has quit IRC | 22:24 | |
| *** ayoung has joined #openstack-kolla | 22:25 | |
| *** ssurana has joined #openstack-kolla | 22:26 | |
| *** Pavo has joined #openstack-kolla | 22:28 | |
| *** dwalsh has joined #openstack-kolla | 22:29 | |
| *** salv-orlando has quit IRC | 22:30 | |
| *** ntpttr has quit IRC | 22:31 | |
| *** ntpttr has joined #openstack-kolla | 22:31 | |
| *** Pavo has quit IRC | 22:33 | |
| *** mewald has left #openstack-kolla | 22:34 | |
| *** quasar has quit IRC | 22:35 | |
| *** msimonin has quit IRC | 22:40 | |
| *** Pavo has joined #openstack-kolla | 22:41 | |
| *** quasar has joined #openstack-kolla | 22:42 | |
| jascott1 | https://blog.buoyant.io/2016/10/04/a-service-mesh-for-kubernetes-part-i-top-line-service-metrics/ | 22:46 |
| *** tonanhngo has joined #openstack-kolla | 22:59 | |
| *** dwalsh has quit IRC | 23:00 | |
| kfox1111 | jascott1: https://xkcd.com/927/ | 23:00 |
| *** lamt has quit IRC | 23:00 | |
| kfox1111 | :) | 23:00 |
| kfox1111 | jascott1: looks interesting. but also seems to have a lot of overlap with what k8s provides itself. | 23:02 |
| jascott1 | haha thats a great one on the short list. yeah idk either but seems to be generating a lot of buzz today Hightower et all | 23:03 |
| *** ayoung has quit IRC | 23:03 | |
| kfox1111 | the gui bits look sweet. | 23:03 |
| jascott1 | the premise is interesting, abstracting the retries and 'communication' management | 23:04 |
| kfox1111 | yeah. | 23:04 |
| jascott1 | also impressive if you really can just run 3 apply commands and get all that goodness | 23:04 |
| kfox1111 | that may speak more to the power of k8s then linkd... but, yeah. :) | 23:05 |
| kfox1111 | linkerd I mean. | 23:05 |
| kfox1111 | the argument feels much like the argument that has been made around most middleware. leave it to us, so you don't have to figure out the hard bits. | 23:06 |
| kfox1111 | which is a good arguement. | 23:06 |
| jascott1 | :) | 23:07 |
| kfox1111 | but has spawned thousands of 'standards' :) | 23:07 |
| jascott1 | yeah I havent even learned all the existing network overlay stuff let alone abstracting pieces of it away! | 23:08 |
| * kfox1111 nods | 23:08 | |
| kfox1111 | I think its going to change quite a bit too once k8s gets multi l2 network support, ala neutron. | 23:09 |
| jascott1 | cool | 23:09 |
| *** Pavo has quit IRC | 23:13 | |
| *** asalkeld has joined #openstack-kolla | 23:13 | |
| *** Pavo has joined #openstack-kolla | 23:14 | |
| *** igordcard has quit IRC | 23:14 | |
| *** igordcard has joined #openstack-kolla | 23:16 | |
| *** awiddersheim has joined #openstack-kolla | 23:18 | |
| *** mtaylor22 has joined #openstack-kolla | 23:19 | |
| mtaylor22 | morning. | 23:19 |
| jascott1 | o/ | 23:19 |
| awiddersheim | having some trouble with RC2 | 23:21 |
| awiddersheim | trying to do a new setup and things fail on keepalived and waiting for the VIP | 23:21 |
| *** v1k0d3n has joined #openstack-kolla | 23:22 | |
| jascott1 | awiddersheim I have the same problem but i dont fully know what I am doing | 23:22 |
| awiddersheim | but the containers aren't making any logs for me to trouble shoot | 23:22 |
| jascott1 | as I understand it, its waiting for the mariadb ip and port to become available | 23:23 |
| jascott1 | I havent figured out how you know what that host is tho | 23:23 |
| awiddersheim | yeah | 23:23 |
| v1k0d3n | hey sdake who was it that we were on a webex a couple of weeks ago that was helping with multi-kube install of kolla-kube? i can't remember the name. | 23:24 |
| awiddersheim | doesn't even seem like it made it far enough to stand that container up | 23:24 |
| *** vhosakot has quit IRC | 23:24 | |
| *** mtaylor22 has quit IRC | 23:27 | |
| *** frzn_ has joined #openstack-kolla | 23:27 | |
| *** frzn_ is now known as mtaylor22 | 23:27 | |
| kfox1111 | v1k0d3n: might have been sbezverk_ or me. | 23:27 |
| *** Pavo has quit IRC | 23:28 | |
| *** Pavo has joined #openstack-kolla | 23:28 | |
| *** srwilkers_ has joined #openstack-kolla | 23:28 | |
| *** lamt has joined #openstack-kolla | 23:28 | |
| *** srwilkers_ has left #openstack-kolla | 23:29 | |
| v1k0d3n | so wanted to get your thoughts on using this repo as a base for developing multi-node kube-kolla... | 23:30 |
| v1k0d3n | https://github.com/att-comdev/halcyon-kubernetes | 23:30 |
| v1k0d3n | kfox1111: it was you :) | 23:30 |
| v1k0d3n | now i remember! lol | 23:30 |
| v1k0d3n | i reached out to sebgoa about merging efforts with his repo too...https://github.com/skippbox/kubeadm-centos | 23:31 |
| *** neilus has joined #openstack-kolla | 23:32 | |
| *** neilus has quit IRC | 23:36 | |
| v1k0d3n | kfox1111: thoughts...good/bad? something that could be useful for kolla-kube perhaps? | 23:37 |
| kfox1111 | looking... | 23:37 |
| kfox1111 | docs talk a lot about vagrent. is that just for testing? | 23:39 |
| kfox1111 | seems to only support ubuntu at the moment, and for some reason installs docker-compose | 23:40 |
| kfox1111 | in general though, looks quite promicing. | 23:40 |
| kfox1111 | the docs make it seem like they really intend it for vagrent setup. | 23:42 |
| v1k0d3n | so here's my actual goal with it. | 23:42 |
| v1k0d3n | break away from vagrant. vagrant will be on it's own...and submodule for the playbooks. | 23:43 |
| * kfox1111 nods | 23:43 | |
| kfox1111 | shouldn't be too hard by the look of it. | 23:43 |
| kfox1111 | though my ansible's pretty green at this point, so take that with a grain of salt. :) | 23:43 |
| v1k0d3n | so they can be maintained separately. however, vagrant is multi-provider. so with the config.rb...you can use the --provider=openstack and deploy directly to openstack for development. | 23:44 |
| v1k0d3n | i will be adding aws either tomorrow or early next week, to deploy directly to aws with some config options. | 23:44 |
| kfox1111 | yeah... Ive tried vagrent pariodically against kvm... never really got it to work well. :/ | 23:44 |
| *** ayoung has joined #openstack-kolla | 23:44 | |
| kfox1111 | kind of gave up on vagrent for a while. | 23:44 |
| v1k0d3n | kfox1111: no worries...it just works out of the box (or should anyway). | 23:45 |
| v1k0d3n | interested if you have issues. | 23:45 |
| kfox1111 | cool. I might have to give it another shot then. :) | 23:45 |
| v1k0d3n | if you do, let me know...i'll correct them. | 23:45 |
| kfox1111 | I just REALLY didn't want to have to install virtualbox. | 23:45 |
| v1k0d3n | i think it would have a good place to start for kolla-kube multi-node development. | 23:46 |
| v1k0d3n | lol | 23:46 |
| v1k0d3n | yeah, i can understand that. but with vbox extensions...it just works so well. | 23:46 |
| v1k0d3n | ^^ hating that i like any product from...*coughs* | 23:46 |
| kfox1111 | I talked with the infra folks and they can't do vm creation in vm without doing non accelerated. | 23:46 |
| kfox1111 | v1k0d3n: until openstack has a virtualbox driver, I can't run openstack and virtualbox on the same laptop. | 23:47 |
| kfox1111 | so multi-node deployment with vagrent won't work in the gate. | 23:47 |
| kfox1111 | but I did get some info out of them how to get two vm's launched at the same time and get ssh between them. | 23:47 |
| kfox1111 | so that should be enough to just ssh in and run the kubeadm join. | 23:48 |
| v1k0d3n | what is your setup again? just curious. | 23:48 |
| kfox1111 | the rest should deploy with daemonsets, so should just work I think. | 23:48 |
| v1k0d3n | yeah daemonsets is good stuff. | 23:49 |
| kfox1111 | for what? my personal dev system, what I t est with, production system, etc? | 23:49 |
| *** sdake_ has joined #openstack-kolla | 23:51 | |
| v1k0d3n | if you'd use something like this i mean. | 23:52 |
| v1k0d3n | i mean, we could just use the playbooks for setting up the kubeadm portion on existing systems. pretty straight forward. | 23:52 |
| *** sdake has quit IRC | 23:52 | |
| kfox1111 | yeah. I think that would be useful. | 23:53 |
| v1k0d3n | or terraform could be used, vagrant can be used....whatever. i do most of my development in openstack projects... | 23:53 |
| v1k0d3n | so i use this vagrant up in an openstack project. but then there are times i need to do local development, local hypervisor. so i use vbox. | 23:53 |
| kfox1111 | my goal right now is to get the gate up and going to the point, where that's the easiest way to test. just submit a review, and it will be tested. :) | 23:53 |
| v1k0d3n | but playbooks at least stay the same...portable. | 23:53 |
| * kfox1111 nods | 23:54 | |
| v1k0d3n | gotcha...makes sense. | 23:54 |
| kfox1111 | kola-kubernetes doesn't have any k8s deployment stuff. not sure it really makes sense to include it directly, as there are so many ways. | 23:54 |
| kfox1111 | but pointing at an ansible based deployment tool sounds good to me. | 23:55 |
| v1k0d3n | btw, i already reached out to sebastian to merge these https://github.com/skippbox/kubeadm-centos | 23:55 |
| kfox1111 | "If you don't aready have a k8s and want something quick that will just work, go here" | 23:55 |
| v1k0d3n | so it would cover both ubuntu and centos deployments of kubeadm | 23:55 |
| v1k0d3n | ^^ exactly...that's the scenario i'm talking about. | 23:56 |
| kfox1111 | yeah. that would be cool. | 23:56 |
| *** cj has quit IRC | 23:56 | |
| v1k0d3n | for instance...my guys wanted to start contributing to kolla-kube...but they were new to everything. so they needed to find a way to deploy kubernetes. | 23:56 |
| kfox1111 | right. :) | 23:57 |
| v1k0d3n | minikube is great...but not the same issues as a multi-node. | 23:57 |
| kfox1111 | yeah. totally. | 23:57 |
| kfox1111 | to be fair, most of that was the focus only on single node for now. | 23:57 |
| kfox1111 | :) | 23:57 |
| kfox1111 | that is changing, as single node is now working pretty well, and has gate checks now. | 23:57 |
| kfox1111 | so I'm moving on to multinode next. | 23:57 |
| kfox1111 | and others are working on the ansible worflow so that should get much easier too. | 23:58 |
| kfox1111 | the multinode stuff really hinges on the ceph stuff getting ironed out, and that wasn't working until a few days ago. | 23:58 |
| kfox1111 | that code merged though, so shouldn't regress. | 23:58 |
| kfox1111 | it just needs a bit more polish and hooking up to the workflow now. | 23:59 |
| kfox1111 | but mariadb backed by ceph, and laungching a vm with image stored in ceph and cinder volume attachment/movement where the volume is in ceph all work. | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
