| jemcevoy | Not that I am aware of... | 00:00 |
|---|---|---|
| inc0 | what is in this giant file? | 00:01 |
| jemcevoy | It will kill prod soon... I am seeing errors in error in kibana from crond? in kibana lots of them | 00:01 |
| *** sp_ has quit IRC | 00:03 | |
| *** Pavo has quit IRC | 00:04 | |
| *** sp_ has joined #openstack-kolla | 00:04 | |
| jemcevoy | inco this is a tail of the file: http://paste.openstack.org/show/601848/ | 00:04 |
| jemcevoy | inc0 ^ | 00:04 |
| *** rhallisey has quit IRC | 00:06 | |
| inc0 | see this /var/log/kolla/...dnsmasq permission denied? | 00:06 |
| inc0 | that's your error right there | 00:06 |
| *** rhallisey has joined #openstack-kolla | 00:07 | |
| *** tonanhngo has joined #openstack-kolla | 00:08 | |
| *** mattmceuen has quit IRC | 00:09 | |
| jemcevoy | [root@cc1-e _data]# ls -al /var/lib/docker/volumes/kolla_logs/_data/neutron/dnsmasq.log | 00:09 |
| jemcevoy | -rw-r----- 1 polkitd polkitd 306643 Mar 7 15:17 /var/lib/docker/volumes/kolla_logs/_data/neutron/dnsmasq.log | 00:09 |
| inc0 | I'm pretty sure polkitd is not service we deploy) | 00:10 |
| *** sp_ has quit IRC | 00:10 | |
| kfox1111 | is that on the host? | 00:10 |
| kfox1111 | uid's on the host don't nessisarily match whats in the container. | 00:10 |
| inc0 | ^ good point | 00:11 |
| jemcevoy | inc0 So I need to give the logs world read? | 00:11 |
| inc0 | exec into neutron container and check permissions within | 00:11 |
| *** sps_ has quit IRC | 00:12 | |
| inc0 | it's strange | 00:12 |
| *** rhallisey has quit IRC | 00:12 | |
| jemcevoy | Which neutron container? | 00:12 |
| jemcevoy | This is on CentOS 7.3 | 00:12 |
| inc0 | uhh can't remmeber name right now | 00:12 |
| inc0 | check l3 agent | 00:12 |
| inc0 | I think this runs dnsmasq | 00:12 |
| *** tonanhngo has quit IRC | 00:13 | |
| jemcevoy | In the neutron_dhcp_agent -rw-r----- 1 neutron neutron 306643 Mar 7 15:17 dnsmasq.log | 00:15 |
| inc0 | and which user runs dnsmasq in your container? | 00:16 |
| jemcevoy | (neutron-l3-agent)[neutron@cc1-e /]$ ls -al /var/log/kolla/neutron/dnsmasq.log | 00:16 |
| jemcevoy | -rw-r----- 1 neutron neutron 306643 Mar 7 15:17 /var/log/kolla/neutron/dnsmasq.log | 00:16 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 00:21 |
| jemcevoy | Can I just echo /dev/null > to the giant json log? After I fix permission on all the logs to all heka to read? But when they rotate will the same issue return? inc0 kfox1111 | 00:23 |
| inc0 | jemcevoy: you can do it right now | 00:23 |
| inc0 | it's buffer overflow for heka rally | 00:23 |
| jemcevoy | Does heka need to be added to a bunch of groups | 00:23 |
| inc0 | hmm, possibly | 00:24 |
| inc0 | I wonder why that's happening...it used to work | 00:24 |
| inc0 | jemcevoy: please submit a bug for it tho | 00:24 |
| inc0 | I have nasty feeling about it | 00:24 |
| jemcevoy | Or should there be an inheritable fattr on the kolla_logs directory that will give heka permissions? | 00:25 |
| jemcevoy | Where do I log the bug? Have not done an openstack bug yet? | 00:26 |
| inc0 | https://bugs.launchpad.net/kolla-ansible/ <- log it here please | 00:26 |
| *** duonghq has joined #openstack-kolla | 00:27 | |
| duonghq | morning guys | 00:27 |
| jemcevoy | I'll play around the fattr's tomorrow and see if I can come up with permissions that work with default fattrs. inc0 | 00:28 |
| inc0 | jemcevoy: that works | 00:28 |
| inc0 | good mornin duonghq | 00:28 |
| duonghq | hi inc0 | 00:30 |
| *** salv-orlando has quit IRC | 00:31 | |
| *** tovin07 has joined #openstack-kolla | 00:33 | |
| *** spsurya has joined #openstack-kolla | 00:33 | |
| *** tovin07_ has joined #openstack-kolla | 00:34 | |
| *** saneax is now known as saneax-_-|AFK | 00:40 | |
| spsurya | morning all | 00:41 |
| *** tonanhngo has joined #openstack-kolla | 00:45 | |
| *** erhudy has quit IRC | 00:50 | |
| *** tonanhngo has quit IRC | 00:50 | |
| sdake | sup inc0 - i am around for about 10 minutes | 00:53 |
| inc0 | nvm, I thought I figured out your docker 17.03 registry issue | 00:53 |
| inc0 | but that was me | 00:53 |
| inc0 | it overriden --insecure-registry config I had | 00:53 |
| sdake | this system was never setup with --insecure-registry flag i think | 00:54 |
| inc0 | but since you're here https://review.openstack.org/#/c/442850/ | 00:54 |
| sdake | although i can't check it atm | 00:54 |
| inc0 | we can't releease without it as far as I'm concerned | 00:54 |
| sdake | i can check the review - i mean i can't check my config | 00:55 |
| *** Pavo has joined #openstack-kolla | 00:55 | |
| sdake | did anyone do the binary release of centos for newton? | 00:55 |
| sdake | rather ocata | 00:55 |
| sdake | and ubuntu | 00:55 |
| sdake | inc0 did you test upgrades with that patch? | 00:56 |
| *** cuongnv has joined #openstack-kolla | 00:56 | |
| inc0 | I'm testing upgrades as we speak, and we might have issue | 00:56 |
| sdake | the gate semi-tests upgrade but not that scenario | 00:56 |
| inc0 | but this patch shouldn't break upgrades | 00:56 |
| *** Manheim has quit IRC | 00:57 | |
| *** Mannheim has quit IRC | 00:57 | |
| inc0 | although I'm inclined to run alternative too - pin do libvirt 1.3 | 00:57 |
| sdake | release team wont wait for the tag, and that patch needs to be backported | 00:57 |
| inc0 | on both centos and ubuntu tbh | 00:58 |
| sdake | any other core reviewers alive? | 00:58 |
| sdake | nah libvirt 1.3 is terribad | 00:58 |
| sdake | i am pretty sure it wont work on centos | 00:58 |
| sdake | it may work on ubuntu | 00:58 |
| inc0 | that's what nova gates runs on | 00:58 |
| sdake | we went to libvirt 2.0 for a reason ;-) | 00:58 |
| inc0 | which means we run on ungated version | 00:58 |
| sdake | the reason was to run against the virt sig's work in centos | 00:59 |
| sdake | not against outdated packages in epel | 00:59 |
| sdake | we should completely remove epel as a dep in our containers | 00:59 |
| sdake | asap | 00:59 |
| *** tovin07_ has quit IRC | 00:59 | |
| sdake | dmsimard had mentioned this ages ago, it has yet to be done because of the repo split chaos ;-) | 00:59 |
| masber | hi | 00:59 |
| *** ssurana has joined #openstack-kolla | 01:01 | |
| spsurya | Jeffrey4l_: duonghq sdake inc0 portdirect SamYaple sayantani pbourke berendt sbezverk . I have chaged my irc nick to avoid conflict with another member who is also register with nick sp prefix. So, For better communication and understanding I am changing this. Problem for this was appendation of more "_" on my nick looks wierd. Hope this won't be a problem | 01:01 |
| spsurya | in communication in meeting. | 01:01 |
| inc0 | spsurya: I figured, thanks for heads up;) | 01:01 |
| *** salv-orlando has joined #openstack-kolla | 01:01 | |
| *** manheim has joined #openstack-kolla | 01:02 | |
| duonghq | spsurya, roger | 01:03 |
| sdake | spsurya works for me | 01:04 |
| sdake | spsurya reigster your nick this time plz :) | 01:04 |
| spsurya | sdake: done | 01:04 |
| sdake | spsurya the eact problem the individual complained about is why I have sdake, sdake_, and sdake__ reigstered ;) | 01:05 |
| *** duonghq is now known as duonghq_ | 01:05 | |
| *** duonghq_ is now known as duonghq__ | 01:05 | |
| *** duonghq__ is now known as duonghq | 01:06 | |
| duonghq | ya, I have some nicks, | 01:06 |
| *** salv-orlando has quit IRC | 01:06 | |
| *** Pavo has quit IRC | 01:07 | |
| *** duonghq is now known as duonghq_afk | 01:08 | |
| *** portdirect is now known as sdake______ | 01:09 | |
| sdake______ | hey :P | 01:09 |
| *** sdake______ is now known as portdirect | 01:09 | |
| inc0 | really | 01:09 |
| inc0 | ? | 01:09 |
| inc0 | btw portdirect go to bed! | 01:10 |
| portdirect | you're not my mum! | 01:10 |
| inc0 | right...well..go drink alcohol, have a life | 01:10 |
| inc0 | not sit on irc with nerds without any social connections whatsoever | 01:11 |
| * inc0 quietly cries under the desk | 01:11 | |
| *** zhubingbing_ has joined #openstack-kolla | 01:11 | |
| portdirect | lol - need to get irccloud off the phone thats true | 01:12 |
| spsurya | sdake: better we should register through IRCcloud on any browser which will remain open, if somebody want that his/her nick would not get appended by "_". Provided that person should not be joined from another s/w or freenode anywhere. | 01:12 |
| *** jrich523 has quit IRC | 01:12 | |
| inc0 | I have something like that...just not irccloud...much much more nerdy | 01:12 |
| inc0 | ssh -> tmux -> weechat | 01:12 |
| inc0 | ncurses for life | 01:12 |
| spsurya | inc0: roger :) | 01:13 |
| spsurya | nice | 01:15 |
| *** zhurong has joined #openstack-kolla | 01:15 | |
| *** rhallisey has joined #openstack-kolla | 01:16 | |
| duonghq_afk | sup rhallisey | 01:16 |
| *** duonghq_afk is now known as duonghq | 01:16 | |
| kfox1111 | bleh. its amazing how much havoc a ups can cause. :/ | 01:16 |
| duonghq | kfox1111, ? | 01:17 |
| *** jrobinson has quit IRC | 01:18 | |
| *** spsurya_ has joined #openstack-kolla | 01:18 | |
| kfox1111 | lost a datacenter level ups a few weeks ago. still dealing with fallout. :/ | 01:18 |
| inc0 | uhh | 01:18 |
| *** jrobinson has joined #openstack-kolla | 01:18 | |
| duonghq | I think that UPS should be redundant? | 01:18 |
| kfox1111 | heh. you'd like to think... | 01:19 |
| kfox1111 | lets just say, they didn't ask me. | 01:19 |
| inc0 | datacenter-grade ups is a room-sized thing | 01:19 |
| kfox1111 | yeah. | 01:19 |
| *** spsurya_ has quit IRC | 01:19 | |
| inc0 | and super expensive | 01:19 |
| duonghq | I mean the solution has redundant feature? | 01:19 |
| inc0 | so yeah...not always redundant | 01:19 |
| kfox1111 | this is a row of racks. but yeah.... | 01:20 |
| kfox1111 | should never have gone out the way it did. | 01:20 |
| duonghq | what's wrong with the ups? or it just went out? | 01:20 |
| kfox1111 | I'm not privy to all the details. | 01:20 |
| kfox1111 | there was arcing involved... and a loud boom.... | 01:21 |
| duonghq | ya | 01:21 |
| kfox1111 | and then weeks of dealing with fallout. :/ | 01:21 |
| duonghq | not so good | 01:23 |
| * kfox1111 nods | 01:24 | |
| inc0 | upgrade failed because of cells, joys of life | 01:26 |
| inc0 | cya around guys, be back tomorrow | 01:27 |
| kfox1111 | l8r | 01:27 |
| duonghq | inc0, cya | 01:27 |
| *** crushil has joined #openstack-kolla | 01:28 | |
| *** eaguilar has quit IRC | 01:29 | |
| masber | hi | 01:30 |
| masber | i would like to install kolla | 01:30 |
| masber | I am following this tutorial --> https://docs.openstack.org/developer/kolla-kubernetes/quickstart.html | 01:31 |
| masber | but the installation is failing | 01:31 |
| masber | http://pastebin.com/raw/gkuZ57bD | 01:31 |
| duonghq | masber, seem that you have not install gcc or g++ yet | 01:33 |
| duonghq | it's need for compile some Python ext | 01:33 |
| masber | ok, no problem, I am using a fresh new centos7 installation | 01:33 |
| masber | also which doc is better the one I am following or this one? https://docs.openstack.org/developer/kolla-kubernetes/host-setup.html | 01:34 |
| masber | this is for testing | 01:34 |
| masber | I have a whole server for this | 01:34 |
| masber | I know kubernetes works better on a cluster with at least 3 nodes but I just want to test | 01:35 |
| *** markmcclain has quit IRC | 01:37 | |
| *** MasterOfBugs has quit IRC | 01:38 | |
| *** pramodrj07 has quit IRC | 01:39 | |
| *** manheim has quit IRC | 01:43 | |
| *** tonanhngo has joined #openstack-kolla | 01:49 | |
| Jeffrey4l_ | inc0, any progress on ubuntu gate red? | 01:49 |
| Jeffrey4l_ | sbezverk, re [scheduler], yep. | 01:51 |
| sbezverk | Jeffrey4l_: thanks | 01:51 |
| sbezverk | Jeffrey4l_: see some wierdmess with ironic | 01:51 |
| Jeffrey4l_ | is it solved? | 01:52 |
| sbezverk | when nova-compute-ironic sees resource of a baremetal, but nova does not see resurces available.. | 01:52 |
| Jeffrey4l_ | spsurya, roger. | 01:52 |
| sbezverk | still trying to debug.. | 01:52 |
| Jeffrey4l_ | sbezverk, try create a ironic node. | 01:53 |
| *** tonanhngo has quit IRC | 01:53 | |
| sbezverk | Jeffrey4l_: yeah the node is created and all property set | 01:55 |
| sbezverk | ironic sees it as available for deployment | 01:55 |
| *** Jeffrey4l_ has left #openstack-kolla | 01:55 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 01:55 | |
| sbezverk | the issue is nova does not see resources offered by this baremetal node.. | 01:55 |
| Jeffrey4l_ | sbezverk, check "nova hypervisort-stats", did u see it? and is the ironic node active? | 01:56 |
| sbezverk | Jeffrey4l_: will add it, but I think it should only nova-compute node stats and not ironic one | 02:02 |
| *** salv-orlando has joined #openstack-kolla | 02:02 | |
| Jeffrey4l_ | sbezverk, no. for ironic case, the resource come from ironic node's cpu/ram/disk. | 02:03 |
| Jeffrey4l_ | if there is no ironic node, then nova hypervisort-stat will be empty. | 02:03 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 02:03 |
| *** ssurana has quit IRC | 02:04 | |
| *** salv-orlando has quit IRC | 02:07 | |
| *** rhallisey has quit IRC | 02:12 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: [NOMERGE]Debug Ubuntu Gate issue https://review.openstack.org/442893 | 02:12 |
| *** tonanhngo has joined #openstack-kolla | 02:19 | |
| *** tonanhngo has quit IRC | 02:24 | |
| *** erlon has joined #openstack-kolla | 02:27 | |
| *** caowei has joined #openstack-kolla | 02:31 | |
| *** bpattewar has joined #openstack-kolla | 02:33 | |
| *** huzhengchuan has joined #openstack-kolla | 02:37 | |
| *** haplo37 has quit IRC | 02:54 | |
| *** g3ek has quit IRC | 02:54 | |
| *** daidv has joined #openstack-kolla | 02:57 | |
| *** g3ek has joined #openstack-kolla | 03:01 | |
| *** haplo37 has joined #openstack-kolla | 03:02 | |
| *** salv-orlando has joined #openstack-kolla | 03:03 | |
| *** tovin07_ has joined #openstack-kolla | 03:03 | |
| *** rwellum_ has joined #openstack-kolla | 03:04 | |
| *** salv-orlando has quit IRC | 03:08 | |
| *** goldyfruit has joined #openstack-kolla | 03:11 | |
| *** goldyfruit has joined #openstack-kolla | 03:12 | |
| *** Guest88863 has joined #openstack-kolla | 03:12 | |
| *** Guest88863 has quit IRC | 03:18 | |
| masber | duonghq, I installed gcc on my server | 03:20 |
| masber | but still fails --> http://pastebin.com/raw/vNVSMvzA | 03:20 |
| masber | gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) | 03:21 |
| openstackgerrit | Steve Baker proposed openstack/kolla master: Replace docker-py with direct docker calls https://review.openstack.org/442923 | 03:22 |
| duonghq | masber, it's python-dev or devel package | 03:23 |
| duonghq | you can google this | 03:23 |
| *** oomichi has quit IRC | 03:30 | |
| *** oomichi has joined #openstack-kolla | 03:33 | |
| *** jrobinson has quit IRC | 03:38 | |
| masber | thank you | 03:47 |
| masber | sorry but ansible is failing now --> http://pastebin.com/raw/MqaMsE5r | 03:48 |
| duonghq | masber, did you setup the network interfaces in globals.yml | 03:49 |
| masber | api_interface_address? | 03:49 |
| duonghq | masber, you should refer to kolla-ansible doc for basic configuration setup | 03:49 |
| duonghq | and then apply the kolla-k8s one | 03:50 |
| *** goldyfruit has quit IRC | 03:52 | |
| *** jrobinson has joined #openstack-kolla | 03:54 | |
| masber | ok, can you help me to get kolla-kubernetes installed and as a gratitude i can update the documentation adding those details for people like me who has limited knowledge? | 03:56 |
| *** haplo37 has quit IRC | 03:58 | |
| *** g3ek has quit IRC | 03:59 | |
| *** haplo37 has joined #openstack-kolla | 03:59 | |
| *** zhurong has quit IRC | 04:01 | |
| *** salv-orlando has joined #openstack-kolla | 04:04 | |
| *** g3ek has joined #openstack-kolla | 04:05 | |
| *** mdnadeem has joined #openstack-kolla | 04:09 | |
| *** salv-orlando has quit IRC | 04:09 | |
| sbezverk | Jeffrey4l_: check this log, according to nova hypervisor-stats, there are two hypervisors http://logs.openstack.org/50/441550/49/experimental/gate-kolla-kubernetes-deploy-centos-source-3-ironic-nv/9b86190/console.html#_2017-03-08_02_35_46_527348 | 04:31 |
| sbezverk | Jeffrey4l_: which is expected as I have nova-compute and nova-compute-ironic | 04:32 |
| sbezverk | but the problem still nova-compute-ironic does not pass baremetal node resources as available to nova scheduler. :( | 04:33 |
| *** rwellum_ has quit IRC | 04:46 | |
| *** jtriley has joined #openstack-kolla | 04:51 | |
| *** skramaja has joined #openstack-kolla | 04:54 | |
| masber | my docker version is 17.03.0-ce is that supported? | 04:55 |
| *** crushil has quit IRC | 04:57 | |
| *** trinaths has joined #openstack-kolla | 05:00 | |
| *** erlon has quit IRC | 05:05 | |
| *** salv-orlando has joined #openstack-kolla | 05:05 | |
| masber | I see that by default you use linuxbridge as neutron_plugin_agent, is there any benefit compared to openvswicth? | 05:07 |
| *** salv-orlando has quit IRC | 05:09 | |
| *** unicell has joined #openstack-kolla | 05:10 | |
| *** daidv has quit IRC | 05:20 | |
| *** trinaths1 has joined #openstack-kolla | 05:23 | |
| *** trinaths has quit IRC | 05:23 | |
| *** daidv has joined #openstack-kolla | 05:25 | |
| *** nhlfr has quit IRC | 05:27 | |
| *** nhlfr has joined #openstack-kolla | 05:27 | |
| SamYaple | masber: linuxbridge has no benefits over openvswitch. some (incorrectly in my opinion) think linuxbridge to be more stable. but if you are using a version of openvswitch that is 2.3.1 or above, it is very very stable | 05:28 |
| SamYaple | plus, openvswitch has several options that will never be available in linuxbridge (like DVR) | 05:29 |
| masber | thanks SamYaple | 05:29 |
| *** Margin has joined #openstack-kolla | 05:29 | |
| masber | hey I am following duonghq advice and installing kolla-ansible | 05:29 |
| masber | when I run kolla-ansible -i all-in-one bootstrap-servers | 05:30 |
| masber | is complains that "ERROR! the playbook: /usr/share/kolla/ansible/kolla-host.yml could not be found" | 05:30 |
| masber | apparently that file is now in --> /usr/share/kolla-ansible/ansible/kolla-host.yml | 05:30 |
| *** daidv has quit IRC | 05:30 | |
| masber | just to let you know | 05:32 |
| masber | I copied the files to the expected folder and everything works | 05:32 |
| SamYaple | could be a bug, sure | 05:35 |
| SamYaple | might be because you havent upgraded kolla-ansible script | 05:35 |
| *** jtriley has quit IRC | 05:42 | |
| *** Andrew_jedi has joined #openstack-kolla | 05:51 | |
| *** unicell has quit IRC | 05:52 | |
| *** mdnadeem has quit IRC | 05:52 | |
| *** mdnadeem has joined #openstack-kolla | 05:54 | |
| jascott1 | kfox1111 ping? | 05:57 |
| SamYaple | o/ jascott1 | 05:57 |
| jascott1 | whatup SamYaple! | 05:57 |
| masber | another issue --> http://pastebin.com/raw/gz1qLBTw | 05:57 |
| jascott1 | hows the good life? | 05:58 |
| masber | kolla-toolbox image not found | 05:58 |
| masber | I don't know how to fix this one | 05:58 |
| SamYaple | jascott1: oh you know | 05:59 |
| inc0 | ok...I'm not the only one wasting perfectly good evening on irc | 06:00 |
| jascott1 | haha | 06:00 |
| inc0 | masber: images are different. run docker images for me please | 06:00 |
| masber | inc0, no docker images pulled yet | 06:04 |
| inc0 | you're running master/ocata? | 06:05 |
| *** salv-orlando has joined #openstack-kolla | 06:06 | |
| masber | that is a good question, I am not sure I just follow https://docs.openstack.org/developer/kolla-ansible/quickstart.html | 06:06 |
| inc0 | then you probably do | 06:06 |
| inc0 | well, 4.0.0 images are still not published | 06:06 |
| inc0 | 3.0.3 should be available on dockerhub | 06:07 |
| masber | ok how can I change from 4.0.0 to 3.0.3? | 06:08 |
| inc0 | you installed with git? | 06:08 |
| inc0 | or pip? | 06:08 |
| Jeffrey4l_ | inc0, yep. | 06:08 |
| masber | pip | 06:09 |
| Jeffrey4l_ | based on your patch, qemu processes will run as nova:nova on all distro. | 06:10 |
| Jeffrey4l_ | inc0, | 06:10 |
| *** daidv has joined #openstack-kolla | 06:10 | |
| inc0 | Jeffrey4l_: I know, but I'd rather have this than inconsistent behavior tbh | 06:10 |
| *** salv-orlando has quit IRC | 06:11 | |
| *** daidv has quit IRC | 06:13 | |
| inc0 | masber: do pip freeze | grep kolla plz | 06:13 |
| masber | kolla-ansible==4.0.0.0rc1 | 06:13 |
| inc0 | ok, so you have 2 options, either build your own images or use newton | 06:14 |
| masber | im happy using newton | 06:14 |
| masber | I have no idea how to build my own images to be honest | 06:14 |
| masber | and i want to see how easy is deploying kolla compared to tripleO | 06:15 |
| masber | so for now I just want to make it working | 06:15 |
| masber | I will se the details later | 06:15 |
| inc0 | kk, so do this | 06:15 |
| inc0 | install kolla==3.0.2 | 06:15 |
| inc0 | pip instakk kolla==3.0.3 | 06:15 |
| inc0 | (3.0.3 is good version, not 2) | 06:15 |
| masber | it says no version 3.0.3 | 06:16 |
| masber | 3,0,2 is the latest | 06:16 |
| masber | form there goes to 4.0.0.0b1 | 06:16 |
| inc0 | Jeffrey4l_: I think we missed milestone on pip;) anyway, 3.0.2 is fine | 06:17 |
| inc0 | so install 3.0.2 | 06:17 |
| masber | np | 06:17 |
| masber | ok done | 06:17 |
| inc0 | in your globals.yaml change "openstack_release" to 3.0.2 | 06:17 |
| inc0 | and run tools/kolla-ansible deploy (from kolla folder) | 06:18 |
| Jeffrey4l_ | user should not use rc or milestone, it is unstable, imo. | 06:18 |
| inc0 | yeah, but we should have version 3.0.3 available on pip | 06:19 |
| Jeffrey4l_ | yes. i will be released. | 06:20 |
| Jeffrey4l_ | typo. it will be released. | 06:20 |
| inc0 | so Jeffrey4l_ back to qemu thing, I'd keep same user on both | 06:20 |
| duonghq | masber, you only need config from kolla-ansible, not whole kolla-ansible for kolla-k8s | 06:21 |
| inc0 | as we now know that 2.5 libvirt breaks | 06:21 |
| Jeffrey4l_ | it seems OK so far. but i do not know whether it may cause other issue. | 06:21 |
| inc0 | well, it's better to have same behavior imho | 06:21 |
| inc0 | we'll spot issue quicker this way | 06:22 |
| inc0 | and in case centos bumps libvirt at some point | 06:22 |
| inc0 | we won't get hurt | 06:22 |
| inc0 | also about bug, let's close this bug and submit another one to remove this workaround | 06:22 |
| Jeffrey4l_ | btw, i sent a mail to ML to ask for help about the libvirt 2.5.0 issue. | 06:22 |
| inc0 | with lower priority | 06:22 |
| inc0 | as it won't be critical (maybe medium)( | 06:23 |
| inc0 | I saw it | 06:23 |
| inc0 | unless someone figure it out, let's merge it before release | 06:23 |
| inc0 | this and we need to fix nova upgrades too:/ | 06:23 |
| inc0 | also with serial mariadb didnt upgrade for me | 06:23 |
| Jeffrey4l_ | if we want to same user, then no need to revert in the future. | 06:23 |
| masber | I get this error when I run tools/kolla-ansible --> http://pastebin.com/raw/Pgedt8Zy | 06:23 |
| Jeffrey4l_ | inc0, mariadb is not serial. | 06:24 |
| Jeffrey4l_ | serial + nova upgrade will be failed | 06:24 |
| inc0 | masber: docker --version please | 06:24 |
| inc0 | yeah, we need to change upgrade play then | 06:24 |
| inc0 | as kolla-ansible upgrade will use serial | 06:24 |
| masber | Docker version 17.03.0-ce, build 60ccb22 | 06:24 |
| masber | which one is the latest docker version supported? | 06:25 |
| inc0 | Jeffrey4l_: check masber's log, I thought we fixed it? | 06:25 |
| inc0 | masber: 1.12 is best tested | 06:25 |
| inc0 | 1.13 and 17.03 might work | 06:25 |
| Jeffrey4l_ | masber, try 1.12 please ;) | 06:26 |
| Jeffrey4l_ | this is fixed but unreleased. | 06:26 |
| inc0 | yeah...duh | 06:26 |
| masber | ok, give few moments | 06:26 |
| Jeffrey4l_ | and kolla master do not work very well with docker 1.13. | 06:26 |
| inc0 | most of the world don't work well with 1.13 | 06:26 |
| inc0 | Jeffrey4l_: this bug was affecting only 1.13+ right? | 06:27 |
| *** bmace has quit IRC | 06:27 | |
| Jeffrey4l_ | inc0, about the qemu issue. i prefer to reduce the impact of you change. we limit it to ubuntu may be better. since we will release recently. | 06:27 |
| Jeffrey4l_ | inc0, yes. | 06:27 |
| inc0 | hmm right...ok I'll wrap it in if | 06:28 |
| Jeffrey4l_ | in the next cycle, we can re-consider whether we should unify the user/group. | 06:29 |
| masber | it is good you have not release version 4.0.0 yet as I would also like to to an upgrade and see how it goes | 06:30 |
| *** manheim has joined #openstack-kolla | 06:30 | |
| masber | I did an upgrade using packstack few days ago and was a nightmare | 06:30 |
| masber | actually I couldn't make it to work | 06:31 |
| inc0 | masber: give us few more days to make proper release:) | 06:31 |
| inc0 | we still battle few bugs before that | 06:31 |
| *** satyar has joined #openstack-kolla | 06:31 | |
| Jeffrey4l_ | masber, i tried upgrade, which works fine ;) | 06:32 |
| Jeffrey4l_ | inc0, back to serial issue. mariadb is not serial, right? | 06:32 |
| masber | inc0, no problem I am not pushing you, just sharing my plans that may be useful to you | 06:32 |
| masber | I am happy to share fedback | 06:32 |
| masber | if needd | 06:32 |
| *** g3ek has quit IRC | 06:32 | |
| inc0 | Jeffrey4l_: it is if we call kolla-ansible upgrade | 06:33 |
| inc0 | btw masber that's all it takes;) kolla-ansible upgrade | 06:33 |
| Jeffrey4l_ | and mariadb upgrade is OK( base on my 3 control test ) | 06:33 |
| masber | thats cool | 06:33 |
| Jeffrey4l_ | serial is bad idea for all role, imo. | 06:34 |
| masber | how many hosts have you run with the control plane running on containers/kolla? | 06:34 |
| Jeffrey4l_ | masber, 100+ | 06:34 |
| inc0 | masber: well...3 for ha | 06:34 |
| inc0 | for control | 06:34 |
| masber | I mean, there any performance degradation from running controllers on containers? | 06:34 |
| Jeffrey4l_ | hrm 100+ openstack cluster i mean | 06:34 |
| *** manheim has quit IRC | 06:35 | |
| inc0 | for compute...biggest I tested was ~200 computes | 06:35 |
| inc0 | masber: none that I know of | 06:35 |
| inc0 | and we did pretty extensive testing | 06:35 |
| masber | nice | 06:35 |
| Jeffrey4l_ | inc0, serial + nova upgrade should be buggy now. | 06:35 |
| inc0 | yeah...I guess we need to remove all serial | 06:35 |
| inc0 | for this release at leat | 06:35 |
| inc0 | least | 06:36 |
| masber | on which network did you run ~200 nodes openstack cluster? 10Gigs? | 06:36 |
| inc0 | yeah, but that was testing, not prod | 06:36 |
| masber | kk | 06:36 |
| inc0 | deployment of 130 nodes took 20min:) | 06:36 |
| masber | nice | 06:36 |
| Jeffrey4l_ | inc0, i did this, but it is block so far https://review.openstack.org/438374 | 06:37 |
| duonghq | Jeffrey4l_, can you re-review that: https://review.openstack.org/#/c/438838/ | 06:37 |
| *** salv-orlando has joined #openstack-kolla | 06:37 | |
| inc0 | yeah I remember | 06:38 |
| Jeffrey4l_ | or we can disable serial in "kolla-ansible" script with a release note. if the operator want to run with serial, just change this. | 06:38 |
| inc0 | Jeffrey4l_: yeah that's probably best | 06:38 |
| inc0 | and for next release we can refactor our plays so upgrade will treat control plane and dataplane differently | 06:38 |
| Jeffrey4l_ | cool. i will push this patch. | 06:38 |
| Jeffrey4l_ | duonghq, checking. | 06:38 |
| Jeffrey4l_ | inc0, agree. | 06:38 |
| Jeffrey4l_ | duonghq, done. nit. | 06:40 |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla-ansible master: Fix Ceilometer cannot create MySQL database https://review.openstack.org/438838 | 06:41 |
| duonghq | Jeffrey4l_, thanks | 06:41 |
| *** g3ek has joined #openstack-kolla | 06:41 | |
| Jeffrey4l_ | inc0, ops. mariadb run with serial.. this is issue.. | 06:42 |
| inc0 | duonghq: I just started voting for including you in core team:) | 06:42 |
| *** skramaja_ has joined #openstack-kolla | 06:42 | |
| inc0 | yeah and upgrade of all computes is also an issue:/ | 06:42 |
| inc0 | bit not as bad | 06:42 |
| duonghq | inc0, thank you | 06:42 |
| inc0 | so yeah, let's merge this, and allow it to be overriden by ops | 06:42 |
| Jeffrey4l_ | why? | 06:42 |
| *** skramaja has quit IRC | 06:43 | |
| inc0 | Jeffrey4l_: with hundreds of them at the same time pulling images can be painful | 06:43 |
| inc0 | but on the other hand, ops can lower amount of forks in ansible | 06:43 |
| Jeffrey4l_ | inc0, ansible has "forks" parameter, which can control the concurrency. | 06:43 |
| *** pcaruana has joined #openstack-kolla | 06:43 | |
| inc0 | yeah | 06:43 |
| Jeffrey4l_ | the default value is only "5" | 06:43 |
| Jeffrey4l_ | so not that bad at this point. | 06:44 |
| Jeffrey4l_ | congrats duonghq ;) | 06:44 |
| duonghq | thank Jeffrey4l_ | 06:44 |
| Margin | hi, when profile is not defined in global.yml , what it stands for ? | 06:46 |
| Jeffrey4l_ | Margin, profile is a kolla-build parameter. | 06:46 |
| Jeffrey4l_ | it is not related to kolla-ansible | 06:46 |
| Margin | I know | 06:47 |
| Margin | will it build all serivce under "docker" directory ? | 06:47 |
| inc0 | Margin: by default, yes | 06:49 |
| inc0 | but you can limit it | 06:49 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Make qemu use nova user https://review.openstack.org/442850 | 06:50 |
| masber | docker is now 1.12.6 but got an error running kolla-ansible deploy --> http://pastebin.com/raw/pAQ1MxS9 | 06:50 |
| inc0 | did you install docker-py? | 06:51 |
| Margin | inc0, do you mean I am right ? | 06:52 |
| inc0 | Margin: yes | 06:52 |
| Margin | ok | 06:52 |
| masber | yes | 06:52 |
| *** Andrew_jedi has quit IRC | 06:53 | |
| *** bmace has joined #openstack-kolla | 06:53 | |
| inc0 | masber: hmm...strange | 06:53 |
| inc0 | try to run python -c "import docker" | 06:53 |
| masber | Docker version 1.12.6, build 78d1802 | 06:53 |
| inc0 | docker != docker-py | 06:53 |
| inc0 | docker-py is python bindings to docker | 06:54 |
| masber | ImportError: No module named docker | 06:54 |
| inc0 | pip install docker-py | 06:54 |
| masber | Requirement already satisfied | 06:54 |
| inc0 | apperantly not :S something is wrong with your pythonpath there.. | 06:55 |
| Jeffrey4l_ | masber, inc0 there is two docker python bind now. docker and docker-py. | 06:55 |
| Jeffrey4l_ | we depends on docker-py, not docker. | 06:55 |
| Margin | inc0, I found I can't define profile=infra,main | 06:55 |
| Jeffrey4l_ | masber, inc0 check https://review.openstack.org/438813 | 06:56 |
| Jeffrey4l_ | Margin, profile only support one parameter. | 06:56 |
| inc0 | Margin: I don't think you can have multiple profiles, but you can define your own | 06:56 |
| Margin | from the words , it is confusing me | 06:57 |
| inc0 | Margin: hold on, let me show you | 06:57 |
| inc0 | Jeffrey4l_: masber's issue is different tho | 06:57 |
| Margin | I want to define different profile for debuging | 06:57 |
| Jeffrey4l_ | Margin, create your own profile. | 06:58 |
| Jeffrey4l_ | Margin, check https://review.openstack.org/#/c/412278/27/tools/setup_gate.sh | 06:58 |
| Jeffrey4l_ | masber, run "pip freeze | grep docker" | 06:58 |
| masber | docker-py==1.7.2 | 06:59 |
| masber | docker-pycreds==0.2.1 | 06:59 |
| Jeffrey4l_ | Margin, "python -c 'import docker'"? | 06:59 |
| inc0 | masber: are you using virtualenv? | 06:59 |
| masber | no | 07:00 |
| inc0 | uhh | 07:00 |
| inc0 | your pip seems to be in different path than python... | 07:01 |
| masber | this is a fresh installation | 07:02 |
| *** saneax-_-|AFK is now known as saneax | 07:05 | |
| inc0 | Jeffrey4l_: today is release deadline:/ | 07:07 |
| *** skramaja_ is now known as skramaja | 07:07 | |
| Jeffrey4l_ | inc0, yes. | 07:07 |
| inc0 | we can't release with this qemu bug | 07:07 |
| inc0 | can we fix it and release tomorrow you think? | 07:07 |
| Jeffrey4l_ | still no with your workaround? | 07:07 |
| inc0 | review and merge it then;) | 07:08 |
| Jeffrey4l_ | np. | 07:08 |
| inc0 | I updated patchset and wrapped this into if | 07:08 |
| inc0 | https://review.openstack.org/#/c/442850/2 | 07:08 |
| Jeffrey4l_ | +2ed. | 07:09 |
| Jeffrey4l_ | wait for gate. | 07:09 |
| inc0 | zhubingbing_: around? | 07:09 |
| inc0 | Jeffrey4l_: I'm going to sleep, can you ping other cores to review it and reply to Dougs questions? | 07:09 |
| Jeffrey4l_ | inc0, 1 sec | 07:09 |
| Jeffrey4l_ | please check the change i made about serial. | 07:10 |
| Jeffrey4l_ | pushing. | 07:10 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Disable serial in default https://review.openstack.org/442974 | 07:11 |
| inc0 | ok | 07:11 |
| *** Andrew_jedi has joined #openstack-kolla | 07:11 | |
| Jeffrey4l_ | inc0, ^^ not fixed, need improve. | 07:11 |
| Jeffrey4l_ | but check the idea. | 07:11 |
| Jeffrey4l_ | not finished.. | 07:11 |
| *** nhlfr has quit IRC | 07:11 | |
| *** nhlfr has joined #openstack-kolla | 07:12 | |
| inc0 | reviewed | 07:12 |
| inc0 | talk to you tomorrow on meeting | 07:12 |
| Jeffrey4l_ | roger. good night. | 07:13 |
| inc0 | bye, thanks | 07:13 |
| duonghq | night inc0 | 07:13 |
| *** magicboiz has joined #openstack-kolla | 07:15 | |
| *** jrobinson has quit IRC | 07:19 | |
| Margin | hello, is it a best practise that openstack_release assigned "4.0.0.1" in global.yml ? | 07:21 |
| zhubingbing_ | i am here | 07:21 |
| zhubingbing_ | inc0 | 07:21 |
| Margin | It is convient for upgrading openstack ? | 07:22 |
| Jeffrey4l_ | zhubingbing_, inc0 is offline. he want you review https://review.openstack.org/442850 | 07:22 |
| zhubingbing_ | ok | 07:22 |
| Jeffrey4l_ | zhubingbing_, please do not +w | 07:22 |
| Jeffrey4l_ | we need more reviewer here. | 07:22 |
| Margin | is it convenient for upgrading openstack ? | 07:23 |
| Jeffrey4l_ | Margin, just assigned the new tag number. | 07:23 |
| *** salv-orl_ has joined #openstack-kolla | 07:23 | |
| Jeffrey4l_ | what the convenient means? | 07:23 |
| Margin | when I used openstack Newton , it is 3.0.0 ,when I want to upgrade openstack from Newton to Ocata , I set it to "4.0.0" , then "kolla-ansible upgrade " | 07:25 |
| Jeffrey4l_ | not exactly. | 07:25 |
| Jeffrey4l_ | need add some items in inventory and passwords.yml file. | 07:26 |
| *** salv-orlando has quit IRC | 07:26 | |
| Margin | that is a big version. when I upgrade different small version in Ocata big version, it is from "4.0.0.1" to "4.0.0.2" , right? | 07:27 |
| Jeffrey4l_ | yep. small version is easy to handle. | 07:27 |
| Margin | ok, I see, do you mean there is more troubles between big version ? | 07:28 |
| *** furlongm has quit IRC | 07:29 | |
| Jeffrey4l_ | need more work between big version. but not that complicated. | 07:29 |
| Margin | ok, thanks | 07:30 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Disable run playbooks in serial by default https://review.openstack.org/442974 | 07:30 |
| *** furlongm has joined #openstack-kolla | 07:33 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Disable running playbooks in serial by default https://review.openstack.org/442974 | 07:34 |
| spsurya | Jeffrey4l_: Just a query why we are running precheck again ?? https://review.openstack.org/#/c/442559/ | 07:34 |
| *** skramaja_ has joined #openstack-kolla | 07:34 | |
| Jeffrey4l_ | make sure prechecks works after first deployment. | 07:35 |
| *** skramaja has quit IRC | 07:35 | |
| Margin | I want to became reviewer , what requirement it have ? | 07:35 |
| Jeffrey4l_ | it will be useful when you want to enable another service after first deploy. | 07:35 |
| Jeffrey4l_ | spsurya, and also useful for upgrade precheck. | 07:36 |
| spsurya | Jeffrey4l_: got that, yes can be run multiple times too | 07:36 |
| Jeffrey4l_ | yep. | 07:36 |
| spsurya | Jeffrey4l_: thnx | 07:37 |
| *** skramaja_ is now known as skramaja | 07:38 | |
| spsurya | Margin: Please follow https://wiki.openstack.org/wiki/How_To_Contribute | 07:38 |
| spsurya | You will all the answer of the primary requite questions | 07:39 |
| spsurya | You will have* | 07:39 |
| Margin | ok | 07:41 |
| *** matrohon has joined #openstack-kolla | 07:51 | |
| spsurya | Jeffrey4l_: does this bug need to be fix https://bugs.launchpad.net/kolla/+bug/1576794 | 07:52 |
| openstack | Launchpad bug 1576794 in kolla "drop root for keystone" [Critical,In progress] - Assigned to Surya Prakash Singh (confisurya) | 07:52 |
| spsurya | I think we cant't provide root previlege to user | 07:52 |
| spsurya | for running keystone | 07:52 |
| Jeffrey4l_ | spsurya, this can be implement i think and nice to fix. | 07:53 |
| spsurya | Jeffrey4l_: ok, I will push the patch for this | 07:53 |
| Jeffrey4l_ | cool. | 07:53 |
| *** huzhengchuan has quit IRC | 07:53 | |
| duonghq | spsurya, Jeffrey4l_ I see pbourke comment in Sharat's patch | 07:54 |
| duonghq | any ideas? | 07:54 |
| *** athomas has joined #openstack-kolla | 07:54 | |
| Jeffrey4l_ | duonghq, hrm ,which patch, link? | 07:54 |
| duonghq | or we can make this bug is won't fix | 07:55 |
| duonghq | https://review.openstack.org/#/c/379332/ | 07:55 |
| spsurya | duonghq: I think we need some kind of handling with keystone port | 07:55 |
| duonghq | spsurya, or break it into some container? | 07:56 |
| Jeffrey4l_ | docker run --cap-add should be helpful for this. | 07:56 |
| duonghq | Jeffrey4l_, so we need update kolla-docker too? | 07:58 |
| Jeffrey4l_ | duonghq, kolla-docker support cap-add now. | 07:59 |
| *** salv-orl_ has quit IRC | 08:00 | |
| duonghq | Jeffrey4l_, seen | 08:00 |
| Jeffrey4l_ | so i think it should be possible | 08:00 |
| spsurya | duonghq: it adds linux capability to docker | 08:01 |
| spsurya | AFAIK | 08:01 |
| duonghq | yup | 08:02 |
| *** gfidente has joined #openstack-kolla | 08:06 | |
| *** gfidente has quit IRC | 08:06 | |
| *** gfidente has joined #openstack-kolla | 08:06 | |
| masber | sorry its me again | 08:07 |
| masber | I started again from scratch | 08:07 |
| masber | and I am getting a different error now --> http://pastebin.com/raw/CwUrPrAB | 08:07 |
| masber | docker conflict? Im using Docker version 1.12.5, build 047e51b/1.12.5 | 08:08 |
| Jeffrey4l_ | masber, uninstall docker-engine-selinux-17.03.0.ce-1.el7.centos.noarch manually. | 08:13 |
| *** mgoddard has joined #openstack-kolla | 08:19 | |
| *** Andrew_jedi has left #openstack-kolla | 08:22 | |
| openstackgerrit | Zeyu Zhu proposed openstack/kolla-ansible master: Modify the directory path in running-tests.rst https://review.openstack.org/435229 | 08:28 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Fix mongodb cluster using wrong network interface https://review.openstack.org/442999 | 08:40 |
| *** shardy has joined #openstack-kolla | 08:43 | |
| *** sudipto has joined #openstack-kolla | 08:50 | |
| *** sudipto_ has joined #openstack-kolla | 08:50 | |
| *** egonzalez has joined #openstack-kolla | 08:50 | |
| openstackgerrit | lyanchih proposed openstack/kolla-kubernetes master: [helm]Create keystone public service failed with nil external vip https://review.openstack.org/443006 | 08:54 |
| *** salv-orlando has joined #openstack-kolla | 09:00 | |
| *** openstackgerrit has quit IRC | 09:03 | |
| *** duankebo has quit IRC | 09:04 | |
| *** salv-orlando has quit IRC | 09:05 | |
| *** hieulq has quit IRC | 09:06 | |
| *** hieulq has joined #openstack-kolla | 09:08 | |
| *** Manheim has joined #openstack-kolla | 09:12 | |
| *** Mannheim has joined #openstack-kolla | 09:12 | |
| *** mgoddard has quit IRC | 09:12 | |
| *** prateek has joined #openstack-kolla | 09:17 | |
| *** salv-orlando has joined #openstack-kolla | 09:31 | |
| *** salv-orlando has quit IRC | 09:35 | |
| egonzalez | anyone running ocata centos binary? | 09:35 |
| duonghq | egonzalez, about 1-2 weeks ago | 09:36 |
| egonzalez | duonghq, can you check in have topology tab in heat stacks? | 09:36 |
| egonzalez | my boss is complaining that want a fancy view of the stacks, but it does not appear | 09:37 |
| duonghq | egonzalez, mins, I need deploying this | 09:37 |
| *** hrw has quit IRC | 09:39 | |
| *** rmart04 has joined #openstack-kolla | 09:40 | |
| *** Margin has quit IRC | 09:42 | |
| *** Margin has joined #openstack-kolla | 09:42 | |
| *** trinaths1 has quit IRC | 09:43 | |
| *** MarginHu has joined #openstack-kolla | 09:44 | |
| *** Margin has quit IRC | 09:44 | |
| *** hrw has joined #openstack-kolla | 09:45 | |
| *** MarginHu has quit IRC | 09:46 | |
| *** mgoddard has joined #openstack-kolla | 09:49 | |
| *** cmceniry has quit IRC | 09:49 | |
| Jeffrey4l_ | egonzalez, please check https://review.openstack.org/442850 | 09:57 |
| Jeffrey4l_ | related to ubuntu gate rede. | 09:57 |
| Jeffrey4l_ | red* | 09:57 |
| *** satyar has quit IRC | 09:57 | |
| *** cmceniry has joined #openstack-kolla | 09:57 | |
| egonzalez | Jeffrey4l_, will changing the owner break old vms with older permissions while upgrading? | 09:59 |
| *** manheim_ has joined #openstack-kolla | 10:00 | |
| *** manheim_ has quit IRC | 10:01 | |
| *** manheim_ has joined #openstack-kolla | 10:01 | |
| Jeffrey4l_ | i guess no. but not sure. | 10:02 |
| egonzalez | after I test other thing I will give a try to the ubuntu upgrade | 10:02 |
| *** manheim_ has quit IRC | 10:02 | |
| Jeffrey4l_ | cool , thanks. | 10:03 |
| *** yingjun has quit IRC | 10:05 | |
| *** manheim_ has joined #openstack-kolla | 10:06 | |
| egonzalez | Jeffrey4l_, you know if there is any reason why ceilometer-*json does not apply permissions at kolla_logs/ceilometer? justs to know before pushing a change | 10:06 |
| *** trinaths has joined #openstack-kolla | 10:10 | |
| *** tovin07_ has quit IRC | 10:11 | |
| *** openstackgerrit has joined #openstack-kolla | 10:14 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Improve developer documentation for Bifrost https://review.openstack.org/436462 | 10:14 |
| *** zhubingbing_ has quit IRC | 10:15 | |
| duonghq | egonzalez, around? | 10:16 |
| egonzalez | duonghq, yep | 10:16 |
| duonghq | about the heat in horizon | 10:16 |
| duonghq | in my installation, in Project -> Orchestration -> Stacks | 10:17 |
| duonghq | seem that no graph appear here, | 10:17 |
| egonzalez | inside a stack, there are different tabs | 10:18 |
| egonzalez | Overview | 10:18 |
| egonzalez | Resources | 10:18 |
| egonzalez | Events | 10:18 |
| egonzalez | Template | 10:18 |
| egonzalez | . Here should be a topology | 10:18 |
| egonzalez | is inside an already created stack(show resources and connections of such stack) | 10:18 |
| *** sambetts|afk is now known as sambetts | 10:19 | |
| duonghq | mins, I need to launch a stack | 10:19 |
| *** salv-orlando has joined #openstack-kolla | 10:20 | |
| *** pbourke has quit IRC | 10:21 | |
| *** pbourke has joined #openstack-kolla | 10:22 | |
| Jeffrey4l_ | egonzalez, may be a bug. | 10:25 |
| *** nhlfr[m] has joined #openstack-kolla | 10:25 | |
| duonghq | egonzalez, my stack creating is in progress for a while | 10:35 |
| *** caowei has quit IRC | 10:35 | |
| egonzalez | duonghq, is a big one? or just an instance? | 10:35 |
| duonghq | just m1.small instance | 10:36 |
| duonghq | in heat-template/hot sample | 10:36 |
| egonzalez | recently i've seen dependencies issues when creating instances with two get_resource: networks | 10:36 |
| duonghq | I use public1 as network | 10:36 |
| egonzalez | is any error in heat-engine? or instance failing in nova? | 10:36 |
| openstackgerrit | Ronan-Alexandre Cherrueau proposed openstack/kolla-ansible master: Multi-regions Support https://review.openstack.org/431658 | 10:37 |
| duonghq | ah, it's failed | 10:37 |
| duonghq | ERROR: Remote error: Conflict Instance 056f071e-64c4-4bdc-9fea-6a101c25ddd4 is not ready (HTTP 409) (Request-ID: req-9ec4afb8-7d1d-418f-ad64-a8c12fe2826c) | 10:37 |
| *** Haaibo has quit IRC | 10:37 | |
| egonzalez | duonghq, nova will say more info about that | 10:38 |
| *** kklimonda has quit IRC | 10:38 | |
| *** kklimonda^ is now known as kklimonda | 10:38 | |
| *** kklimonda is now known as Guest71046 | 10:39 | |
| duonghq | libvirtError: invalid argument: could not find capabilities for domaintype=kvm -> this error | 10:39 |
| duonghq | min, I'll try to fix it | 10:39 |
| egonzalez | guess the host does not support vmx right? maybe forgot to add virt_type = qemu and cpu_mode =none at nova.conf | 10:40 |
| duonghq | yup | 10:40 |
| *** kbaegis has joined #openstack-kolla | 10:41 | |
| *** Guest71046 is now known as kklimonda | 10:41 | |
| *** kklimonda has joined #openstack-kolla | 10:41 | |
| kbaegis | Hi all. Has anyone else seen fluentd say "container_linux.go:247: starting container process caused "exec: \"-E\": executable file not found in $PATH""? | 10:41 |
| des_consolado | what calls extend_start.sh ? | 10:42 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Fix booting from volume failure https://review.openstack.org/442519 | 10:47 |
| duonghq | egonzalez, complete | 10:48 |
| duonghq | egonzalez, I cannot find any topology | 10:48 |
| *** cuongnv has quit IRC | 10:48 | |
| egonzalez | duonghq, thats for trying, I cannot find any error in logs about that | 10:49 |
| egonzalez | *thanks | 10:49 |
| duonghq | egonzalez, you want a graph? | 10:49 |
| duonghq | kind of graph with node connect to network? | 10:49 |
| egonzalez | duonghq, there is a tab in horizon to show all the stack topology https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/stacks/tabs.py#L32 | 10:51 |
| egonzalez | duonghq, check https://www.google.co.uk/search?q=heat+stack+topology&safe=off&source=lnms&tbm=isch&sa=X&ved=0ahUKEwiwwYLI3sbSAhVLDZoKHXukAkEQ_AUICCgB&biw=1313&bih=564#imgrc=nGT-Mz-GrSTx9M: | 10:52 |
| des_consolado | what calls extend_start.sh ? it's not documented, it's not in any grep of the repo's... bit lost here... | 10:52 |
| duonghq | des_consolado, it's copied to container under other name | 10:52 |
| duonghq | egonzalez, so, I only get 4 tabs | 10:52 |
| des_consolado | oh shit yeah it's then used under the RUN, I'm an idiot, sorry | 10:53 |
| duonghq | des_consolado, COPY extend_start.sh /usr/local/bin/kolla_extend_start | 10:53 |
| duonghq | egonzalez, des_consolado I should come back to home now, bye guys | 10:54 |
| egonzalez | duonghq, yep, may be an horizon bug, i cannot find any warn or error message | 10:54 |
| egonzalez | duonghq, thanks for the test :D | 10:54 |
| duonghq | egonzalez, np ;) | 10:54 |
| *** duonghq has quit IRC | 10:54 | |
| des_consolado | egonzalez: I saw that you put about putting work items on that blueprint to do with the bgp-dragent btw, wasn't really sure what to put under work items so I thought I'd just do the work first and then try to put it into work items. I'm working full time on this now though so hopefully will be done soon (spent the last few weeks playing with ironic/bifrost etc) :p | 10:57 |
| egonzalez | des_consolado, roger, need help on that? what is the bp? | 10:58 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Apply correct permissions to kolla_logs/ceilometer https://review.openstack.org/443066 | 10:59 |
| des_consolado | egonzalez: I think I'm okay, this is the bp on kolla https://blueprints.launchpad.net/kolla/+spec/neutron-bgp-dragent - I guess a separate bp is needed for kolla-ansible if I add playbooks for running this way? | 11:01 |
| egonzalez | des_consolado, what changes are needed? create bgp agent dockerfiles, change ironic or other services? | 11:02 |
| egonzalez | des_consolado, i would add as work items: create neutron bgp dockerfile, implement bgp agent in <service> image, etc | 11:03 |
| egonzalez | and for kolla-ansible a similar pattern | 11:03 |
| des_consolado | okay I will - and no ironic stuff no, that's just what I was busy with the last couple weeks so why this was on hold | 11:04 |
| egonzalez | des_consolado, hrm, ok. work items is just for know what is the state of implementation of a blueprint. so we can change to good progress, implemented, etc | 11:05 |
| *** zhurong has joined #openstack-kolla | 11:07 | |
| *** masber has quit IRC | 11:09 | |
| *** masber has joined #openstack-kolla | 11:10 | |
| *** athomas has quit IRC | 11:18 | |
| masber | docker from centos repos breaks the kolla-ansible installation | 11:21 |
| *** matrohon has quit IRC | 11:25 | |
| *** athomas has joined #openstack-kolla | 11:25 | |
| egonzalez | masber, where breaks and docker version? | 11:25 |
| *** g3ek has quit IRC | 11:31 | |
| *** satyar has joined #openstack-kolla | 11:33 | |
| openstackgerrit | Martin André proposed openstack/kolla master: Add heat-api-cloudwatch image https://review.openstack.org/443077 | 11:37 |
| *** g3ek has joined #openstack-kolla | 11:40 | |
| *** trinaths has quit IRC | 12:03 | |
| *** bjolo has joined #openstack-kolla | 12:25 | |
| sdake | morning peeps | 12:27 |
| sdake | i am out :) | 12:28 |
| sdake | just wanted to drop in and say hi | 12:28 |
| egonzalez | hi sdake | 12:28 |
| *** rwallner has joined #openstack-kolla | 12:28 | |
| sdake | inc0 spsurya I use znc + virtualbox which is much more nerdy even that what inc0 has :) | 12:29 |
| * sdake nerd for life :) | 12:29 | |
| sdake | rather Textual not virtualbox | 12:29 |
| *** david-lyle has quit IRC | 12:30 | |
| *** david-lyle has joined #openstack-kolla | 12:33 | |
| spsurya | Morning sdake | 12:34 |
| *** shardy has quit IRC | 12:34 | |
| spsurya | sdake: +1 znc + virtualbox | 12:35 |
| *** zhurong has quit IRC | 12:35 | |
| *** erlon has joined #openstack-kolla | 12:37 | |
| spsurya | :-) | 12:38 |
| *** rhallisey has joined #openstack-kolla | 12:46 | |
| *** goldyfruit has joined #openstack-kolla | 12:47 | |
| *** goldyfruit has quit IRC | 12:47 | |
| *** salv-orlando has quit IRC | 12:48 | |
| *** schwicht has quit IRC | 12:52 | |
| *** salv-orlando has joined #openstack-kolla | 12:54 | |
| *** rhallisey has quit IRC | 12:54 | |
| *** bjolo has quit IRC | 13:02 | |
| *** MarginHu has joined #openstack-kolla | 13:05 | |
| *** manheim_ has quit IRC | 13:06 | |
| *** rhallisey has joined #openstack-kolla | 13:07 | |
| *** spsurya__ has joined #openstack-kolla | 13:08 | |
| *** yingjun has joined #openstack-kolla | 13:09 | |
| *** manheim_ has joined #openstack-kolla | 13:10 | |
| *** mattmceuen has joined #openstack-kolla | 13:10 | |
| *** manheim_ has quit IRC | 13:14 | |
| *** manheim_ has joined #openstack-kolla | 13:14 | |
| *** duonghq has joined #openstack-kolla | 13:17 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Fix booting from volume failure https://review.openstack.org/442519 | 13:17 |
| duonghq | hi guys, which docker version are you using? | 13:17 |
| Jeffrey4l_ | 1.12.6 | 13:18 |
| duonghq | roger | 13:18 |
| *** kbaegis has quit IRC | 13:19 | |
| egonzalez | 1.12.6 too | 13:19 |
| egonzalez | the first thing I do after install docker is downgrade to 1.12 | 13:19 |
| duonghq | I just try build image in Arch linux | 13:20 |
| duonghq | the docker version is 17.03 | 13:20 |
| duonghq | and got strange error: | 13:20 |
| duonghq | http://paste.openstack.org/show/601928/ | 13:20 |
| Jeffrey4l_ | egonzalez, could u take a looks this ? https://review.openstack.org/442974 | 13:20 |
| Jeffrey4l_ | no idea what's it :( haven't test again 17.03 | 13:21 |
| duonghq | just find that docker changed their version convention | 13:22 |
| *** salv-orl_ has joined #openstack-kolla | 13:23 | |
| *** kbaegis has joined #openstack-kolla | 13:24 | |
| *** salv-orlando has quit IRC | 13:26 | |
| *** rwellum_ has joined #openstack-kolla | 13:26 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Disable running playbooks in serial by default https://review.openstack.org/442974 | 13:27 |
| *** matrohon has joined #openstack-kolla | 13:33 | |
| duonghq | egonzalez, can you review this one: https://review.openstack.org/#/c/438838/ | 13:33 |
| *** kbaegis has quit IRC | 13:34 | |
| *** kbaegis has joined #openstack-kolla | 13:34 | |
| *** prateek has quit IRC | 13:34 | |
| *** kbaegis has quit IRC | 13:38 | |
| *** kbaegis has joined #openstack-kolla | 13:39 | |
| *** kbaegis has quit IRC | 13:43 | |
| *** lamt has joined #openstack-kolla | 13:44 | |
| *** rhallisey has quit IRC | 13:46 | |
| *** kbaegis has joined #openstack-kolla | 13:47 | |
| *** rabel has joined #openstack-kolla | 13:50 | |
| *** haplo37 has quit IRC | 13:50 | |
| *** goldyfruit has joined #openstack-kolla | 13:52 | |
| *** haplo37 has joined #openstack-kolla | 13:59 | |
| *** bjolo has joined #openstack-kolla | 14:00 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix keystone idempotency https://review.openstack.org/437900 | 14:00 |
| *** zhurong has joined #openstack-kolla | 14:01 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Enable run nova-compute-ironic with nova-compute on the same node https://review.openstack.org/443138 | 14:01 |
| *** rhallisey has joined #openstack-kolla | 14:01 | |
| *** strigazi_AFK is now known as strigazi | 14:04 | |
| *** bjolo has quit IRC | 14:05 | |
| *** spsurya__ has quit IRC | 14:07 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 14:10 |
| *** shardy has joined #openstack-kolla | 14:13 | |
| *** m1dev has joined #openstack-kolla | 14:15 | |
| magicboiz | Hi, I'm working on this bug: https://bugs.launchpad.net/kolla/+bug/1668303. It turns out that mongodb.conf defines "storage.dbPath = /var/lib/mongodb" while mongodb/config.json only grants access to "/varlib/kolla/mongodb" | 14:15 |
| openstack | Launchpad bug 1668303 in kolla "upgrade from newton to ocata: error with mongodb" [Undecided,New] | 14:15 |
| magicboiz | Does it make sense? | 14:15 |
| magicboiz | Moreover, I don't know how this works in newton.... | 14:15 |
| *** salv-orl_ has quit IRC | 14:18 | |
| *** lamt has quit IRC | 14:19 | |
| egonzalez | magicboiz, in newton probably was using /var/lib/mongo, while upgrade, container is replaced by new one losing content on there since is not in the volume | 14:19 |
| kbaegis | Hey guys, is there a streaming site for openstack boston? | 14:20 |
| *** spsurya__ has joined #openstack-kolla | 14:20 | |
| kbaegis | The view summit schedule links on openstack.org aren't working for me | 14:20 |
| *** mattmceuen has quit IRC | 14:20 | |
| *** dave-mccowan has joined #openstack-kolla | 14:20 | |
| egonzalez | kbaegis, all talks are published on youtube | 14:21 |
| magicboiz | egonzalez: checking ocata: https://github.com/openstack/kolla-ansible/blob/stable/ocata/ansible/roles/mongodb/templates/mongodb.json.j2 (defines "path": "/var/lib/kolla/mongodb") and https://github.com/openstack/kolla-ansible/blob/stable/ocata/ansible/roles/mongodb/templates/mongodb.conf.j2 (defines dbPath: /var/lib/mongodb") | 14:22 |
| egonzalez | magicboiz, is a bug | 14:25 |
| egonzalez | while refactoring mongodb.conf probably that path was missed | 14:25 |
| *** MarginHu has quit IRC | 14:28 | |
| *** eaguilar has joined #openstack-kolla | 14:29 | |
| *** schwicht has joined #openstack-kolla | 14:29 | |
| Jeffrey4l_ | magicboiz, this is correct. the permissions is used for log. | 14:29 |
| magicboiz | yes, bug1668303. Upgrade actually fails from newton to ocata in my lab (mongo deployed). How should we fix it? | 14:29 |
| magicboiz | Jeffrey4l_: you mean "/var/log/kolla" or "/var/lib/kolla"? | 14:30 |
| *** kbaegis has quit IRC | 14:31 | |
| *** erhudy has joined #openstack-kolla | 14:31 | |
| Jeffrey4l_ | hrm, sorry, you are correct. | 14:31 |
| Jeffrey4l_ | /var/lib/kolla/mongodb should be /var/lib/mongodb in mongodb.json.j2 | 14:32 |
| Jeffrey4l_ | could u push a fix? magicboiz | 14:32 |
| *** jtriley has joined #openstack-kolla | 14:32 | |
| Jeffrey4l_ | btw, i moved bug 1668303 to kolla-ansible project. | 14:33 |
| openstack | bug 1668303 in kolla-ansible ocata "upgrade from newton to ocata: error with mongodb" [Undecided,New] https://launchpad.net/bugs/1668303 | 14:33 |
| magicboiz | Jeffrey4l_: sure, just let me know what is the strategy for ocata: /var/lib/mongodb vs /var/lib/kolla/mongodb. In the first case it's quite easy, just change mongodb.json to match mondodb.conf dbPath. In the second case, as egonzalez pointed out, we should move data from /var/lib/mongodb to /var/lib/kolla/mongodb during upgrade process.... | 14:34 |
| egonzalez | Jeffrey4l_, should not be the other way. /var/lib/mongodb changed to /var/lib/kolla/mongodb as it was like that in newton? | 14:34 |
| Jeffrey4l_ | we should use /var/lib/mongodb always. | 14:34 |
| Jeffrey4l_ | /var/lib/kolla/mongodb is a typo. | 14:34 |
| Jeffrey4l_ | magicboiz, no need to move. /var/lib/kolla/mongodb just exist in config.json file. | 14:35 |
| Jeffrey4l_ | it did nothing actually. | 14:35 |
| magicboiz | in newton, the bug is present too, but mongodb container works (don't ask me why) | 14:35 |
| Jeffrey4l_ | yes. it should works. | 14:35 |
| Jeffrey4l_ | permissions section in config.json only used for upgrade ( static uid/gid issue ) | 14:36 |
| egonzalez | Jeffrey4l_, yes, the volume is mounted at /var/lib/mongodb | 14:36 |
| magicboiz | aaaaahhhh...ok | 14:36 |
| Jeffrey4l_ | /var/lib/kolla/mongod is a typo. | 14:36 |
| magicboiz | ok, I'll push a fix | 14:36 |
| Jeffrey4l_ | cool. thanks. | 14:36 |
| Jeffrey4l_ | egonzalez, any progress for ubuntu libvirt issue testing? | 14:37 |
| *** g3ek has quit IRC | 14:37 | |
| Jeffrey4l_ | we have to release ocata in the next few hours. so we have to merged the critical patch ASAP. | 14:38 |
| *** manheim_ has quit IRC | 14:41 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 14:42 |
| *** srwilkers has joined #openstack-kolla | 14:42 | |
| egonzalez | Jeffrey4l_, i'm rebuilding atm, tried with tag 3.0.3-beta.1 from DH bug not have the path for simple_cell_setup | 14:45 |
| Jeffrey4l_ | cool. thanks. | 14:45 |
| egonzalez | hope to finish in ~1 hour | 14:46 |
| *** g3ek has joined #openstack-kolla | 14:47 | |
| egonzalez | Jeffrey4l_, any idea on what can I do with ceilometer permissions. If i add /var/log/kolla/ceilometer only on one container will apply also permissions to ceilometer-api | 14:47 |
| Jeffrey4l_ | egonzalez, check keystone.json file, you can specify the filename . | 14:48 |
| Jeffrey4l_ | but i think you patch should works. | 14:48 |
| Jeffrey4l_ | root process write file into ceilometer:ceilometer works. | 14:49 |
| *** lamt has joined #openstack-kolla | 14:49 | |
| egonzalez | cool, did't check that | 14:49 |
| *** ioannis has joined #openstack-kolla | 14:49 | |
| Jeffrey4l_ | event though it looks weird. | 14:49 |
| egonzalez | yes, I tested and ceilometer api works, but didn't check logs | 14:49 |
| egonzalez | well, never saw that error before, this time was in a special case. reconfigured with --tag ceilometer to apply the fix of python modules. maybe something changed in between | 14:51 |
| egonzalez | will try to replicate in fresh deploys | 14:52 |
| Jeffrey4l_ | what error? | 14:53 |
| openstackgerrit | Javier Castillo proposed openstack/kolla-ansible master: Fix dbPath typo in templates/mongodb.json.j2 https://review.openstack.org/443160 | 14:53 |
| *** mattmceuen has joined #openstack-kolla | 14:53 | |
| egonzalez | log permission https://bugs.launchpad.net/kolla-ansible/+bug/1671020 | 14:53 |
| openstack | Launchpad bug 1671020 in kolla-ansible "ceilometer cannot read/write logs" [Undecided,In progress] - Assigned to Eduardo Gonzalez (egonzalez90) | 14:53 |
| egonzalez | Jeffrey4l_, ^^ | 14:54 |
| Jeffrey4l_ | aha, it bug should exist. we changed all uid/gid during ocata. | 14:54 |
| *** mdnadeem has quit IRC | 14:57 | |
| *** saneax is now known as saneax-_-|AFK | 15:04 | |
| *** zhubingbing has joined #openstack-kolla | 15:05 | |
| *** fguillot has joined #openstack-kolla | 15:05 | |
| *** zhurong has quit IRC | 15:05 | |
| *** crushil has joined #openstack-kolla | 15:08 | |
| des_consolado | is there a way to tell kolla-build not to bother re-building the base, openstack-base etc. kind of images before doing an image build? | 15:15 |
| *** yingjun has quit IRC | 15:15 | |
| egonzalez | des_consolado, that PS is under review yet | 15:17 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: [TEST] TEST UBUNTU OCATA PROPOSE REPO https://review.openstack.org/443166 | 15:17 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 15:17 |
| des_consolado | egonzalez: PS? | 15:18 |
| egonzalez | https://review.openstack.org/#/c/426322/ | 15:18 |
| *** salv-orlando has joined #openstack-kolla | 15:18 | |
| des_consolado | oh I see. PS stands for something like pull specification maybe? | 15:19 |
| des_consolado | lol | 15:19 |
| egonzalez | patch set | 15:19 |
| Jeffrey4l_ | egonzalez, could u review this again? | 15:19 |
| Jeffrey4l_ | https://review.openstack.org/442519 | 15:19 |
| Jeffrey4l_ | sup pbourke | 15:21 |
| pbourke | hey Jeffrey4l_ | 15:21 |
| Jeffrey4l_ | could u review this patch ? https://review.openstack.org/442519 | 15:21 |
| *** manheim_ has joined #openstack-kolla | 15:21 | |
| *** matrohon has quit IRC | 15:23 | |
| *** salv-orlando has quit IRC | 15:24 | |
| pbourke | done | 15:25 |
| *** duonghq has quit IRC | 15:26 | |
| egonzalez | Jeffrey4l_, approved libvirt patch | 15:28 |
| egonzalez | upgrade worked | 15:28 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix booting from volume failure https://review.openstack.org/442519 | 15:28 |
| *** duonghq has joined #openstack-kolla | 15:29 | |
| Jeffrey4l_ | egonzalez, roger. thanks. | 15:29 |
| openstackgerrit | Merged openstack/kolla-ansible master: Make qemu use nova user https://review.openstack.org/442850 | 15:30 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: [TEST] TEST UBUNTU OCATA PROPOSE REPO https://review.openstack.org/443166 | 15:32 |
| egonzalez | Jeffrey4l_, can you remove -2 https://review.openstack.org/443144 | 15:37 |
| Jeffrey4l_ | sure. done. | 15:37 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix dbPath typo in templates/mongodb.json.j2 https://review.openstack.org/443160 | 15:38 |
| egonzalez | Jeffrey4l_, cherry-pick maintain track of the change in master(being said, a new change is made in parent PS, cherry-pick still using the older or the new change) | 15:39 |
| Jeffrey4l_ | egonzalez, which patch are u talking? | 15:40 |
| Jeffrey4l_ | cherry-pick share the same change id. this is the key point. | 15:41 |
| Jeffrey4l_ | (cherry picked from commit d06efcecc52b00f3e26dce3016accd04f6a52793) in commit message is unnecessary. | 15:41 |
| *** matrohon has joined #openstack-kolla | 15:42 | |
| duonghq | egonzalez, about the Ceilometer ps, in theory, db user can be granted Create privilege, so I think it does not only check for present of the database? | 15:42 |
| Jeffrey4l_ | zuul is very busy today ;/ | 15:43 |
| egonzalez | Jeffrey4l_, just have that doubt, didn't know that git behaviour. Thanks | 15:44 |
| Jeffrey4l_ | np ;) | 15:44 |
| Jeffrey4l_ | change id is used for gerrit and "cherry picked from ..." is useful for human ;p | 15:45 |
| *** adrian_otto has joined #openstack-kolla | 15:46 | |
| egonzalez | duonghq, what i'm saying in ceilometer ps is | 15:46 |
| *** adrian_otto has quit IRC | 15:47 | |
| egonzalez | duonghq, the first task checks with ceilometer user if have access. If yes, do nothing. | 15:47 |
| inc0 | good morning | 15:47 |
| Jeffrey4l_ | egonzalez, another patch need review ;) https://review.openstack.org/442974 | 15:47 |
| *** mattmceuen has quit IRC | 15:47 | |
| Jeffrey4l_ | morning inc0 | 15:47 |
| egonzalez | duonghq, if not, the next task create db and the next create user and permissions | 15:47 |
| inc0 | egonzalez: did you check nova upgrades by any chance? | 15:47 |
| egonzalez | inc0, yep | 15:47 |
| inc0 | Jeffrey4l_: how are we going with release? | 15:47 |
| Jeffrey4l_ | inc0, still wait for merged last few patch. | 15:48 |
| duonghq | egonzalez, the 1st task also create db if the given user has enough privileges? | 15:48 |
| duonghq | egonzalez, I think it's a possible | 15:48 |
| egonzalez | duonghq, is using ceilometer user, not admin user. if ceilometer user is already created in the external db, the db should be already created imo | 15:49 |
| duonghq | egonzalez, it's normally in that case | 15:50 |
| duonghq | but not 100% | 15:50 |
| Jeffrey4l_ | egonzalez, what duonghq said is possible. | 15:50 |
| Jeffrey4l_ | imo | 15:50 |
| *** mkoderer has joined #openstack-kolla | 15:51 | |
| Jeffrey4l_ | inc0, check this https://review.openstack.org/442974 | 15:51 |
| Jeffrey4l_ | ignore. i see. | 15:51 |
| egonzalez | well, is just a task name :) | 15:52 |
| *** sudipto has quit IRC | 15:52 | |
| *** sudipto_ has quit IRC | 15:52 | |
| duonghq | I want changing it due to it was misleading for me | 15:53 |
| *** bpattewar_ has joined #openstack-kolla | 15:54 | |
| inc0 | meeting in 5min | 15:55 |
| Jeffrey4l_ | duonghq, you are in utc+7 right? | 15:55 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix Ceilometer cannot create MySQL database https://review.openstack.org/438838 | 15:55 |
| duonghq | Jeffrey4l_, yup | 15:56 |
| duonghq | morning inc0 | 15:56 |
| *** kbaegis has joined #openstack-kolla | 15:56 | |
| *** l4yerffeJ has joined #openstack-kolla | 15:56 | |
| egonzalez | duonghq, guess that need to be backported to ocata right? | 15:56 |
| kbaegis | :'( Well, after much deliberation I decided to nuke my gentoo box. You guys forced me onto ubuntu | 15:57 |
| Jeffrey4l_ | got. | 15:57 |
| duonghq | egonzalez, yup | 15:57 |
| inc0 | egonzalez: all that stuff has to be :S | 15:57 |
| duonghq | kbaegis, I'm still using a arch box | 15:57 |
| Jeffrey4l_ | arch +1 | 15:57 |
| kbaegis | duonghq: Might have more luck with arch. Emerge doesn't like pip | 15:57 |
| kbaegis | The paths all get messed up. | 15:58 |
| duonghq | ah, | 15:58 |
| duonghq | guess that slot of emerge do not like pip? | 15:58 |
| des_consolado | I've added this neutron-bgp-dragent image and it should all work... everything is running through (kolla-ansible) to the end with no errors, but then when it's finished it keeps restarting because neutron-bgp-agent the binary isn't there, because none of the source seems to be installed into the image... | 15:58 |
| des_consolado | I've added the tarball location to the common/config.py file and I'm kolla-build'ing with -t source | 15:59 |
| spsurya__ | meeting starting | 15:59 |
| kbaegis | duonghq: Yeah, who knew that a python package management system wouldn't get along with another python package management system without venv? | 15:59 |
| egonzalez | duonghq, dunno why but gerrit dont let me cherry-pick, says is already closed | 15:59 |
| kbaegis | Oh, I'll shut up :) | 15:59 |
| kbaegis | Meeting | 15:59 |
| *** Jeffrey4l_ is now known as Jeffrey4l | 15:59 | |
| *** adrian_otto has joined #openstack-kolla | 15:59 | |
| duonghq | egonzalez, https://review.openstack.org/#/c/438842/ -> picked :P | 16:00 |
| egonzalez | duonghq, oo thanks was already cherry-picked :D | 16:01 |
| duonghq | egonzalez, can you review this too? | 16:01 |
| zhubingbing | meeting? | 16:02 |
| duonghq | zhubingbing, @ #openstack-meeting-4, yes | 16:02 |
| des_consolado | maybe a better question: is there anything else I need to do apart from adding something like this into this file: https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L462-L465 | 16:02 |
| des_consolado | in order to get software installed in an image when installing from source | 16:03 |
| Jeffrey4l | any idea about https://bugs.launchpad.net/kolla-ansible/+bug/1671143 egonzalez | 16:03 |
| openstack | Launchpad bug 1671143 in kolla-ansible pike "openstack_release auto error with pull images" [Undecided,New] | 16:03 |
| egonzalez | Jeffrey4l, had no time to look into it | 16:04 |
| Jeffrey4l | roger. | 16:04 |
| egonzalez | dunno why but the fact is set with the correct version, seems like default/main of the roles is not catching the fact | 16:04 |
| *** adrian_otto1 has joined #openstack-kolla | 16:05 | |
| *** adrian_otto has quit IRC | 16:07 | |
| *** ioannis is now known as chthon | 16:08 | |
| *** chthon is now known as yannis | 16:08 | |
| *** Murali has joined #openstack-kolla | 16:09 | |
| *** yannis has left #openstack-kolla | 16:10 | |
| *** rabel has quit IRC | 16:12 | |
| inc0 | right, so let's see where are we | 16:13 |
| Jeffrey4l | egonzalez, i never saw this before. and gate is green too. | 16:13 |
| *** nathharp has joined #openstack-kolla | 16:13 | |
| inc0 | egonzalez: frankly? ceilometer changes shouldnt block release | 16:13 |
| Jeffrey4l | egonzalez, can u reproduce this issue? | 16:13 |
| inc0 | imho | 16:13 |
| Jeffrey4l | which ansible version are u using? | 16:13 |
| duonghq | egonzalez, https://review.openstack.org/#/c/438842/ -> can you W+1? | 16:13 |
| inc0 | if we merge all else, let's tag rc and release | 16:13 |
| egonzalez | Jeffrey4l, 2.2.1.0 | 16:14 |
| inc0 | I think we'll release 4.0.1 sooner than usual | 16:14 |
| *** Jeffrey4l has quit IRC | 16:15 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 16:15 | |
| egonzalez | Jeffrey4l_, what issue are you refering if I can reproduce? | 16:15 |
| Jeffrey4l_ | inc0, yes and on the other hand, we should release kolla minor version regularly. like two month? | 16:16 |
| Jeffrey4l_ | egonzalez, openstack_release:auto | 16:16 |
| egonzalez | yes, tried now in centos source and the same | 16:16 |
| Jeffrey4l_ | and deploy works? | 16:17 |
| egonzalez | nope | 16:17 |
| Jeffrey4l_ | you can try add a - debug task after the set_facts. | 16:18 |
| egonzalez | The requested image does not exist: 192.168.100.215:4000/lokolla/centos-source-fluentd:auto | 16:18 |
| Jeffrey4l_ | i see. openstack_release is not configure properly. | 16:18 |
| inc0 | Jeffrey4l_: I think we can do both, I'd rather break 2-month rule than keep people with broken code | 16:19 |
| *** trinaths has joined #openstack-kolla | 16:19 | |
| Jeffrey4l_ | agree. | 16:19 |
| wirehead_ | inc0: https://www.flickr.com/photos/cyberspace/albums/72157676133724500 | 16:20 |
| *** salv-orlando has joined #openstack-kolla | 16:20 | |
| duonghq | wirehead_, which camera are you using? | 16:21 |
| wirehead_ | Olympus E-M5 Mk II | 16:21 |
| trinaths | wirehead_: nice snaps | 16:21 |
| egonzalez | Jeffrey4l_, http://paste.openstack.org/show/601958/ not even in the same task work http://paste.openstack.org/show/601958/ | 16:21 |
| wirehead_ | I finally finished editing everything from Barcelona. Really wish I would have gotten around to lining up the Kolla cores for a group shot. | 16:21 |
| egonzalez | note, first debug is inside the same Play, second outside | 16:22 |
| Jeffrey4l_ | weird | 16:22 |
| egonzalez | weird thing is that pbr works, get the version properly | 16:22 |
| Jeffrey4l_ | 4.0.0 is correct. | 16:23 |
| *** trinaths1 has joined #openstack-kolla | 16:23 | |
| Jeffrey4l_ | because 4.0.0 is not release. | 16:23 |
| inc0 | nice wirehead_ ,thanks, some of these looks familiar;) | 16:23 |
| Jeffrey4l_ | when 4.0.0 tag is created. pbr will return 4.0.1 | 16:23 |
| Jeffrey4l_ | set_fact is bust? ;( | 16:24 |
| egonzalez | yep, saying that works fine. in debug get the 4.0.0 | 16:24 |
| egonzalez | maybe is burst set_fact to the same var name, cannot override | 16:25 |
| Jeffrey4l_ | it is impossible. | 16:25 |
| *** salv-orlando has quit IRC | 16:25 | |
| Jeffrey4l_ | how abut you comment openstack_release:auto ? | 16:25 |
| Jeffrey4l_ | and remove the when: openstack_release == "auto" | 16:26 |
| *** trinaths has quit IRC | 16:26 | |
| *** zhubingbing has quit IRC | 16:26 | |
| egonzalez | commenting in globals works | 16:26 |
| *** crushil has quit IRC | 16:26 | |
| egonzalez | in group_vars is auto too but works | 16:27 |
| Jeffrey4l_ | set_fact should have the highest priority. | 16:27 |
| Jeffrey4l_ | wait. | 16:29 |
| *** crushil has joined #openstack-kolla | 16:30 | |
| Jeffrey4l_ | i got what's going on. | 16:30 |
| Jeffrey4l_ | globals.yml override the variable in another playbook. | 16:30 |
| Jeffrey4l_ | egonzalez, check http://docs.ansible.com/ansible/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable | 16:31 |
| Jeffrey4l_ | extra vars (-e in the command line) always win | 16:31 |
| Jeffrey4l_ | so globals.yml file win | 16:31 |
| Jeffrey4l_ | this is what ansible expected. we can do nothing in kolla. | 16:32 |
| Jeffrey4l_ | But a better way may be remove the confused "openstack_release: auto" in globals.yml file. | 16:33 |
| Jeffrey4l_ | or just leave "#openstack_release: " | 16:33 |
| inc0 | agree | 16:34 |
| inc0 | that being said, please not as release blocker;) | 16:34 |
| inc0 | Jeffrey4l_: lets make it medium bug plz | 16:35 |
| *** gfhellma has joined #openstack-kolla | 16:35 | |
| inc0 | can I have review on this please? https://review.openstack.org/#/c/442974/ | 16:35 |
| inc0 | and, when we merge it, on backport? | 16:35 |
| *** rmart04 has quit IRC | 16:35 | |
| inc0 | after this I think we're good to release | 16:36 |
| inc0 | (sorry egonzalez, mongodb patches are important, but we're past due with release already:() | 16:36 |
| egonzalez | inc0, the important is this which break mongo upgrade https://review.openstack.org/#/c/443180/ and is merging | 16:37 |
| inc0 | right, ok, we need to backport it too | 16:38 |
| egonzalez | that was a backport ;) | 16:38 |
| Jeffrey4l_ | is this critical https://review.openstack.org/439024 ? | 16:38 |
| Jeffrey4l_ | egonzalez, | 16:39 |
| egonzalez | nope, is in pike | 16:39 |
| Jeffrey4l_ | roger | 16:39 |
| egonzalez | but let me ensure | 16:40 |
| inc0 | so review+merge this backport and wait for all the gates? https://review.openstack.org/#/c/443228/1 | 16:41 |
| inc0 | Jeffrey4l_: can you handle this few more minutes to tag a release?:) | 16:41 |
| egonzalez | Jeffrey4l_, in my ocata env I can change quotas properly. In master fails | 16:41 |
| egonzalez | *nova quotas | 16:41 |
| l4yerffeJ | inc0, np. | 16:42 |
| l4yerffeJ | egonzalez, roger. thanks. | 16:42 |
| egonzalez | there is another issue in nova with quotas not cleaned in db, but is on their side | 16:42 |
| l4yerffeJ | wjat | 16:42 |
| l4yerffeJ | what's the result? egonzalez ? | 16:42 |
| egonzalez | let me find the log | 16:42 |
| inc0 | l4yerffeJ: https://review.openstack.org/#/c/443228/ | 16:44 |
| l4yerffeJ | inc0, cool. wait for +w | 16:45 |
| egonzalez | Jeffrey4l_, https://bugs.launchpad.net/nova/+bug/1668267 | 16:45 |
| openstack | Launchpad bug 1668267 in OpenStack Compute (nova) "Nova fails to correctly update the quota when deleting lots of VMs, some of which previously failed due to a quota error" [High,In progress] - Assigned to Pushkar Umaranikar (pushkar-umaranikar) | 16:45 |
| egonzalez | IRC log http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-03-06.log.html#t2017-03-06T13:43:12 | 16:45 |
| l4yerffeJ | egonzalez, great. i saw this issue too. | 16:46 |
| inc0 | l4yerffeJ: gates are slow today:/ | 16:46 |
| Jeffrey4l_ | inc0, yes. a busy day ;) | 16:47 |
| *** JoseMello has joined #openstack-kolla | 16:47 | |
| inc0 | yay timeout on docs check https://review.openstack.org/#/c/441030/ | 16:49 |
| Jeffrey4l_ | rechecking. | 16:49 |
| openstackgerrit | Merged openstack/kolla-ansible master: Disable running playbooks in serial by default https://review.openstack.org/442974 | 16:51 |
| *** fooliouno has joined #openstack-kolla | 16:51 | |
| inc0 | ok, updated etherpad | 16:52 |
| inc0 | 2 changes just waiting for gates | 16:52 |
| inc0 | pbourke: around? | 16:53 |
| pbourke | inc0: yup | 16:53 |
| inc0 | I've seen few checks failing on timeout on oraclelinux | 16:54 |
| Jeffrey4l_ | inc0, could u review https://review.openstack.org/#/c/442538/ ? even though it is not critical. | 16:54 |
| inc0 | keep an eye on it will you? | 16:54 |
| pbourke | inc0: will do | 16:54 |
| inc0 | it's not all of them, but always oracle | 16:54 |
| *** rhallisey has quit IRC | 16:54 | |
| pbourke | the only ones I've seen failing lately have been ubuntu | 16:54 |
| *** spsurya__ has quit IRC | 16:55 | |
| inc0 | yeah ubuntu was constant fail and we fixed it already | 16:55 |
| inc0 | let me find an example | 16:56 |
| kfox1111 | morning. | 16:56 |
| inc0 | nvm I'll get back to you, that might be hallucination from malnutrition on my side | 16:57 |
| egonzalez | inc0 pbourke , can you review https://review.openstack.org/#/c/443144/ | 16:58 |
| *** athomas has quit IRC | 16:58 | |
| des_consolado | surely there's some piece of code that uses pip to install any software downloaded as a tarball when using kolla-build with -t source? | 16:58 |
| *** salv-orlando has joined #openstack-kolla | 16:59 | |
| fooliouno | Hello .. I'm trying to deploy using kolla-kubernetes and am stuck with a mariadb error. Could someone please help point to what may be wrong. Here is the log from the mariadb pod: | 16:59 |
| fooliouno | {"log":"Enter current password for root (enter for none): \r\n","stream":"stdout","time":"2017-03-08T16:50:54.101498954Z"} {"log":"ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)\r\n","stream":"stdout","time":"2017-03-08T16:50:54.110725571Z"} {"log":"Enter current password for root (enter for none): \n","stream":"stdout","time":"2017-03-08T16:51:04.11425458Z"} {"log":"Failed to get 'Set ro | 17:00 |
| *** duonghq has quit IRC | 17:00 | |
| fooliouno | I'm just following instructions from the kolla-kubernetes guides | 17:00 |
| des_consolado | does kolla-kubernetes use kolla-genpwd? | 17:00 |
| des_consolado | fooliouno: ^^ | 17:00 |
| fooliouno | yes .. I used the script to generate the passwords | 17:01 |
| fooliouno | I see an entry for database password as well | 17:01 |
| des_consolado | maybe try manually logging into the db with creds from your passwords.yml to confirm that works | 17:02 |
| fooliouno | des_consolado: Thanks! I will try that | 17:02 |
| Jeffrey4l_ | last one pbourke please ack https://review.openstack.org/#/c/443228/ | 17:03 |
| inc0 | pbourke: https://review.openstack.org/#/c/441030/ <- this one timed out but that might be bad luck | 17:05 |
| pbourke | inc0: seems pulling the image tarball took over 20 minutes... | 17:11 |
| inc0 | right... | 17:11 |
| pbourke | only 5 mins in the centos case | 17:12 |
| Jeffrey4l_ | it depends. | 17:12 |
| pbourke | so I guess yeah just unlucky, if you spot another one search for 'wget -q -c -O /tmp' and look at the timestamps | 17:13 |
| *** skramaja has quit IRC | 17:15 | |
| Jeffrey4l_ | seems all critical patches are +w, simple wait for the merge. | 17:16 |
| *** rwellum_ has quit IRC | 17:16 | |
| *** rwellum_ has joined #openstack-kolla | 17:16 | |
| *** rabel has joined #openstack-kolla | 17:18 | |
| *** mnasiadka has joined #openstack-kolla | 17:20 | |
| inc0 | cool | 17:20 |
| inc0 | on the bright note, one of our pike priorities is full upgrade gates on multinode | 17:20 |
| inc0 | and god, we need this. | 17:21 |
| openstackgerrit | Merged openstack/kolla-kubernetes master: Helm charts for heat/heat-cfn microservices https://review.openstack.org/414496 | 17:21 |
| Jeffrey4l_ | +++ | 17:22 |
| inc0 | with tarballs-registry we can do this properly | 17:23 |
| inc0 | deploy stable and upgrade to master | 17:23 |
| inc0 | also I'm working with few people to create proper test set for it | 17:23 |
| *** gfidente has quit IRC | 17:25 | |
| Jeffrey4l_ | inc0, that will be cool. | 17:27 |
| *** mnasiadka has quit IRC | 17:33 | |
| *** trinaths1 has quit IRC | 17:33 | |
| *** trinaths has joined #openstack-kolla | 17:34 | |
| *** nathharp has quit IRC | 17:35 | |
| *** trinaths1 has joined #openstack-kolla | 17:35 | |
| *** trinaths has quit IRC | 17:38 | |
| egonzalez | Jeffrey4l_, all changes in queue are merged | 17:39 |
| Jeffrey4l_ | egonzalez, cool. | 17:40 |
| Jeffrey4l_ | i have three test patch are running. seem not bad ;) | 17:40 |
| jemcevoy | inc0 I just logged bug 167117 on on the heka jason.log issue https://bugs.launchpad.net/kolla-ansible/+bug/1671177 | 17:41 |
| openstack | Launchpad bug 1671177 in kolla-ansible "heka generating huge json log with read permission errors" [Undecided,New] | 17:41 |
| openstack | bug 167117 in Inkscape "Kerning lost on editing" [Medium,Fix released] https://launchpad.net/bugs/167117 - Assigned to Richard Hughes (cyreve) | 17:41 |
| *** deadnull has joined #openstack-kolla | 17:41 | |
| *** manheim_ has quit IRC | 17:41 | |
| Jeffrey4l_ | hrm i am thinking to run heka/fluentd by using root user. | 17:42 |
| inc0 | Jeffrey4l_: I think all the stuff merged | 17:43 |
| Jeffrey4l_ | yes. | 17:43 |
| inc0 | sooo...release?:) | 17:43 |
| Jeffrey4l_ | OK. | 17:44 |
| *** shardy has quit IRC | 17:45 | |
| *** vhosakot has joined #openstack-kolla | 17:47 | |
| *** manheim_ has joined #openstack-kolla | 17:48 | |
| *** manheim_ has quit IRC | 17:48 | |
| Jeffrey4l_ | inc0, done. https://review.openstack.org/443249 https://review.openstack.org/443252 | 17:49 |
| *** srwilkers has quit IRC | 17:52 | |
| inc0 | cool, thanks | 17:52 |
| inc0 | also unblock https://review.openstack.org/#/c/442137/ please | 17:53 |
| inc0 | I'll take it from there, thanks Jeffrey4l_ | 17:54 |
| Jeffrey4l_ | seems the final release is done by tc. | 17:54 |
| inc0 | yeah, I'll work with Doug on this | 17:54 |
| inc0 | just remove -1 | 17:54 |
| inc0 | and get some rest | 17:54 |
| inc0 | :) | 17:54 |
| Jeffrey4l_ | done. | 17:55 |
| egonzalez | ups found a missing backport to kolla ocata, is not that critical (centos binary barbican issue) https://review.openstack.org/#/c/443253/ | 17:55 |
| inc0 | egonzalez: we'll do it in 4.0.1 | 17:55 |
| egonzalez | yep | 17:55 |
| inc0 | thank you all for this! | 17:55 |
| Jeffrey4l_ | check this https://review.openstack.org/434637 :) | 17:56 |
| Jeffrey4l_ | not bad, right? | 17:56 |
| inc0 | nice! | 17:58 |
| egonzalez | Jeffrey4l_, kolla-ansible is not using all docker ps -a filters, and also barbican_api fails in centos binary and gate is green | 17:58 |
| *** mgoddard has quit IRC | 17:58 | |
| inc0 | we also need multinode+ceph+cinder | 17:58 |
| Jeffrey4l_ | oh, right. | 17:58 |
| inc0 | so I'll wait for zuulv3 with all the multinode stuff | 17:59 |
| Jeffrey4l_ | we need lots of jobs for patch ( another issue ) | 17:59 |
| Jeffrey4l_ | another one, heat related services https://review.openstack.org/434636 | 18:02 |
| *** matrohon has quit IRC | 18:03 | |
| Jeffrey4l_ | egonzalez, wether is the docker filter patch? seem i can not find it. | 18:04 |
| egonzalez | nor me, im looking for it :/ | 18:05 |
| *** pramodrj07 has joined #openstack-kolla | 18:06 | |
| *** MasterOfBugs has joined #openstack-kolla | 18:06 | |
| egonzalez | Jeffrey4l_, https://review.openstack.org/#/c/438227/ | 18:06 |
| egonzalez | really is this https://review.openstack.org/#/c/438228/ | 18:06 |
| *** rabel has quit IRC | 18:06 | |
| Jeffrey4l_ | yes. thanks. | 18:07 |
| Jeffrey4l_ | let me backport it. | 18:07 |
| egonzalez | OK | 18:08 |
| Jeffrey4l_ | aha, i got what is happening. | 18:09 |
| Jeffrey4l_ | for the first patch, i pushed into wrong branch , then abandoned. | 18:09 |
| egonzalez | hrm, the same happend to me with keystone upgrade right? | 18:10 |
| Jeffrey4l_ | so following cherry-pick do not works. | 18:10 |
| Jeffrey4l_ | iirc, you are using cycle depends. | 18:10 |
| egonzalez | yeah | 18:10 |
| Jeffrey4l_ | not the same. but the same result ;( | 18:11 |
| *** egonzalez has quit IRC | 18:14 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 18:17 |
| *** Murali has quit IRC | 18:18 | |
| *** srwilkers has joined #openstack-kolla | 18:20 | |
| *** unicell has joined #openstack-kolla | 18:23 | |
| *** kollaFun has joined #openstack-kolla | 18:27 | |
| *** kollaFun has quit IRC | 18:28 | |
| *** m1dev has quit IRC | 18:32 | |
| *** sambetts is now known as sambetts|afk | 18:36 | |
| *** kbaegis has quit IRC | 18:44 | |
| *** mgoddard has joined #openstack-kolla | 18:47 | |
| *** kbaegis has joined #openstack-kolla | 18:50 | |
| *** kbaegis has quit IRC | 18:50 | |
| *** srwilkers has quit IRC | 18:51 | |
| *** srwilkers has joined #openstack-kolla | 18:52 | |
| *** crushil has left #openstack-kolla | 18:54 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Remove cell setup and db creation nova upgrade https://review.openstack.org/443290 | 18:56 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Set empty openstack_release option https://review.openstack.org/443291 | 19:04 |
| *** vhosakot has quit IRC | 19:08 | |
| *** vhosakot has joined #openstack-kolla | 19:08 | |
| *** rwsu has quit IRC | 19:09 | |
| *** pcaruana has quit IRC | 19:13 | |
| inc0 | also, to all out ladies here, happy International Womans Day! | 19:21 |
| *** salv-orl_ has joined #openstack-kolla | 19:23 | |
| *** breitz has quit IRC | 19:25 | |
| *** salv-orlando has quit IRC | 19:26 | |
| *** breitz has joined #openstack-kolla | 19:26 | |
| *** srwilkers has quit IRC | 19:27 | |
| *** mkoderer has quit IRC | 19:27 | |
| *** srwilkers has joined #openstack-kolla | 19:30 | |
| Mannheim | hello | 19:30 |
| *** Manheim has quit IRC | 19:31 | |
| *** g3ek has quit IRC | 19:32 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Copy pxelinux.0 file to tftpboot https://review.openstack.org/443300 | 19:37 |
| *** salv-orl_ has quit IRC | 19:37 | |
| *** vhosakot has quit IRC | 19:40 | |
| *** mgoddard has quit IRC | 19:40 | |
| *** g3ek has joined #openstack-kolla | 19:41 | |
| *** sayantan_ has joined #openstack-kolla | 19:41 | |
| *** vhosakot has joined #openstack-kolla | 19:43 | |
| *** vhosakot has quit IRC | 19:44 | |
| *** kollaFUn has joined #openstack-kolla | 19:46 | |
| *** kollaFUn has quit IRC | 19:47 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 19:48 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Copy pxelinux.0 file to tftpboot https://review.openstack.org/443300 | 19:50 |
| *** srwilkers has quit IRC | 19:50 | |
| sbezverk | kfox1111: ping | 19:50 |
| *** alop has joined #openstack-kolla | 19:53 | |
| *** vhosakot has joined #openstack-kolla | 19:54 | |
| kfox1111 | sbezverk: hi | 19:56 |
| kfox1111 | whats up? | 19:56 |
| sbezverk | kfox1111: well while testing ironic I hit interesting issue | 19:57 |
| sbezverk | would like to know your opinion | 19:57 |
| kfox1111 | sure | 19:57 |
| sbezverk | kfox1111: when baremetal vp gets started neutron tries to bind a port and tries to communicate with ovs agent which runs on the host where the resource is located | 19:58 |
| sbezverk | it works fine for normal compute | 19:58 |
| sbezverk | but ironic compute runs as a regules not net=host container | 19:59 |
| sbezverk | when neutron tries to contact it for openvswitch agent | 19:59 |
| sbezverk | it fails because there is none | 20:00 |
| kfox1111 | right... | 20:00 |
| sbezverk | it seems associatation between compute resource and openvswitch is by hostname | 20:00 |
| kfox1111 | is this ironic-compute or nova-ironic-compute? | 20:00 |
| sbezverk | nova-compute-ironic | 20:00 |
| kfox1111 | hmm... | 20:01 |
| sbezverk | I chatted with ironic folks and one crazy idea was to run neutron-openswitch-agent as a side car container to nova-compute-ironic | 20:01 |
| sbezverk | it will be kind of dummy container | 20:01 |
| sbezverk | just to register with neutron | 20:01 |
| sbezverk | so this check for agent presence will not fail | 20:02 |
| kfox1111 | I'm a bit weirded out why it woudl even try talking to neutron. | 20:02 |
| sbezverk | also this container will not run with net-host and will share the same name as nova-compute-ironic | 20:02 |
| sbezverk | kfox1111: well that part I cannot explain other than it happens | 20:03 |
| kfox1111 | I'm wondering if its just a misconfiguration though. | 20:03 |
| sbezverk | kfox1111: does not seem to be the case as I ironic people see the same results | 20:04 |
| kfox1111 | that maybe a configmap is generated a bit too vm specific right now. | 20:04 |
| kfox1111 | ah. | 20:04 |
| *** vhosakot has quit IRC | 20:04 | |
| kfox1111 | so, known bug that needs to be worked around... | 20:04 |
| sbezverk | kfox1111: they confirmed that communicating with neutron for port binding is expected | 20:04 |
| kfox1111 | oh, I buy it needs to talk to neutron. just not that it needs ovs. | 20:05 |
| sbezverk | it was by design, and in future it will change | 20:05 |
| kfox1111 | ah. | 20:05 |
| sbezverk | they will come up with ironic ml2 driver | 20:05 |
| kfox1111 | so, we need to work around it... | 20:05 |
| kfox1111 | ah. | 20:05 |
| sbezverk | so no more dependency on ovs | 20:05 |
| kfox1111 | ok. so... it needs a ovs agent to talk to... | 20:05 |
| sbezverk | but for now there is a need for solution :) | 20:06 |
| sbezverk | kfox1111: my thinking is if it needs just to talk to | 20:06 |
| kfox1111 | sidecar would work, but would run into the too many rpc services thing. so two possible options I see. | 20:06 |
| sbezverk | then maybe neutron-ovs-agent running as a side car might actually work | 20:06 |
| kfox1111 | statefulset + sidecar, or daemonset net=host ? | 20:06 |
| kfox1111 | can you run more then one nova-compute per ironic set yet? | 20:07 |
| sbezverk | kfox1111: net=host will not work for sure | 20:08 |
| sbezverk | as then we cannot have nova-copmute and nova-compute-ironic on the same box | 20:08 |
| *** nathharp has joined #openstack-kolla | 20:08 | |
| kfox1111 | why not? we do nova-compute's... | 20:08 |
| kfox1111 | oh. your trying to put them in the same system... | 20:08 |
| sbezverk | since they will use the same name and nova goes crazy | 20:08 |
| sbezverk | I already tried it | 20:08 |
| *** srwilkers has joined #openstack-kolla | 20:08 | |
| kfox1111 | yeah. if vm and ironic are on the same system, that would be a problem. | 20:09 |
| kfox1111 | hmm... | 20:09 |
| kfox1111 | so... here's the rub... | 20:10 |
| sbezverk | kfox1111: it should work, there is no any arch thing peventing it | 20:10 |
| sbezverk | only software design, which needs to be bend a little ;) | 20:10 |
| kfox1111 | having two neutron-openvswitch-agents on the same host talking to the same ovs still may cause issues. even if you could isolate nova-computes. | 20:10 |
| kfox1111 | and I think there can only be one ovs per kernel. | 20:11 |
| kfox1111 | it may be an architectural limitation that nova-compute-vm and nova-compute-ironic just cant be made to co'exist on the same box. | 20:11 |
| sbezverk | kfox1111: ovs will be only one | 20:11 |
| sbezverk | but neutron-openvswitcvh-agent could be several | 20:11 |
| kfox1111 | until they get an ironicc ml2 driver. | 20:11 |
| kfox1111 | but will neutron-openvswitch-agent fight with each other for bridge ownerships? | 20:12 |
| sbezverk | kfox1111: for sure driver looks like much clean solution | 20:12 |
| kfox1111 | and openflow rules. | 20:12 |
| sbezverk | kfox1111: not sure | 20:13 |
| kfox1111 | I would bet they never considered that use case. :/ | 20:13 |
| kfox1111 | so until ironic gets an ml2 driver, I'm thinking we probably should just recommend people not mix nova-compute-vm and nova-compute-ironic on the same host. | 20:14 |
| kfox1111 | if thats a documented limitation, both options are still on the table. | 20:14 |
| sbezverk | kfox1111: right but you see putting nova-compute-ironic in net=host does not make sense since this process does not won any hardware | 20:14 |
| sbezverk | it is just a proxy | 20:14 |
| kfox1111 | it does want hardware though, as it requires openvswitch-agent. :/ | 20:14 |
| kfox1111 | it directly doesn't want it, but it is asking for more then it needs. :/ | 20:15 |
| sbezverk | kfox1111: yeah at least now it does :( | 20:15 |
| kfox1111 | if you really really want to try and get them to co'exist, I'd say try a statefulset with an openvswitch sidecar. but I think its probably playing with fire, sharing that openvswitch daemon. :/ | 20:15 |
| kfox1111 | if we just put it as a known limitation until ironic fixes the issue, | 20:16 |
| sbezverk | kfox1111: oh no, I think openvswitch is so damn delicate | 20:16 |
| sbezverk | it will bring so much grief I do not want even consider it | 20:16 |
| kfox1111 | and more then one nova-compute for ironic can exist, daemonset would be preffered I think. | 20:16 |
| *** sayantan_ has quit IRC | 20:16 | |
| kfox1111 | yeah. I agree. | 20:16 |
| *** sayantan_ has joined #openstack-kolla | 20:17 | |
| kfox1111 | https://blueprints.launchpad.net/nova/+spec/ironic-multiple-compute-hosts | 20:18 |
| sbezverk | kfox1111: I will experiment switching nova-compute-ironic to stateful set and try neutron-ovs-agent just for fun.. to see where it hicks up next | 20:18 |
| sbezverk | but hopefully ml2 driver will resolve it cleanly | 20:18 |
| kfox1111 | k. | 20:18 |
| kfox1111 | I think there is a slight chance it might work, if you can reconfigure all the bridge names it wants to lookat, and you configure it for vlans. | 20:19 |
| kfox1111 | +1 to ml2 driver fix. | 20:19 |
| *** rabel has joined #openstack-kolla | 20:19 | |
| kfox1111 | if not, the daemonset looks more attractive to me, as it should support rolling upgrades soon. | 20:19 |
| kfox1111 | until the ml2 driver is finished. | 20:20 |
| kfox1111 | then we can move it back to a deployment. | 20:20 |
| sbezverk | kfox1111: do you recall why we needed neutron-ovs-agent to run in net=host mode? | 20:20 |
| kfox1111 | for the hostname, and I think for socket sharing with ovs. | 20:21 |
| kfox1111 | but its been a while. | 20:21 |
| sbezverk | kfox1111: it talks to ovsdb or vswitchd via socket | 20:21 |
| kfox1111 | yeah. | 20:21 |
| sbezverk | so I can expose that socket inside a pod | 20:21 |
| kfox1111 | I believe so. | 20:22 |
| *** sayantan_ has quit IRC | 20:22 | |
| kfox1111 | hmmm.. that leads to an interesting idea... | 20:22 |
| kfox1111 | does ovs have a dummy/debug mode? | 20:22 |
| sbezverk | kfox1111: no idea, never heard though | 20:25 |
| sbezverk | kfox1111: just recalled ironic people told me that actually no ports gets plugged into the bridge | 20:27 |
| sbezverk | it is only to create a port in neutron with mac address of baremetal server | 20:27 |
| sbezverk | so maybe they will not conflict | 20:27 |
| sbezverk | and fight for bridges | 20:27 |
| kfox1111 | yeah. thats not what I'm worried about, but more the, openvswitch-agent comes up and tries to plumb in br-int, br-tun, etc. | 20:29 |
| kfox1111 | I don't think anything needs to bind for it to actively try creating bridges/flows. | 20:29 |
| *** oomichi has left #openstack-kolla | 20:29 | |
| sbezverk | kfox1111: right right it will try to create br-int and all other things :( | 20:30 |
| kfox1111 | I think its possible to rename br-int to something else that won't conflict. | 20:30 |
| kfox1111 | if you don't configure it to use vxlan's it shouldn't touch br-tun. | 20:30 |
| kfox1111 | not sure about the flow rules though. | 20:30 |
| kfox1111 | it may not mange any until it has ports to manage. | 20:31 |
| sbezverk | kfox1111: more we discuss, more I see that a driver is the way to go. because all other solutions will look like unstable hacks | 20:33 |
| *** Murali has joined #openstack-kolla | 20:33 | |
| kfox1111 | yeah. :/ | 20:34 |
| kfox1111 | I guess there's one more possibility. | 20:34 |
| kfox1111 | must it be ovs, or can it be linuxbridge? | 20:35 |
| sbezverk | kfox1111: well they asked about linuxbridge possibility, but we do not support it in kolla-k8s | 20:36 |
| *** salv-orlando has joined #openstack-kolla | 20:38 | |
| *** Murali has quit IRC | 20:39 | |
| *** kbaegis has joined #openstack-kolla | 20:42 | |
| kbaegis | Is the meeting over? | 20:42 |
| jascott1 | kfox1111 ping | 20:42 |
| kbaegis | I'll take that as a yes | 20:42 |
| *** salv-orlando has quit IRC | 20:42 | |
| kbaegis | Has anyone seen "AnsibleUndefinedVariable: 'dict object' has no attribute 'solum-application-deployment"? | 20:43 |
| kbaegis | I nuked gentoo and am playing with kolla as a good little boy | 20:43 |
| *** adrian_otto1 has quit IRC | 20:43 | |
| *** sayantan_ has joined #openstack-kolla | 20:46 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 20:47 |
| *** MasterOfBugs has quit IRC | 20:48 | |
| *** pramodrj07 has quit IRC | 20:48 | |
| *** manheim has joined #openstack-kolla | 20:50 | |
| *** MasterOfBugs has joined #openstack-kolla | 20:52 | |
| *** satyar has quit IRC | 20:59 | |
| *** kbaegis has quit IRC | 21:03 | |
| *** kbaegis has joined #openstack-kolla | 21:03 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Copy pxelinux.0 file to tftpboot https://review.openstack.org/443300 | 21:07 |
| *** rabel has quit IRC | 21:08 | |
| *** jtriley_ has joined #openstack-kolla | 21:17 | |
| *** fooliouno has quit IRC | 21:17 | |
| *** jtriley has quit IRC | 21:20 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 21:20 |
| *** rwallner has quit IRC | 21:22 | |
| *** jtriley_ has quit IRC | 21:23 | |
| *** sayantan_ has quit IRC | 21:24 | |
| *** adrian_otto has joined #openstack-kolla | 21:24 | |
| *** sayantan_ has joined #openstack-kolla | 21:25 | |
| *** JoseMello has quit IRC | 21:25 | |
| *** salv-orlando has joined #openstack-kolla | 21:31 | |
| *** l4yerffeJ has quit IRC | 21:33 | |
| *** l4yerffeJ has joined #openstack-kolla | 21:34 | |
| *** Jeffrey4l_ has quit IRC | 21:34 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 21:35 | |
| *** srwilkers has quit IRC | 21:46 | |
| *** adrian_otto has quit IRC | 21:50 | |
| *** fguillot has quit IRC | 21:50 | |
| *** MarginHu has joined #openstack-kolla | 21:54 | |
| *** MarginHu has joined #openstack-kolla | 21:54 | |
| *** mgoddard has joined #openstack-kolla | 22:03 | |
| *** l4yerffeJ_ has joined #openstack-kolla | 22:13 | |
| *** dave-mccowan has quit IRC | 22:13 | |
| *** l4yerffeJ has quit IRC | 22:13 | |
| *** nathharp has quit IRC | 22:16 | |
| *** kbaegis has quit IRC | 22:19 | |
| *** mgoddard has quit IRC | 22:25 | |
| *** jrobinson has joined #openstack-kolla | 22:27 | |
| kfox1111 | back. | 22:29 |
| *** rwsu has joined #openstack-kolla | 22:31 | |
| *** Murali has joined #openstack-kolla | 22:35 | |
| *** Murali has quit IRC | 22:38 | |
| *** Pavo has joined #openstack-kolla | 22:41 | |
| *** haplo37 has quit IRC | 22:42 | |
| *** Pavo has quit IRC | 22:44 | |
| *** Pavo has joined #openstack-kolla | 22:47 | |
| *** vhosakot has joined #openstack-kolla | 22:49 | |
| *** japestinho has quit IRC | 22:50 | |
| *** haplo37 has joined #openstack-kolla | 22:51 | |
| *** manheim has quit IRC | 22:53 | |
| *** rwallner has joined #openstack-kolla | 22:55 | |
| *** manheim has joined #openstack-kolla | 22:56 | |
| *** vhosakot has quit IRC | 22:58 | |
| *** vhosakot has joined #openstack-kolla | 22:59 | |
| *** rwallner has quit IRC | 22:59 | |
| masber | hi | 23:00 |
| masber | question | 23:00 |
| masber | according to this doc https://docs.openstack.org/developer/kolla-ansible/quickstart.html#deploy-kolla | 23:00 |
| masber | I can isntall kolla running this command --> pip install kolla-ansible | 23:01 |
| masber | however i am after version 3.0.2 because the 4.0 has not been released yet | 23:01 |
| masber | so, do I have to run this command instead? pip install kolla==3.0.2 | 23:01 |
| masber | is that right? | 23:02 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate https://review.openstack.org/441550 | 23:09 |
| *** Pavo has quit IRC | 23:10 | |
| *** Pavo has joined #openstack-kolla | 23:11 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Copy pxelinux.0 file to tftpboot https://review.openstack.org/443300 | 23:12 |
| *** manheim has quit IRC | 23:13 | |
| *** adrian_otto has joined #openstack-kolla | 23:15 | |
| *** manheim has joined #openstack-kolla | 23:16 | |
| *** Pavo has quit IRC | 23:19 | |
| *** saneax-_-|AFK is now known as saneax | 23:21 | |
| *** MarginHu has quit IRC | 23:22 | |
| *** MarginHu has joined #openstack-kolla | 23:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla master: Updated from global requirements https://review.openstack.org/443352 | 23:24 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements https://review.openstack.org/443353 | 23:24 |
| *** sayantan_ has quit IRC | 23:25 | |
| *** rwellum_ has quit IRC | 23:25 | |
| *** sayantan_ has joined #openstack-kolla | 23:25 | |
| *** manheim has quit IRC | 23:26 | |
| *** MarginHu has quit IRC | 23:27 | |
| *** adrian_otto has quit IRC | 23:29 | |
| *** schwicht has quit IRC | 23:30 | |
| *** g3ek has quit IRC | 23:30 | |
| *** adrian_otto has joined #openstack-kolla | 23:31 | |
| *** gfhellma_ has joined #openstack-kolla | 23:32 | |
| *** gfhellma has quit IRC | 23:35 | |
| *** g3ek has joined #openstack-kolla | 23:39 | |
| jemcevoy | inc0 - I am seeing lots and lots of ceilometer errors in kibaba. Do you know what might cause this to be logged? http://paste.openstack.org/show/602000/ | 23:39 |
| *** manheim has joined #openstack-kolla | 23:42 | |
| *** manheim has quit IRC | 23:43 | |
| jemcevoy | masber - last time I installed kolla I simply ran "pip install kolla" and after I copied the globals.yml to /etc/kolla I changed the openstack release to 'openstack_release: "3.0.2"' | 23:45 |
| *** jrobinson has quit IRC | 23:46 | |
| *** jmccarthy has quit IRC | 23:47 | |
| *** manheim has joined #openstack-kolla | 23:48 | |
| *** Mannheim has quit IRC | 23:49 | |
| *** jrich523 has joined #openstack-kolla | 23:51 | |
| jemcevoy | inc0 I updated the paste after seeing it start right after the rabbitmq came back... How do I clear that pesky wabbit | 23:55 |
| *** david-lyle has quit IRC | 23:56 | |
| *** gfhellma_ has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!