| manjeets | sdake not yet let me get some results from lb will try afterwards | 00:01 |
|---|---|---|
| sdake | cool when you try leave info in the review please | 00:02 |
| *** athomas has quit IRC | 00:03 | |
| manjeets | sdake sure | 00:03 |
| sdake | thanks manjeets | 00:04 |
| *** vhosakot has joined #openstack-kolla | 00:09 | |
| dave-mcc_ | sdake should vm boots now with latest patch? | 00:11 |
| sdake | dave-mcc_ my vm waits in build state | 00:12 |
| sdake | i am building from source | 00:12 |
| dave-mcc_ | is the behavior different with TLS? maybe that's enough for me to debug. | 00:12 |
| sdake | i have tls disabled | 00:13 |
| sdake | with tls enabled it definately never booted before | 00:13 |
| sdake | i think i have tls disabled let me double check | 00:13 |
| gmmaha | sdake: going to try and enable magnum on kolla deploy.. any pitfalls i should look for? | 00:13 |
| sdake | it doesn't work | 00:14 |
| gmmaha | aah :) | 00:14 |
| sdake | bug in 2..0.0 bug list | 00:14 |
| sdake | tls is disabled dave | 00:14 |
| gmmaha | and i am trying to deploy linux bridge again after manjeets got it working.. i am 84 commits ahead of him butnothing in the neutron roles directory | 00:15 |
| sdake | so not sure if this is my patch or what | 00:15 |
| sdake | what did manjeets have to do to get linux bridge working | 00:15 |
| sdake | gmmaha we are about t-10 days from having all these problems solved, based upon my professional experience | 00:15 |
| sdake | except possibly magnum | 00:16 |
| gmmaha | sdake: he said he had to clean everything in /etc/kolla/* in all the dpeloyment targets | 00:16 |
| gmmaha | for both neutron and nova | 00:16 |
| sdake | interesting | 00:17 |
| sdake | did that work for you? | 00:17 |
| gmmaha | going to test it now | 00:17 |
| sdake | when i clean up i copy my yml files to /tmp and rm -rf /et/ckolla and copy y eml files back in | 00:17 |
| sdake | gmmaha arey ou on master? | 00:17 |
| sdake | more or less? | 00:18 |
| gmmaha | sdake: more of less.. | 00:18 |
| sdake | and your vms boot? | 00:18 |
| gmmaha | i am 3 commits away | 00:18 |
| gmmaha | yeah | 00:18 |
| sdake | dave-mcc_ do you ahve tls enabled? with tls enabled, the symptom I saw was vms woudl not boot | 00:18 |
| pabelanger | sdake: ask this earlier but you were offline. | 00:18 |
| pabelanger | Greetings, wanted to see when kolla is expecting to tag its mitaka release | 00:18 |
| sdake | wound me | 00:18 |
| pabelanger | over in openstack-infra we're looking at migrating your devstack-centos7 jobs to centos-7 dibs. Which requires some downtime | 00:18 |
| gmmaha | i was able to get 150 Vms up.. though i only got 4 compute nodes | 00:18 |
| sdake | pabelanger we just slipped a week to the 15th | 00:18 |
| sdake | in today's meeting | 00:18 |
| pabelanger | Hmm... okay | 00:19 |
| dave-mcc_ | my vms are not booting (time out error) even with no tls and only single vip. | 00:19 |
| sdake | with your patch applied? | 00:19 |
| sdake | or with master | 00:19 |
| dave-mcc_ | master + your patch. | 00:19 |
| sdake | master also didn't work? | 00:20 |
| sdake | centos source or binary? | 00:20 |
| dave-mcc_ | same behavior for both. centos binary. | 00:20 |
| manjeets | with linux bridge first run of 100 vms and error rate is 100% | 00:20 |
| gmmaha | manjeets: holy cow | 00:21 |
| sdake | dave let me check if sbezverk's patch has merged | 00:21 |
| gmmaha | whats the error | 00:21 |
| manjeets | the problem is not even a single port got created | 00:21 |
| manjeets | debugging logs | 00:21 |
| gmmaha | manjeets: check if linux_bridgeagent container is running | 00:21 |
| gmmaha | that was the error i was hitting before | 00:21 |
| manjeets | it is | 00:21 |
| *** cbits has quit IRC | 00:23 | |
| sdake | did yo ureboot before switching deployment types? | 00:24 |
| sdake | oh i have linuxbridge enabled | 00:24 |
| sdake | my vms are not starting | 00:24 |
| manjeets | sdake i did 2 reboots | 00:24 |
| sdake | let me go back to ovs | 00:24 |
| manjeets | http://paste.openstack.org/show/493251/ | 00:24 |
| manjeets | i see warning in docker logs | 00:24 |
| *** daneyon has quit IRC | 00:25 | |
| manjeets | some version mess may be happening setuptools | 00:25 |
| sdake | ya setuptools is installed twice | 00:25 |
| sdake | i am aware of it | 00:26 |
| manjeets | what how it can be twice | 00:26 |
| manjeets | you mean 2 versions ? | 00:26 |
| sdake | yup pip upgrde setuptools will not upgrade setuptools that ships with centos | 00:26 |
| sdake | its a bug in pip | 00:26 |
| sdake | but really we shouldn't be installing setuptools | 00:26 |
| manjeets | are you sure of ubuntu | 00:27 |
| *** rhallisey has joined #openstack-kolla | 00:27 | |
| sdake | oh this is a centos problem | 00:27 |
| sdake | not sure on ubuntu | 00:27 |
| sdake | but uubuntu also installs binary setuptools i think | 00:27 |
| sdake | through jinja2 dependency | 00:27 |
| sdake | i can't fix 20 bugs a day guys | 00:27 |
| sdake | hang tight | 00:27 |
| sdake | i'm luckky if i cn do 2 a dy | 00:28 |
| manjeets | sdake ptl's should be ready to fix 100 a day | 00:28 |
| manjeets | lol | 00:28 |
| *** daneyon has joined #openstack-kolla | 00:28 | |
| sdake | ptl no differnet then any other contributor to the project | 00:32 |
| manjeets | setup tools where should be updated | 00:33 |
| manjeets | ? | 00:33 |
| manjeets | and which version is required | 00:33 |
| sdake | it can't be updated | 00:33 |
| manjeets | lb is messing with | 00:33 |
| manjeets | what ? | 00:33 |
| sdake | the only way to fix it is to remove it | 00:33 |
| sdake | in openstack-base/Dockerfile.j2 | 00:33 |
| manjeets | means it won't work with lb then | 00:33 |
| manjeets | ? | 00:33 |
| manjeets | if i don't have that lot of thing in neutron code are importing setup tools | 00:34 |
| sdake | manjeets i've got about 20 patches outstanding thats about the most i can have and maintain context on them all | 00:34 |
| *** absubram has quit IRC | 00:34 | |
| sdake | i just told you how to fix it, remove it with dpkg | 00:35 |
| sdake | in ubuntu or centos using rpm | 00:35 |
| manjeets | ok | 00:35 |
| sdake | in openstack-bae dockerfile | 00:36 |
| sdake | i did this rcently for kolla-toolbox | 00:36 |
| sdake | becaue it buildss from base | 00:36 |
| sdake | but it also needs to be done for openstack-base | 00:36 |
| manjeets | where is openstack-base i mean is it on all targets ? | 00:37 |
| sdake | in the docke rdirectory | 00:37 |
| sdake | openstack-base is used as a base image for all other openstack iamges | 00:38 |
| sdake | base is used as a bse image for non-openstack images | 00:39 |
| manjeets | in /var/lib/docker ? | 00:39 |
| *** jrist has quit IRC | 00:39 | |
| *** achanda has quit IRC | 00:40 | |
| sdake | manjeets i am helping dave atm | 00:40 |
| sdake | he is blocked on fixing bugs | 00:40 |
| sdake | look in th source tree | 00:40 |
| sdake | docker/openstack-base/dockerfile.j2 | 00:40 |
| vhosakot | manjeets: help me catch up.... i will help you... what is the isuue ? linux bridge ? | 00:40 |
| sdake | look at recent changes to dockerfile.j2 for kolla-toolbox | 00:41 |
| vhosakot | manjeets: i will check irc logs | 00:41 |
| gmmaha | manjeets: can you send us the full log | 00:41 |
| gmmaha | sdake: i will try and help vhosakot here | 00:41 |
| *** jrist has joined #openstack-kolla | 00:41 | |
| *** jrist has joined #openstack-kolla | 00:41 | |
| sdake | tia :) | 00:41 |
| vhosakot | cool... digging irc logs | 00:42 |
| sdake | dave-mcc_ do you hve this ocmmit in your repo? Adds migration step from nova to nova_api database | 00:43 |
| *** Marga_ has quit IRC | 00:43 | |
| gmmaha | vhosakot: when i deployed linuxbridge on ubuntu i got this. https://bugs.launchpad.net/kolla/+bug/1566543 | 00:43 |
| openstack | Launchpad bug 1566543 in kolla "linuxbridge deployment is broken with latest master (2f4a3f35c00277cc97e887ba647395a702268e84)" [Undecided,New] | 00:43 |
| gmmaha | hoping manjeets is the same. then its one issue to debug | 00:43 |
| manjeets | i don't even see setup tools in Dockerfile.J2 | 00:44 |
| sdake | yes its a dep of jinja2 | 00:44 |
| vhosakot | gmmaha: cool.. manjeets are you seeing http://paste.openstack.org/show/493076/ | 00:44 |
| vhosakot | linux bridge worked for me 2 weeks ago on AIO | 00:45 |
| vhosakot | i will check with master now | 00:45 |
| manjeets | vhosakot deployment is done properly | 00:45 |
| manjeets | gmmaha is having issue with deployment | 00:45 |
| manjeets | i don't see that | 00:46 |
| sdake | --shared | 00:46 |
| sdake | will cause deployment issue | 00:46 |
| gmmaha | manjeets: my deployment finished.. but netutron _linuxbridge agent wasnt running and i coldnt get the container to run | 00:46 |
| manjeets | mine is running | 00:46 |
| manjeets | let me double check | 00:46 |
| *** Marga_ has joined #openstack-kolla | 00:47 | |
| vhosakot | i will see if I can repro https://bugs.launchpad.net/kolla/+bug/1566543 with mater as we speak | 00:47 |
| openstack | Launchpad bug 1566543 in kolla "linuxbridge deployment is broken with latest master (2f4a3f35c00277cc97e887ba647395a702268e84)" [Undecided,New] | 00:47 |
| *** Jeffrey4l has joined #openstack-kolla | 00:48 | |
| manjeets | there is definitely a issue with lb i see that container getting stopped when i schedule vms | 00:48 |
| manjeets | it was running when i first started | 00:48 |
| dave-mcc_ | i need to take off for a team dinner. i'll check back in later. | 00:51 |
| *** dave-mcc_ has quit IRC | 00:51 | |
| *** Marga_ has quit IRC | 00:51 | |
| gmmaha | vhosakot: on a side Q.. i did a cleanup of hosts and i still see ovs-system, br-ex, br-int and br-tun in al the targets | 00:52 |
| gmmaha | any tips on flushing it? | 00:52 |
| gmmaha | i remember inc0 suggested something a while back and i see to have completely missed it | 00:52 |
| vhosakot | a linux bridge deployment after an OVS deployment is not the right way | 00:52 |
| *** weiyu has joined #openstack-kolla | 00:53 | |
| vhosakot | gmmaha: cleanup everything, reboot, run linux bridge | 00:53 |
| vhosakot | gmmaha: yes, ovs-system should not even be seen in the first place with linux bridge | 00:53 |
| gmmaha | vhosakot: right.. not doing linuxbridge now | 00:53 |
| gmmaha | that error was from a previous deploy | 00:53 |
| vhosakot | ah ok... | 00:54 |
| gmmaha | its now manjeets who is running is linuxbridge :) | 00:54 |
| vhosakot | you see ovs-system, br-ex, br-int and br-tun after cleanup ? | 00:54 |
| gmmaha | i am re-deploying OVS, but was curious how i can clean out all those. | 00:54 |
| gmmaha | ChanServ: yeah | 00:54 |
| gmmaha | vhosakot: yeah | 00:54 |
| gmmaha | :P | 00:54 |
| gmmaha | damn... | 00:54 |
| *** dave-mccowan has joined #openstack-kolla | 00:54 | |
| manjeets | gmmaha http://paste.openstack.org/show/493253/ | 00:54 |
| sdake | dave-mccowan with my patch and ovs, my instances hit active state | 00:54 |
| manjeets | cleanup | 00:55 |
| manjeets | suggested by inc0 | 00:55 |
| sdake | dave-mccowan with my patch and linuxbridge, my instance never hits active state | 00:55 |
| sdake | make sure you dont hae linuxbridge enabled it appears doa | 00:55 |
| dave-mccowan | sdake which is default? | 00:55 |
| sdake | ovs | 00:55 |
| sdake | so linuxbridge is busted | 00:56 |
| sdake | an indepdnent problem is that enable_vip is busted | 00:56 |
| sdake | dave-mccowan paste your globals.yml | 00:56 |
| gmmaha | manjeets: thanks | 00:57 |
| dave-mccowan | http://paste.fedoraproject.org/350694/59990645 | 00:57 |
| *** daneyon has quit IRC | 00:57 | |
| sdake | dave-mccowan i'd higly recommend against using centos binary at this point in time | 00:59 |
| sdake | the rest of your config looks good | 00:59 |
| sdake | the delta in your conifg nd mine is my patch (possibly) and build from source | 00:59 |
| vhosakot | gmmaha: I dont think cleanup should delete docker0 interface.. it is created and used by docker daemon... if you're saying cleanup needs to delete ovs-system, br-ex, br-int and br-tun, yes, I agree this point... | 01:00 |
| vhosakot | gmmaha: https://github.com/openstack/kolla/blob/master/tools/cleanup-host does not delete ovs-system, br-ex, br-int and br-tun | 01:00 |
| gmmaha | vhosakot: yeah.. docker0 is the daemon and thats cool.. its the other stuff | 01:00 |
| vhosakot | gmmaha: agreed, I will submit a fix and add you as co-author | 01:00 |
| gmmaha | vhosakot: thanks.. | 01:01 |
| sdake | dave-mccowan which git hash are you on | 01:01 |
| gmmaha | let me see if i can test a fix now on my machine | 01:01 |
| gmmaha | if so, i cna throw up a patch | 01:01 |
| gmmaha | save you some time | 01:01 |
| sdake | dave-mccowan its essential you hae this commit in your repo 7676b079e6b812b16ac63d753d27853e75732d69 | 01:02 |
| vhosakot | gmmaha: cleanup order would be : delete nova VMs (this deletes the tap interfaces), cleanup-containers, cleanup-images, and cleanup-host | 01:02 |
| vhosakot | gmmaha: at this point, ovs-system, br-ex, br-int and br-tun.. will submit fix... thanks for noticing this | 01:02 |
| gmmaha | vhosakot: right.. thats the order of deletion | 01:02 |
| vhosakot | i meant, at this point, ovs-system, br-ex, br-int and br-tun should not be there | 01:02 |
| manjeets | vhosakat did you catch up ? | 01:02 |
| dave-mccowan | i just updated, built, and i'm deploying. i'll try from source next if this doesn't work. | 01:02 |
| *** dave-mccowan has quit IRC | 01:03 | |
| manjeets | gmmaha clean up host will clean all roueters tap no matter what order you follow | 01:03 |
| vhosakot | manjeets: yes, cathing up on LB issue... | 01:03 |
| *** iceyao has joined #openstack-kolla | 01:03 | |
| gmmaha | manjeets: yes agreed | 01:04 |
| gmmaha | i meant the cleanup of dumping VMS, containers, image and then the host residuals | 01:05 |
| vhosakot | gmmaha: here is what I'll do... cleanup-host will check is there are nova VMs, neutron network, subnet, router,... if anything is there, flag an error "delete vm/network/subnet/router first before cleaningup"... if they are not there, cleanup-host will delete ovs-system, br-ex, br-int and br-tun | 01:06 |
| vhosakot | gmmaha: does this sound good ? | 01:07 |
| gmmaha | vhosakot: that seems fair | 01:07 |
| vhosakot | cool | 01:07 |
| sdake | vhosakot linuxbridge is broken | 01:09 |
| sdake | it doesnt work for me either | 01:09 |
| *** iceyao_ has joined #openstack-kolla | 01:09 | |
| sdake | Jeffrey4l is looking into it | 01:09 |
| sdake | he said he would this morning at 2am my time | 01:09 |
| sdake | so i expect he will fix it today | 01:09 |
| *** Marga_ has joined #openstack-kolla | 01:10 | |
| vhosakot | sdake: ok...thanks for letting me know | 01:11 |
| vhosakot | sdake: I'm pulled latest master.. currently building images | 01:12 |
| sdake | ya if you can fix it before more poewr to you :) | 01:12 |
| sdake | linuxbridge is highest prioity bug atm | 01:12 |
| Jeffrey4l | sdake, yea. I will check that. | 01:12 |
| *** iceyao has quit IRC | 01:13 | |
| vhosakot | cool | 01:13 |
| *** iceyao_ has quit IRC | 01:13 | |
| vhosakot | bbiab 30 mins | 01:13 |
| gmmaha | vhosakot: running into a interesting issue.. not able to dlete the br-* | 01:13 |
| *** iceyao has joined #openstack-kolla | 01:13 | |
| gmmaha | RTNETLINK answers: Operation not supported | 01:13 |
| sdake | dinner time - bbl | 01:13 |
| *** Marga_ has quit IRC | 01:14 | |
| Jeffrey4l | gmmaha, what command r u using? | 01:17 |
| *** absubram has joined #openstack-kolla | 01:17 | |
| gmmaha | Jeffrey4l: ip link delete br-ex | 01:17 |
| gmmaha | and also tried ip link delete br-ex type bridge | 01:18 |
| *** iceyao has quit IRC | 01:18 | |
| *** iceyao has joined #openstack-kolla | 01:18 | |
| *** achanda has joined #openstack-kolla | 01:18 | |
| Jeffrey4l | gmmaha, I do not think that will work. that device is created by ovs. So i think we need remove that by using ovs-vsctl command. | 01:18 |
| gmmaha | Jeffrey4l: aah | 01:19 |
| *** vhosakot has quit IRC | 01:21 | |
| gmmaha | Jeffrey4l: i dont think so | 01:23 |
| gmmaha | ovs-vsctl doesnt even see the bridges | 01:23 |
| gmmaha | ovs-vsctl list-br | 01:23 |
| gmmaha | returns empty | 01:23 |
| Jeffrey4l | you need `docker exec openvswitch_db ovs-vsctl show` | 01:24 |
| Jeffrey4l | gmmaha, ^ | 01:25 |
| gmmaha | Jeffrey4l: ohh.. | 01:25 |
| gmmaha | hm.. | 01:25 |
| gmmaha | tats tricky now that we have dumped all the containers alrady :( | 01:25 |
| Jeffrey4l | gmmaha, if so i have no idea about that. But if we leave them be, it will be no harm i think. | 01:26 |
| gmmaha | Jeffrey4l: yeah, you might be right.. i am sure i can re-deploy without any issues.. | 01:26 |
| gmmaha | but just curious that we have residue.. tahts all | 01:26 |
| gmmaha | maybe a special cleanup in the cotnainer if you are using ovs.. | 01:27 |
| gmmaha | let me read up on containers more before i yap anymore | 01:27 |
| Jeffrey4l | gmmaha, at least, re-deploy a openvswitch env is OK. I often do that. | 01:27 |
| Jeffrey4l | gmmaha, cool | 01:27 |
| *** sdake has quit IRC | 01:28 | |
| gmmaha | Jeffrey4l: thanks for the help.. now atleast its a little ore clear | 01:29 |
| gmmaha | more* | 01:29 |
| *** sdake has joined #openstack-kolla | 01:32 | |
| *** daneyon has joined #openstack-kolla | 01:40 | |
| *** vhosakot has joined #openstack-kolla | 01:44 | |
| vhosakot | gmmaha: ping | 01:44 |
| vhosakot | can you try "ip link delete bt-int" | 01:44 |
| *** daneyon has quit IRC | 01:45 | |
| gmmaha | vhosakot: sure | 01:46 |
| gmmaha | vhosakot: actually have a deployment in play | 01:46 |
| gmmaha | let me delete it and try it out | 01:46 |
| vhosakot | gmmaha: try it when you are ready to cleanup | 01:46 |
| gmmaha | vhosakot: cleaning up now.. ' | 01:46 |
| gmmaha | well, i neednt have.. but bah! | 01:46 |
| gmmaha | vhosakot: same error | 01:47 |
| vhosakot | gmmaha: what command did you run > | 01:47 |
| vhosakot | ? | 01:47 |
| gmmaha | ip link delete br-ex | 01:47 |
| gmmaha | root@jfz1r02h08:~# ip link delete br-ex | 01:48 |
| gmmaha | RTNETLINK answers: Operation not supported | 01:48 |
| gmmaha | root@jfz1r02h08:~# ip link delete br-int | 01:48 |
| gmmaha | RTNETLINK answers: Operation not supported | 01:48 |
| vhosakot | sudo ? | 01:48 |
| gmmaha | i am root | 01:49 |
| gmmaha | vhosakot: i have to run out now.. will try to be back online later tonight | 01:49 |
| vhosakot | that means the interface is in use | 01:49 |
| vhosakot | cool | 01:49 |
| gmmaha | but this isnt such a big issue.. | 01:49 |
| vhosakot | thanks for checking | 01:49 |
| vhosakot | yep | 01:49 |
| gmmaha | vhosakot: no worries.. | 01:49 |
| gmmaha | its showing as down | 01:49 |
| gmmaha | so wasnt sure wher it was in use.. | 01:49 |
| gmmaha | but yeah, we can debug this soon.. | 01:49 |
| gmmaha | maybe after the releae cycle.. | 01:50 |
| gmmaha | i know you have bigger issues to focus on. | 01:50 |
| gmmaha | will try and see if i can figure out a solution in the meantime | 01:50 |
| sdake | gmmaha yo uhave to use ip netns | 01:51 |
| Jeffrey4l | sdake, about the linuxbridge, what's your issue? r u have any info about? | 01:54 |
| sdake | no detaills | 01:55 |
| sdake | turn on linux bridge | 01:55 |
| sdake | start a nova vm | 01:55 |
| sdake | it wont go into active state | 01:55 |
| sdake | i think because it can't get a port | 01:55 |
| sdake | didn't debug it at all | 01:55 |
| sdake | just noticed behavior | 01:55 |
| sdake | there is a bug filed | 01:55 |
| Jeffrey4l | OK. I will debug this today(now) | 01:56 |
| sdake | cool | 01:56 |
| mark-casey | is Ubuntu Wily supported or only Trusty? I think Wily uses systemd so I think I'm having trouble getting /run to be a shared mount (well, assuming shared should be showing as a mount flag... it isn't) | 01:57 |
| Jeffrey4l | sdake, are u testing with? or without? your qemu vesion patch? | 01:57 |
| sdake | mark-casey wily shoudl work in the same way as cento7 wrt systemd | 01:57 |
| sdake | Jeffrey4l with qemu patch | 01:57 |
| Jeffrey4l | ok | 01:57 |
| sdake | Jeffrey4l i am retty sure it h appen without | 01:57 |
| sdake | because eveyrone else is complaining about it | 01:58 |
| sdake | and they re not running my patch | 01:58 |
| vhosakot | mark-casey: I think inc0 uses wily that has systemd.... search for MountFlags=shared in http://docs.openstack.org/developer/kolla/quickstart.html and update docker.service | 01:58 |
| Jeffrey4l | sdake, i am testing it, it not work for sure. the has some permission issue. debuging~ | 01:58 |
| mark-casey | thanks all. I'd modified the unit directly and tried the 'mount --make-shared' and am still getting "APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/images/create?tag=2.0.0&fromImage=10.101.0.15%3A5000%2Fkollaglue%2Fubuntu-source-heka',),)" on "TASK: [common | Starting heka container] " for all hosts. | 01:59 |
| mark-casey | gonna try the drop in '.d' version of the setting instead | 01:59 |
| *** rhallisey has quit IRC | 02:02 | |
| vhosakot | mark-casey: "cat /proc/self/mountinfo | grep "/run" | grep shared" should show if /run is shared or not | 02:02 |
| *** salv-orlando has joined #openstack-kolla | 02:03 | |
| mark-casey | that returns 6 lines, most in /run, some in /sys/fs... | 02:04 |
| mark-casey | 23 25 0:19 / /run rw,nosuid,noexec,relatime shared:5 - tmpfs tmpfs rw,size=3284196k,mode=755 | 02:04 |
| mark-casey | 28 23 0:22 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs tmpfs rw,size=5120k | 02:04 |
| mark-casey | so I see shared there | 02:04 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 02:04 |
| *** kjelly has joined #openstack-kolla | 02:04 | |
| mark-casey | does that have to also be done in the deployer/operator host or only targets? | 02:04 |
| vhosakot | mark-casey: just /run... | 02:05 |
| vhosakot | mark-casey: can you do this ? cat /proc/self/mountinfo | grep "/run " | grep shared | 02:05 |
| vhosakot | all hosts... deplot and target both | 02:05 |
| vhosakot | add a space after /run in in first grep | 02:06 |
| mark-casey | 23 25 0:19 / /run rw,nosuid,noexec,relatime shared:5 - tmpfs tmpfs rw,size=3284196k,mode=755 | 02:06 |
| vhosakot | mark-casey: so it is shared... can you do the same on the node on which the error is seen | 02:07 |
| vhosakot | did the dhcp container fail ? | 02:07 |
| mark-casey | vhosakot: that was the node, it isn't there on the deployment host. will chase that down :) | 02:08 |
| vhosakot | the node's IP address is seen in the output of the play | 02:08 |
| mark-casey | right. the line I pasted was when I was ssh'ed into it | 02:09 |
| mark-casey | same command had no output on the machine I'm running kolla-ansible from | 02:09 |
| vhosakot | ah, that means deploy node's /run is not shared | 02:09 |
| *** jtriley has joined #openstack-kolla | 02:10 | |
| vhosakot | share /run on all the nodes... deploy (on which kolla-ansible is run) as well as the target nodes (network, compute, strorage, control) | 02:11 |
| *** wuhg has joined #openstack-kolla | 02:12 | |
| *** jtriley has quit IRC | 02:15 | |
| *** mwheckmann has joined #openstack-kolla | 02:18 | |
| *** sdake_ has joined #openstack-kolla | 02:20 | |
| *** sdake has quit IRC | 02:22 | |
| *** salv-orlando has quit IRC | 02:23 | |
| *** salv-orlando has joined #openstack-kolla | 02:28 | |
| *** salv-orlando has quit IRC | 02:33 | |
| *** jtriley has joined #openstack-kolla | 02:44 | |
| *** ccesario___ has quit IRC | 02:54 | |
| *** ccesario___ has joined #openstack-kolla | 02:55 | |
| *** mdnadeem has joined #openstack-kolla | 02:59 | |
| *** openstackstatus has quit IRC | 03:01 | |
| sbezverk | vhosakot ping | 03:02 |
| vhosakot | pong | 03:02 |
| sbezverk | vhosakot I am trying to setup registry on ubuntu to test iscsi on ubuntu, I cannot find where on ubuntu | 03:03 |
| sbezverk | I can configure docker for insecure connectivity, I know how | 03:03 |
| sbezverk | to do it on Centos but not on ubuntu | 03:03 |
| vhosakot | use a non-local docker registry over the network ? | 03:04 |
| sbezverk | vhosakot yes | 03:04 |
| sbezverk | On centos we change /etc/sysconfig/docker | 03:05 |
| sbezverk | but which file needs to be changed on ubuntu?? | 03:05 |
| vhosakot | ah.. docker config file on UBuntu.. 1 sec | 03:05 |
| ccesario___ | Ubuntu is on /etc/default/docker ... If Im not wrong | 03:06 |
| *** sdake_ has quit IRC | 03:06 | |
| vhosakot | sbezverk: yep, change DOCKER_OPTS= in /etc/default/docker | 03:06 |
| *** jasonsb has joined #openstack-kolla | 03:06 | |
| *** sdake has joined #openstack-kolla | 03:07 | |
| vhosakot | sbezverk: have you setup Nginx for secure remote docker registry ? | 03:07 |
| sbezverk | vhosakot nope | 03:08 |
| vhosakot | sbezverk: also, point docker_registry in globals.yml to <remote registry IP>:<remote docker registry port> | 03:08 |
| vhosakot | sbezverk: should be ok... remote registry can be insecure... I have not tested it | 03:08 |
| sbezverk | I just loaded docker's registry 2.3 | 03:09 |
| *** achanda has quit IRC | 03:09 | |
| *** iceyao has quit IRC | 03:09 | |
| sbezverk | I see, ok I will check with oother folks who uses ubuntu | 03:09 |
| *** iceyao has joined #openstack-kolla | 03:09 | |
| vhosakot | cool | 03:11 |
| ccesario___ | vhosakot: if possible please... Comment it ... https://review.openstack.org/#/c/297984/ ... I would like to receive comments about my last reply.... If possible of course ... :) | 03:12 |
| patchbot | ccesario___: patch 297984 - kolla - Added hostname param into kolla_docker | 03:12 |
| ccesario___ | I found a wayt to do it ... Using another way | 03:13 |
| vhosakot | ccesario___: yes, will look into it.. | 03:13 |
| mark-casey | does the docker registry listen on http and https? | 03:15 |
| mark-casey | my docker logs re: APIError(HTTPError say that it can't ping the registry but it is trying https | 03:15 |
| mark-casey | Apr 07 03:15:01 deep-sea docker[13753]: time="2016-04-07T03:15:01.452693177Z" level=error msg="Handler for POST /v1.22/images/create returned error: unable to ping registry endpoint https://10.101.0.15:5000/v0/\nv2 ping attempt failed with error: Get https://10.101.0.15:5000/v2/: tls: oversized record received with length 20527\n v1 ping attempt failed with error: Get https://10.101.0.15:5000/v1/_ping: tls: oversized record recei | 03:16 |
| sbezverk | mark-casey Ubuntu or Centos? | 03:16 |
| mark-casey | sorry Ubuntu | 03:17 |
| *** sdake has quit IRC | 03:17 | |
| sbezverk | mark-casey /etc/default/docker | 03:17 |
| sbezverk | add this line: | 03:17 |
| mark-casey | DOCKER_OPTS='--insecure-registry 10.101.0.15:5000' | 03:17 |
| sbezverk | DOCKER_OPTS="--insecure-registry myregistrydomain.com:4000" | 03:17 |
| mark-casey | :D | 03:18 |
| *** sdake has joined #openstack-kolla | 03:18 | |
| sbezverk | netstat -tunlp | 03:18 |
| sbezverk | check which port docker is listen for | 03:18 |
| mark-casey | run this on a target or the deployment host? | 03:18 |
| vhosakot | mark-casey: docker listens on a unix socket | 03:18 |
| sbezverk | on the server where the resgistry runs | 03:19 |
| mark-casey | this the one? tcp6 0 0 :::5000 :::* LISTEN 13805/docker-proxy | 03:20 |
| vhosakot | tep.. | 03:20 |
| vhosakot | yep | 03:20 |
| ccesario___ | mark-casey, I advise you usage port 4000 instead 5000 | 03:20 |
| vhosakot | mark-casey: ps aux | grep 13805 | 03:20 |
| ccesario___ | http://docs.openstack.org/developer/kolla/image-building.html | 03:20 |
| mark-casey | it is non-AIO but I'm willing to change it | 03:20 |
| ccesario___ | Ohhh ok ok | 03:20 |
| mark-casey | root 13805 0.0 0.1 264988 14284 ? Sl 21:56 0:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5000 -container-ip 172.17.0.2 -container-port 5000 | 03:21 |
| ccesario___ | Ignore me :p | 03:21 |
| mark-casey | not at all I'll take all the help I can get :D | 03:21 |
| vhosakot | mark-casey: looks like your doing 5000:5000 (host port:container port) | 03:22 |
| mark-casey | yes | 03:22 |
| sdake | vhosakot registry has a tls implementation no need for nginx | 03:22 |
| sbezverk | vhosakot kolla expect host port to be 4000 | 03:22 |
| vhosakot | ccesario___ is right... our guide suugests http://docs.openstack.org/developer/kolla/image-building.html#setting-up-docker-local-registry 4000:5000.. | 03:23 |
| *** iceyao has quit IRC | 03:23 | |
| sbezverk | at least on my setup I have 4000:5000 and it works now fine | 03:23 |
| sdake | kolla doen't care | 03:23 |
| *** iceyao has joined #openstack-kolla | 03:23 | |
| vhosakot | me too.. works fine.. local insecure registry | 03:23 |
| sdake | but keystone on aio will cause problems | 03:23 |
| mark-casey | docker_registry: "10.101.0.15:5000" | 03:24 |
| mark-casey | #docker_namespace: "companyname" | 03:24 |
| mark-casey | #docker_registry_username: "sam" | 03:24 |
| mark-casey | #docker_registry_password: "correcthorsebatterystaple" | 03:24 |
| mark-casey | FWIW | 03:24 |
| sbezverk | sdake I am brining up ubuntu to test iscsi, is it safe now to use your qemu patch on ubuntu? | 03:24 |
| sdake | mark what problem are you having | 03:24 |
| sdake | sbezverk i dont know i need someone to test it | 03:24 |
| vhosakot | mark-casey: https://github.com/openstack/kolla/blob/master/ansible/group_vars/all.yml#L104.... that is why host port is 4000 on deploy node | 03:24 |
| sbezverk | sdake then it will be me but tomorrow morning, still building images | 03:25 |
| mark-casey | sdake: TASK: [common | Starting heka container] ************************************** | 03:25 |
| mark-casey | failed: [control03] => {"changed": true, "failed": true} | 03:25 |
| mark-casey | msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/images/create?tag=2.0.0&fromImage=10.101.0.15%3A5000%2Fkollaglue%2Fubuntu-source-heka',),) | 03:25 |
| mark-casey | slash run shows shared on targets and deployer | 03:25 |
| sdake | docker --version shows what? | 03:25 |
| mark-casey | all hosts Docker version 1.10.3, build 20f81dd | 03:26 |
| sdake | run ps -ef | grep docker | 03:26 |
| vhosakot | mark-casey: did you remove heka, share /run, re-deploy ? | 03:26 |
| mark-casey | did not remove heka from target | 03:26 |
| vhosakot | mark-casey: was that the one that caused the error before /run was shaed ? | 03:27 |
| mark-casey | yep | 03:27 |
| sdake | run tools/cleanup-containers | 03:27 |
| sdake | and redeploy | 03:27 |
| vhosakot | yep | 03:27 |
| sdake | yu will need to do this on all target nodes | 03:27 |
| vhosakot | precehcks! | 03:27 |
| mark-casey | I did pass prechecks | 03:28 |
| sdake | prechecks undcoumented | 03:28 |
| sdake | i'm gong to crash | 03:28 |
| sdake | night folks | 03:28 |
| *** jtriley has quit IRC | 03:28 | |
| vhosakot | mark-casey: tools/cleanup-containers, kolla-ansible prechecks, kolla-ansible deploy | 03:28 |
| vhosakot | night sdake | 03:28 |
| sdake | sbezverk when yo utet ubuntu plese report in the review if it woked or you | 03:28 |
| mark-casey | night, ty | 03:28 |
| *** sdake has quit IRC | 03:28 | |
| vhosakot | sdake: precehks is there is qsg | 03:28 |
| mark-casey | vhosakot: I don't seem to have cleanup-containers on the targets | 03:30 |
| mark-casey | should I have? or just scp it over? | 03:31 |
| ccesario___ | Yes... | 03:31 |
| ccesario___ | Scp rsync ;p | 03:31 |
| *** iceyao has quit IRC | 03:31 | |
| *** iceyao has joined #openstack-kolla | 03:32 | |
| vhosakot | mark-casey: yep, cleanup is a raw bash script and not ansible based, and shows up in the node on which kolla is cloned... so, yes, scp to target that needs to be claned | 03:32 |
| ccesario___ | Choose and copy to nodes and run : | 03:32 |
| vhosakot | cleaned* | 03:32 |
| vhosakot | gmmaha: so you have the ansible command you ran the other day to run "cleanup scripts" on remote hosts without scp'ing the scripts ? | 03:33 |
| vhosakot | so==do | 03:33 |
| vhosakot | mark-casey: ccesario___: since ansible already knows all the nodes' IPs, ansible can be used to run the cleanup scripts without scp'ing them using ansbile's adhoc commands... gmmaha has done it | 03:35 |
| vhosakot | http://docs.ansible.com/ansible/intro_adhoc.html | 03:35 |
| vhosakot | gmmaha: we'll need to document those steps | 03:35 |
| mark-casey | Apr 07 03:35:02 deep-sea docker[13753]: time="2016-04-07T03:35:02.867625317Z" level=error msg="Handler for POST /v1.22/images/create returned error: unable to ping registry endpoint https://10.101.0.15:5000/v0/\nv2 ping attempt failed with error: Get https://10.101.0.15:5000/v2/: tls: oversized record received with length 20527\n v1 ping attempt failed with error: Get https://10.101.0.15:5000/v1/_ping: tls: oversized record recei | 03:36 |
| ccesario___ | vhosakot, ohhh its nice | 03:36 |
| ccesario___ | Send the patch for us :) | 03:37 |
| vhosakot | yep | 03:37 |
| vhosakot | mark-casey: when is this seen ? | 03:37 |
| mark-casey | I did 'journalctl | grep docker' on target host after deploy exploded | 03:38 |
| ccesario___ | This minimize the steps and prevent end user errors | 03:38 |
| ccesario___ | On the target hosts .... Did you configure the docker options too? | 03:39 |
| mark-casey | oh... yea, but some googling I was just doing also says maybe docker isn't picking up --insecure-registry | 03:39 |
| ccesario___ | mark-casey, ^^ | 03:39 |
| vhosakot | yep, DOCKER_OPTS on target hosts must have registry config | 03:40 |
| *** coolsvap has joined #openstack-kolla | 03:40 | |
| mark-casey | I do have that but I'm seeing references to /etc/systemd/system/multi-user.target.wants/docker.service too | 03:40 |
| mark-casey | huzzah | 03:41 |
| ccesario___ | Sorry by my short phrases... Smart phone is not so good to type :P | 03:41 |
| mark-casey | that insecure flag is missing on targets | 03:41 |
| mark-casey | hence the https | 03:41 |
| mark-casey | well... missing when I ps -ef | grep docker | 03:41 |
| ccesario___ | You need configure into /etc/default/docker | 03:42 |
| mark-casey | root@deep-sea:~# grep OPTS /etc/default/docker | 03:42 |
| mark-casey | # Use DOCKER_OPTS to modify the daemon startup options. | 03:42 |
| mark-casey | DOCKER_OPTS='--insecure-registry 10.101.0.15:5000' | 03:42 |
| vhosakot | mark-casey: that means, the targets are not seeing registry.. the insecure registry flag must be seen in the output of "ps -ef | grep docker" on all targets | 03:42 |
| vhosakot | mark-casey: is DOCKER_OPTS='--insecure-registry 10.101.0.15:5000' in /etc/default/docker on all targets ? | 03:43 |
| vhosakot | ccesario___: can you curl <docker registry>:<port> form all targets ? | 03:45 |
| mark-casey | yes. I'm running on Ubuntu Wily though and it uses systemd | 03:45 |
| mark-casey | so the docs thing about "Docker Inc’s packaged version of docker-engine for CentOS is defective and does not read the other_args configuration options from “/etc/sysconfig/docker”." applied to me. Except I had to change /etc/systemd/system/multi-user.target.wants/docker.service | 03:46 |
| mark-casey | this may just be fixing something else I've broken, or it could be that way for anyone who tried it. not sure | 03:46 |
| ccesario___ | vhosakot, I'm not on my env.... I'm in my home now.... :( | 03:46 |
| mark-casey | but the target node I did it on now says it can't find the heka image so I'm making progress | 03:47 |
| vhosakot | mark-casey: so, in that case, on all targets, /etc/systemd/system/docker.service.d/kolla.conf must have --insecure-registry, and all targets must show the insecure registry flag in "ps -ef | grep docker" | 03:47 |
| mark-casey | *when I run kolla-ansible that node says... | 03:47 |
| vhosakot | ccesario___: cool :) | 03:47 |
| mark-casey | awesome | 03:48 |
| vhosakot | mark-casey: search for "ExecStart=/usr/bin/docker daemon -H fd:// $other_args" in http://docs.openstack.org/developer/kolla/quickstart.html | 03:48 |
| vhosakot | mark-casey: did it work ? | 03:48 |
| ccesario___ | Need sleep now... | 03:49 |
| vhosakot | ccesario___: night! | 03:49 |
| ccesario___ | See you guys! | 03:49 |
| mark-casey | night, thanks! | 03:49 |
| ccesario___ | Night....! | 03:49 |
| mark-casey | vhosakot: yes, that node changed error from can't ping registry to heka image not found. so progress. I think I'm going to rebuild to make sure all the hacking at it isn't affecting further work | 03:49 |
| *** salv-orlando has joined #openstack-kolla | 03:50 | |
| mark-casey | now that that issue is understood | 03:50 |
| ccesario___ | vhosakot, if possible... Comment the patch... :) | 03:50 |
| vhosakot | ccesario___: yep :) | 03:50 |
| ccesario___ | See you guys! | 03:50 |
| vhosakot | mark-casey: yep, heka image must be built.. docker images | grep heka | 03:50 |
| *** sekrit has quit IRC | 03:51 | |
| *** salv-orlando has quit IRC | 03:52 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Add prechecks to validate kolla_internal_vip_address and kolla_external_vip_address https://review.openstack.org/302026 | 03:53 |
| mark-casey | vhosakot: cool. I'll check back later if more trouble after getting it back to this point but cleaner. thanks for all the help! | 03:53 |
| vhosakot | mark-casey: cool | 03:53 |
| *** salv-orlando has joined #openstack-kolla | 03:53 | |
| *** Marga_ has joined #openstack-kolla | 03:56 | |
| vhosakot | Jeffrey4l: I addresed your comments and uploaded new PS.. thanks for the comments https://review.openstack.org/302026 | 03:56 |
| vhosakot | Jeffrey4l: ^^^ | 03:56 |
| *** dave-mccowan has joined #openstack-kolla | 03:57 | |
| *** Marga_ has quit IRC | 03:57 | |
| *** Marga_ has joined #openstack-kolla | 03:57 | |
| Jeffrey4l | vhosakot, cool, will check that later. | 03:57 |
| vhosakot | cool, thanks! | 03:57 |
| *** salv-orlando has quit IRC | 03:58 | |
| *** dave-mcc_ has joined #openstack-kolla | 03:58 | |
| *** mwheckmann has quit IRC | 04:00 | |
| *** dave-mccowan has quit IRC | 04:01 | |
| *** Marga_ has quit IRC | 04:01 | |
| *** Marga_ has joined #openstack-kolla | 04:02 | |
| *** sekrit has joined #openstack-kolla | 04:05 | |
| *** absubram_ has joined #openstack-kolla | 04:14 | |
| *** absubram_ has quit IRC | 04:14 | |
| *** absubram has quit IRC | 04:15 | |
| *** achanda has joined #openstack-kolla | 04:22 | |
| *** daneyon has joined #openstack-kolla | 04:23 | |
| *** banix has joined #openstack-kolla | 04:26 | |
| *** daneyon has quit IRC | 04:28 | |
| *** weiyu has quit IRC | 04:34 | |
| *** achanda has quit IRC | 04:37 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 04:42 |
| *** salv-orlando has joined #openstack-kolla | 04:57 | |
| *** salv-orlando has quit IRC | 05:01 | |
| *** achanda has joined #openstack-kolla | 05:03 | |
| *** salv-orlando has joined #openstack-kolla | 05:11 | |
| openstackgerrit | Rajath Agasthya proposed openstack/kolla: Added Kibana instructions to quickstart.rst https://review.openstack.org/302458 | 05:14 |
| *** weiyu has joined #openstack-kolla | 05:15 | |
| *** ccesario___ has quit IRC | 05:19 | |
| *** weiyu has quit IRC | 05:25 | |
| gmmaha | vhosakot: for sure.. | 05:25 |
| vhosakot | gmmaha: thanks! | 05:25 |
| *** banix has quit IRC | 05:25 | |
| gmmaha | vhosakot: i basically have a clone of the kolla project in all the machines at a known location | 05:26 |
| vhosakot | ah ok | 05:26 |
| gmmaha | ansible -i ansible/inventory/multinode -m shell -a '/<path to kolla project>/kolla/tools/cleanup-host' all | 05:26 |
| gmmaha | vhosakot: if you know of a place i should put this down as documentaiton, i can do that | 05:27 |
| vhosakot | cool, will let you know | 05:27 |
| gmmaha | vhosakot: i didnt understand what sdake meant by 'you have to use ip netns' | 05:27 |
| vhosakot | to cehck namespaces | 05:28 |
| gmmaha | right.. i thought he mentioned that in reference to the left of network interfaces | 05:28 |
| gmmaha | hence was curious how that command will help | 05:28 |
| mark-casey | if you use ansible's script module it will copy that script over to a tmp path and then run it for you | 05:30 |
| mark-casey | but if there are other uses for having the project on machines maybe less cool | 05:31 |
| gmmaha | mark-casey: doubt there is any other reason to have the whole git project | 05:31 |
| gmmaha | i am sure we can conjure up a ansible script that can copy this script module to the targets and run them | 05:31 |
| gmmaha | didnt know about ansible's script modue | 05:31 |
| gmmaha | module* | 05:31 |
| mark-casey | yea i had to use it for something.. I forget what. was surprised http://docs.ansible.com/ansible/script_module.html | 05:32 |
| gmmaha | aah this is a much easier ansible module to write then | 05:33 |
| gmmaha | and stick it into the kolla project | 05:33 |
| *** achanda has quit IRC | 05:33 | |
| gmmaha | vhosakot: what do you think? | 05:33 |
| *** iceyao has quit IRC | 05:38 | |
| *** iceyao has joined #openstack-kolla | 05:38 | |
| vhosakot | gmmaha: it would be used to run cleanup scripts on remote hosts ? | 05:38 |
| vhosakot | mark-casey: good find! | 05:38 |
| *** iceyao has quit IRC | 05:39 | |
| gmmaha | vhosakot: yeah | 05:39 |
| *** iceyao has joined #openstack-kolla | 05:39 | |
| mark-casey | vhosakot: thanks | 05:39 |
| gmmaha | simple ansible script, it will do the copy over of the local script to the targets and run it | 05:39 |
| gmmaha | mark-casey: +1 on vhosakot's comment | 05:39 |
| mark-casey | :) | 05:40 |
| *** iceyao has quit IRC | 05:40 | |
| *** iceyao_ has joined #openstack-kolla | 05:40 | |
| *** tfukushima has joined #openstack-kolla | 05:41 | |
| coolsvap | vhosakot: http://logs.openstack.org/36/302036/14/experimental/gate-kolla-dsvm-build-oraclelinux-binary/392bc91/console.html#_2016-04-07_05_28_04_870 | 05:44 |
| coolsvap | oraclelinux gate failure | 05:44 |
| coolsvap | not able to install epel-release | 05:45 |
| coolsvap | pbourke: around? | 05:46 |
| vhosakot | coolsvap: pbourke: yep, I edited that patch set... looks like epel does not work on oraclelinux | 05:51 |
| *** achanda has joined #openstack-kolla | 05:59 | |
| coolsvap | vhosakot: yes | 06:03 |
| *** gfidente has joined #openstack-kolla | 06:03 | |
| openstackgerrit | Kuo-tung Kao proposed openstack/kolla: add unit tests for build script https://review.openstack.org/290423 | 06:04 |
| *** weiyu has joined #openstack-kolla | 06:04 | |
| *** mikelk has joined #openstack-kolla | 06:10 | |
| *** Serlex has joined #openstack-kolla | 06:11 | |
| *** unicell has quit IRC | 06:13 | |
| *** unicell has joined #openstack-kolla | 06:15 | |
| *** achanda has quit IRC | 06:22 | |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 06:36 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Add default flavors during init-runonce https://review.openstack.org/302534 | 06:36 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Add default flavors during init-runonce https://review.openstack.org/302534 | 06:37 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Configure Neutron to use internalURL for contacting Nova https://review.openstack.org/302536 | 06:38 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Configure Neutron to use internalURL for contacting Nova https://review.openstack.org/302536 | 06:39 |
| dave-mcc_ | vhosakot ping | 06:40 |
| vhosakot | pong | 06:41 |
| dave-mcc_ | i accidentally uploaded a patchset on the CentOS repo bug. Can you re-upload your patch set/ | 06:41 |
| dave-mcc_ | ? | 06:41 |
| coolsvap | vhosakot: i can do or are you doing it? | 06:45 |
| vhosakot | dave-mcc_: coolsvap: I can revert it... no worries | 06:47 |
| dave-mcc_ | vhosakot thanks | 06:47 |
| openstackgerrit | MD NADEEM proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/255450 | 06:47 |
| vhosakot | cool, np :) | 06:47 |
| coolsvap | vhosakot: sounds good | 06:47 |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 06:49 |
| openstackgerrit | Merged openstack/kolla: Fix manila keystone v3 issues for all clients https://review.openstack.org/299534 | 06:51 |
| *** salv-orlando has quit IRC | 06:52 | |
| *** Marga_ has quit IRC | 07:04 | |
| *** dave-mcc_ has quit IRC | 07:09 | |
| *** jmccarthy has left #openstack-kolla | 07:11 | |
| *** mlima_ has joined #openstack-kolla | 07:17 | |
| *** mlima__ has quit IRC | 07:21 | |
| mark-casey | anyone up for taking a look at this? I'm unable to have control01 and network01 resolve to the same target host or else I get conflicts. The multinode inventory's comments suggest it shouldn't be an issue | 07:28 |
| mark-casey | http://paste.openstack.org/show/493264/ | 07:28 |
| *** mbound has quit IRC | 07:31 | |
| *** jmccarthy has joined #openstack-kolla | 07:42 | |
| *** jmccarthy has left #openstack-kolla | 07:42 | |
| *** shardy has joined #openstack-kolla | 07:47 | |
| *** weiyu has quit IRC | 07:48 | |
| *** salv-orlando has joined #openstack-kolla | 07:54 | |
| *** ccesario___ has joined #openstack-kolla | 07:57 | |
| *** mark-casey has quit IRC | 08:02 | |
| *** weiyu_ has joined #openstack-kolla | 08:02 | |
| *** mbound has joined #openstack-kolla | 08:07 | |
| *** athomas has joined #openstack-kolla | 08:10 | |
| *** salv-orlando has quit IRC | 08:10 | |
| *** Jeffrey4l has quit IRC | 08:16 | |
| *** Jeffrey4l has joined #openstack-kolla | 08:22 | |
| *** salv-orlando has joined #openstack-kolla | 08:26 | |
| *** openstackstatus has joined #openstack-kolla | 08:28 | |
| *** ChanServ sets mode: +v openstackstatus | 08:28 | |
| *** vhosakot has quit IRC | 08:38 | |
| *** rmart04 has joined #openstack-kolla | 08:45 | |
| *** iceyao_ has quit IRC | 08:49 | |
| *** iceyao has joined #openstack-kolla | 08:50 | |
| *** pbourke has quit IRC | 08:51 | |
| *** iceyao has quit IRC | 08:51 | |
| *** iceyao has joined #openstack-kolla | 08:51 | |
| *** pbourke has joined #openstack-kolla | 08:52 | |
| *** salv-orl_ has joined #openstack-kolla | 08:52 | |
| *** iceyao has quit IRC | 08:53 | |
| *** iceyao has joined #openstack-kolla | 08:53 | |
| *** salv-orlando has quit IRC | 08:55 | |
| *** iceyao has quit IRC | 08:58 | |
| *** iceyao_ has joined #openstack-kolla | 08:59 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Add Heka support https://review.openstack.org/286301 | 09:04 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add all *-openrc files to .gitignore https://review.openstack.org/302608 | 09:04 |
| *** jhesketh has joined #openstack-kolla | 09:24 | |
| *** iceyao has joined #openstack-kolla | 09:26 | |
| *** ccesario___ has quit IRC | 09:27 | |
| *** iceyao_ has quit IRC | 09:27 | |
| *** ccesario___ has joined #openstack-kolla | 09:30 | |
| *** iceyao has quit IRC | 09:55 | |
| *** iceyao_ has joined #openstack-kolla | 09:55 | |
| *** allen_gao has quit IRC | 09:56 | |
| *** iceyao_ has quit IRC | 09:56 | |
| *** achanda has joined #openstack-kolla | 09:56 | |
| *** iceyao has joined #openstack-kolla | 09:56 | |
| *** allen_gao has joined #openstack-kolla | 10:01 | |
| *** achanda has quit IRC | 10:02 | |
| *** weiyu_ has quit IRC | 10:02 | |
| *** athomas has quit IRC | 10:03 | |
| *** iceyao has quit IRC | 10:04 | |
| *** iceyao has joined #openstack-kolla | 10:04 | |
| *** iceyao has quit IRC | 10:06 | |
| *** iceyao has joined #openstack-kolla | 10:07 | |
| *** weiyu_ has joined #openstack-kolla | 10:08 | |
| *** athomas has joined #openstack-kolla | 10:11 | |
| *** ccesario___ has quit IRC | 10:23 | |
| *** iceyao has quit IRC | 10:25 | |
| *** iceyao has joined #openstack-kolla | 10:25 | |
| *** rhallisey has joined #openstack-kolla | 10:35 | |
| *** ccesario___ has joined #openstack-kolla | 10:52 | |
| *** achanda has joined #openstack-kolla | 10:58 | |
| *** achanda has quit IRC | 11:03 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Drop root for swift https://review.openstack.org/259374 | 11:07 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add Heka bootstraping https://review.openstack.org/286301 | 11:13 |
| *** coolsvap has quit IRC | 11:17 | |
| *** pmisiak has joined #openstack-kolla | 11:18 | |
| *** iceyao has quit IRC | 11:18 | |
| *** iceyao has joined #openstack-kolla | 11:18 | |
| *** iceyao has quit IRC | 11:19 | |
| pmisiak | Hi, does Kolla support multiple network nodes ? Can I use the same nodes in the control and network host group in inventory? | 11:19 |
| *** iceyao has joined #openstack-kolla | 11:19 | |
| *** weiyu_ has quit IRC | 11:20 | |
| pmisiak | or network node is a SPOF? | 11:21 |
| *** mbound has quit IRC | 11:24 | |
| *** tfukushima has quit IRC | 11:25 | |
| openstackgerrit | Merged openstack/kolla: Configure Neutron to use internalURL for contacting Nova https://review.openstack.org/302536 | 11:28 |
| *** mbound has joined #openstack-kolla | 11:29 | |
| *** coolsvap has joined #openstack-kolla | 11:40 | |
| ccesario | morning | 11:43 |
| *** Marga_ has joined #openstack-kolla | 11:49 | |
| *** banix has joined #openstack-kolla | 11:50 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Fix manila keystone v3 issues for all clients https://review.openstack.org/302694 | 11:52 |
| *** mbound has quit IRC | 11:54 | |
| *** mbound has joined #openstack-kolla | 11:54 | |
| *** mbound has quit IRC | 11:56 | |
| *** mbound has joined #openstack-kolla | 12:01 | |
| *** dwalsh has joined #openstack-kolla | 12:03 | |
| *** jhesketh has quit IRC | 12:07 | |
| *** achanda has joined #openstack-kolla | 12:07 | |
| *** jhesketh has joined #openstack-kolla | 12:07 | |
| *** cbits has joined #openstack-kolla | 12:11 | |
| *** achanda has quit IRC | 12:15 | |
| *** karthiks has joined #openstack-kolla | 12:22 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 12:23 |
| *** salv-orl_ has quit IRC | 12:28 | |
| *** wuhg has quit IRC | 12:29 | |
| *** banix has quit IRC | 12:31 | |
| *** banix has joined #openstack-kolla | 12:32 | |
| *** banix has quit IRC | 12:32 | |
| *** cbits has quit IRC | 12:32 | |
| *** salv-orlando has joined #openstack-kolla | 12:40 | |
| *** dwalsh has quit IRC | 12:43 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add iscsid/tgtd specific parameters to Kolla infrastructure https://review.openstack.org/300004 | 12:43 |
| *** achanda has joined #openstack-kolla | 12:47 | |
| *** iceyao has quit IRC | 12:48 | |
| sbezverk | Jeffrey4l ping | 12:49 |
| *** jtriley has joined #openstack-kolla | 12:50 | |
| *** andrewvwebber has joined #openstack-kolla | 12:57 | |
| *** dwalsh has joined #openstack-kolla | 12:57 | |
| *** diogogmt has quit IRC | 12:58 | |
| sbezverk | Appreciate a ping back from folks who attempted ubuntu based multinode installation.. | 12:58 |
| *** mbound has quit IRC | 12:58 | |
| *** achanda has quit IRC | 12:58 | |
| *** diogogmt has joined #openstack-kolla | 12:59 | |
| *** mbound has joined #openstack-kolla | 13:00 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla: Create and chown /etc/heka directory https://review.openstack.org/302742 | 13:01 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add Heka support https://review.openstack.org/286301 | 13:02 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add Heka support https://review.openstack.org/286301 | 13:02 |
| *** achanda has joined #openstack-kolla | 13:08 | |
| *** diogogmt has quit IRC | 13:08 | |
| *** shakamunyi has quit IRC | 13:10 | |
| *** shakamunyi has joined #openstack-kolla | 13:12 | |
| *** diogogmt has joined #openstack-kolla | 13:12 | |
| *** achanda has quit IRC | 13:13 | |
| *** SiRiuS has joined #openstack-kolla | 13:14 | |
| *** SiRiuS has quit IRC | 13:15 | |
| *** inc0 has joined #openstack-kolla | 13:19 | |
| inc0 | good morning | 13:19 |
| inc0 | sbezverk, I'll be with you in just a moment, needs coffee | 13:20 |
| inc0 | badly | 13:20 |
| sbezverk | inc0 good morning, sure thing, ping me when you are ready | 13:20 |
| *** mbound has quit IRC | 13:22 | |
| *** cbits has joined #openstack-kolla | 13:24 | |
| *** SiRiuS has joined #openstack-kolla | 13:28 | |
| *** mbound has joined #openstack-kolla | 13:28 | |
| *** inc0 has quit IRC | 13:30 | |
| *** inc0 has joined #openstack-kolla | 13:31 | |
| *** banix has joined #openstack-kolla | 13:34 | |
| *** mgoddard_ has joined #openstack-kolla | 13:44 | |
| *** mgoddard has quit IRC | 13:48 | |
| *** mbound has quit IRC | 13:51 | |
| *** mbound_ has joined #openstack-kolla | 13:51 | |
| *** cbits has quit IRC | 13:51 | |
| *** iceyao has joined #openstack-kolla | 13:54 | |
| *** cbits has joined #openstack-kolla | 13:56 | |
| andrewvwebber | after deploying kolla on coreos, creating an instance with nova fails within "validate_networks" | 14:01 |
| andrewvwebber | could this be due to an invalid global.yml by myself? i am attempted the veth pair approach | 14:02 |
| *** mgoddard has joined #openstack-kolla | 14:04 | |
| *** mgoddard_ has quit IRC | 14:04 | |
| inc0 | sbezverk, soo, whats up? | 14:05 |
| *** sdake has joined #openstack-kolla | 14:07 | |
| *** thomasem_ is now known as thomasem | 14:08 | |
| sbezverk | inc0 when I start a deployment from my deployment ubuntu server, I see an error< here is the paste: http://paste.openstack.org/show/493319/ | 14:08 |
| inc0 | I assume whole dir /etc/kolla has correct perms? | 14:09 |
| *** diogogmt has quit IRC | 14:09 | |
| inc0 | don't forget it want's to save there as well | 14:10 |
| sbezverk | inc0 I sent you email earlier with ls -al /etc/kolla | 14:10 |
| sbezverk | I think they are ok | 14:10 |
| inc0 | hmm maybe try changing it with -R to 777 | 14:11 |
| inc0 | and see if it helps | 14:11 |
| inc0 | chmod -R 777 /etc/kolla | 14:11 |
| *** sdake has quit IRC | 14:12 | |
| sbezverk | inc0 ok will try it | 14:12 |
| sbezverk | inc0 nope, did not help | 14:13 |
| inc0 | oO | 14:13 |
| inc0 | you have perms set up on every node right? | 14:13 |
| inc0 | but no | 14:14 |
| inc0 | you don't need ot | 14:14 |
| inc0 | I...have no idea | 14:15 |
| inc0 | never had perms issues | 14:15 |
| inc0 | like that | 14:15 |
| inc0 | sudo su and run deploy? | 14:15 |
| inc0 | btw what's your inventory | 14:15 |
| inc0 | ? | 14:15 |
| inc0 | you are ssh'ing to sbezverk user right? | 14:15 |
| inc0 | are you using become or sth there? | 14:15 |
| sbezverk | inc0 ok let me try to do "sudo su" | 14:16 |
| *** sdake has joined #openstack-kolla | 14:21 | |
| *** blahRus has joined #openstack-kolla | 14:23 | |
| sbezverk | inc0 it does not help :-( | 14:23 |
| inc0 | something is wrong in your buntu then | 14:24 |
| inc0 | terribly wrong | 14:24 |
| inc0 | if you can't make dir as root | 14:24 |
| sbezverk | inc0 no I can mkdir :-) | 14:24 |
| sbezverk | but when I try to run deploy | 14:24 |
| inc0 | what's your inventory? | 14:25 |
| sbezverk | in sudo su mode it fails to connect to remote host as I suspect it does not allow root to ssh in.. I use all in one inventory but all in one is run from deployment server.. | 14:26 |
| sdake | morning | 14:26 |
| inc0 | yeah, it won't allow ssh to root | 14:26 |
| inc0 | can change it in sshd.conf tho | 14:26 |
| inc0 | you'll also need to setup password for root | 14:27 |
| inc0 | alternatively you can run become sudo | 14:27 |
| sbezverk | inc0 but somebody surely has done ubuntu multinode I am surprised it did not pop up earlier. | 14:28 |
| inc0 | sbezverk, I'm doing ubuntu multinode several times a day | 14:28 |
| sbezverk | and you have not prepped in any way your test bed?? | 14:28 |
| inc0 | what you need to do is to change configs in sshd | 14:28 |
| inc0 | to allow root login | 14:29 |
| sbezverk | inc0 but there is nothing about it in documentation | 14:29 |
| inc0 | nothing too fancy tbh | 14:29 |
| sdake | sbezverk did you get results ont he qemu patch yet | 14:29 |
| *** SiRiuS_ has joined #openstack-kolla | 14:29 | |
| inc0 | sbezverk, well, we assume you can access to your machine over ssh;) | 14:29 |
| inc0 | in docs | 14:29 |
| sbezverk | sdake I cannot launch damn deployment on ubuntu | 14:29 |
| *** mwheckmann has joined #openstack-kolla | 14:29 | |
| sdake | oh, i thought or some reason you were using ubuntu all the time | 14:29 |
| sdake | but yur right your on centos | 14:29 |
| sbezverk | inc0 if requirements clearly stated then for sure | 14:29 |
| sdake | ori wouldn't have even asked you to try it | 14:30 |
| sdake | my bad | 14:30 |
| inc0 | well, that's ansible | 14:30 |
| inc0 | but by all means, add a paragraph | 14:30 |
| sdake | sbezverk ya we should document that sudoless root is required and the config needs to be set | 14:30 |
| inc0 | let's first get your stuff fixed | 14:30 |
| sbezverk | sdake I was using centos and red hat only | 14:30 |
| sdake | sbezverk my bad - i should have not asked yo uto test ubuntu | 14:31 |
| *** vhosakot has joined #openstack-kolla | 14:31 | |
| sbezverk | sdake no worries I need to get ubuntu going to get iscsi tested | 14:31 |
| *** SiRiuS has quit IRC | 14:31 | |
| sdake | sbezverk if you read th einternet it tells hwo to turn on root sudo | 14:31 |
| sdake | hwo/how | 14:31 |
| sdake | which version of ubuntu did you go with | 14:32 |
| sbezverk | 14.04.4 lts | 14:32 |
| sdake | make sure to ugprade the kernel | 14:32 |
| sdake | it needs to be 4.20 | 14:32 |
| sbezverk | it comes by default with 4.2 | 14:33 |
| sdake | ok | 14:33 |
| sdake | i think that hsould be fine then | 14:33 |
| *** iceyao has left #openstack-kolla | 14:37 | |
| vhosakot | morning! sdake: I edited the CentOS/qemu patch a bit last night, and looks like epel installation step works on CentOS and not on OracleLinux... pbourke: could you check https://review.openstack.org/#/c/302036/ | 14:38 |
| patchbot | vhosakot: patch 302036 - kolla - Use proper CentOS repositories for ceph and qemu | 14:39 |
| pbourke | vhosakot: sure, thanks | 14:39 |
| *** JoseMello has joined #openstack-kolla | 14:39 | |
| vhosakot | cool, thanks! | 14:39 |
| *** banix has quit IRC | 14:42 | |
| *** vhosakot has quit IRC | 14:43 | |
| *** vhosakot has joined #openstack-kolla | 14:43 | |
| pbourke | vhosakot: the cert on mirror.centos.org seems dodgy | 14:44 |
| pbourke | vhosakot: is seems to install ok if I use http:// | 14:44 |
| vhosakot | pbourke: ah ok, instead of https://, on OL ? | 14:45 |
| pbourke | yes - just trying to figure out why centos is ok with the cert but OL is not | 14:45 |
| inc0 | sbezverk, no it does not | 14:46 |
| inc0 | 14.04 doesn't have kernel new enough | 14:46 |
| inc0 | uname -a;) | 14:46 |
| pbourke | if you go to https://mirror.centos.org/ do you get a cert warning? | 14:46 |
| *** cbits has quit IRC | 14:46 | |
| vhosakot | pbourke: ok, the https://github.com/openstack/kolla/blob/master/docker/base/oraclelinux-extras.repo#L4-L6 is not affecting this PS, is it ? | 14:48 |
| pbourke | i dont think so | 14:49 |
| pbourke | vhosakot: tbh Im thinking of dropping OL support | 14:49 |
| pbourke | nobody is using it | 14:49 |
| pbourke | will need to discuss with sdake | 14:49 |
| vhosakot | yep, check with sdake | 14:50 |
| sdake | pbourke sup? | 14:50 |
| pbourke | sdake: hey | 14:50 |
| sdake | yo guys dont neeed to consult with me on everything | 14:50 |
| sdake | ust make a decision and moveo n ;) | 14:50 |
| pbourke | this is a little more fundamental | 14:50 |
| sdake | oh centos | 14:50 |
| pbourke | sdake: Im considering just dropping ol from kolla | 14:50 |
| sdake | extra repo we need | 14:50 |
| pbourke | kind of waiting for someone to talk me out of it ;) | 14:52 |
| *** salv-orl_ has joined #openstack-kolla | 14:52 | |
| sdake | oh drop ole support | 14:52 |
| sdake | why would weo that? | 14:52 |
| sbezverk | inc0 uname -a | 14:52 |
| sbezverk | Linux sbezverk-osp-5 4.2.0-27-generic #32~14.04.1-Ubuntu SMP Fri Jan 22 15:32:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | 14:52 |
| sbezverk | inc0 it comes with the image | 14:52 |
| inc0 | oO | 14:52 |
| sdake | ole support is easy to implement and as long as your around to maintain it i see zero reason to drop it | 14:52 |
| sdake | unless you plan on not being arund :( | 14:53 |
| pbourke | no Im around | 14:53 |
| pbourke | :) | 14:53 |
| inc0 | today is not my day | 14:53 |
| *** salv-orl_ has quit IRC | 14:53 | |
| inc0 | but I am fairly sure it doesn't come with an image! | 14:53 |
| sdake | so what is rationale for dropping ole | 14:53 |
| sdake | pbourke ^& | 14:53 |
| inc0 | they might have changed it...in last few weeks | 14:53 |
| inc0 | but doubt | 14:53 |
| sbezverk | inc0 I fixed permission, to get into another problem: http://paste.openstack.org/show/493348/ | 14:54 |
| *** salv-orlando has quit IRC | 14:54 | |
| inc0 | sbezverk, what was the issue? root ssh? | 14:54 |
| inc0 | sbezverk, tail /var/log/syslog plz | 14:55 |
| pbourke | sdake: I feel there's overhead in maintaining it but nobody is using it. Oracle are of course but we're very far behind upstream right now with no rebase in the near future | 14:55 |
| sbezverk | inc0 yeah, I copied ssh id of my regular user, but for the installation I need root, so I had to enable root access in sshd_config/reset root password/generate ssh key for root/copy root ssh key to the target | 14:56 |
| sdake | pbourke it costs little to maintian | 14:56 |
| sdake | pbourke you guys have forked - iget that, but you will rebase at some point i hope :) | 14:56 |
| pbourke | true | 14:56 |
| pbourke | trying to get the gates working is frustrating me | 14:56 |
| sdake | medication nation ftw | 14:57 |
| pbourke | but I think we're close in fairness | 14:57 |
| *** Serlex has quit IRC | 14:57 | |
| sdake | how do you mean close on fiarness | 14:57 |
| sdake | you mean parity? | 14:57 |
| pbourke | close to getting the OL gates passing | 14:57 |
| sdake | ya partiy i think is the word youwere after ;) | 14:57 |
| sbezverk | inc0 here you go http://paste.openstack.org/show/493349/ | 14:57 |
| pbourke | rabbitmq is fubared in OL | 14:57 |
| sdake | thats bad news | 14:58 |
| *** cristicalin has joined #openstack-kolla | 14:58 | |
| sdake | get em to fix it ;) | 14:58 |
| *** cristicalin has quit IRC | 14:58 | |
| pbourke | we were seeing similar problems in centos though I think? | 14:58 |
| *** cristicalin has joined #openstack-kolla | 14:59 | |
| sdake | rabbitmq works well for me | 14:59 |
| sdake | what problem do you see in ol? | 14:59 |
| *** mgoddard_ has joined #openstack-kolla | 15:00 | |
| pbourke | https://bugs.launchpad.net/kolla/+bug/1562701 | 15:00 |
| openstack | Launchpad bug 1562701 in kolla mitaka "rabbitmq crashed on centos binary deploy" [Critical,Confirmed] | 15:00 |
| pbourke | though im seeing a different issue locally | 15:00 |
| vhosakot | I added my observations at the end in that bug... found many links about it online | 15:00 |
| pbourke | yeah thanks for that vhosakot | 15:00 |
| inc0 | vhosakot, not much there... | 15:01 |
| vhosakot | i dont see that rabbitmq crash locally though.. only in gate | 15:01 |
| pbourke | vhosakot: for your other patch, would it be a problem to use http? | 15:01 |
| inc0 | are you using registry? | 15:01 |
| sdake | looks like vhosakot debugged it a bit | 15:01 |
| vhosakot | it is not mine :) sdake did all the work | 15:01 |
| vhosakot | I jsut debugged OLE errors | 15:01 |
| vhosakot | sdake: can we use http instead of https:// ? | 15:01 |
| sdake | debugged which ole work | 15:02 |
| sdake | i was talking about the rabbitmq gate crashes | 15:02 |
| vhosakot | https://review.openstack.org/#/c/302036/ | 15:02 |
| patchbot | vhosakot: patch 302036 - kolla - Use proper CentOS repositories for ceph and qemu | 15:02 |
| vhosakot | I debugged rabbitmq crash as well | 15:02 |
| sbezverk | inc0 yes, it is hosted on my deployment server | 15:02 |
| sdake | oh ya https is fine | 15:02 |
| vhosakot | pbourke: could you change it to http:// and check... | 15:03 |
| pbourke | vhosakot: want me to submit a PS? | 15:03 |
| sdake | i know why it fais | 15:03 |
| *** dougs1 has joined #openstack-kolla | 15:03 | |
| vhosakot | pbourke: I can do it.. 1 sec | 15:03 |
| pmisiak | hi, kolla-build is trying to push images to registry using https protocol | 15:03 |
| *** mgoddard has quit IRC | 15:03 | |
| pbourke | pmisiak: google docker insecure registry | 15:03 |
| *** wmiller has quit IRC | 15:03 | |
| *** dougs has quit IRC | 15:03 | |
| *** wmiller has joined #openstack-kolla | 15:04 | |
| pmisiak | should I set insecure registry on the operator node too? | 15:04 |
| inc0 | sbezverk, insecure registry is all set? | 15:04 |
| sbezverk | inc0 never mind, my bad.. | 15:05 |
| sdake | pmisiak wherever your pushing and pulling from | 15:05 |
| pmisiak | ok, thanks | 15:05 |
| pmisiak | I missed that, I set it only on target nodes | 15:05 |
| inc0 | sbezverk, don't worry, it's not best of my days either;0 | 15:06 |
| *** ccesario___ has quit IRC | 15:08 | |
| sbezverk | inc0 I am planning to turn this day into a good day with iscsi working on ubuntu as well ;-) | 15:09 |
| *** diogogmt has joined #openstack-kolla | 15:09 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 15:10 |
| inc0 | you sir have my blessing and support, I really want to see it working for some of my own egoistic reasons | 15:10 |
| pmisiak | I suppose master is unstable branch? | 15:10 |
| pmisiak | should I use stable/* branches ? | 15:11 |
| inc0 | pmisiak, we try to keep it stable | 15:11 |
| inc0 | stable/mitaka is around master now | 15:11 |
| pmisiak | ok | 15:11 |
| inc0 | and stable liberty is not advised | 15:11 |
| pmisiak | so master will be my choice :) | 15:12 |
| pmisiak | do you support libvirt and ironic in the same time? | 15:12 |
| pmisiak | few months ago when I enabled ironic, nova-compute didn't have libvirt enabled (only ironic) | 15:13 |
| *** daneyon has joined #openstack-kolla | 15:13 | |
| pmisiak | what about centos vs ubuntu? | 15:13 |
| pmisiak | I prefer ubuntu, is it stable as well as centos? | 15:14 |
| sdake | pmisiak centos ole ubuntu are all in good shape, debian fedora atm are not quit ethere yet | 15:15 |
| sdake | pmisiak we have gates for centos and ubuntu that test our software per commit and ole gat ecoming soon | 15:16 |
| pmisiak | great | 15:17 |
| sdake | as far as ironic goes, that is tech preview | 15:17 |
| sdake | we are not quite sure what to do with it yet | 15:17 |
| *** daneyon has quit IRC | 15:17 | |
| sdake | its inthe code it sort of work | 15:17 |
| sdake | s | 15:18 |
| pmisiak | what about ironic&libvirt ? will I be able to spawn libvirt VM when I enable ironic ? | 15:18 |
| sdake | but it doesnt work entirely the way i think i'd like it to | 15:18 |
| sdake | pmisiak i dont know the exact answer to that jpeeler may be able to expand | 15:18 |
| openstackgerrit | Merged openstack/kolla: Fix the Neutron reconfigure https://review.openstack.org/301737 | 15:18 |
| pmisiak | ERROR:kolla.cmd.build:unauthorized: authentication required | 15:18 |
| pmisiak | is it something with registry? | 15:18 |
| openstackgerrit | Merged openstack/kolla: Added Kibana instructions to quickstart.rst https://review.openstack.org/302458 | 15:19 |
| vhosakot | pmisiak: can you paste "ps aux | grep docker" | 15:20 |
| pmisiak | root 17594 24.9 2.1 2695204 88092 ? Ssl 17:08 3:33 /usr/bin/docker daemon --insecure-registry 10.94.7.2:4000 | 15:22 |
| pmisiak | root 17683 0.0 0.7 363828 28696 ? Sl 17:08 0:00 docker-proxy -proto tcp -host-ip 10.94.7.2 -host-port 4000 -container-ip 172.17.0.2 -container-port 5000 | 15:22 |
| pmisiak | root 17722 0.0 0.5 257120 21892 ? Ssl 17:08 0:00 /bin/registry /etc/docker/registry/config.yml | 15:22 |
| *** achanda has joined #openstack-kolla | 15:24 | |
| sdake | pmisiak docker 10.0.3? | 15:24 |
| sdake | rather 1.10.3 | 15:24 |
| pmisiak | Docker version 1.10.3, build 20f81dd | 15:24 |
| sdake | did you use the qsg instructions to create your registry? | 15:25 |
| sdake | or some other instructions from docker inc | 15:25 |
| pmisiak | I used instruction from this site: http://docs.openstack.org/developer/kolla/quickstart.html | 15:25 |
| vhosakot | pmisiak: are you passing --registry 10.94.7.2:4000 to kolla-build ? | 15:25 |
| pmisiak | ups :/ | 15:28 |
| pmisiak | I had registry variable in the kolla-build.conf | 15:28 |
| pmisiak | but it's commented | 15:28 |
| pmisiak | sorry | 15:28 |
| *** cristicalin has quit IRC | 15:29 | |
| sdake | sierra happens ) | 15:29 |
| jpeeler | pmisiak: the problem i recall is that ironic required dedicated networking, so there's work to be done to support both ironic deployments along with VMs | 15:31 |
| jpeeler | and the fact that ironic itself probably needs some love | 15:31 |
| pmisiak | ;) | 15:31 |
| jpeeler | maybe one day i'll get a chance to look at it | 15:31 |
| pmisiak | my goal is to have ironic with multitenancy networking :D | 15:31 |
| sdake | pmisiak even ironic doesn't relally do that yet ;) | 15:32 |
| sdake | with or without kolla | 15:32 |
| pmisiak | yeahh | 15:32 |
| pmisiak | :) | 15:32 |
| pmisiak | i know | 15:32 |
| pmisiak | what can i say.. | 15:32 |
| sdake | that is the same thing i want out of ironic | 15:32 |
| pmisiak | i have to :D | 15:32 |
| vhosakot | pmisiak: yep, please uncomment it in kolla-build.conf :) | 15:32 |
| pmisiak | as I see in the code I can have all controllers listed in network group in the inventory? | 15:34 |
| pmisiak | I don't want to have a separate and single network node | 15:34 |
| sdake | yup that should be fine | 15:35 |
| *** SiRiuS_ has quit IRC | 15:36 | |
| *** achanda has quit IRC | 15:37 | |
| andrewvwebber | is this anything to be concerned about - "No protocol specified in for api_server 'http://172.20.30.167:9292', please update [glance] api_servers with fully qualified url including scheme (http / https)" | 15:39 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add iscsid/tgtd specific parameters to Kolla infrastructure https://review.openstack.org/300004 | 15:39 |
| pmisiak | what is the difference between kolla_external_vip_interface vs neutron_external_interface | 15:41 |
| coolsvap | sdake: https://bugs.launchpad.net/kolla/+bug/1530415 | 15:41 |
| openstack | Launchpad bug 1530415 in kolla "Override interface name in inventory" [High,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 15:41 |
| pmisiak | neutron_external_interface will be bridged by OVS ? | 15:41 |
| coolsvap | the review for it is already in master & mitaka | 15:41 |
| pmisiak | and on the kolla_external_vip_interface public API will be available ? | 15:42 |
| coolsvap | i think it was waiting to be merged in liberty, but we have bug for it marked for master & mitaka | 15:42 |
| sdake | coolsvap can yu put links to the bugs in launchpad | 15:43 |
| sdake | i dont know why launchpad isn't tracking the changes for that patch properly | 15:43 |
| *** mikelk has quit IRC | 15:43 | |
| pmisiak | so I can't have OS API and floating IP on the same network/subnet? | 15:44 |
| pmisiak | hmm I can, but I have configure public IPs on the OVS bridge interface and on the neutron_external_interface directly | 15:45 |
| coolsvap | sdake: updated | 15:46 |
| coolsvap | please check i think we can mark it completed or whatever | 15:46 |
| sdake | coolsvap you got links to the bugs | 15:48 |
| sdake | to the revies | 15:48 |
| coolsvap | yes | 15:48 |
| coolsvap | 1 min | 15:48 |
| coolsvap | sdake: this is merged https://review.openstack.org/#/c/247999 | 15:49 |
| andrewvwebber | pmisiak: i am experimenting with a veth bridge with the following scripts - https://gist.github.com/andrewwebber/f788790772cef47ed5bff39aae2212cf | 15:49 |
| coolsvap | sdake: https://review.openstack.org/#/c/289241 is targeted for liberty | 15:49 |
| sdake | wow this merged in december | 15:49 |
| sdake | coolsvap that is wrong | 15:50 |
| sdake | look at this review https://review.openstack.org/#/c/289241/ | 15:51 |
| patchbot | sdake: patch 289241 - kolla (stable/liberty) - Fix retrieving api_interface | 15:51 |
| sdake | this patch has a -2 from jeffrey | 15:51 |
| vhosakot | pmisiak: https://github.com/openstack/kolla/blob/master/etc/kolla/globals.yml has all the info about kolla_external_vip_interface and neutron_external_interface | 15:51 |
| coolsvap | sdake: yes but the bug is already merged in mitaka and newton :) | 15:51 |
| coolsvap | I am just checking through all the 2.0.0 bugs | 15:52 |
| sdake | ya sometiems launchpad times out and gerrit doesn't update the bug status | 15:52 |
| pmisiak | vhosakot: so neutron_external_interface will be part of "br-ex" bridge ? | 15:54 |
| pmisiak | (i dont know what name is it in kolla) | 15:54 |
| pmisiak | and it will be used for routers, floating ips etc. | 15:55 |
| pbourke | vhosakot: im going to try build that patch locally | 15:56 |
| *** shakamunyi has quit IRC | 15:56 | |
| vhosakot | pbourke: it failed again with http ? | 15:57 |
| pbourke | vhosakot: no its failing with another error now | 15:57 |
| pbourke | https://jenkins01.openstack.org/job/gate-kolla-dsvm-build-oraclelinux-binary/7/console | 15:57 |
| openstackgerrit | Merged openstack/kolla: Make Heka write Keepalived logs to dedicated directory https://review.openstack.org/301452 | 15:57 |
| vhosakot | pbourke: ah ok... | 15:58 |
| pmisiak | so kolla_external_vip_interface can not be the same as neutron_external_interface | 15:58 |
| vhosakot | pbourke: yes, excatly, neutron_external_interface (eth1) will be a member of the br-ex bridge .. http://paste.openstack.org/show/493360/ | 15:59 |
| vhosakot | pmisiak: ^^^ | 16:00 |
| vhosakot | pmisiak: yes, kolla_external_vip_interface (that defaults to network_interface which is eth0) can not be the same as neutron_external_interface (eth1) | 16:00 |
| vhosakot | pmisiak: http://paste.openstack.org/show/493360/ | 16:00 |
| vhosakot | pbourke: http://logs.openstack.org/36/302036/17/experimental/gate-kolla-dsvm-build-centos-7-source/74de379/console.html#_2016-04-07_15_40_12_672 | 16:02 |
| vhosakot | pbourke: I will do a recheck | 16:02 |
| *** jasonsb has quit IRC | 16:02 | |
| pbourke | vhosakot: there's no point | 16:03 |
| vhosakot | ah ok | 16:03 |
| vhosakot | what is the issue... http caused this ? | 16:03 |
| pbourke | give me a few mins, I have it reproduced locally | 16:03 |
| *** jasonsb has joined #openstack-kolla | 16:03 | |
| pbourke | no i think missing packages | 16:03 |
| *** dougs has joined #openstack-kolla | 16:03 | |
| pmisiak | vhosakot: thanks! | 16:03 |
| *** dave-mccowan has joined #openstack-kolla | 16:03 | |
| *** mbound_ has quit IRC | 16:04 | |
| vhosakot | pbourke: ah ok | 16:04 |
| *** dougs1 has quit IRC | 16:05 | |
| pmisiak | neutron-openvswitch-agent is not building | 16:06 |
| pmisiak | INFO:kolla.cmd.build:neutron-openvswitch-agent:The following packages have unmet dependencies: | 16:06 |
| pmisiak | INFO:kolla.cmd.build:neutron-openvswitch-agent: neutron-plugin-openvswitch : Depends: neutron-common (= 1:2014.1.5-0ubuntu4) but 2:8.0.0~b2-0ubuntu7~cloud0 is to be installed | 16:06 |
| *** mgoddard_ has quit IRC | 16:06 | |
| pmisiak | base=ubuntu source=binary | 16:07 |
| *** mgoddard has joined #openstack-kolla | 16:08 | |
| pbourke | vhosakot: we may have to stick with the existing ceph repos for OL | 16:08 |
| pbourke | the new ones seem heavily bound to centos | 16:08 |
| pbourke | the issue would be if there were version discrepencies | 16:09 |
| pbourke | what is this centos-release rpm | 16:10 |
| pbourke | not sure why ceph needs to be dependent on it | 16:11 |
| vhosakot | pbourke: ah ok.. I think all those are needed for ceph/hammer | 16:11 |
| pbourke | http://paste.openstack.org/show/493365/ | 16:11 |
| vhosakot | pbourke: isn't there an OL equivalent for these three lines ? | 16:11 |
| vhosakot | RUN yum install -y http://mirror.centos.org/centos-7/7/extras/x86_64/Packages/epel-release-7-5.noarch.rpm | 16:11 |
| vhosakot | RUN yum install -y http://mirror.centos.org/centos-7/7/extras/x86_64/Packages/centos-release-ceph-hammer-1.0-5.el7.centos.noarch.rpm | 16:11 |
| vhosakot | RUN yum install -y http://mirror.centos.org/centos-7/7/extras/x86_64/Packages/centos-release-qemu-ev-1.0-1.el7.noarch.rpm | 16:11 |
| vhosakot | RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 | 16:11 |
| pbourke | unfortunately no | 16:12 |
| vhosakot | ignore last line pasted... | 16:12 |
| vhosakot | ah ok | 16:12 |
| pbourke | tbh these repos may have issues on RHEL also | 16:12 |
| vhosakot | pbourke: is http://paste.openstack.org/show/493365/ from you dev machine or gate ? | 16:13 |
| *** coolsvap has quit IRC | 16:13 | |
| pbourke | gate | 16:13 |
| pbourke | oh wait, sorry | 16:13 |
| pbourke | from my machine, I just wanted to see what actually was in centos-release | 16:13 |
| *** unicell has quit IRC | 16:13 | |
| vhosakot | ah ok | 16:14 |
| pmisiak | quit | 16:14 |
| pmisiak | :) | 16:14 |
| *** pmisiak has quit IRC | 16:14 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Fix the Neutron reconfigure https://review.openstack.org/302930 | 16:15 |
| *** unicell has joined #openstack-kolla | 16:15 | |
| pbourke | sdake: do you know if these packages install ok on RHEL? | 16:16 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Adds migration step from nova to nova_api database https://review.openstack.org/302931 | 16:16 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/302856 | 16:19 |
| *** pmisiak has joined #openstack-kolla | 16:20 | |
| sdake | which packages | 16:21 |
| sdake | pbourke the problem with them is they use releasever variable | 16:21 |
| sdake | releasever on oracle linux is 7Server | 16:21 |
| pbourke | the qemu/ceph ones | 16:21 |
| sdake | on centos is 7 | 16:21 |
| sdake | relesaever needs to be seed into the number 7 | 16:21 |
| pbourke | yeah the releasever thing is annoying | 16:21 |
| sdake | in the repo files | 16:21 |
| sdake | seed/sed | 16:21 |
| pbourke | but I dont think its relevant to this particular patch | 16:21 |
| sdake | that is the solution | 16:21 |
| sdake | to making the gate pass | 16:22 |
| pbourke | what im saying is, these new qemu/ceph rpms are heavily dependant on centos | 16:22 |
| pbourke | they dont install on OL and im not sure they would on RHEL either | 16:22 |
| sdake | they are dependent on epel which we already install | 16:22 |
| sdake | they install on ol | 16:22 |
| pbourke | they depend on a package called "centos-release" | 16:22 |
| pbourke | they dont :) | 16:22 |
| sdake | oh ok ;) | 16:22 |
| sdake | well i haven't debugged it yet | 16:22 |
| pbourke | i'll put my findings on the review | 16:22 |
| sdake | rhel already ha qemu 2.3 | 16:22 |
| sdake | i wil debug it before patch merges | 16:23 |
| pbourke | we could maybe just install them for centos but then we'll be in a situation where different bases have different versions | 16:23 |
| pbourke | which I dont think we want | 16:23 |
| sdake | wecoudl nstall them with --nodeps ;) | 16:23 |
| pbourke | heh yeah it crossed my mind | 16:23 |
| sdake | all they do is insall repo files and gpg keys | 16:23 |
| pbourke | seems like a bad idea though I dunno | 16:24 |
| sdake | its not a terrible idea | 16:24 |
| pbourke | ok let me try locally with that | 16:24 |
| sdake | the idea is to use mirrors.* insted of hardcoded paths | 16:24 |
| pbourke | those packages shouldnt depend on that rpm though | 16:24 |
| pbourke | in the first place | 16:24 |
| sdake | it depends on that rpm because of key signing | 16:25 |
| sdake | it rquires the centos key to install i suspect | 16:26 |
| *** coolsvap has joined #openstack-kolla | 16:26 | |
| *** daneyon_ has quit IRC | 16:28 | |
| *** daneyon has joined #openstack-kolla | 16:29 | |
| *** unicell has quit IRC | 16:31 | |
| *** dwalsh has quit IRC | 16:33 | |
| pbourke | sdake: are the 'rpm --imports' important? | 16:33 |
| sdake | no idea who put those in or why | 16:33 |
| sdake | i didn't do that | 16:33 |
| pbourke | vhosakot: did you add the rpm --import lines? | 16:33 |
| pbourke | im guessing they're if you want to verify the package integrity after the fact | 16:34 |
| vhosakot | pbourke: yes, for OL | 16:34 |
| pbourke | one of them is missing on OL | 16:34 |
| pbourke | so if they're not super important Id suggest leaving them out? | 16:34 |
| sdake | i added a bunch of them for ol but i was just copying what was already there | 16:34 |
| vhosakot | pbourke: sdake added it initially in PS 13, and I broke the single RUN command into multiple RUNs to isoltae error | 16:34 |
| sdake | use git blame on pristine master to see who added them | 16:34 |
| sdake | and what the commit message/bugid says | 16:34 |
| pbourke | vhosakot: do you mind if I submit a PS? | 16:35 |
| vhosakot | pbourke: sure... :) | 16:35 |
| vhosakot | pbourke: i meant I wont mind :) go ahead submit the PS | 16:35 |
| pbourke | :) | 16:35 |
| sdake | i dont mind either | 16:36 |
| sdake | the reason people care is because committer is used in stackalytics | 16:36 |
| sdake | rather then author line | 16:36 |
| sdake | for counting stats | 16:36 |
| sdake | as stupid as that is | 16:36 |
| *** haplo37 has joined #openstack-kolla | 16:37 | |
| vhosakot | sdake: I thought all co-authors get numbers | 16:37 |
| vhosakot | sdake: anyway, numbers is secondary... we'll try to fix it on OL :) | 16:37 |
| vhosakot | pbourke: it is all sdake's work.. all I did was to break down the single RUN into multiple RUNs to isolate error on OL | 16:38 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Configure Neutron to use internalURL for contacting Nova https://review.openstack.org/302948 | 16:40 |
| pbourke | actually I forgot another question | 16:42 |
| pbourke | why are we installing ceph in base | 16:42 |
| pbourke | and qem | 16:42 |
| pbourke | *qemu | 16:42 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 16:44 |
| *** daneyon_ has joined #openstack-kolla | 16:46 | |
| dougs | anyone got a Kolla spec file (for rpm)? | 16:48 |
| *** JoseMello has quit IRC | 16:49 | |
| sdake | we are installling the repos in base | 16:49 |
| sdake | this is our modle pbourke | 16:49 |
| pbourke | sdake: yeah it just dawned on me those are not the actual packages | 16:49 |
| sdake | that way we can substitute repos in one palce instead of all over the repo | 16:49 |
| pbourke | that's cool | 16:49 |
| sdake | dougs i wil shortly | 16:50 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/302856 | 16:50 |
| sdake | dougs curious - may I ask why you desire such a thing :) | 16:50 |
| dougs | sdake: we want to use it as part of our distro | 16:50 |
| sdake | dougs which distro | 16:51 |
| sdake | dougs i will likely have the spec file done this weekish and happy to share | 16:51 |
| *** daneyon_ has quit IRC | 16:52 | |
| dougs | sdake: thanks, pmed | 16:52 |
| sdake | dougs i assume you have tried out kolla if you want ot rpm'ize it | 16:55 |
| sdake | what are the top 3 pain points and top 3 gaps you see with it? | 16:56 |
| manjeets | vhosakot ping | 16:57 |
| mgoddard | sdake, dougs is with me at Cray. We decided to go with Kolla over OSA to provide Cray's OpenStack deployment framework | 16:58 |
| sdake | mgoddard interesting | 16:58 |
| vhosakot | manjeets: pong | 16:58 |
| sdake | cool well the model of kolla is people use it as an upstream for their downstream products | 16:58 |
| sdake | (just like the rest of the openstack model) | 16:58 |
| vhosakot | manjeets: bbib 20 mins.. leave msg.. will come back and catch up | 16:59 |
| sdake | this is a model i want to actively promote - oracle has already done it | 16:59 |
| sdake | if an rpm would help - makes sense to produce one ;) | 16:59 |
| mgoddard | biggest pain point so far is image build times. We've not spent much time investigating yet, but I've seen it take 5 hours to build all the images | 16:59 |
| dougs | mgoddard: pain points, build time no.1? | 16:59 |
| mgoddard | admittedly on a VM on my laptop | 17:00 |
| manjeets | still having a linux brdige updated pip and setuptools manually on one of network nodes | 17:00 |
| manjeets | vhoasakot | 17:00 |
| manjeets | issue** | 17:00 |
| sdake | vm laptoop lots of memcpy ;) | 17:00 |
| mgoddard | I'm sure there are things we can do to improve | 17:00 |
| sdake | fwiw it takes me 14 minutes to build centos source on intel 750 nvme ssd | 17:00 |
| mgoddard | sure - it's not our final environment | 17:00 |
| sdake | and i have gige intenet at my home | 17:01 |
| mgoddard | that's more like what I'm hoping for | 17:01 |
| mgoddard | perfect | 17:01 |
| sdake | solution is to put it on a intel 750 ssd - 350 bucks for 400gb disk | 17:01 |
| sdake | and get a fast intenet connection ;) | 17:01 |
| mgoddard | we'll be building from a local set of yum repos, so hopefully BW won't be an issue | 17:01 |
| sdake | a local mirror will speed things up dramatically | 17:01 |
| sbezverk | inc0 ping | 17:02 |
| sdake | by put it, i mean put /var/libdocker | 17:02 |
| sdake | /var/lib/docker | 17:02 |
| inc0 | go ahead sbezverk | 17:02 |
| mgoddard | any hints on docker performance tuning for builds? | 17:02 |
| sdake | i ru nwith overlayfs | 17:02 |
| sdake | although its a bit flakey | 17:02 |
| sdake | i wouldn't recommend that for production | 17:02 |
| sbezverk | inc0 I told you this day is going to change for better ;-) check this out: http://paste.openstack.org/show/493375/ | 17:02 |
| sdake | run docker 1.10 at minimum and registry 2.3 at minimum | 17:02 |
| inc0 | btrfs is not far away performance wise and it's stable | 17:03 |
| sdake | agree with inc0 btrfs is the best solution | 17:03 |
| sdake | if your uncomfortable with that graph driver, use lvm+thin | 17:03 |
| sdake | the performance of docker is heavily affected by the graph driver | 17:03 |
| sdake | we use natie networkign in kolla | 17:03 |
| sdake | although the build doesnt 'use native networking | 17:03 |
| sdake | by native networking i mean --net=host | 17:04 |
| *** andrewvwebber has quit IRC | 17:04 | |
| inc0 | cool sbezverk | 17:04 |
| inc0 | now get this iscsi working;) | 17:04 |
| inc0 | and I'll be happier person | 17:04 |
| *** dwalsh has joined #openstack-kolla | 17:05 | |
| sdake | to make your registry go as fast as possible bindmount the registry store | 17:05 |
| sdake | but that wont work if you want to scale out your registry | 17:05 |
| sdake | what about gaps | 17:05 |
| sdake | any you see? | 17:05 |
| sdake | mgoddard ^^ | 17:05 |
| sdake | dougs ^^ | 17:06 |
| sbezverk | inc0 it is working centos vm is using iscsi/lvm2 | 17:06 |
| sbezverk | cinder list | 17:07 |
| sbezverk | +--------------------------------------+--------+----------------+------+-------------+----------+--------------------------------------+ | 17:07 |
| sbezverk | | ID | Status | Name | Size | Volume Type | Bootable | Attached to | | 17:07 |
| sbezverk | +--------------------------------------+--------+----------------+------+-------------+----------+--------------------------------------+ | 17:07 |
| sbezverk | | 7cbb2752-90ce-4774-969d-b5fdfb0fab46 | in-use | centos7-1-disk | 40 | - | true | 07ef480a-7b73-4258-97e4-261d62aedad7 | | 17:07 |
| sbezverk | | b97140db-6de3-44b0-957c-5619bf4ee459 | in-use | | 5 | - | true | 07ef480a-7b73-4258-97e4-261d62aedad7 | | 17:07 |
| sbezverk | +--------------------------------------+--------+----------------+------+-------------+----------+--------------------------------------+ | 17:07 |
| *** rmart04 has quit IRC | 17:07 | |
| sdake | pastebin ftw | 17:07 |
| sbezverk | sdake sorry | 17:07 |
| sdake | nobodoy can actually read that output in irc anyway :) | 17:08 |
| sdake | need to view raw in a paste service to see oepnstack cli output | 17:08 |
| inc0 | sbezverk, any issues? | 17:08 |
| inc0 | did you ran into issue I had | 17:08 |
| inc0 | ? | 17:08 |
| sdake | mgoddard curious what decision point was on osa vs kolla | 17:09 |
| sdake | mind sharing? | 17:09 |
| dougs | sdake: HA is probably the main missing part | 17:09 |
| dougs | (for all services) | 17:09 |
| *** bmotz has joined #openstack-kolla | 17:09 | |
| sdake | kolla has ha | 17:09 |
| sbezverk | inc0 I did not because I fixed it beforehand | 17:09 |
| inc0 | by this fstab stuff? | 17:09 |
| sdake | kolla is nway-active except for the database which is n+m (1/2) | 17:10 |
| sbezverk | inc0 http://paste.openstack.org/show/493377/ | 17:10 |
| dougs | sdake: cool - an oversight on my part. Any support for active/passive? | 17:10 |
| mgoddard | last I checked, there was no SSL support, although I understand that was on the cards for Mitaka | 17:10 |
| *** unicell has joined #openstack-kolla | 17:10 | |
| sdake | the database is n+m (active/passive) | 17:10 |
| sdake | we hae external tls implmeented | 17:11 |
| inc0 | sdake, not true, db is galera | 17:11 |
| inc0 | which is a/a | 17:11 |
| sdake | although apparently buggy :) | 17:11 |
| inc0 | with few caveats to it | 17:11 |
| sdake | i thought db was n+m | 17:11 |
| inc0 | galera is a/a | 17:11 |
| sdake | shows what I knwo :) | 17:11 |
| sdake | ok then our entire system is nway active | 17:11 |
| sdake | active/active implies 2 nodes, we requrie 3 for clustering | 17:12 |
| inc0 | dougs, which part you'd like to be a/p? | 17:12 |
| sdake | we can actually use more - one cat tried 7 | 17:12 |
| dougs | sdake: mainly the DB for 2 node HA | 17:12 |
| *** mwheckmann has quit IRC | 17:12 | |
| inc0 | so neutron l3 is still a/p but not much we can do about it | 17:12 |
| *** salv-orlando has joined #openstack-kolla | 17:12 | |
| inc0 | dougs, yeah, if you want ha you need at least 3 | 17:12 |
| dougs | cinder-volume | 17:12 |
| inc0 | but that's well...how you do ha | 17:12 |
| sdake | where our ha is weak is we dont detect failures and recover from them | 17:13 |
| inc0 | dougs, we use ceph as main cinder backend so not really a/p | 17:13 |
| sdake | which is steps 1-2 of an ha system ;) | 17:13 |
| *** jasonsb has quit IRC | 17:13 | |
| mgoddard | external TLS is fine for us. we are looking to use certificate based auth also | 17:13 |
| inc0 | but yeah, we won't be more HA than openstack | 17:13 |
| sdake | detect/isolate/recover/notify | 17:13 |
| sdake | we have none of this | 17:13 |
| sdake | what we do is deploy in nway-active mode | 17:14 |
| sdake | we need to detect/isolate/recover/notify on faults | 17:14 |
| dougs | sdake: hence my questions regarding pacemaker a few weeks back! | 17:14 |
| sdake | i see that as a gap vs some other deployment solutions | 17:14 |
| sdake | dougs see millions of questios here, dont recall pacemaker question | 17:14 |
| sdake | but ya, i wrote most of corosync | 17:15 |
| sdake | so I'm familiar with pacemaker | 17:15 |
| dougs | sdake: I think the conclusion was the network-isolation was required and that pacemaker was overkill | 17:15 |
| dougs | *that | 17:15 |
| sdake | since andrew and i partnered on making linux highly available l:) | 17:15 |
| sdake | pacemaker is overkilll | 17:15 |
| *** salv-orlando has quit IRC | 17:15 | |
| coolsvap | sdake: should we abandon this patch https://review.openstack.org/#/c/300888 and mark the bug https://bugs.launchpad.net/kolla/+bug/1565373 as invalid ? | 17:16 |
| openstack | Launchpad bug 1565373 in kolla mitaka "haproxy container does not have /var/log/kolla directory" [High,Confirmed] - Assigned to Carlos Cesario (ccesario) | 17:16 |
| inc0 | dougs, what kind of net isolation you're reffering to? | 17:17 |
| *** mwheckmann has joined #openstack-kolla | 17:17 | |
| dougs | inc0: possibility of VIPs active on two instances at one time | 17:17 |
| inc0 | ah, you mean split brain? | 17:18 |
| dougs | yeah | 17:18 |
| inc0 | yeah, keepalived sux in this matter | 17:18 |
| inc0 | and pacemaker is overkill | 17:18 |
| sdake | coolsvap not yet | 17:18 |
| sdake | see last comment | 17:18 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/302856 | 17:19 |
| *** Marga_ has quit IRC | 17:19 | |
| inc0 | we had precisely this discussion in Vancouver | 17:19 |
| mgoddard | on the reasons for choosing Kolla over OSA - it was close. It mostly came down to ease of use, simplicity (esp networking), use of Docker, immutable images philosophy, separation of build and deployment | 17:19 |
| mgoddard | support for CentOS was a big one also | 17:19 |
| sdake | that was close? :) | 17:19 |
| sdake | what did osa have going for it then | 17:19 |
| coolsvap | sdake: we do have the /var/log/haproxy let me check on heka | 17:19 |
| inc0 | mgoddard, speed of deployment and fact that kolla doesn't eat up 7 ips per node might be good consideration as well | 17:20 |
| inc0 | ;) | 17:20 |
| mgoddard | inc0 - yes, both good | 17:20 |
| inc0 | one thing mgoddard don't deploy stable liberty yet | 17:21 |
| mgoddard | OSA has more stability, support for multiple containers of a given type on a single node, support for Kilo | 17:21 |
| mgoddard | inc0, why not? | 17:21 |
| inc0 | kinda big backport is coming up, it will kick stability tremendously | 17:21 |
| mgoddard | we were planning to use stable/mitaka, and port back to kilo OpenStack | 17:21 |
| inc0 | that would work | 17:21 |
| mgoddard | will all features be ported back to L? | 17:21 |
| inc0 | issue was in liberty we were bound to docker 1.8 and it didn't have features we needed for data persistance | 17:22 |
| mgoddard | another gap - Monasca. We use it for monitoring | 17:22 |
| inc0 | should be quick merge if you guys want to contribute | 17:22 |
| mgoddard | we'll have to develop it first ;) | 17:22 |
| mgoddard | but yes, we'll be keen to contribute | 17:23 |
| inc0 | deployment of big tent is one of our priorities, so Monaska fit it perfectly | 17:23 |
| inc0 | or Monasca even | 17:23 |
| mgoddard | comes with a lot of baggage - kafka, zookeeper, apache storm... | 17:23 |
| mgoddard | but at least it's not ceilometer :) | 17:23 |
| inc0 | not that easy then, I think we already had zookeeper containers | 17:24 |
| inc0 | :D | 17:24 |
| sdake | baggage is ok in a container deployment system | 17:24 |
| *** rajathagasthya has joined #openstack-kolla | 17:24 | |
| sdake | because it doesnt make a mess of your system in teh first place if its not turned on | 17:24 |
| coolsvap | sdake: updated review with comments, it think we can abandon this change set | 17:25 |
| mgoddard | in theory | 17:25 |
| sdake | coolsvap no ccesario can abandon if he chooses | 17:25 |
| sdake | we shouldn't abandon other peopels work | 17:25 |
| inc0 | mgoddard, well, unless it messes with kernel that is | 17:25 |
| coolsvap | sdake: alright | 17:25 |
| sdake | lets look at the bug next | 17:25 |
| coolsvap | np | 17:25 |
| mgoddard | very much like the config templating & layering | 17:25 |
| inc0 | if it doesn't touch kernel, you're usually good to just purge it and it's volumes | 17:25 |
| *** slagle has quit IRC | 17:26 | |
| mgoddard | would you be open to conf.d support for the config layers? | 17:26 |
| inc0 | mgoddard, as long as it makes sense and doesnt cause tons of mess | 17:26 |
| *** Marga_ has joined #openstack-kolla | 17:26 | |
| inc0 | which it should not if you do it correctly | 17:27 |
| mgoddard | context: we have multiple platforms that sit on top of us, each wanting to configure us in some way. Conf.d comes up a lot... | 17:27 |
| sdake | what is conf.d | 17:27 |
| inc0 | so about being open to features, ansible gives us enough flexibility to turn off and on features as we please | 17:27 |
| mgoddard | directory of config files | 17:27 |
| sdake | you can override configuation via /etc/kolla/config | 17:28 |
| inc0 | so if you have need for some feature we can always merge it and turn it off as default | 17:28 |
| sdake | make an /etc/kolla/config/nova.conf | 17:28 |
| sdake | and put in what you like | 17:28 |
| inc0 | sdake, not really what mdnadeem means | 17:28 |
| inc0 | but yeah, we should be ok | 17:28 |
| sdake | are we back to the plugins discussio nthen inc0? | 17:28 |
| mgoddard | /etc/kolla/config/nova.conf.d/abc.conf | 17:28 |
| inc0 | and confs are merged I assume | 17:29 |
| inc0 | or included in main conf | 17:29 |
| sdake | what goes in abc.conf? | 17:29 |
| *** banix has joined #openstack-kolla | 17:30 | |
| * coolsvap will be sleeping now will check remaining 2.0.0 bugs in morning | 17:31 | |
| sdake | coolsvap good job on catching that bug - one less on our plates ;) | 17:32 |
| mgoddard | inc0, confs are merged | 17:35 |
| sdake | mgoddard we got that arleady bro ;) | 17:35 |
| inc0 | mgoddard, well, depends what you need | 17:35 |
| sdake | just put your conig options in /etc/kolla/config/nova.conf | 17:35 |
| inc0 | but as sdake says, we have confs overrides already | 17:36 |
| sdake | and it overrides the default | 17:36 |
| inc0 | one thing we might need is to have centos.conf and ubuntu.conf and somehow put only correct one in place | 17:36 |
| inc0 | but that's possible and don't forget that these confs lives inside container | 17:37 |
| mgoddard | right. in some cases we have multiple sources for Nova configuration, which each need to own their own config file | 17:37 |
| inc0 | so you will in general have exactly same environment | 17:37 |
| vhosakot | mgoddard: steps to reconfigure services / override defauls configs are in https://etherpad.openstack.org/p/kolla-mitaka-testing-reconfigure | 17:37 |
| inc0 | besides things like interface names, but these are possible to override in inventory | 17:37 |
| mgoddard | it's possible that we can manage this part ourselves, and provide kolla with a single file. Still TBD. | 17:37 |
| inc0 | I'm sure we can think of something | 17:38 |
| sdake | mgoddard whre olla is weak is managing polciy files nad the like | 17:38 |
| mgoddard | sure - it's not an immediate requirement | 17:38 |
| inc0 | btw mgoddard out of curiosity, which company are you from? | 17:38 |
| *** coolsvap has quit IRC | 17:38 | |
| mgoddard | I work for Cray. | 17:39 |
| mgoddard | inc0, and you? | 17:40 |
| inc0 | Intel | 17:40 |
| mgoddard | I saw something on thenewstack recently about Intel + Mirantis wanting to containerise OpenStack. Is that Kolla based? | 17:40 |
| inc0 | that's...a complicated question;) | 17:41 |
| inc0 | Mirantis does contribute to kolla heavily | 17:41 |
| sdake | mgoddard everyoone wnats to containerize openstack because they see how simple kolla is | 17:42 |
| sdake | mgoddard and every deployment tool in existence is complex by comprison | 17:42 |
| sdake | mgoddard so i think its natural for vendors to want to follow what works- whether mirantis or red hat or anyone else end up using kolla for an upstream is an unknown at this point | 17:43 |
| sdake | the known is oracle uses kolla for an usptream | 17:43 |
| inc0 | TripleO also works with kolla, and RH is playing around it, but as for commercial appliances...you can never really tell;) | 17:44 |
| inc0 | mgoddard, so if you want to backport kilo to mitaka deploy code | 17:46 |
| inc0 | https://review.openstack.org/#/c/299563/ look at this | 17:46 |
| patchbot | inc0: patch 299563 - kolla - Making Mitaka deploying liberty | 17:46 |
| inc0 | this is latest master deploying liberty, this will eventually be backported to stable/liberty | 17:47 |
| inc0 | (eventually == ~ week or 2 from now) | 17:47 |
| inc0 | as you can see not that many changes, I expect Kilo to be similar | 17:47 |
| mgoddard | excellent - thanks | 17:48 |
| mgoddard | my colleague has combed through Oracle's changes to deploy Kilo on Liberty - he said there didn't seem to be too much involved | 17:48 |
| inc0 | I suggest using stable/mitaka | 17:49 |
| inc0 | or stable/liberty once we will backport it | 17:49 |
| inc0 | it will be tagged as 1.1.0 | 17:49 |
| inc0 | keep track on this patchset, this will be it | 17:49 |
| inc0 | plan is - once we release mitaka, we'll cherrypick it to stable/mitaka and tag as stable/liberty | 17:50 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add Kuryr ansible role https://review.openstack.org/298894 | 17:50 |
| inc0 | few differences tho, it will deploy liberty services we deployed in liberty | 17:50 |
| inc0 | in any case, do test it please;) | 17:51 |
| *** aginwala has joined #openstack-kolla | 17:52 | |
| *** rmart04 has joined #openstack-kolla | 17:52 | |
| sdake | we can't tag as stable liberty inc0 | 18:02 |
| sdake | stable/liberty is a branch, we need to backport our work into stable/librety | 18:02 |
| sdake | probably through some monsterous mega-patch | 18:02 |
| sdake | which we rubber stamp | 18:02 |
| sdake | followed by heavy teting | 18:02 |
| inc0 | sdake, I'll leave git magic to you | 18:02 |
| sdake | fantastic | 18:03 |
| sdake | i have no idea how to make that patch ;) | 18:03 |
| sdake | i was hoping you did | 18:03 |
| inc0 | lol | 18:03 |
| inc0 | git diff between 2 branches? | 18:03 |
| inc0 | or just checkout this code, copy all the files to liberty directory and git commit;) | 18:04 |
| sdake | uusally for this i use git export | 18:04 |
| sdake | then diff the two trees | 18:04 |
| sdake | then make a patch out of the diff | 18:04 |
| sdake | inc0 right thats anothe rmodel | 18:04 |
| sdake | the problem to deal with is the addition of files | 18:04 |
| inc0 | but then we're losing git history | 18:04 |
| sdake | which git status should help with | 18:05 |
| sdake | yup git history is lost on liberty | 18:05 |
| sdake | the alternative is to backport the 1k patches in mitaka to liberty | 18:05 |
| sdake | thats alot of +2 clicking | 18:05 |
| sdake | i leave the backport magic to you ;) | 18:05 |
| inc0 | ehh | 18:05 |
| sdake | but we can't tag liberty off of stable/mitaka :) | 18:05 |
| inc0 | ok, let's start by tagging stable/mitaka | 18:06 |
| sdake | right | 18:06 |
| inc0 | and cherry-pick this code | 18:06 |
| sdake | what will happen is on the 15th 2.0.0 will be tagged from stable/mitaka | 18:06 |
| sdake | then stable/mitaka needs to somehow be merged into stable/liberty | 18:07 |
| sdake | it would be fantastic not to lose history | 18:07 |
| sdake | but the trees are divergent enoug hthati think there will be collisions in the 1k patchset | 18:07 |
| inc0 | can we do normal git merge? | 18:07 |
| sdake | i never use git merge | 18:07 |
| sdake | so i dont know what it does | 18:07 |
| inc0 | ever used git pull? | 18:07 |
| sdake | duh of course | 18:07 |
| sdake | i use git pull --rebase | 18:07 |
| *** haplo37 has quit IRC | 18:08 | |
| inc0 | git pull == git fetch && git merge | 18:08 |
| inc0 | merge means 2 branches got merged | 18:08 |
| sdake | does it maintain history? | 18:08 |
| inc0 | yup | 18:08 |
| *** blahRus has quit IRC | 18:08 | |
| sdake | and what about patch collisiions | 18:08 |
| inc0 | not sure how it will deal with all the cherry picks we backported | 18:08 |
| sdake | let me try that locally what command should I type in | 18:08 |
| inc0 | https://git-scm.com/docs/git-merge | 18:09 |
| inc0 | check out liberty code | 18:09 |
| inc0 | checkout my code to a branch | 18:09 |
| inc0 | git merge branches | 18:09 |
| sdake | so | 18:10 |
| sdake | git clone repo | 18:10 |
| sdake | git checkout stable/liberty | 18:10 |
| sdake | git merge branches? | 18:10 |
| sdake | or git merge stable/mitaka? | 18:10 |
| *** aginwala has quit IRC | 18:10 | |
| inc0 | clone repo | 18:11 |
| sdake | STDAKE-M-J2VL:kolla sdake$ git merge stable/mitaka | 18:11 |
| sdake | merge: stable/mitaka - not something we can merge | 18:11 |
| inc0 | well...I'll play aronund it | 18:11 |
| inc0 | no clue really | 18:11 |
| sdake | right | 18:11 |
| inc0 | but I'll fugire sth out | 18:11 |
| sdake | ok sounds good :) | 18:12 |
| *** shardy has quit IRC | 18:12 | |
| sdake | im focused on 2.0.0 atm ;) | 18:12 |
| sdake | tbh i think git expprt is our best option | 18:14 |
| sdake | inc0 try to sync with asalkeld | 18:14 |
| *** aginwala has joined #openstack-kolla | 18:14 | |
| sdake | he is a git wizard | 18:14 |
| sdake | and solved a similar problem for corosync a long time ago | 18:15 |
| sdake | without losing history | 18:15 |
| inc0 | will do | 18:16 |
| *** aginwala has quit IRC | 18:16 | |
| Jeffrey4l | sdake, around? | 18:16 |
| sdake | Jeffrey4l shoot | 18:17 |
| Jeffrey4l | sdake, I want to talk about the qemu permission issue. i think just add the qemu to the nova group is enough. | 18:17 |
| Jeffrey4l | no just to change to the kolla group. | 18:18 |
| Jeffrey4l | it is a little wired. and the end-ops will be confused. | 18:18 |
| sdake | Jeffrey4l good idea | 18:18 |
| sdake | Jeffrey4l i dont know if it will work | 18:18 |
| Jeffrey4l | moreover, in the Cento Distro, it will the solution. | 18:19 |
| sdake | becuase the permissions still need to be 775 | 18:19 |
| Jeffrey4l | sdake, it works. | 18:19 |
| sdake | Jeffrey4l idd you test a patch? | 18:19 |
| Jeffrey4l | actually, I use this in my `fix neutron-linuxbridge-agent` PS. | 18:19 |
| Jeffrey4l | :D | 18:19 |
| Jeffrey4l | \ | 18:19 |
| sdake | Jeffrey4l ok i'll test it | 18:19 |
| Jeffrey4l | sdake, it has other issue. | 18:20 |
| Jeffrey4l | But i can fix it in 1 minute. :D | 18:20 |
| sdake | Jeffrey4l i'm not quie sure what you suggest, do you mean get rid of the chmod 775 ? | 18:21 |
| sdake | can you leave comment in the review about what ou think i shoudl change | 18:21 |
| Jeffrey4l | sdake, no. Just no need run the qemu process in kolla group. | 18:21 |
| Jeffrey4l | sdake, np. | 18:21 |
| *** cbits has joined #openstack-kolla | 18:22 | |
| *** Marga_ has quit IRC | 18:24 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 18:24 |
| *** mlima__ has joined #openstack-kolla | 18:24 | |
| *** sdake_ has joined #openstack-kolla | 18:26 | |
| *** sdake has quit IRC | 18:26 | |
| sdake_ | let me get us on a stable commit | 18:26 |
| sdake_ | moment | 18:26 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 18:27 |
| *** mlima_ has quit IRC | 18:28 | |
| sbezverk | sdake I am ready to test your qemu patch, do you still need it? | 18:30 |
| *** ssurana has joined #openstack-kolla | 18:31 | |
| *** salv-orlando has joined #openstack-kolla | 18:32 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 18:32 |
| sdake_ | sbezverk there ya go ;) | 18:33 |
| sdake_ | Jeffrey4l ok add review commnets there on what you think should change | 18:33 |
| sdake_ | i'll try it out | 18:33 |
| Jeffrey4l | sdake_, cool | 18:33 |
| sdake_ | if it works we can use that technique | 18:33 |
| daneyon | sdake_ I am following the kolla install instructions and have come up to this optional step: Optionally, you can install tox and generate the build configuration using following steps. | 18:34 |
| daneyon | can u explain a bit more what ^ does? | 18:34 |
| sdake_ | daneyon i suspect it generates build-conf.conf fiel | 18:35 |
| sdake_ | or something like that ;) | 18:35 |
| sdake_ | daneyon never tried it | 18:35 |
| *** athomas has quit IRC | 18:35 | |
| sdake_ | i dont use buildconf | 18:35 |
| sdake_ | because my machine is super fast at building i dont care to chane htings ;) | 18:35 |
| daneyon | sdake_ what is that file for? | 18:35 |
| sdake_ | lets you select htings out of the build i think | 18:35 |
| sdake_ | pbourke would know more | 18:35 |
| sdake_ | daneyon for your work i'd ignore it | 18:36 |
| daneyon | ok | 18:36 |
| daneyon | sdake_ are there any known issues outside of the install steps for using 1 node, i.e. 1 server as the build and target? | 18:37 |
| sdake_ | use from source for the moment | 18:37 |
| sdake_ | centos binary may have trouble | 18:37 |
| sdake_ | --type source as the build.py option | 18:37 |
| *** SiRiuS_ has joined #openstack-kolla | 18:38 | |
| sdake_ | tls is broken | 18:38 |
| sdake_ | so dont enable it | 18:38 |
| sdake_ | dave is working on it | 18:38 |
| *** salv-orlando has quit IRC | 18:38 | |
| sdake_ | in master flavors r eno longer created by nova automatically | 18:39 |
| *** nihilifer has quit IRC | 18:39 | |
| sdake_ | you so hae to creat a flavor | 18:39 |
| sdake_ | qemu may be bust in genral, if you want to avoid that use the qemu 2.3 patchset | 18:39 |
| vhosakot | daneyon: build-conf has inputs paramters for kolla-build | 18:39 |
| *** slagle has joined #openstack-kolla | 18:39 | |
| daneyon | ok, thanks sdake_ and vhosakot for the info | 18:40 |
| *** nihilifer has joined #openstack-kolla | 18:40 | |
| sdake_ | ok pbourke vhosakot re change to get ol to buld leave comments in teh patch set | 18:41 |
| sdake_ | i dont want it getting busted along the wy - it looks like atleast 4 people have pushed patches and i dont even know what changed there | 18:41 |
| daneyon | _sdake vhosakot ansible still has to be < 2.0.0 | 18:41 |
| vhosakot | daneyon: https://github.com/openstack/kolla/blob/master/doc/image-building.rst#build-openstack-from-source has an example kolla-build.conf, and how it is used to fee input parameters for kolla-build when building images | 18:41 |
| sdake_ | i'll address the cmments nteh patchset | 18:41 |
| vhosakot | daneyon: yes | 18:41 |
| vhosakot | daneyon: fee==feed | 18:42 |
| inc0 | hey, quick linux question | 18:42 |
| daneyon | ok, thx | 18:42 |
| inc0 | I see disk in lsblk, in /dev/disk/by-id | 18:42 |
| vhosakot | sdake_: cool, will leave comment sin PS after OL gate results are out | 18:42 |
| *** aginwala has joined #openstack-kolla | 18:43 | |
| inc0 | but can't see it in /dev/vd* and can't run parted on /dev/disk/by-d | 18:43 |
| ccesario | sdake_, sorry by delay..... | 18:43 |
| sdake_ | which delay is that | 18:43 |
| ccesario | about ps https://review.openstack.org/#/c/300888/ yes, the vhosakot is right... we have haproxy log in heka container | 18:44 |
| patchbot | ccesario: patch 300888 - kolla - Added missing haproxy log dir. | 18:44 |
| ccesario | since the opened file bug..... the question it was the log directory into haproxy container | 18:45 |
| sdake_ | comon guys quit apologizing | 18:45 |
| vhosakot | ccesario: as Eric commented, haproxy does not need access to /var/log/kolla as haproxy writes its logs to the /var/lib/kolla/heka/log Unix socket. | 18:46 |
| sdake_ | its a team effort everyone is obviously doign the best they can | 18:46 |
| vhosakot | ^^^ +1 | 18:46 |
| ccesario | rigth ! | 18:48 |
| ccesario | CShould I abandon it !? | 18:48 |
| ccesario | Should* | 18:48 |
| sdake_ | since bug is invalid i think it makes snese to do so | 18:49 |
| ccesario | vhosakot, are you agree with it? (I'm asking you too because you are aware about it) | 18:50 |
| *** aginwala has quit IRC | 18:50 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 18:50 |
| vhosakot | ccesario: yes, as there is nothing to fix ccesario, and we see haproxy log in the output of "docker exec heka ls -lrt /var/log/kolla/haproxy/haproxy.log" | 18:50 |
| vhosakot | ccesario: plus, as elemoine_ commented in the review, haproxy container does not need access to /var/log/kolla as it writes its logs to the /var/lib/kolla/heka/log Unix socket | 18:51 |
| ccesario | rigth...! | 18:52 |
| elemoine_ | I just commented in the CR | 18:52 |
| *** salv-orlando has joined #openstack-kolla | 18:52 | |
| ccesario | only click into abandon.... right!? | 18:52 |
| elemoine_ | I hope this will end the discussion :) | 18:52 |
| *** rstarmer has joined #openstack-kolla | 18:53 | |
| elemoine_ | (friendly hope) | 18:53 |
| *** mlima__ has quit IRC | 18:55 | |
| *** mlima has joined #openstack-kolla | 18:55 | |
| dave-mccowan | sdake_ tls is fixed. patch merged. | 18:57 |
| *** aginwala has joined #openstack-kolla | 18:57 | |
| dave-mccowan | sdake_ also, i have a patch out to add flavors during init-runonce | 18:58 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 18:58 |
| sdake_ | dave-mccowan nice1 | 18:59 |
| sdake_ | dave nice again! :) | 19:00 |
| dave-mccowan | what's the plan for binary centos being broken? | 19:00 |
| *** salv-orlando has quit IRC | 19:01 | |
| sdake_ | is it confirmed broken? | 19:01 |
| sdake_ | we are witing on rdo to cut a mitaka release for rdo | 19:01 |
| sdake_ | dmsimard is working on that | 19:01 |
| Jeffrey4l | sdake_, see the comments https://review.openstack.org/#/c/302036/21/ansible/roles/nova/templates/qemu.conf.j2 :D | 19:01 |
| patchbot | Jeffrey4l: patch 302036 - kolla - Use proper CentOS repositories for ceph and qemu | 19:01 |
| dave-mccowan | on my server, it started working when i switch to build from source. | 19:02 |
| *** cbits has quit IRC | 19:03 | |
| sdake_ | Jeffrey4l we already have a kolla group | 19:03 |
| Jeffrey4l | sdake_, i know that. | 19:04 |
| Jeffrey4l | sdake_, I means to introduce that kolla group to the /var/lib/nova folder. | 19:04 |
| sdake_ | Jeffrey4l the issue is this | 19:04 |
| sdake_ | qemu 1.5 runs as root:root | 19:04 |
| sdake_ | qemu 2.3 runs as qemu;qemu | 19:04 |
| sdake_ | qemu cannto write to that nova directory and apparently it needs to be able to do so | 19:05 |
| vhosakot | dave-mccowan: binary centos works in gate.. is it broken with TLS enabled ? | 19:05 |
| *** aginwala has quit IRC | 19:05 | |
| Jeffrey4l | sdake_, end-user may confused that: "why here has a kolla group owner? it must be wrong. I saw my centos deploy, it should be nova:nova. Let change that" | 19:05 |
| sdake_ | how do you propose we fix it then? | 19:06 |
| *** rajathagasthya has quit IRC | 19:06 | |
| sdake_ | you said add qemu to nova group | 19:06 |
| Jeffrey4l | sdake_, see this https://review.openstack.org/#/c/302721/2/docker/nova/nova-libvirt/Dockerfile.j2 | 19:06 |
| patchbot | Jeffrey4l: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 19:06 |
| sdake_ | but the file is still not writeable | 19:06 |
| Jeffrey4l | yup. | 19:06 |
| Jeffrey4l | sdake_, also need change the mod in /var/lib/nova/ and /var/lib/nova/instance | 19:07 |
| Jeffrey4l | my changes just like the distro used. When install both libvirt and nova-compute: in CentOS: it will add qemu to nova group. In Ubuntu, it will add nova to libvirtd group. | 19:08 |
| Jeffrey4l | sdake_, btw, in qemu 1.5 ( centos + source, my 10 nodes env ), the qemu process is run under qemu user for sure. | 19:09 |
| Jeffrey4l | sdake_, see this http://paste.openstack.org/show/493402/ | 19:10 |
| sdake_ | Jeffrey4l i had thought of the nova-base solution too | 19:10 |
| sdake_ | to unify nnova and qemu user and group permissions | 19:10 |
| sdake_ | but it makes libvirt fatter | 19:10 |
| Jeffrey4l | sdake_, if not, it may have bug. | 19:10 |
| sdake_ | line 1 of your dockerifle.j2 | 19:11 |
| Jeffrey4l | the nova user uid and gid may be different in nova-compute and nova-libvirt. | 19:11 |
| Jeffrey4l | sdake_, yea. I got what are you means. | 19:11 |
| sdake_ | i think we shoud lmerge your patch first | 19:12 |
| sdake_ | and my patch second | 19:12 |
| Jeffrey4l | sdake_, thanks for that :D | 19:12 |
| sdake_ | but before we do that, we should unif y the global uids we need in some base package | 19:12 |
| sdake_ | such as base itself | 19:12 |
| Jeffrey4l | sdake_, I also thought that. it should one thing we must to do. | 19:13 |
| sdake_ | should do now | 19:13 |
| Jeffrey4l | sdake_, what's your thought? create the nova user in base? | 19:13 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Set db connection retry to infinity https://review.openstack.org/302434 | 19:13 |
| sdake_ | nova and qemu uerin base | 19:13 |
| sdake_ | they are bot hshard between many containers | 19:13 |
| dave-mccowan | vhosakot does the gate boot a VM? i could not boot a VM with no TLS using binary centos. | 19:14 |
| *** salv-orlando has joined #openstack-kolla | 19:14 | |
| *** Marga_ has joined #openstack-kolla | 19:14 | |
| sdake_ | they need to be created with the same uid and gid that the packaging uses for debuntian and centos | 19:14 |
| vhosakot | dave-mccowan: let me check gate logs... 2 mins | 19:14 |
| sdake_ | here is suggested plan of action1 | 19:15 |
| sdake_ | 1. merge your ptach | 19:15 |
| sdake_ | 2 .i'll rebse and fix whtever necessary which will be super painful | 19:15 |
| sdake_ | 3. while i'm rebasing you can make a followon patch to fix the libvirt fatness | 19:15 |
| Jeffrey4l | cool. sdake_ | 19:15 |
| Jeffrey4l | agreed. | 19:15 |
| sdake_ | uids and gids need to match whatever is in the distro | 19:16 |
| sdake_ | we will end up with more of these global uid/gids over time | 19:16 |
| sdake_ | heka is another | 19:16 |
| *** cbits has joined #openstack-kolla | 19:16 | |
| Jeffrey4l | sdake_, `heka is another` mean? | 19:17 |
| *** aginwala has joined #openstack-kolla | 19:17 | |
| sdake_ | aother uid/gid requiring unification | 19:17 |
| Jeffrey4l | heka can not read the log file due to the file owner/permission? | 19:17 |
| Jeffrey4l | sdake_, yea. | 19:18 |
| sdake_ | just pointing out that over time wee will wnat to unify all the uids/gids in the bse file if the packaging tools will permit it | 19:18 |
| *** stvnoyes has left #openstack-kolla | 19:20 | |
| Jeffrey4l | +1 | 19:20 |
| Jeffrey4l | no. +2 :D | 19:20 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add Kuryr ansible role https://review.openstack.org/298894 | 19:20 |
| sdake_ | Jeffrey4l https://review.openstack.org/#/c/302721/2 is busted | 19:21 |
| patchbot | sdake_: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 19:21 |
| sdake_ | pleae address the comment | 19:21 |
| sdake_ | so we cn get it merged now | 19:21 |
| sdake_ | so I can get on with qemu work | 19:21 |
| Jeffrey4l | sdake_, ok | 19:21 |
| vhosakot | dave-mccowan: yep, centos binary not booting VM in gate ? do you see rabbitmq crash ? | 19:22 |
| sdake_ | vhosakot its on his personal machine - it doesn't go to active state | 19:22 |
| vhosakot | without TLS ? | 19:22 |
| sdake_ | yup | 19:23 |
| sdake_ | i have same problem i think | 19:23 |
| dave-mccowan | without TLS, with qemu patch, i could not boot vm from binary. i can with source. | 19:23 |
| vhosakot | ok.. gate is... without TLS, wihtout qemu patch, rabbitmq crash, vm does not boot, centos source.. different issue.. I am checking this gate issue now | 19:25 |
| *** daneyon_ has joined #openstack-kolla | 19:29 | |
| vhosakot | dave-mccowan: just so we know rabbitmq is the common issue or not... is rabbitmq container running ? | 19:29 |
| dave-mccowan | vhosakot i'll check. i'll have to rebuild/redeploy. | 19:30 |
| vhosakot | dave-mccowan: ah ok, thnkas! | 19:30 |
| vhosakot | sdake_: is rabbitmq container running in your env ? | 19:31 |
| sdake_ | Jeffrey4l if you remoe lines 34 and 44 your patch should be good t o go | 19:31 |
| dave-mccowan | i have build from source on centos now. anything i should check related to build from source, before i blow it away? | 19:31 |
| sdake_ | that logic should happen for all install types | 19:31 |
| sdake_ | dave-mccowan you might try out Jeffrey4l 's ptch once he has it working from build from binary - it may fix your boot problem | 19:32 |
| *** Jeffrey4l_ has joined #openstack-kolla | 19:32 | |
| sdake_ | or it may not ;) | 19:32 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 19:33 |
| Jeffrey4l_ | sdake_, PS pushed ^ | 19:33 |
| inc0 | duh, after fresh deploy - rabbit is crying that no exchange | 19:33 |
| *** daneyon_ has quit IRC | 19:34 | |
| *** Jeffrey4l has quit IRC | 19:34 | |
| Jeffrey4l_ | I need continue to sleep. Bye guys. | 19:34 |
| sdake_ | Jeffrey4l_ your patch makes no sense | 19:35 |
| Jeffrey4l_ | sdake_, what? | 19:36 |
| sdake_ | && usermod -a -G nova qemu | 19:36 |
| sdake_ | && usermod -a -G libvirtd nova | 19:36 |
| Jeffrey4l_ | sdake_, why? | 19:36 |
| sdake_ | one adds qemu to nova the other adds nova to libvirtd | 19:36 |
| *** banix has quit IRC | 19:36 | |
| sdake_ | or something like that | 19:37 |
| Jeffrey4l_ | sdake_, because there is no qemu user in Ubuntu. In Ubuntu just has libvirtd-qemu user. | 19:37 |
| Jeffrey4l_ | Ubuntu and CentOS do not handle this the same. | 19:37 |
| sdake_ | was this last patch tested on ubuntu and centos? | 19:38 |
| daneyon | sdake_ vhosakot I continue to work my way through the install guide. Question re local registry, the install guide provides an example for systemd but not initd. I added --insecure-registry 10.30.118.91:4000 to DOCKER_OPTS= in /etc/init.d/docker. Can anyone confirm if this the correct way for initd? | 19:38 |
| Jeffrey4l_ | sdake_, Just the centos. it works. never test the Ubuntu, because I have no ubuntu env. | 19:38 |
| Jeffrey4l_ | But let's wait the CI reason? | 19:38 |
| vhosakot | daneyon: initd meaning UBuntu 14.04 ? | 19:39 |
| sdake_ | so adding qemu to nova works | 19:39 |
| daneyon | yes | 19:39 |
| sdake_ | why not add libvirtd-user to nova on ubuntu? | 19:39 |
| sdake_ | adding qemu grou pto nova user works | 19:39 |
| sdake_ | is there no libvirt group? | 19:39 |
| vhosakot | daneyon: add DOCKER_OPTS=--insecure-registry 10.30.118.91:4000 in /etc/default/docker | 19:40 |
| Jeffrey4l_ | sdake_, http://paste.openstack.org/show/493406/ | 19:40 |
| sdake_ | vhosakot can you paste your /etc/group file | 19:40 |
| Jeffrey4l_ | Because the Ubuntu packaging do like that. | 19:40 |
| daneyon | vhosakot will do | 19:40 |
| Jeffrey4l_ | sdake_, I just follow what the Distro packaging does. | 19:41 |
| vhosakot | sdake_: http://paste.openstack.org/show/493407/ | 19:41 |
| sdake_ | Jeffrey4l_ mind if i take over your patch set? | 19:41 |
| vhosakot | daneyon: the quickstart guide does not have steps to do it ? | 19:41 |
| Jeffrey4l_ | sdake_, no. | 19:41 |
| sdake_ | so you can reitre to bed ;) | 19:41 |
| Jeffrey4l_ | sdake_, please | 19:41 |
| vhosakot | daneyon: adding DOCKER_OPTS=--insecure-registry 10.30.118.91:4000 in /etc/default/docker on UBunut 14.04 ? | 19:41 |
| daneyon | vhosakot just centos example | 19:43 |
| vhosakot | daneyon: thanks for letting me know.. I will update the docs | 19:44 |
| daneyon | sure, thx for the help | 19:44 |
| *** aginwala has quit IRC | 19:45 | |
| vhosakot | np | 19:45 |
| *** rajathagasthya has joined #openstack-kolla | 19:45 | |
| vhosakot | bbiab 30 mins | 19:46 |
| * Jeffrey4l_ heading to bed, say bye to everyone. | 19:46 | |
| sdake_ | who has ubuntu that can test a patch | 19:47 |
| sdake_ | https://review.openstack.org/#/c/302721/ | 19:48 |
| patchbot | sdake_: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 19:48 |
| sdake_ | need nova boot tested on this patch with linuxbridge network driver | 19:48 |
| sdake_ | on ubuntu | 19:48 |
| *** diogogmt has quit IRC | 19:52 | |
| *** diogogmt has joined #openstack-kolla | 19:52 | |
| inc0 | and I need someone to test rabbitmq on multinode | 19:52 |
| inc0 | and run ocker exec -it rabbitmq rabbitmqctl eval 'rabbit_clusterer:status().' | 19:53 |
| inc0 | on rabbitmq nodes | 19:53 |
| *** vhosakot has quit IRC | 19:58 | |
| sdake_ | i wonder what would happen in a scrum team run by darth vadar | 19:58 |
| *** vhosakot has joined #openstack-kolla | 19:58 | |
| sdake_ | general veers PREPARE YOUR MEN! | 20:01 |
| *** vhosakot has quit IRC | 20:01 | |
| sbezverk | inc0 about the issue you hit trying iscsi, I we should address it in the documentation, I do not see a way to do it during the kolla deployment.. | 20:04 |
| inc0 | sbezverk, if you say so, can you drop a patchset with that note? | 20:08 |
| sbezverk | inc0 I have another patch set which add documentation for lvm/iscsi, I can add paragraph there, what do you think? | 20:09 |
| inc0 | yeah, that'd do | 20:10 |
| *** aginwala has joined #openstack-kolla | 20:11 | |
| manjeets | inc0 sdake a neutron core asked me a question about kolla which i think is true but will still like to confirm | 20:12 |
| manjeets | does kolla manages system dependencies as well ? | 20:12 |
| inc0 | depends what you mean by that | 20:14 |
| inc0 | yes we do manage rpm packages, not only python ones | 20:14 |
| inc0 | but kernel is shared | 20:14 |
| sdake_ | trunk.rdoproject.org is bust | 20:15 |
| * sdake_ groans | 20:15 | |
| rhallisey | inc0, I'll check rabbit for you | 20:15 |
| rhallisey | just a sec | 20:15 |
| inc0 | thanks rhallisey it's strange behaviour | 20:16 |
| manjeets | ebtables is missing | 20:17 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 20:17 |
| *** ayoung has quit IRC | 20:18 | |
| *** reddy has quit IRC | 20:19 | |
| sdake_ | asteroids do not concern me admieral. | 20:20 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add mongodb in cleanup containers script https://review.openstack.org/303058 | 20:21 |
| sdake_ | rdorpoejct.org will lbe up in 5-10 mins | 20:22 |
| sdake_ | wars make one not great! | 20:22 |
| *** sdake_ is now known as sdake | 20:23 | |
| *** jtriley has quit IRC | 20:25 | |
| rhallisey | inc0, https://paste.fedoraproject.org/351288/60823146/ | 20:27 |
| inc0 | so your case is worse than mine | 20:27 |
| rhallisey | inc0, my config could be wrong here.. I've been seeing this for a little now | 20:27 |
| inc0 | I will check one more thing in my node and if that wont help I'm going to panic, then get a drink and start debugging | 20:28 |
| inc0 | strong drink | 20:28 |
| inc0 | rhallisey, check if nodes have ntp setup | 20:28 |
| mlima | bye to everyone :) | 20:29 |
| *** mlima has quit IRC | 20:29 | |
| rhallisey | inc0, they don't | 20:29 |
| rhallisey | inc0, I'm new to multinode. So I've been doing an operator crash course the last few days | 20:30 |
| inc0 | welcome to the real world rhallisey ;) | 20:30 |
| rhallisey | I'm still not really there tbh | 20:30 |
| *** aginwala has quit IRC | 20:30 | |
| rhallisey | virt-multinode ftw | 20:30 |
| rhallisey | thanks to ironic | 20:31 |
| *** vhosakot has joined #openstack-kolla | 20:31 | |
| inc0 | did you get kolla-based undercloud worked out? | 20:31 |
| rhallisey | no. Just using stock undercloud | 20:31 |
| rhallisey | with a heat template for node setup | 20:31 |
| *** reddy has joined #openstack-kolla | 20:32 | |
| *** gfidente has quit IRC | 20:35 | |
| *** aginwala has joined #openstack-kolla | 20:40 | |
| inc0 | ok...drink and debug it is | 20:43 |
| rhallisey | :) | 20:43 |
| openstackgerrit | Merged openstack/kolla: Fix the Neutron reconfigure https://review.openstack.org/302930 | 20:44 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 20:53 |
| *** nihilifer has quit IRC | 20:54 | |
| *** nihilifer has joined #openstack-kolla | 20:54 | |
| *** aginwala has quit IRC | 20:55 | |
| *** aginwala has joined #openstack-kolla | 20:56 | |
| openstackgerrit | Merged openstack/kolla: Increase max scheduling attempts to 10 https://review.openstack.org/300914 | 20:56 |
| inc0 | sooo...I'm worried guys, can anyone deploy running multinode and confirm that rabbit cluster is being built correctly? | 20:59 |
| rhallisey | new error | 21:01 |
| rhallisey | https://paste.fedoraproject.org/351310/14600628/ | 21:01 |
| rhallisey | inc0, what are you seeing? | 21:01 |
| inc0 | so for me it's not error | 21:02 |
| inc0 | but each node is running on it's own | 21:02 |
| *** rajathagasthya has quit IRC | 21:07 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add LVM2/iSCSI documentation https://review.openstack.org/300824 | 21:13 |
| *** rstarmer has quit IRC | 21:13 | |
| *** daneyon_ has joined #openstack-kolla | 21:16 | |
| *** shakamunyi has joined #openstack-kolla | 21:17 | |
| *** salv-orl_ has joined #openstack-kolla | 21:18 | |
| *** salv-orlando has quit IRC | 21:22 | |
| *** shakamunyi has quit IRC | 21:24 | |
| *** daneyon_ has quit IRC | 21:29 | |
| *** ayoung has joined #openstack-kolla | 21:32 | |
| sbezverk | sdake I got a kernel crash on my ubuntu box with the latest patch set for 302721 | 21:35 |
| *** dwalsh has quit IRC | 21:37 | |
| *** rstarmer has joined #openstack-kolla | 21:38 | |
| *** rajathagasthya has joined #openstack-kolla | 21:38 | |
| *** SiRiuS_ has quit IRC | 21:46 | |
| vhosakot | sbezverk: ping | 21:48 |
| sbezverk | vhosakot pong | 21:52 |
| vhosakot | sbezverk: should docker/iscsid/Dockerfile.j2 run as root ? | 21:52 |
| vhosakot | sbezverk: https://review.openstack.org/#/c/291285/53/docker/iscsid/Dockerfile.j2 | 21:52 |
| patchbot | vhosakot: patch 291285 - kolla - Adds iscsid and tgtd docker containers | 21:52 |
| sbezverk | vhosakot yes | 21:53 |
| vhosakot | sbezverk: docker/tgtd/Dockerfile.j2 also ? | 21:53 |
| vhosakot | https://review.openstack.org/#/c/291285/53/docker/tgtd/Dockerfile.j2 | 21:53 |
| patchbot | vhosakot: patch 291285 - kolla - Adds iscsid and tgtd docker containers | 21:53 |
| vhosakot | sbezverk: ^^ | 21:53 |
| vhosakot | sbezverk: should both iscsid and tgtd containers run as root ? | 21:54 |
| sbezverk | vhosakot these two services requires run as root | 21:54 |
| sbezverk | at least at this point | 21:54 |
| vhosakot | sbezverk: could you exlpain why ? | 21:54 |
| vhosakot | sbezverk: binaries cannot run as non-root ? | 21:55 |
| sbezverk | because they access /sys/kernel/config | 21:55 |
| sbezverk | because they bind to priv sockets | 21:55 |
| vhosakot | sbezverk: ah ok... | 21:56 |
| vhosakot | sbezverk: these two touch /sys/kernel/config ? | 21:56 |
| sbezverk | yep | 21:56 |
| sbezverk | vhosakot I can file a techdebt bug to come back and investigate | 21:57 |
| vhosakot | sbezverk: yes, please... | 21:57 |
| vhosakot | sbezverk: /sys/kernel is owned by root on any distro | 21:57 |
| sbezverk | but at this point I do not want to anything as it is already big and difficult to manage | 21:57 |
| sbezverk | I meant PS is big and difficult to manage | 21:58 |
| sbezverk | vhosakot btw I got it running on ubuntu, so you can try it | 21:59 |
| vhosakot | nice! | 21:59 |
| vhosakot | same docs you added work for Ubuntu as well ? | 21:59 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add LVM2/iSCSI documentation https://review.openstack.org/300824 | 22:01 |
| *** ayoung has quit IRC | 22:01 | |
| sbezverk | vhosakot in general yes, but there is one particularity on ubuntu, it does not mount by default configfs | 22:01 |
| sbezverk | I am planning to add one paragraph about that | 22:02 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 22:02 |
| vhosakot | sbezverk: where... add in the same doc PS ? or a new doc PS for ubunutu part | 22:02 |
| openstackgerrit | Manjeet Singh Bhatia proposed openstack/kolla: Add ebtables dependency for ubuntu. https://review.openstack.org/303094 | 22:02 |
| manjeets | sdake inc0 it tested linux bridge for distro ubuntu type source worked fine after above patch | 22:03 |
| manjeets | I tested ** | 22:03 |
| sbezverk | vhosakot the same PS for doc which you just reviewed | 22:04 |
| vhosakot | sbezverk: nice.. cool.. thanks! | 22:04 |
| *** dave-mccowan has quit IRC | 22:06 | |
| inc0 | manjeets, good to know | 22:10 |
| *** cbits has quit IRC | 22:10 | |
| *** daneyon_ has joined #openstack-kolla | 22:11 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/255450 | 22:11 |
| inc0 | rhallisey, something for you;) ^ | 22:12 |
| rhallisey | ya | 22:13 |
| sdake | yo cats | 22:13 |
| sdake | it didn't build for me on centos | 22:13 |
| sdake | so i had to push a second patch | 22:14 |
| sdake | manjeets ^^ | 22:14 |
| *** ntpttr_ has joined #openstack-kolla | 22:14 | |
| *** stvnoyes has joined #openstack-kolla | 22:15 | |
| *** daneyon_ has quit IRC | 22:16 | |
| *** shakamunyi has joined #openstack-kolla | 22:18 | |
| *** dave-mccowan has joined #openstack-kolla | 22:18 | |
| *** inc0 has quit IRC | 22:19 | |
| *** barra204 has joined #openstack-kolla | 22:20 | |
| sbezverk | sdake I am rebuilding neutron as it was missing ebtables | 22:20 |
| sbezverk | and was flapping | 22:20 |
| *** mwheckmann has quit IRC | 22:20 | |
| openstackgerrit | Manjeet Singh Bhatia proposed openstack/kolla: Add ebtables dependency for ubuntu. https://review.openstack.org/303094 | 22:23 |
| *** shakamunyi has quit IRC | 22:24 | |
| *** barra204 has quit IRC | 22:25 | |
| *** ayoung has joined #openstack-kolla | 22:26 | |
| *** vhosakot has quit IRC | 22:26 | |
| *** aginwala has quit IRC | 22:28 | |
| *** aginwala has joined #openstack-kolla | 22:28 | |
| manjeets | sdake what | 22:29 |
| manjeets | you mean ebtables ? | 22:29 |
| manjeets | thing | 22:29 |
| sdake | no the linuxbridge patch | 22:29 |
| sdake | i havent herd anything aboutebtables | 22:29 |
| manjeets | i thought you are talking regarding this https://review.openstack.org/#/c/303094 | 22:30 |
| manjeets | after this patch i can see instances active with lb and lb working fine for me on ubuntu type source | 22:30 |
| *** aginwala_ has joined #openstack-kolla | 22:31 | |
| *** aginwala has quit IRC | 22:32 | |
| *** rmart04 has quit IRC | 22:32 | |
| sbezverk | sdake wihtout manjeets patch lb is complaining about missing ebtables binary | 22:35 |
| sdake | which patch | 22:35 |
| manjeets | https://review.openstack.org/#/c/303094/1 | 22:35 |
| patchbot | manjeets: patch 303094 - kolla - Add ebtables dependency for ubuntu. | 22:35 |
| sdake | it needs to be merged into this patch | 22:35 |
| *** ntpttr_ has quit IRC | 22:35 | |
| sdake | manjeets i am going to merge thtin to the linuxridge patch and add you as a coauthor | 22:36 |
| sdake | so i will abandon your patch for now | 22:36 |
| sdake | ok? | 22:36 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/255450 | 22:39 |
| *** ntpttr_ has joined #openstack-kolla | 22:40 | |
| sbezverk | sdake I think there is still a problem with your lb patch, it seems you do not install openstack plugin binaries for the case of ubuntu/source and centos/source | 22:40 |
| sdake | which plugins | 22:40 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 22:41 |
| *** aginwala_ has quit IRC | 22:43 | |
| sbezverk | centos - openstack-neutron-linuxbridge, Ubuntu - neutron-plugin-linuxbridge | 22:44 |
| sbezverk | I see them installed only in binary case | 22:45 |
| sbezverk | I think they are missing in source | 22:45 |
| sdake | sbezverk see last patch i just proposed | 22:46 |
| sdake | it merges manjeets work into Jeffrey4l_ 's work | 22:46 |
| sdake | so give that a spin | 22:46 |
| sdake | you just have to bulid the agent container | 22:46 |
| sbezverk | sdake If I am right, latest patch has the same issue neither openstack-neutron-linuxbridge nor neutron-plugin-linuxbridge are installed for "source" case | 22:47 |
| sdake | what is the source of that file? | 22:48 |
| sdake | sbezverk i think in from source that just gets installed | 22:49 |
| sbezverk | please check docker/neutron/neutron-linuxbridge-agent/Dockerfile.j2 | 22:49 |
| sbezverk | sdake if you are sure, then I will remove -1 | 22:50 |
| sdake | i am not sure | 22:51 |
| sdake | but source for example doesn't install "ml2" | 22:51 |
| sdake | or "metadata" repos from anywhere | 22:51 |
| sdake | yet those work | 22:51 |
| sbezverk | sdake if I had a choice I would always go for explicit installation :-) cause god know what is going to happen later | 22:52 |
| *** aginwala has joined #openstack-kolla | 22:53 | |
| *** sdake has quit IRC | 22:54 | |
| *** aginwala has quit IRC | 22:56 | |
| *** sdake has joined #openstack-kolla | 22:58 | |
| *** aginwala has joined #openstack-kolla | 23:00 | |
| *** rajathagasthya is now known as agasthya | 23:05 | |
| *** rstarmer has quit IRC | 23:06 | |
| *** agasthya is now known as agasthy | 23:10 | |
| *** agasthy is now known as agasthya | 23:10 | |
| *** agasthya has quit IRC | 23:11 | |
| *** banix has joined #openstack-kolla | 23:11 | |
| *** rajathagasthya has joined #openstack-kolla | 23:11 | |
| *** sdake has quit IRC | 23:12 | |
| *** aginwala has quit IRC | 23:13 | |
| *** aginwala has joined #openstack-kolla | 23:13 | |
| *** rajathagasthya has left #openstack-kolla | 23:13 | |
| *** rajathagasthya has joined #openstack-kolla | 23:13 | |
| *** rajathagasthya has left #openstack-kolla | 23:13 | |
| *** aginwala has quit IRC | 23:17 | |
| *** aginwala has joined #openstack-kolla | 23:20 | |
| *** jtriley has joined #openstack-kolla | 23:22 | |
| *** ntpttr_ has quit IRC | 23:22 | |
| *** salv-orl_ has quit IRC | 23:24 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 23:25 |
| *** ayoung has quit IRC | 23:26 | |
| *** banix has quit IRC | 23:27 | |
| gmmaha | Jeffrey4l_: https://review.openstack.org/#/c/302721/4/docker/neutron/neutron-linuxbridge-agent/Dockerfile.j2 | 23:33 |
| patchbot | gmmaha: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 23:33 |
| gmmaha | it was not getting installed via neutron-base.. thats what manjeets was proposing with https://review.openstack.org/303094 | 23:33 |
| manjeets | sdake credit for gmmaha is missing | 23:34 |
| manjeets | he was one who suggested me | 23:34 |
| gmmaha | Jeffrey4l_: nevermind.. I see the new patch has that change.. apologies | 23:38 |
| *** Jeffrey4l_ has quit IRC | 23:42 | |
| *** vhosakot has joined #openstack-kolla | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!