| *** ayoung has joined #openstack-kolla | 00:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/302856 | 00:10 |
|---|---|---|
| *** mwheckmann has joined #openstack-kolla | 00:13 | |
| *** banix has joined #openstack-kolla | 00:16 | |
| openstackgerrit | Carlos Cesario proposed openstack/kolla: Identify container through PS1 shell variable https://review.openstack.org/297984 | 00:18 |
| *** shakamunyi has joined #openstack-kolla | 00:18 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 00:20 |
| *** mwheckmann has quit IRC | 00:20 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/255450 | 00:20 |
| *** ccesario_ has joined #openstack-kolla | 00:21 | |
| ccesario | vhosakot, ping ? | 00:22 |
| vhosakot | ccesario: pong | 00:22 |
| *** shakamunyi has quit IRC | 00:23 | |
| ccesario | https://review.openstack.org/297984 if possible could you ppls check it? | 00:23 |
| *** rhallisey has quit IRC | 00:24 | |
| ccesario | now it is working without errors :) | 00:24 |
| *** salv-orlando has joined #openstack-kolla | 00:26 | |
| *** salv-orlando has quit IRC | 00:30 | |
| *** weiyu_ has joined #openstack-kolla | 00:33 | |
| *** jtriley has quit IRC | 00:41 | |
| *** jtriley has joined #openstack-kolla | 00:50 | |
| openstackgerrit | Merged openstack/kolla: Add mongodb in cleanup containers script https://review.openstack.org/303058 | 01:01 |
| *** sdake has joined #openstack-kolla | 01:02 | |
| *** ccesario___ has joined #openstack-kolla | 01:04 | |
| *** ccesario_ has quit IRC | 01:06 | |
| vhosakot | ccesario: excellent work! :) | 01:14 |
| vhosakot | ccesario: let me test couple of times please | 01:14 |
| *** sdake has quit IRC | 01:14 | |
| *** diogogmt has quit IRC | 01:16 | |
| *** shakamunyi has joined #openstack-kolla | 01:18 | |
| *** sdake has joined #openstack-kolla | 01:19 | |
| ccesario___ | vhosakot, sure :) | 01:21 |
| *** jtriley has quit IRC | 01:22 | |
| *** sdake_ has joined #openstack-kolla | 01:22 | |
| *** shakamunyi has quit IRC | 01:23 | |
| *** vhosakot has quit IRC | 01:23 | |
| *** sdake has quit IRC | 01:25 | |
| *** jtriley has joined #openstack-kolla | 01:28 | |
| sbezverk | sdake tested ubuntu with patch it wored ok | 01:41 |
| *** dave-mccowan has quit IRC | 01:45 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add mongodb in cleanup containers script https://review.openstack.org/303180 | 02:04 |
| *** diogogmt has joined #openstack-kolla | 02:04 | |
| *** ccesario___ has quit IRC | 02:08 | |
| *** pbourke has quit IRC | 02:15 | |
| *** pbourke has joined #openstack-kolla | 02:16 | |
| *** nihilifer has quit IRC | 02:16 | |
| *** nihilifer has joined #openstack-kolla | 02:17 | |
| *** shakamunyi has joined #openstack-kolla | 02:19 | |
| *** openstackstatus has quit IRC | 02:19 | |
| *** aginwala has quit IRC | 02:19 | |
| *** unicell has quit IRC | 02:20 | |
| *** ssurana has quit IRC | 02:21 | |
| *** weiyu_ has quit IRC | 02:21 | |
| *** vhosakot has joined #openstack-kolla | 02:23 | |
| *** shakamunyi has quit IRC | 02:24 | |
| *** banix has quit IRC | 02:25 | |
| *** rstarmer has joined #openstack-kolla | 02:25 | |
| *** weiyu_ has joined #openstack-kolla | 02:29 | |
| *** aginwala has joined #openstack-kolla | 02:29 | |
| *** weiyu_ has quit IRC | 02:30 | |
| openstackgerrit | Kuo-tung Kao proposed openstack/kolla: add unit tests for build script https://review.openstack.org/290423 | 02:31 |
| *** rstarmer has quit IRC | 02:31 | |
| *** weiyu_ has joined #openstack-kolla | 02:32 | |
| *** sdake_ is now known as sdake | 02:33 | |
| *** aginwala has quit IRC | 02:39 | |
| vhosakot | can folks ack https://review.openstack.org/#/c/302026/ I added validations to reduce errors seen by new openrators | 02:41 |
| patchbot | vhosakot: patch 302026 - kolla - Add prechecks to validate kolla_internal_vip_addre... | 02:41 |
| *** yuanying has quit IRC | 02:46 | |
| *** ccesario___ has joined #openstack-kolla | 02:51 | |
| *** aginwala has joined #openstack-kolla | 02:52 | |
| *** Marga_ has quit IRC | 02:53 | |
| *** aginwala has quit IRC | 02:59 | |
| *** kjelly_ has joined #openstack-kolla | 03:01 | |
| *** cbits has joined #openstack-kolla | 03:03 | |
| vhosakot | daneyon: could you retry building horizon container on CentOS source... I was seeing https://bugs.launchpad.net/kolla/+bug/1567657 | 03:11 |
| openstack | Launchpad bug 1567657 in kolla "Build Fails Using Tunk" [Undecided,New] | 03:11 |
| *** diogogmt has quit IRC | 03:12 | |
| *** shakamunyi has joined #openstack-kolla | 03:18 | |
| *** shakamunyi has quit IRC | 03:23 | |
| *** vhosakot has quit IRC | 03:23 | |
| *** d_code has joined #openstack-kolla | 03:26 | |
| *** Marga_ has joined #openstack-kolla | 03:27 | |
| *** Marga_ has quit IRC | 03:31 | |
| *** diogogmt has joined #openstack-kolla | 03:37 | |
| *** Marga_ has joined #openstack-kolla | 03:45 | |
| *** jasonsb has joined #openstack-kolla | 03:47 | |
| *** yuanying has joined #openstack-kolla | 03:49 | |
| *** jasonsb has quit IRC | 03:49 | |
| *** jasonsb has joined #openstack-kolla | 03:51 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 04:00 | |
| *** tfukushima has joined #openstack-kolla | 04:00 | |
| *** ssurana has joined #openstack-kolla | 04:03 | |
| *** reddy has quit IRC | 04:10 | |
| *** shakamunyi has joined #openstack-kolla | 04:18 | |
| *** sdake_ has joined #openstack-kolla | 04:19 | |
| *** sdake has quit IRC | 04:19 | |
| *** shakamunyi has quit IRC | 04:22 | |
| daneyon | vhosakot rebuilding now | 04:27 |
| *** weiyu_ has quit IRC | 04:31 | |
| *** aginwala has joined #openstack-kolla | 04:39 | |
| *** shakamunyi has joined #openstack-kolla | 04:40 | |
| *** shakamunyi has quit IRC | 04:45 | |
| *** kjelly_ has quit IRC | 04:48 | |
| *** cbits has quit IRC | 04:50 | |
| *** salv-orlando has joined #openstack-kolla | 04:52 | |
| *** d_code has quit IRC | 04:54 | |
| *** sdake has joined #openstack-kolla | 05:03 | |
| *** salv-orlando has quit IRC | 05:04 | |
| *** d_code has joined #openstack-kolla | 05:04 | |
| *** sdake_ has quit IRC | 05:06 | |
| *** weiyu_ has joined #openstack-kolla | 05:09 | |
| *** jtriley has quit IRC | 05:09 | |
| *** d_code has quit IRC | 05:12 | |
| *** sdake_ has joined #openstack-kolla | 05:13 | |
| *** sdake has quit IRC | 05:17 | |
| *** shakamunyi has joined #openstack-kolla | 05:18 | |
| openstackgerrit | MD NADEEM proposed openstack/kolla: Update vagrant-dev-env.rst https://review.openstack.org/303218 | 05:19 |
| *** shakamunyi has quit IRC | 05:23 | |
| *** reddy has joined #openstack-kolla | 05:25 | |
| *** weiyu_ has quit IRC | 05:31 | |
| *** salv-orlando has joined #openstack-kolla | 05:32 | |
| *** Jeffrey4l_ has quit IRC | 05:50 | |
| *** aginwala has quit IRC | 05:53 | |
| *** sdake has joined #openstack-kolla | 05:55 | |
| *** sdake_ has quit IRC | 05:57 | |
| *** kjelly_ has joined #openstack-kolla | 05:59 | |
| *** mikelk has joined #openstack-kolla | 06:00 | |
| *** banix has joined #openstack-kolla | 06:02 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 06:03 | |
| *** kjelly1 has joined #openstack-kolla | 06:07 | |
| *** weiyu has joined #openstack-kolla | 06:07 | |
| *** kjelly1 has left #openstack-kolla | 06:08 | |
| *** sdake_ has joined #openstack-kolla | 06:09 | |
| *** sdake has quit IRC | 06:11 | |
| sdake_ | Jeffrey4l_ you done sleeping yet | 06:11 |
| Jeffrey4l_ | sdake_, yea :D | 06:11 |
| sdake_ | nice | 06:11 |
| sdake_ | so i am testing out your patch | 06:11 |
| sdake_ | and it doesn't work | 06:11 |
| sdake_ | centos source | 06:12 |
| sdake_ | here is the output | 06:12 |
| Jeffrey4l_ | sdake_, what happen? any log | 06:12 |
| Jeffrey4l_ | ok | 06:12 |
| Jeffrey4l_ | let me check. | 06:12 |
| sdake_ | note i had to push an update | 06:12 |
| sdake_ | your patch didn't build | 06:12 |
| sdake_ | centos binary | 06:12 |
| sdake_ | and you left out ebtables | 06:13 |
| sdake_ | http://paste.fedoraproject.org/351503/9596014/ -> http://paste.fedoraproject.org/351503/09596014 | 06:13 |
| *** SiRiuS_ has joined #openstack-kolla | 06:14 | |
| Jeffrey4l_ | sdake_, i will test it locally, thanks. | 06:15 |
| Jeffrey4l_ | sdake_, are u have any fix for this? | 06:15 |
| sdake_ | no i just got done testing it | 06:15 |
| sdake_ | i took a nap | 06:15 |
| sdake_ | and just woke up too ;) | 06:15 |
| sdake_ | the patch yu submittied didn't build at all | 06:16 |
| sdake_ | note i am using ceph aio | 06:16 |
| sdake_ | with linuxbridge set | 06:16 |
| sdake_ | check the errors | 06:16 |
| sdake_ | cant read instance | 06:16 |
| Jeffrey4l_ | sdake_, saw that. | 06:16 |
| sdake_ | i think libvirt needs to be addded to nova and nova to libvirt | 06:17 |
| Jeffrey4l_ | sdake_, build should be OK. the build in the gate is all green. | 06:17 |
| sdake_ | yes because i fixed it | 06:17 |
| sdake_ | you were doign a useradd nova | 06:17 |
| sdake_ | which failed | 06:17 |
| sdake_ | this cause the build to fail | 06:17 |
| Jeffrey4l_ | ah, thanks | 06:17 |
| sdake_ | so you were testing an "old" image | 06:17 |
| sdake_ | not the one you actually built for nova* | 06:18 |
| *** shakamunyi has joined #openstack-kolla | 06:18 | |
| mdnadeem | sdake_, hello, I would like to work on this bp : https://blueprints.launchpad.net/kolla/+spec/enable-congress-container | 06:18 |
| sdake_ | mdnadeem feel fre | 06:19 |
| mdnadeem | sdake_, can you please assign it to me(mail2nadeem92) | 06:19 |
| sdake_ | e | 06:19 |
| *** coolsvap has joined #openstack-kolla | 06:19 | |
| mdnadeem | sdake_, Thanks :) | 06:20 |
| sdake_ | enjoy | 06:21 |
| *** Jeffrey4l__ has joined #openstack-kolla | 06:21 | |
| sdake_ | http://paste.fedoraproject.org/351511/46009658/ -> http://paste.fedoraproject.org/351511/46009658 | 06:23 |
| sdake_ | Jeffrey4l_ more debug | 06:23 |
| sdake_ | libvirt cnt writ to /var/lib/nova | 06:23 |
| *** shakamunyi has quit IRC | 06:23 | |
| sdake_ | thats why console.log can't be found | 06:23 |
| sdake_ | in the compute log | 06:23 |
| *** andrewliu117__ has joined #openstack-kolla | 06:24 | |
| Jeffrey4l__ | sdake_, i am testing the latest code. And will try to fix it. | 06:24 |
| sdake_ | no try | 06:24 |
| sdake_ | do ;) | 06:24 |
| Jeffrey4l__ | sdake_, yea. i can fix it ;D | 06:25 |
| *** Jeffrey4l__ is now known as Jeffrey4l | 06:25 | |
| *** Jeffrey4l_ has quit IRC | 06:25 | |
| sdake_ | if we add libvirt to the nova group, the permissions still must chane to 775 | 06:25 |
| *** ccesario___ has quit IRC | 06:35 | |
| *** banix_ has joined #openstack-kolla | 06:35 | |
| *** banix_ has quit IRC | 06:36 | |
| *** banix has quit IRC | 06:36 | |
| *** fangfenghua has joined #openstack-kolla | 06:44 | |
| *** fangfenghua has left #openstack-kolla | 06:44 | |
| *** salv-orlando has quit IRC | 06:54 | |
| *** yuanying has quit IRC | 06:57 | |
| *** Serlex has joined #openstack-kolla | 07:09 | |
| *** shardy has joined #openstack-kolla | 07:13 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/303267 | 07:13 |
| *** kjelly_ has quit IRC | 07:16 | |
| *** shakamunyi has joined #openstack-kolla | 07:18 | |
| *** unicell has joined #openstack-kolla | 07:20 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Increase max pool size so conductor doesn't implode https://review.openstack.org/302415 | 07:21 |
| *** gfidente has joined #openstack-kolla | 07:23 | |
| *** gfidente has joined #openstack-kolla | 07:23 | |
| *** rmart04 has joined #openstack-kolla | 07:23 | |
| openstackgerrit | Merged openstack/kolla: Add mongodb in cleanup containers script https://review.openstack.org/303180 | 07:23 |
| *** shakamunyi has quit IRC | 07:23 | |
| openstackgerrit | Merged openstack/kolla: Update vagrant-dev-env.rst https://review.openstack.org/303218 | 07:39 |
| *** kjelly has quit IRC | 07:39 | |
| *** jmccarthy has joined #openstack-kolla | 07:42 | |
| openstackgerrit | Merged openstack/kolla: Adds migration step from nova to nova_api database https://review.openstack.org/302931 | 07:48 |
| Jeffrey4l | sdake_, need go out. will be back in 1 hour. I have found the reason why `fix neutron linuxbridge agent` failed. Will push the fix when I am back. | 07:54 |
| sdake_ | how about push now | 07:55 |
| sdake_ | so i can get to work on qemu | 07:55 |
| Jeffrey4l | sdake_, no. I have no idea why it happen and the solution. | 07:56 |
| sdake_ | you said you found reason | 07:56 |
| sdake_ | you said you have no idea why it happen | 07:56 |
| sdake_ | which one is it ;) | 07:56 |
| Jeffrey4l | sdake_, the /var/lib/nova have wrong permission. http://paste.openstack.org/show/493439/ | 07:57 |
| Jeffrey4l | this is the reason. when fix it , it will be Ok. | 07:57 |
| Jeffrey4l | But i have no idea why it happen. We have set the 755 permission to the /var/lib/nova folder in the extend_start.sh script. | 07:58 |
| Jeffrey4l | sdake_, ^& | 07:58 |
| sdake_ | what permission do you expect it to have | 07:58 |
| sdake_ | oh 755 | 07:58 |
| sdake_ | and its 700 | 07:58 |
| Jeffrey4l | If i run the kolla_nova_extend_start in the nova_compute container, it fix the permission. | 07:58 |
| Jeffrey4l | sdake_, correct. | 07:58 |
| Jeffrey4l | I have no idea why it is 700 now. | 07:59 |
| sdake_ | there is code in nova-api to bootstrap too | 07:59 |
| sdake_ | have you made nay changes | 07:59 |
| sdake_ | if not i'll look at it while your out | 07:59 |
| sdake_ | qmeu i bocked on this work | 07:59 |
| sdake_ | blocked | 07:59 |
| Jeffrey4l | i changed this file https://review.openstack.org/#/c/302721/6/docker/nova/nova-compute/extend_start.sh sdake_ | 07:59 |
| patchbot | Jeffrey4l: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 07:59 |
| Jeffrey4l | I know that. | 08:00 |
| Jeffrey4l | this file do not work as expected. | 08:00 |
| Jeffrey4l | need go. If you can fix it, please do it. Otherwise, i will fix it when i am back in 1 hours. | 08:02 |
| *** jtriley has joined #openstack-kolla | 08:05 | |
| sdake_ | i might just hit the sack | 08:06 |
| *** andrewvwebber has joined #openstack-kolla | 08:13 | |
| coolsvap | sdake_: i am checking the centos source gate failure on stable.mitaka | 08:16 |
| openstackgerrit | Merged openstack/kolla: Updated from global requirements https://review.openstack.org/303267 | 08:16 |
| coolsvap | filing a bug for the same | 08:17 |
| sdake_ | i think vhosakot is looking into that | 08:17 |
| sdake_ | i left a comment for your ssh requests bug | 08:17 |
| sdake_ | that needs fixing | 08:17 |
| sdake_ | ls | 08:18 |
| *** shakamunyi has joined #openstack-kolla | 08:18 | |
| coolsvap | the centos source issue on the security fix does not seem related | 08:19 |
| coolsvap | since there is only ubuntu package updates | 08:19 |
| coolsvap | i tried building the failed images locally earlier they didnt fail for centos source | 08:19 |
| sdake_ | i explained what the problem is | 08:20 |
| sdake_ | requests is already installed | 08:20 |
| sdake_ | uninstall it | 08:20 |
| sdake_ | the gate will pass | 08:20 |
| coolsvap | hmm | 08:20 |
| sdake_ | cpio copy errors are alawysa package already installed | 08:20 |
| sdake_ | pip vs rpm conflict | 08:21 |
| coolsvap | hmm | 08:22 |
| coolsvap | checking | 08:22 |
| *** shakamunyi has quit IRC | 08:23 | |
| pmisiak | hi guys | 08:27 |
| pmisiak | I have problem with building neutron-openvswitch-agent container | 08:28 |
| pmisiak | neutron-plugin-openvswitch : Depends: neutron-common (= 1:2014.1.5-0ubuntu4) but 2:8.0.0~b2-0ubuntu7~cloud0 is to be installed | 08:28 |
| andrewvwebber | pmisiak: i had no problems then the base was fedora, yesterday | 08:29 |
| pmisiak | my base is ubuntu | 08:29 |
| pmisiak | im in neutron-base container and I'm trying to debug this | 08:29 |
| sdake_ | pmisiak ubuntu binary? | 08:30 |
| pmisiak | yes | 08:30 |
| sdake_ | ya that probably doesn't work too well at this point | 08:30 |
| sdake_ | it is failing the build gate | 08:30 |
| sdake_ | and the deploy gate | 08:30 |
| sdake_ | try ubuntu source or centos source | 08:31 |
| *** kjejlly_ has joined #openstack-kolla | 08:32 | |
| *** openstackgerrit has quit IRC | 08:33 | |
| *** openstackgerrit has joined #openstack-kolla | 08:33 | |
| *** athomas has joined #openstack-kolla | 08:33 | |
| *** timonwong has joined #openstack-kolla | 08:33 | |
| andrewvwebber | when i try to create an instance via nova i get a timeout, within nova-api.log i see a timeout while trying to "validate networks". Does anyone have any advise where to look next? | 08:33 |
| *** jmccarthy has left #openstack-kolla | 08:33 | |
| sdake_ | andrewvwebber centos binary? | 08:34 |
| andrewvwebber | yes | 08:34 |
| *** ccesario___ has joined #openstack-kolla | 08:34 | |
| sdake_ | ya dont use centos binary | 08:34 |
| andrewvwebber | :) | 08:34 |
| andrewvwebber | thanks | 08:34 |
| sdake_ | its broken because rdo project has not cut a new releease | 08:34 |
| sdake_ | in like 3 weeks | 08:34 |
| sdake_ | its been broken for that long | 08:34 |
| andrewvwebber | ok thanks | 08:35 |
| sdake_ | dmsimard can add mor econtext if you need | 08:35 |
| andrewvwebber | so centos source is the way | 08:35 |
| sdake_ | for the moment | 08:35 |
| sdake_ | we re waiting on rdo packaging to work correctly | 08:36 |
| sdake_ | its not a kolla bug | 08:36 |
| sdake_ | its a centos rdo problem with using old upstream packages | 08:36 |
| andrewvwebber | ok i will rebuild the containers, thanks | 08:36 |
| *** Serlex has left #openstack-kolla | 08:37 | |
| andrewvwebber | im just supprised that i was able to get so far on coreos :) | 08:38 |
| andrewvwebber | however i imagine success has also been made on atomic i presume... | 08:38 |
| timonwong | Hello, our company are seeking deploy OpenStack cloud (with customizations) using kolla, is kolla now production ready? | 08:39 |
| sdake_ | timonwong define production ready | 08:40 |
| sdake_ | we have our 2.0.0 release on the 15th | 08:40 |
| sdake_ | so at this time, 2.0.0 rcs are not ready for deployment | 08:40 |
| sdake_ | it does support customizations | 08:41 |
| sdake_ | i'd feel comfortable deploying 2.0.0 when its releaed for a private cloud up to 100 node scale | 08:42 |
| sdake_ | (e.g. 3 racks of gear) | 08:42 |
| timonwong | Thanks for you reply, sdake_, for now, because we need to use ldap (for company AD), using a custom domain, so I find there is some manual steps for both keystone and horizon. | 08:44 |
| sdake_ | ya plugins | 08:45 |
| sdake_ | we are orking on that for newton | 08:45 |
| *** Serlex has joined #openstack-kolla | 08:46 | |
| timonwong | Good to know :), so for now I think we may have a patched kolla ansible scripts to do those jobs. | 08:48 |
| andrewliu117__ | hi, sdake_, new guy for openstack and kolla, we are planing to deploy openstack for production environment, is it ok to use kolla to deploy? | 08:50 |
| andrewliu117__ | is there anything we should pay attention to? | 08:51 |
| sdake_ | andrewliu117__ I'd wait until 2.0.0 persoanlly | 08:51 |
| sdake_ | and I'd do alot of deploys up fron tto mak sure your familiar with the software and how it opertes | 08:51 |
| sdake_ | rcs re not redy yet for deployment | 08:51 |
| sdake_ | there are about 30 critical bugs | 08:52 |
| andrewliu117__ | ok, got the idea | 08:53 |
| andrewliu117__ | 2.0.0 will be ready at | 08:53 |
| andrewliu117__ | April 15th, right? | 08:53 |
| sdake_ | thats the theory | 08:53 |
| sdake_ | but i am not tagging soething that isn't woring | 08:54 |
| andrewliu117__ | that sounds promising | 08:54 |
| *** timonwong has left #openstack-kolla | 08:56 | |
| *** timonwong_ has joined #openstack-kolla | 08:56 | |
| andrewliu117__ | sdake_, thanks for your time, really help us a lot | 08:57 |
| sdake_ | sure if you have more questions feel free to ask anyone in the channel not just me | 08:57 |
| *** timonwong_ is now known as timonwong | 08:59 | |
| ccesario___ | Morning guys ! | 09:05 |
| sdake_ | night guys | 09:05 |
| sdake_ | Jeffrey4l no idea | 09:05 |
| sdake_ | need to hit the rack 2am | 09:06 |
| sdake_ | ttyl | 09:06 |
| sdake_ | plese leave a not ein th review log on linuxbridge patch | 09:06 |
| sdake_ | Jeffrey4l my first inclination is to think the bootstrap isn't run for compute ater looking at the ansible playbooks | 09:06 |
| *** timonwong_ has joined #openstack-kolla | 09:12 | |
| *** timonwong_ has left #openstack-kolla | 09:12 | |
| *** timonwong has quit IRC | 09:14 | |
| *** sdake_ has quit IRC | 09:17 | |
| *** salv-orlando has joined #openstack-kolla | 09:18 | |
| *** andrewliu117_ has joined #openstack-kolla | 09:19 | |
| *** andrewliu117__ has quit IRC | 09:22 | |
| *** andrewliu117_ is now known as andrewliu117 | 09:24 | |
| pmisiak | after applying this patch: http://paste.openstack.org/show/493452/ neutron-openvswitch-agent has been built succesfully | 09:24 |
| pmisiak | ubuntu binary | 09:24 |
| *** subscope has joined #openstack-kolla | 09:38 | |
| *** ssurana has quit IRC | 09:42 | |
| *** ssurana has joined #openstack-kolla | 09:42 | |
| *** subscope has quit IRC | 09:44 | |
| *** gfidente has quit IRC | 09:45 | |
| *** subscope has joined #openstack-kolla | 09:45 | |
| openstackgerrit | Merged openstack/kolla: Change waiting on mariadb.pid https://review.openstack.org/301974 | 09:45 |
| *** gfidente has joined #openstack-kolla | 09:47 | |
| *** gfidente has quit IRC | 09:47 | |
| *** gfidente has joined #openstack-kolla | 09:47 | |
| openstackgerrit | Merged openstack/kolla: Updated from global requirements https://review.openstack.org/302856 | 09:50 |
| *** yuanying has joined #openstack-kolla | 09:50 | |
| andrewvwebber | after building centos (source), starting an instance results in "No valid host was found. There are not enough hosts available". | 09:50 |
| *** ssurana has quit IRC | 09:51 | |
| *** ccesario___ has quit IRC | 09:53 | |
| andrewvwebber | i managed to hunt down - "libvirtError: Cannot access storage file '/var/lib/nova/instances/890061e7-1e17-43f8-8d72-f25adad53c04/disk' (as uid:107, gid:107): Permission denied" | 09:56 |
| openstackgerrit | Merged openstack/kolla: Fix manila keystone v3 issues for all clients https://review.openstack.org/302694 | 09:57 |
| *** athomas has quit IRC | 10:07 | |
| *** ccesario___ has joined #openstack-kolla | 10:07 | |
| *** subscope has quit IRC | 10:10 | |
| *** weiyu has quit IRC | 10:10 | |
| *** athomas has joined #openstack-kolla | 10:11 | |
| *** cbits has joined #openstack-kolla | 10:12 | |
| *** weiyu_ has joined #openstack-kolla | 10:15 | |
| *** subscope has joined #openstack-kolla | 10:15 | |
| *** ccesario___ has quit IRC | 10:20 | |
| *** subscope has quit IRC | 10:20 | |
| *** ccesario___ has joined #openstack-kolla | 10:26 | |
| *** weiyu_ has quit IRC | 10:29 | |
| *** ccesario___ has quit IRC | 10:38 | |
| *** ccesario___ has joined #openstack-kolla | 10:41 | |
| pmisiak | the cmd: mount --make-shared /run | 10:46 |
| pmisiak | should I add this command to startup scripts? | 10:46 |
| pmisiak | after reboot /run is not shared any more | 10:47 |
| *** ccesario___ has quit IRC | 10:47 | |
| *** ccesario___ has joined #openstack-kolla | 10:47 | |
| *** ccesario___ has quit IRC | 10:48 | |
| *** ccesario___ has joined #openstack-kolla | 10:49 | |
| *** rhallisey has joined #openstack-kolla | 10:58 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add LVM2/iSCSI documentation https://review.openstack.org/300824 | 11:01 |
| *** weiyu has joined #openstack-kolla | 11:03 | |
| *** weiyu has quit IRC | 11:04 | |
| *** ccesario___ has quit IRC | 11:12 | |
| *** weiyu_ has joined #openstack-kolla | 11:13 | |
| *** weiyu_ has quit IRC | 11:15 | |
| sbezverk | pmisiak I added mount --make-shared /run to /etc/rc.local and it seems it is doing the job right | 11:15 |
| *** shakamunyi has joined #openstack-kolla | 11:18 | |
| andrewvwebber | libvirtd.log - "virStorageFileGetMetadataRecurse:2998 : Cannot access storage file '/var/lib/nova/instances/c1bff7c5-491f-4f9f-ae37-66adea1d88b5/disk' (as uid:107, gid:107): Permission denied" | 11:21 |
| *** shakamunyi has quit IRC | 11:23 | |
| *** tfukushima has quit IRC | 11:23 | |
| rhallisey | andrewvwebber, check the user:group of /var/lib/nova/instances I'm curious | 11:24 |
| rhallisey | brb amin | 11:24 |
| *** cbits has quit IRC | 11:26 | |
| *** cbits has joined #openstack-kolla | 11:27 | |
| Jeffrey4l | andrewvwebber, rhallisey i am working on a PS fix that issue. | 11:27 |
| *** tfukushima has joined #openstack-kolla | 11:34 | |
| *** jtriley has quit IRC | 11:35 | |
| *** cbits has quit IRC | 11:36 | |
| *** weiyu_ has joined #openstack-kolla | 11:38 | |
| *** subscope has joined #openstack-kolla | 11:39 | |
| *** weiyu_ has quit IRC | 11:40 | |
| pmisiak | I have problem with keystone bootstrapping | 11:42 |
| *** mlima has joined #openstack-kolla | 11:42 | |
| mlima | morning guys | 11:43 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Install proper security libs https://review.openstack.org/295602 | 11:44 |
| pmisiak | error: http://paste.openstack.org/show/493464/ | 11:44 |
| ccesario | mlima, morning :) | 11:44 |
| pmisiak | keystone package version: 2:9.0.0~b2-0ubuntu1~cloud0 | 11:45 |
| *** salv-orlando has quit IRC | 11:46 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 11:46 |
| andrewvwebber | Jeffrey4l does this mean that this is an issue and that i have not forgotten an installation step? | 11:47 |
| Jeffrey4l | andrewvwebber, yep. It is a bug. | 11:47 |
| andrewvwebber | phew :) | 11:48 |
| andrewvwebber | is there a work around ? | 11:48 |
| *** weiyu_ has joined #openstack-kolla | 11:48 | |
| andrewvwebber | i guess qemu does not have rights to the folder? | 11:48 |
| *** subscope has quit IRC | 11:50 | |
| *** ayoung has quit IRC | 11:54 | |
| *** Marga_ has quit IRC | 11:54 | |
| *** Marga_ has joined #openstack-kolla | 11:54 | |
| *** SiRiuS_ has quit IRC | 11:59 | |
| *** SiRiuS_ has joined #openstack-kolla | 12:00 | |
| *** subscope has joined #openstack-kolla | 12:03 | |
| *** weiyu_ has quit IRC | 12:04 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Set db connection retry to infinity https://review.openstack.org/302434 | 12:05 |
| *** rhallisey is now known as rhallisey-brb | 12:07 | |
| *** dwalsh has joined #openstack-kolla | 12:09 | |
| *** andrewliu117 has quit IRC | 12:11 | |
| *** shakamunyi has joined #openstack-kolla | 12:18 | |
| *** salv-orlando has joined #openstack-kolla | 12:22 | |
| *** banix has joined #openstack-kolla | 12:23 | |
| *** shakamunyi has quit IRC | 12:23 | |
| *** Marga_ has quit IRC | 12:26 | |
| *** banix has quit IRC | 12:27 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add LVM2/iSCSI documentation https://review.openstack.org/300824 | 12:30 |
| *** d_code has joined #openstack-kolla | 12:30 | |
| *** banix has joined #openstack-kolla | 12:32 | |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Set SELinux mode to permissive mode https://review.openstack.org/303381 | 12:33 |
| Jeffrey4l | andrewvwebber, 1. set the /var/lib/nova and /var/lib/nova/instances to 755 2. add the qemu to nova group `usermod -a -G nova qemu` in nova_libvirt container. | 12:34 |
| *** rhallisey-brb is now known as rhallisey | 12:35 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Drop root for swift https://review.openstack.org/259374 | 12:36 |
| *** Jeffrey4l has quit IRC | 12:42 | |
| *** Jeffrey4l has joined #openstack-kolla | 12:42 | |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Enable unbuffered output for ansible https://review.openstack.org/303389 | 12:45 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Update tox to use the experimental JJB template https://review.openstack.org/303391 | 12:46 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Update tox to use the experimental JJB template https://review.openstack.org/303391 | 12:49 |
| andrewvwebber | Jeffrey4l: thanks, i will give that a try | 12:51 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: [DNM] TEsting https://review.openstack.org/303395 | 12:54 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: add curlrc file to base image https://review.openstack.org/301911 | 12:54 |
| *** cbits has joined #openstack-kolla | 12:55 | |
| *** coolsvap has quit IRC | 12:56 | |
| *** jtriley has joined #openstack-kolla | 12:57 | |
| openstackgerrit | Paul Belanger proposed openstack/kolla: [DNM] TEsting https://review.openstack.org/303395 | 12:58 |
| pmisiak | what is the most tested and working installation mode? centos:binary, centos:source, ubuntu:binary? | 13:03 |
| *** Jeffrey4l has quit IRC | 13:05 | |
| andrewvwebber | pmisiak: this is a great question | 13:06 |
| *** Jeffrey4l has joined #openstack-kolla | 13:06 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Put the variables in alphabetical order https://review.openstack.org/303409 | 13:07 |
| pmisiak | hehe, containers builded yesterday are not deployable | 13:08 |
| pmisiak | but after rebuilding then today its promising, deployment is running ;) | 13:09 |
| pmisiak | *them | 13:09 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Enable unbuffered output for ansible https://review.openstack.org/303389 | 13:10 |
| pmisiak | ubuntu packages has been changed meanwhile | 13:10 |
| andrewvwebber | pimisiak: i was able to deploy cento successfully, i have only issue post deployment | 13:10 |
| andrewvwebber | pimisiak: centos/source was successful, i am not trying ubuntu/source... | 13:11 |
| pmisiak | in ubuntu:binary one important container doesn't build | 13:11 |
| pmisiak | after a small patch its ok | 13:12 |
| andrewvwebber | pmisiak: i found building from source to be safe as sdake suggested | 13:12 |
| pmisiak | yeah one dependency less (packages versions from official repos) | 13:13 |
| *** banix has quit IRC | 13:14 | |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Make sure /etc/hostname is valid for SELinux https://review.openstack.org/303414 | 13:15 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: [DNM] Update tox to use the experimental JJB template https://review.openstack.org/303391 | 13:17 |
| *** shakamunyi has joined #openstack-kolla | 13:18 | |
| *** d_code has quit IRC | 13:19 | |
| *** cbits has quit IRC | 13:22 | |
| *** shakamunyi has quit IRC | 13:23 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 13:28 |
| andrewvwebber | pmisiak: ubuntu/source also successful | 13:29 |
| pmisiak | ubuntu:binary too :) | 13:29 |
| pmisiak | finally :) | 13:30 |
| *** subscope has quit IRC | 13:30 | |
| pmisiak | but this patch is required: http://paste.openstack.org/show/493452/ | 13:30 |
| *** banix has joined #openstack-kolla | 13:32 | |
| andrewvwebber | ok i will try and confirm that | 13:35 |
| *** dims is now known as dimsum__ | 13:41 | |
| andrewvwebber | Jeffrey4l: after switching from centos/source to ubuntu/source i am able to launch instances using nova | 13:42 |
| andrewvwebber | pmisiak: without your patch i was able to deploy and launch an instance. Did you need this change for another reason or just to build the containers | 13:44 |
| Jeffrey4l | andrewvwebber, the root cause is that: on the centos the UMASK is 077, and on the ubuntu the UMASK is 022. when creating the nova home folder, they are different. | 13:44 |
| andrewvwebber | understood, thanks | 13:44 |
| *** mgoddard_ has joined #openstack-kolla | 13:45 | |
| pmisiak | andrewvwebber: without this patch neutron-openvswitch-agent container does not build in ubuntu:binary | 13:46 |
| *** allen_gao has quit IRC | 13:46 | |
| andrewvwebber | ok so your patch was specific to binary not source, i am still coming to terms with the differences :) | 13:47 |
| *** cbits has joined #openstack-kolla | 13:48 | |
| *** inc0 has joined #openstack-kolla | 13:48 | |
| *** mgoddard has quit IRC | 13:49 | |
| *** allen_gao has joined #openstack-kolla | 13:52 | |
| pabelanger | Greetings kolla, if core reviewiers would mind +A https://review.openstack.org/#/c/303389/ and https://review.openstack.org/#/c/303414/ it will help openstack-infra migrate your project from devstack-centos7 to centos-7 dibs. | 13:53 |
| patchbot | pabelanger: patch 303389 - kolla - Enable unbuffered output for ansible | 13:53 |
| patchbot | pabelanger: patch 303414 - kolla - Make sure /etc/hostname is valid for SELinux | 13:53 |
| pabelanger | first one is a nice to have, makes debugging better in the gate, other is an issue with centos-7 dib | 13:54 |
| *** subscope has joined #openstack-kolla | 13:57 | |
| *** ayoung has joined #openstack-kolla | 13:57 | |
| *** diogogmt has quit IRC | 13:58 | |
| *** diogogmt has joined #openstack-kolla | 14:00 | |
| andrewvwebber | pmisiak: my instances have no dhcp assigned IPs once created, so i will try your patch and see if that helps | 14:01 |
| *** ayoung has quit IRC | 14:04 | |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Make sure /etc/hostname is valid for SELinux https://review.openstack.org/303414 | 14:08 |
| pmisiak | my patch is not related to ubuntu:source | 14:08 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MASTER BRANCH https://review.openstack.org/294906 | 14:09 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MITAKA BRANCH https://review.openstack.org/299182 | 14:10 |
| pabelanger | As a follow up to above, this is the patch -infra will be landing to migrate kolla to centos-7: https://review.openstack.org/#/c/301320 | 14:11 |
| pabelanger | results of the testing can be seen: http://logs.openstack.org/91/303391/3/experimental/gate-kolla-bindep-build-centos-7-binary/24ef296/console.html and http://logs.openstack.org/91/303391/3/experimental/gate-kolla-bindep-build-centos-7-source/a6fc0a7/console.html | 14:11 |
| inc0 | hey guys | 14:12 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Enable unbuffered output for ansible https://review.openstack.org/303389 | 14:13 |
| *** diogogmt has quit IRC | 14:15 | |
| *** dave-mccowan has joined #openstack-kolla | 14:16 | |
| inc0 | sooo, this day isn't a starting to be a particulary good one | 14:18 |
| inc0 | I deployed rabbitmq on fresh rebuild | 14:18 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Make sure /etc/hostname is valid for SELinux https://review.openstack.org/303414 | 14:19 |
| inc0 | ss -l | grep 5672 doesn't show it's listening on it's usual port-.- | 14:19 |
| *** tfukushima has quit IRC | 14:19 | |
| pmisiak | glance-api does not work in my env | 14:20 |
| pmisiak | ImportError: No module named memcache | 14:20 |
| *** diogogmt has joined #openstack-kolla | 14:22 | |
| andrewvwebber | pmisiak: genius | 14:23 |
| *** AJaeger has joined #openstack-kolla | 14:24 | |
| *** mwheckmann has joined #openstack-kolla | 14:25 | |
| *** vhosakot has joined #openstack-kolla | 14:25 | |
| pmisiak | look at Dockerfile for openstack-base | 14:31 |
| pmisiak | so many packages installed on centos and only few on ubuntu :) | 14:31 |
| pabelanger | What time does sdake usually get online at? | 14:31 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Add default flavors during init-runonce https://review.openstack.org/302534 | 14:33 |
| *** subscope has quit IRC | 14:34 | |
| inc0 | pabelanger, mornings are not his forte really | 14:37 |
| inc0 | any problem specifically? maybe I can help? | 14:37 |
| vhosakot | pabelanger: thanks for adding fix for selinux and unbuffered ouput for ansible... | 14:37 |
| pabelanger | inc0: thanks | 14:38 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/302036 | 14:38 |
| vhosakot | pabelanger: per your comment, i will add CONTRIBUTING.rst to the repo | 14:38 |
| vhosakot | Jeffrey4l: https://review.openstack.org/#/c/302026/ | 14:38 |
| patchbot | vhosakot: patch 302026 - kolla - Add prechecks to validate kolla_internal_vip_addre... | 14:38 |
| pabelanger | inc0: just want to make sure he is okay with us switching gate jobs from devstack-centos7 to centos-7 dibs today | 14:38 |
| pabelanger | we are removing devstack-* nodes from the gate and replacing them with dibs | 14:39 |
| *** haplo37 has joined #openstack-kolla | 14:39 | |
| inc0 | ok, can't answer that | 14:39 |
| vhosakot | pabelanger: is there any difference between devstack and dibs ? | 14:39 |
| pmisiak | how can I update only one container? | 14:40 |
| pmisiak | I have new version in my local repository | 14:40 |
| pmisiak | glance-api | 14:40 |
| vhosakot | pmisiak: have you built the new version using kolla-build ? | 14:40 |
| pabelanger | vhosakot: yes, basically centos-7 has no dependencies installed by default. We now using bindep to add the packages a job launch, so what is will mean for kolla you'll eventually add an empty other-requirements.txt file to your project and you should get an almost empty image to then run ansible on | 14:41 |
| pabelanger | but default, we install all the same packages as devstack-centos7 if other-requirements.txt is missing | 14:42 |
| vhosakot | pabelanger: ah ok, if we add packages in other-requirements.txt, will those be installed in dibs at job launch ? | 14:43 |
| pabelanger | vhosakot: correct | 14:43 |
| inc0 | cool, in our case this will be almost empty | 14:43 |
| pabelanger | yup | 14:43 |
| vhosakot | right | 14:44 |
| inc0 | so it will loose a bit of load you have guys | 14:44 |
| pabelanger | that's what I do for windmill and have ansible just deal with packages | 14:44 |
| *** mikelk has quit IRC | 14:44 | |
| pabelanger | it will also make you gate jobs run faster with an empty other-requirements.txt | 14:44 |
| pabelanger | since we don't need to bootstrap the jobs | 14:44 |
| vhosakot | ah ok | 14:44 |
| inc0 | yup | 14:44 |
| *** sdake has joined #openstack-kolla | 14:45 | |
| pabelanger | https://github.com/openstack/windmill/blob/master/other-requirements.txt | 14:45 |
| pabelanger | that's what I only have | 14:45 |
| pmisiak | vhosakot: yes | 14:45 |
| pabelanger | libselinux-python is needed for yum module | 14:45 |
| pabelanger | since is modifies SELinex | 14:45 |
| pmisiak | vhosakot: using kolla-build | 14:45 |
| vhosakot | pmisiak: so, re-deply now... "kolla-ansible deploy" | 14:46 |
| pmisiak | vhosakot: can I use some tags to short deploy time? | 14:47 |
| vhosakot | pmisiak: yes... can you try "kolla-ansible deploy -t glance" | 14:47 |
| vhosakot | pabelanger: thanks for the info... sdake is here | 14:48 |
| *** sdake_ has joined #openstack-kolla | 14:49 | |
| pmisiak | TASK: [glance | Starting glance-api container] ******************************** | 14:49 |
| pmisiak | ok: [ctrl3] | 14:49 |
| pmisiak | ok: [ctrl1] | 14:49 |
| pmisiak | nothing changed | 14:49 |
| pmisiak | ok: [ctrl2] | 14:49 |
| vhosakot | pmisiak: looks like three nodes should get the new glance-api conatiner now.. can you check uptime of them ? | 14:50 |
| vhosakot | pmisiak: "docke ps -a | grep glance" | 14:50 |
| pmisiak | hour ago | 14:50 |
| vhosakot | pmisiak: not right then... | 14:50 |
| pmisiak | hmm can i force to use the newest image? | 14:51 |
| vhosakot | pmisiak: can you stop glance-api and remove it on each controller, and re-try | 14:51 |
| pmisiak | in some way? | 14:51 |
| *** sdake has quit IRC | 14:51 | |
| *** mgoddard has joined #openstack-kolla | 14:51 | |
| *** mgoddard_ has quit IRC | 14:51 | |
| *** banix has quit IRC | 14:52 | |
| pmisiak | the newest image is in registry | 14:52 |
| pmisiak | but not on the nodes | 14:52 |
| pmisiak | so maybe he don't know about the newest image | 14:52 |
| inc0 | pmisiak, tools/cleanup-images | 14:53 |
| inc0 | will do the job | 14:53 |
| vhosakot | pmisiak: deploy does not stop, remove... it does not touch an already running container... https://github.com/openstack/kolla/blob/master/ansible/roles/glance/tasks/deploy.yml#L19-L20 | 14:53 |
| inc0 | on every node | 14:53 |
| vhosakot | yep, cleanup on all nodes | 14:53 |
| *** Serlex has quit IRC | 14:54 | |
| vhosakot | pabelanger: is this review running dibs already ? https://review.openstack.org/#/c/303055/ I see bindep gate jobs | 14:57 |
| patchbot | vhosakot: patch 303055 - kolla - Remove /etc/kolla configuration folders | 14:57 |
| vhosakot | pmisiak: did you pass --registry registry_ip_address:registry_ip_port to kolla-build when building the new glance-api image ? | 14:58 |
| pmisiak | yes | 14:58 |
| vhosakot | pmisiak: cool.. so, cleanup and deploy should run the new container on all controller, yep | 14:59 |
| *** cbits has quit IRC | 15:03 | |
| vhosakot | pabelanger: can you ping you if we have questions about gate/infra ? :) | 15:04 |
| vhosakot | pabelanger: can we ping you if we have questions about gate/infra ? :) | 15:05 |
| *** subscope has joined #openstack-kolla | 15:05 | |
| *** SiRiuS_ has quit IRC | 15:08 | |
| pabelanger | vhosakot: yes, we added an experimental job yesterday to test kolla. Once we land https://review.openstack.org/#/c/301320/ we'll remove them and migrate check / gate jobs to it. | 15:08 |
| patchbot | pabelanger: patch 301320 - openstack-infra/project-config - Migrate kolla jobs to centos-7 dib | 15:08 |
| pabelanger | vhosakot: yes, feel free to ping me in #openstack-infra with problems | 15:08 |
| vhosakot | pabelanger: cool, thanks! | 15:08 |
| pbourke | what is dibs? | 15:09 |
| pabelanger | diskimage-builder | 15:11 |
| pbourke | oh cool | 15:13 |
| *** banix has joined #openstack-kolla | 15:13 | |
| *** ayoung has joined #openstack-kolla | 15:16 | |
| *** vhosakot has quit IRC | 15:16 | |
| *** vhosakot has joined #openstack-kolla | 15:17 | |
| *** vhosakot has quit IRC | 15:17 | |
| *** vhosakot has joined #openstack-kolla | 15:18 | |
| *** daneyon_ has joined #openstack-kolla | 15:20 | |
| *** daneyon_ has quit IRC | 15:24 | |
| AJaeger | pbourke: if you're interested in the big picture, read https://www.technovelty.org/openstack/image-building-in-openstack-ci.html | 15:25 |
| *** aginwala has joined #openstack-kolla | 15:25 | |
| AJaeger | pbourke: it's a lenghty read... | 15:25 |
| *** AJaeger has left #openstack-kolla | 15:28 | |
| *** mark-casey1 has joined #openstack-kolla | 15:28 | |
| *** aginwala has quit IRC | 15:29 | |
| *** aginwala has joined #openstack-kolla | 15:30 | |
| andrewvwebber | pmisiak: thank you, you made my day :) | 15:30 |
| pmisiak | :) | 15:31 |
| *** rmart04 has quit IRC | 15:34 | |
| mark-casey1 | hey anyone... I was out yesterday but was looking at eavesdrop re: Ubuntu multinode. I just finished a multinode Ubuntu deploy to metal. | 15:37 |
| mark-casey1 | I think I missed what the actual issue was but mine is up. I'm on Wily | 15:37 |
| *** jtriley_ has joined #openstack-kolla | 15:38 | |
| *** jtriley has quit IRC | 15:41 | |
| andrewvwebber | mark-casey1: awesome | 15:42 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add LVM2/iSCSI documentation https://review.openstack.org/300824 | 15:43 |
| andrewvwebber | i just completed a metal coreos all-in-one with a single veth nic - https://gist.github.com/andrewwebber/f788790772cef47ed5bff39aae2212cf | 15:43 |
| *** sdake has joined #openstack-kolla | 15:43 | |
| *** sdake_ has quit IRC | 15:44 | |
| *** jtriley_ has quit IRC | 15:44 | |
| mark-casey1 | andrewvwebber: https://github.com/ropsoft/kolla_from_vagrant | 15:44 |
| *** dwalsh has quit IRC | 15:45 | |
| mark-casey1 | I actually wish I was on coreos might like to work on getting that to multinode | 15:45 |
| andrewvwebber | my next task is multinode coreos on monday | 15:45 |
| andrewvwebber | i will keep you posted | 15:45 |
| mark-casey1 | any interest on having a kubernetes layer in there? not sure whose baby that one is... | 15:46 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 15:47 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 15:47 |
| andrewvwebber | my understanding is that the kolla stopped trying to deploy open stack on kubernetes. however the coreos team have achieved this - https://tectonic.com/blog/openstack-and-kubernetes-come-together.html | 15:48 |
| mark-casey1 | andrewvwebber: ^ | 15:48 |
| mark-casey1 | right. I think there was some talk of picking it back up | 15:48 |
| andrewvwebber | oh really?.. | 15:49 |
| rhallisey | andrewvwebber, https://review.openstack.org/#/c/255450/ | 15:49 |
| patchbot | rhallisey: patch 255450 - kolla - Spec: Deploy Kolla images using Kubernetes | 15:49 |
| andrewvwebber | that would be great | 15:49 |
| mark-casey1 | I'd help and use it | 15:50 |
| rhallisey | I don't think there's a session for it at summit, but if there's interest I'm sure we can find a place for it | 15:50 |
| *** jtriley has joined #openstack-kolla | 15:51 | |
| *** rhallisey has left #openstack-kolla | 15:52 | |
| *** rhallisey has joined #openstack-kolla | 15:53 | |
| andrewvwebber | i think for many there is a lack of guidance on openstack/kubernetes convergence options. Kubernetes on openstack, openstack on kubernetes or openstack running side by side with kubernetes. This would make a great session | 15:53 |
| *** aginwala has quit IRC | 15:53 | |
| *** aginwala has joined #openstack-kolla | 15:54 | |
| *** aginwala has quit IRC | 15:56 | |
| *** aginwala_ has joined #openstack-kolla | 15:56 | |
| * mark-casey1 goes to ask to go to Summit | 15:58 | |
| mark-casey1 | heh | 15:58 |
| *** SiRiuS_ has joined #openstack-kolla | 15:59 | |
| sdake | Jeffrey4l i dont think that compute bootstrap code is ever being executd | 15:59 |
| *** jtriley has quit IRC | 16:01 | |
| *** vhosakot has quit IRC | 16:02 | |
| *** Jeffrey4l has quit IRC | 16:04 | |
| *** andrewvwebber has quit IRC | 16:04 | |
| sdake | pbourke i specifically left ou t those repo fiels on purpose | 16:09 |
| sdake | pbourke it was complained about in a previous review | 16:09 |
| sdake | your also missing the gpg keys required | 16:11 |
| sdake | which will causetehbuild to fail | 16:11 |
| sdake | tihs is why i didn't use this method - because have to install 4 different files in thebase directory | 16:11 |
| *** unicell has quit IRC | 16:13 | |
| *** unicell has joined #openstack-kolla | 16:13 | |
| *** banix has quit IRC | 16:14 | |
| *** Jeffrey4l has joined #openstack-kolla | 16:14 | |
| *** Jeffrey4l has joined #openstack-kolla | 16:14 | |
| sdake | Jeffrey4l i think the compute bootstrap is not being executed at all | 16:14 |
| Jeffrey4l | sdake, no. | 16:14 |
| Jeffrey4l | I have got the root cause and fixed it. | 16:15 |
| sdake | which is what | 16:15 |
| Jeffrey4l | the root cause is: /var/lib/nova has wrong permission in the images. In centos it is 700, in ubuntu it is 755 | 16:15 |
| sdake | ye but bootstrap is supposed to rwrite that right? | 16:16 |
| sdake | and it isn't | 16:16 |
| Jeffrey4l | bootstrap just change the volume permission. not the folder permission in the image. | 16:17 |
| sdake | are they not one in the same | 16:17 |
| Jeffrey4l | no. | 16:17 |
| Jeffrey4l | a little hard to explain. let me write a long explain. wait for a minutes, please. | 16:17 |
| sdake | show me where in the code the bootstrap is triggered | 16:17 |
| *** unicell has quit IRC | 16:18 | |
| sdake | i can't find such a thing in ansible forcompute | 16:18 |
| Jeffrey4l | sdake, it is not the issue of bootstrap. bootstrap works as expect. | 16:18 |
| sdake | i dont think bootstrpa is working at all because its never being called | 16:18 |
| Jeffrey4l | it is the `start nova-libvirt image` cause the issue. | 16:18 |
| sdake | hence the question, where is it being called | 16:18 |
| *** shakamunyi has joined #openstack-kolla | 16:18 | |
| Jeffrey4l | no, it is called. | 16:18 |
| * Jeffrey4l is seaching | 16:18 | |
| Jeffrey4l | sdake, https://github.com/openstack/kolla/blob/master/ansible/roles/nova/tasks/bootstrap_service.yml#L31 | 16:19 |
| *** aginwala_ has quit IRC | 16:19 | |
| Jeffrey4l | here | 16:19 |
| sdake | ok thanks | 16:19 |
| *** aginwala has joined #openstack-kolla | 16:19 | |
| sdake | so why even need the bootstrap code at all if the permissions are inherited in the volume mouting? | 16:20 |
| sbezverk | inc0 I tried to deploy liberty using mitaka and I see some issues with swift.. | 16:20 |
| inc0 | sbezverk, note them in patch plz | 16:20 |
| Jeffrey4l | sdake, wait for a minute. writing a long explanation. bootstrap is needed, for sure. | 16:21 |
| *** shakamunyi has quit IRC | 16:23 | |
| sdake | inc0 lets get jeffrey's patches merged please | 16:23 |
| sdake | so we have linuxbridge fixed and i can get on with qemu fixing | 16:23 |
| *** aginwala has quit IRC | 16:24 | |
| inc0 | sdake, ok, but rabbitmq is broken:( | 16:24 |
| sdake | https://review.openstack.org/#/c/303509 | 16:25 |
| sdake | rabbitmq works for me | 16:25 |
| sdake | in what way is it broken | 16:25 |
| sdake | rhallisey need revies on above and its children | 16:26 |
| sdake | and parents | 16:26 |
| Jeffrey4l | sdake, check this http://paste.openstack.org/show/493507/ | 16:26 |
| Jeffrey4l | sdake, btw, i have split my patch into two. one for qemu permission fix, another for neutron-linuxbridge-agent fix. | 16:27 |
| sdake | Jeffrey4l that doesn't answer why we still need bootstrap | 16:27 |
| Jeffrey4l | sdake, we need create instances folder at least. | 16:28 |
| sdake | this should inherit its arent direcotry perissions | 16:29 |
| sdake | ok lets sort out the bootstrap thing as a separate patch | 16:29 |
| sdake | can you file a bug for that please | 16:29 |
| Jeffrey4l | sdake, seems you are right at this point. | 16:30 |
| sdake | Jeffrey4l i dont know - it needs teting | 16:31 |
| sdake | but its not a priority | 16:31 |
| Jeffrey4l | sdake, I can make some test for this. And try to see whether it is really needed. | 16:31 |
| Jeffrey4l | yea | 16:31 |
| sdake | in eed my box now atm ;) | 16:31 |
| *** SiRiuS_ has quit IRC | 16:31 | |
| *** aginwala has joined #openstack-kolla | 16:31 | |
| Jeffrey4l | sdake, I can test in my laptop. Just need more time. :D | 16:31 |
| Jeffrey4l | no worry. | 16:32 |
| sdake | ok sounds good | 16:32 |
| sdake | do sa a separate bug/patch | 16:32 |
| Jeffrey4l | sdake, btw, the rabbitmq is broken for sure. I saw it crashed many times in the gate. ( But I can not reproduce that) i do not know why. | 16:33 |
| Jeffrey4l | sure | 16:33 |
| sdake | i see it crashing in gate too | 16:34 |
| sdake | possibly my regex patch triggers itt o crash | 16:35 |
| *** subscope has quit IRC | 16:35 | |
| Jeffrey4l | fyi, here is the related bug https://bugs.launchpad.net/ubuntu/+source/erlang/+bug/1374109 | 16:35 |
| openstack | Launchpad bug 1374109 in erlang (Ubuntu) "epmd does not support binding to an IPv4 address anymore" [High,Confirmed] - Assigned to Dariusz Gadomski (dgadomski) | 16:35 |
| Jeffrey4l | sorry | 16:35 |
| Jeffrey4l | here is the right link https://bugs.launchpad.net/kolla/+bug/1562701 | 16:35 |
| openstack | Launchpad bug 1562701 in kolla mitaka "rabbitmq crashed on centos binary deploy" [Critical,Confirmed] | 16:35 |
| *** andrewvwebber has joined #openstack-kolla | 16:36 | |
| *** salv-orlando has quit IRC | 16:38 | |
| *** mark-casey has joined #openstack-kolla | 16:41 | |
| *** Marga_ has joined #openstack-kolla | 16:43 | |
| *** mark-casey1 has quit IRC | 16:43 | |
| *** mark-casey1 has joined #openstack-kolla | 16:44 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 16:45 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 16:45 |
| *** SiRiuS_ has joined #openstack-kolla | 16:45 | |
| pabelanger | sdake: do you mind helping land https://review.openstack.org/#/c/303414/ ? This is needed for the migration to centos-7 dib | 16:45 |
| patchbot | pabelanger: patch 303414 - kolla - Make sure /etc/hostname is valid for SELinux | 16:45 |
| *** rmart04 has joined #openstack-kolla | 16:47 | |
| *** mark-casey has quit IRC | 16:47 | |
| *** rmart04 has quit IRC | 16:48 | |
| sdake | pabelanger kolla does not yet work with selinux | 16:48 |
| sdake | pabelanger did the gate images change to enable it by default? | 16:48 |
| *** vhosakot has joined #openstack-kolla | 16:50 | |
| pabelanger | well, SELinux is enabled for devstack-centos7, I confirmed that | 16:51 |
| sdake | pabelanger interesting | 16:51 |
| pabelanger | so, 2 ways to fix it for centos-7. 1) sudo setenforce 0 in your gate script (which was -1 already) or 2) the above patch | 16:51 |
| sdake | pabelanger that patch looks fine but is failing the ubuntu deploy gate and needs a bug id | 16:51 |
| pabelanger | we are actually working on a fix in -infra too for this, but will take some time to get a proper patch | 16:51 |
| sbezverk | inc0 instance also fails to launch http://paste.openstack.org/show/493513/ | 16:52 |
| pabelanger | sdake: job failures are git failures, so not relation. Don't have a bug for this | 16:52 |
| inc0 | sbezverk, I think this is due to rabbitmq being broken | 16:52 |
| inc0 | you're running multinode? | 16:53 |
| pabelanger | ongoing effort from openstack-infra | 16:53 |
| inc0 | and do you have "-" in your hostnames?;) | 16:53 |
| sdake | pabelanger you need to file a bug so Ican backport the change | 16:53 |
| sdake | file and link and i'll handle thte rest | 16:53 |
| pabelanger | sdake: see: http://specs.openstack.org/openstack-infra/infra-specs/specs/dib-nodepool.html | 16:54 |
| sdake | the fact that the git fails is irrelevant, gate should be green ;) | 16:54 |
| sbezverk | inc0 of course | 16:54 |
| sbezverk | the time for one word hostname is gone ;-) | 16:54 |
| sdake | pabelanger would you prefer i file a bug for you | 16:54 |
| sdake | inc0 ther eis a bug and fix for - in hostname | 16:55 |
| sdake | its in the 2.0.0 tracker | 16:55 |
| inc0 | sdake, one that was backported? | 16:56 |
| *** stvnoyes has left #openstack-kolla | 16:56 | |
| sdake | inc0 its not merged | 16:56 |
| sdake | sam didn't like it for whatever reason | 16:56 |
| sdake | he said it broke ubuntu | 16:56 |
| sdake | but others reported it worked fine on ubuntu | 16:56 |
| inc0 | I just did it in ubuntu | 16:56 |
| inc0 | it's broken without it | 16:56 |
| sdake | with the patch? | 16:56 |
| sdake | yes we know its broken withou tthe patch | 16:57 |
| sdake | the question is - is the patch correct | 16:57 |
| pabelanger | sdake: https://storyboard.openstack.org/#!/story/2000544 | 16:57 |
| inc0 | https://bugs.launchpad.net/kolla/+bug/1540234 where is patch? | 16:57 |
| openstack | Launchpad bug 1540234 in kolla mitaka "rabbitmq cluster does not work with dashes in the hostname" [Critical,Confirmed] | 16:57 |
| sdake | pabelanger will merge patch - need a bug id | 16:57 |
| sbezverk | sdake can you please give a link I want to try to apply this patch on my cenots | 16:57 |
| sdake | inc0 hiding out in the queue somewhere | 16:57 |
| pabelanger | sdake: 2000544 | 16:57 |
| sdake | launchpad sink ftw | 16:57 |
| pabelanger | we don't use launchpad for openstack-infa | 16:58 |
| pabelanger | just storyboard | 16:58 |
| sdake | ok well kolal needs bugs to track backports | 16:58 |
| sdake | as I sai dwoud lyou prefer i filea bug for you? | 16:58 |
| sdake | bug doesn't have to be detailed | 16:59 |
| sdake | it can be simple as "/etc/hosts has wrong credentials for selinux" | 16:59 |
| pabelanger | sure, if you need a bug for your workflow that's fine with me. I have no issue with your co-authoring the patch | 16:59 |
| sdake | pabelanger i dontneed to coauthor the patch | 16:59 |
| sdake | just need a bug id to make the backporting process not a nightmare ;) | 17:00 |
| *** aginwala has quit IRC | 17:00 | |
| sdake | ok i'll file bug moment | 17:00 |
| *** aginwala has joined #openstack-kolla | 17:00 | |
| *** aginwala has quit IRC | 17:01 | |
| rhallisey | inc0, https://review.openstack.org/#/c/275111/ | 17:01 |
| patchbot | rhallisey: patch 275111 - kolla - fix miss rabbitmq config when hostname include spe... (MERGED) | 17:01 |
| sdake | pabelanger https://bugs.launchpad.net/kolla/+bug/1568047 | 17:02 |
| openstack | Launchpad bug 1568047 in kolla mitaka "/etc/hostname has wrong selinux context" [Low,Confirmed] | 17:02 |
| *** aginwala has joined #openstack-kolla | 17:02 | |
| sdake | what is yur launchpad id please pabelanger | 17:02 |
| pabelanger | pabelanger | 17:02 |
| inc0 | rhallisey, yeah I know it got reverted | 17:03 |
| inc0 | we need to re-revert it | 17:03 |
| pabelanger | sdake: do you mind also adding a bug to add CONTRIBUTING.rst to your project? That will help new contributors better understand your workflow. All these steps are a surprise to me | 17:03 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Make sure /etc/hostname is valid for SELinux https://review.openstack.org/303414 | 17:04 |
| openstackgerrit | Merged openstack/kolla: Add default flavors during init-runonce https://review.openstack.org/302534 | 17:04 |
| openstackgerrit | Paul Belanger proposed openstack/kolla: Make sure /etc/hostname is valid for SELinux https://review.openstack.org/303414 | 17:05 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Make sure /etc/hostname is valid for SELinux https://review.openstack.org/303414 | 17:05 |
| sdake | pabelanger that patch will merge naturally through our review processes | 17:06 |
| sdake | enjoy | 17:06 |
| inc0 | sdake, I think this patchset is abandoned | 17:06 |
| inc0 | let me revive it | 17:06 |
| inc0 | and I'm going to merge it unless someone else tell me it's broken in ubuntu | 17:07 |
| inc0 | I can't get clusterer to work without it | 17:07 |
| sdake | inc0 did you test iton ubuntu multinode? | 17:07 |
| inc0 | sdake, yes | 17:07 |
| sdake | ok well that is good enough for me | 17:07 |
| *** andrewvwebber has quit IRC | 17:07 | |
| pabelanger | sdake: okay, we'd like to land it today so we can finialize our migrate from devstack-centos7 | 17:07 |
| inc0 | spent good portion of yesterday trying to fix rabbit, this is only one that works | 17:08 |
| *** vhosakot has quit IRC | 17:09 | |
| *** jtriley has joined #openstack-kolla | 17:11 | |
| sdake | pabelanger fwiw we get mixed reports about selinux working or not working wrt kolla | 17:11 |
| sdake | its interesting its enabled in the gate by default | 17:11 |
| sdake | i didn't know that | 17:11 |
| *** mdnadeem_ has joined #openstack-kolla | 17:12 | |
| sdake | Jeffrey4l re your patch set | 17:13 |
| sdake | whats the word on the bootstrap | 17:13 |
| sdake | i am totally blocked on your work | 17:13 |
| Jeffrey4l | sdake, are you meaning my current PatchSet? Or my test for bootstrap? | 17:14 |
| pabelanger | sdake: Right, I remember you saying it was disabled before. But somebody -1'd my patch to setenforce Permissive | 17:14 |
| *** wmiller has quit IRC | 17:15 | |
| *** mdnadeem has quit IRC | 17:15 | |
| sdake | pabelanger which patch was that | 17:15 |
| *** wmiller has joined #openstack-kolla | 17:16 | |
| pabelanger | sdake: another option would be do call fixfiles relabel before you start your gate jobs too, resetting SELinux to defaults: https://www.centos.org/docs/5/html/5.2/Deployment_Guide/sec-sel-fsrelabel.html | 17:16 |
| sdake | yes i am familar with relabel | 17:16 |
| Jeffrey4l | sdake, current PatchSet should be OK. and my bootstrap test tell me that: no need to call `chown nova:nova /var/lib/nova` but need `mkdir -p /var/lib/nova/instances` | 17:16 |
| sdake | takes too long imo | 17:16 |
| *** aginwala has quit IRC | 17:16 | |
| pabelanger | sdake: I abandoned it 303381 | 17:17 |
| sdake | pabelanger can you give me a link plz | 17:17 |
| * sdake in enumberoverloader | 17:17 | |
| pabelanger | http://review.openstack.org/303381 | 17:18 |
| *** shakamunyi has joined #openstack-kolla | 17:18 | |
| *** shardy has quit IRC | 17:20 | |
| *** dwalsh has joined #openstack-kolla | 17:20 | |
| sdake | Jeffrey4l can you fix your patchset to remove the cruft then | 17:20 |
| *** aginwala has joined #openstack-kolla | 17:20 | |
| Jeffrey4l | sdake, np | 17:21 |
| sbezverk | inc0 please let me know when hostname patch is merged I will re-try liberty then.. | 17:22 |
| sdake | where is this mythical hostname patch | 17:22 |
| sdake | lets get a link in the bug to the patch itself ;) | 17:22 |
| *** mark-casey has joined #openstack-kolla | 17:22 | |
| inc0 | sbezverk, https://review.openstack.org/#/c/288415/ | 17:22 |
| patchbot | inc0: patch 288415 - kolla - Add quotes for RBMQ nodes definition | 17:22 |
| sdake | since the launchpad syncing with gerrit infra sometimes times out | 17:23 |
| *** shakamunyi has quit IRC | 17:23 | |
| sdake | inc0 can you fix tht patch with the bug id | 17:23 |
| sdake | or shall I | 17:23 |
| inc0 | well it has -W | 17:23 |
| sdake | we can override that | 17:24 |
| sdake | but there was actually soemone else who fixed this problem | 17:24 |
| *** gfidente has quit IRC | 17:24 | |
| sdake | in the same way | 17:24 |
| inc0 | weiyu | 17:24 |
| inc0 | yes | 17:24 |
| inc0 | but then Sam overriden it because ubuntu was broken | 17:24 |
| *** mark-casey1 has quit IRC | 17:25 | |
| inc0 | and now it doesn't look like it | 17:25 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1540234 | 17:25 |
| openstack | Launchpad bug 1540234 in kolla mitaka "rabbitmq cluster does not work with dashes in the hostname" [Critical,Confirmed] | 17:25 |
| sdake | sam removed all his votes on everything in kolla | 17:25 |
| sdake | the workflow is settable by anyone | 17:25 |
| sdake | but the patch needs a bug id instead of trivial fix | 17:25 |
| sdake | inc0 this patch already merged: | 17:27 |
| sdake | https://review.openstack.org/#/c/275111/ | 17:27 |
| patchbot | sdake: patch 275111 - kolla - fix miss rabbitmq config when hostname include spe... (MERGED) | 17:27 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Add quotes for RBMQ nodes definition https://review.openstack.org/288415 | 17:28 |
| sdake | someone rverted weiyu's patch | 17:28 |
| inc0 | Sam | 17:28 |
| sdake | add him as a coauthor since he acctually fixed the damn thing | 17:28 |
| *** jasonsb has quit IRC | 17:29 | |
| *** mark-casey1 has joined #openstack-kolla | 17:29 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Add quotes for RBMQ nodes definition https://review.openstack.org/288415 | 17:29 |
| *** athomas has quit IRC | 17:29 | |
| inc0 | I hope I remembered tags correctly | 17:29 |
| *** unicell has joined #openstack-kolla | 17:30 | |
| *** mark-casey has quit IRC | 17:31 | |
| *** rajathagasthya has joined #openstack-kolla | 17:33 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 17:35 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 17:35 |
| sbezverk | Jeffrey4l why you removed sudoers from nova base? | 17:38 |
| sbezverk | nova uses rootwrap extensively | 17:39 |
| *** cbits has joined #openstack-kolla | 17:43 | |
| *** sdake_ has joined #openstack-kolla | 17:44 | |
| *** sdake has quit IRC | 17:45 | |
| sbezverk | inc0 I cherrypicked hostname patch and tried deploying again, see the same Unexpected API Error when launcing an instance.. | 17:46 |
| inc0 | hmm | 17:47 |
| inc0 | centos? | 17:47 |
| sbezverk | inc0 yep, source | 17:49 |
| inc0 | I guess I need to install it at some point;) | 17:49 |
| sbezverk | inc0 I can go through api longs to narrow down what is broken | 17:50 |
| inc0 | would be helpful, thanks | 17:50 |
| sbezverk | inc0 oh well, I had to install ubuntu, now it is pay back ;-) you have to install centos | 17:50 |
| inc0 | sbezverk, one thing tho | 17:50 |
| inc0 | did you ntp your machines | 17:51 |
| inc0 | ? | 17:51 |
| sbezverk | inc0 it is AIO, I have only 1 server for this exercise | 17:52 |
| sdake_ | Jeffrey4l did you intend to remove nova sudoers file? | 17:53 |
| *** Marga_ has quit IRC | 17:57 | |
| *** achanda has joined #openstack-kolla | 17:57 | |
| *** sdake has joined #openstack-kolla | 17:59 | |
| sbezverk | inc0 I see tons of error in rabbitmq container logs. who is local rabbitmq expert? | 17:59 |
| *** cbits has quit IRC | 18:00 | |
| *** sdake_ has quit IRC | 18:01 | |
| *** daneyon_ has joined #openstack-kolla | 18:02 | |
| inc0 | sbezverk, we need to develop one | 18:02 |
| inc0 | I experienced same rabbimq behavtiour yesterday | 18:03 |
| inc0 | something is broken | 18:03 |
| inc0 | also ss -l | grep 5672 | 18:03 |
| sdake | sbezverk fixing Jeffrey4l's patch | 18:04 |
| sbezverk | sdake good idea | 18:05 |
| sdake | i thnk he passed outat his keyboard ;) | 18:05 |
| *** daneyon_ has quit IRC | 18:07 | |
| * sdake wtb vacation | 18:08 | |
| sbezverk | sdake nooo, he has done it on puprose to check if we can spot it ;-) | 18:09 |
| sdake | Jeffrey4l ' ability to test is impaired - his cluster i sunderuse atm | 18:10 |
| *** banix has joined #openstack-kolla | 18:11 | |
| *** achanda has quit IRC | 18:11 | |
| *** ayoung has quit IRC | 18:14 | |
| sbezverk | inc0 http://paste.openstack.org/show/493525/ | 18:15 |
| sbezverk | inc0 rabbitmq errors http://paste.openstack.org/show/493522/ | 18:16 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 18:20 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 18:20 |
| *** aginwala has quit IRC | 18:24 | |
| *** Marga_ has joined #openstack-kolla | 18:26 | |
| mark-casey1 | so this is a more generic os question... but does anyone know what happened to being able to use '-1' as the protocol when adding a security group rule in Horizon? The tooltip still says -1 means wildcard but I'm getting "Not a valid IP protocol number" | 18:27 |
| *** aginwala has joined #openstack-kolla | 18:28 | |
| sbezverk | mark-casey1 I noticed that too, very annoying, I guess we should file a bug in Horizon to address it.. | 18:39 |
| *** rajathagasthya has quit IRC | 18:41 | |
| mark-casey1 | sbezverk: I think they may have also changed the default rules to allow ingress. Unless that is a Kolla thing. so that helps a little. I'm adding networks ATM but I'll ping you later with whether or not I can get to the instances w/out changing defaults | 18:41 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add a role to cleanup containers after deployment https://review.openstack.org/303575 | 18:41 |
| *** sdake has quit IRC | 18:42 | |
| inc0 | sooo good news | 18:42 |
| inc0 | with rabbitmq patchset stable mitaka on 10 nodes and ubuntu and ceph deployed and vm is active | 18:43 |
| *** banix has quit IRC | 18:49 | |
| *** aginwala has quit IRC | 18:49 | |
| *** mark-casey has joined #openstack-kolla | 18:50 | |
| *** aginwala has joined #openstack-kolla | 18:50 | |
| *** sdake has joined #openstack-kolla | 18:52 | |
| *** mark-casey1 has quit IRC | 18:52 | |
| *** d_code has joined #openstack-kolla | 18:57 | |
| *** aginwala has quit IRC | 18:57 | |
| *** salv-orlando has joined #openstack-kolla | 19:01 | |
| *** dwalsh has quit IRC | 19:04 | |
| *** mark-casey1 has joined #openstack-kolla | 19:05 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add etcd Docker container https://review.openstack.org/298451 | 19:05 |
| *** mark-casey has quit IRC | 19:06 | |
| *** ayoung has joined #openstack-kolla | 19:09 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 19:10 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 19:10 |
| *** salv-orlando has quit IRC | 19:12 | |
| *** dave-mccowan has quit IRC | 19:15 | |
| sbezverk | inc0 I had a chat with rabbitmq folks and my specific issue is related to this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1324922 | 19:18 |
| openstack | bugzilla.redhat.com bug 1324922 in erlang "Log handler repeatedly crashes" [Urgent,Modified] - Assigned to jeckersb | 19:18 |
| sbezverk | inc0 how it is fixed in this version of erlang-R16B-03.17.el7 | 19:18 |
| sbezverk | sdake anyway to get this version of erlang used for liberty? | 19:20 |
| sdake | make my life harder | 19:29 |
| sbezverk | sdake here is the link where we can get updated source rpm | 19:31 |
| sbezverk | http://koji.fedoraproject.org/koji/buildinfo?buildID=752065 | 19:31 |
| *** Jeffrey4l_ has joined #openstack-kolla | 19:31 | |
| sbezverk | bb in 2 hours | 19:32 |
| inc0 | sbezverk, liberty rabbit == mitaka | 19:33 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 19:34 |
| *** Jeffrey4l has quit IRC | 19:34 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add a role to cleanup containers after deployment https://review.openstack.org/303575 | 19:35 |
| *** rajathagasthya has joined #openstack-kolla | 19:40 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 19:42 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 19:42 |
| *** mark-casey has joined #openstack-kolla | 19:44 | |
| *** mark-casey1 has quit IRC | 19:46 | |
| sdake | manjeets ping | 19:47 |
| *** daneyon_ has joined #openstack-kolla | 19:50 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix the qemu has nova permission in nova instances folder https://review.openstack.org/303509 | 19:53 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix launch vm failed in neutron-linuxbridge-agent env https://review.openstack.org/302721 | 19:53 |
| *** daneyon_ has quit IRC | 19:55 | |
| *** vhosakot has joined #openstack-kolla | 20:01 | |
| *** zeus has joined #openstack-kolla | 20:03 | |
| *** zeus is now known as Guest80019 | 20:03 | |
| *** rajathagasthya has quit IRC | 20:03 | |
| *** salv-orlando has joined #openstack-kolla | 20:03 | |
| *** salv-orlando has joined #openstack-kolla | 20:04 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 20:06 |
| *** dave-mccowan has joined #openstack-kolla | 20:10 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add a role to cleanup containers after deployment https://review.openstack.org/303575 | 20:16 |
| *** rajathagasthya has joined #openstack-kolla | 20:20 | |
| *** d_code has quit IRC | 20:22 | |
| mlima | good bye guys | 20:25 |
| *** mlima has quit IRC | 20:25 | |
| *** sdake_ has joined #openstack-kolla | 20:26 | |
| *** sdake has quit IRC | 20:27 | |
| *** mark-casey1 has joined #openstack-kolla | 20:31 | |
| manjeets | sdake pong | 20:31 |
| *** jtriley has quit IRC | 20:32 | |
| *** mark-casey has quit IRC | 20:33 | |
| *** cbits has joined #openstack-kolla | 20:36 | |
| *** rajathagasthya has quit IRC | 20:41 | |
| *** daneyon_ has joined #openstack-kolla | 20:42 | |
| *** sdake_ has quit IRC | 20:42 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Fix missing libvirt python module in ceilometer-compute https://review.openstack.org/300650 | 20:43 |
| *** mark-casey1 has quit IRC | 20:43 | |
| *** rajathagasthya has joined #openstack-kolla | 20:46 | |
| *** daneyon_ has quit IRC | 20:58 | |
| *** vhosakot has quit IRC | 21:05 | |
| *** rajathagasthya has quit IRC | 21:06 | |
| sbezverk | inc0 by some reason I do not see this issue in stable mitaka.. | 21:08 |
| sbezverk | is there any way to use one source rpm? just as a PoC | 21:09 |
| *** inc0 has quit IRC | 21:10 | |
| sbezverk | anybody could point me to the place where I could nail a specific version of RPM? | 21:22 |
| *** haplo37 has quit IRC | 21:24 | |
| *** rhallisey has quit IRC | 21:24 | |
| *** cfarquhar_ has quit IRC | 21:33 | |
| *** cfarquhar has joined #openstack-kolla | 21:35 | |
| *** cfarquhar has joined #openstack-kolla | 21:35 | |
| *** daneyon_ has joined #openstack-kolla | 21:37 | |
| *** mwheckmann has quit IRC | 21:40 | |
| *** daneyon_ has quit IRC | 21:41 | |
| *** ccesario_ has joined #openstack-kolla | 21:53 | |
| *** ccesario_ has quit IRC | 22:01 | |
| *** vhosakot has joined #openstack-kolla | 22:01 | |
| dave-mccowan | sbezverk i'm seeing the same API error booting an image. works fine with centos-source, but fails with centos-binary. do we know what version centos-binary is? i wondering if it is still liberty. i am also seeing lots of error messages from rabbit. | 22:01 |
| *** ccesario_ has joined #openstack-kolla | 22:01 | |
| vhosakot | sbezverk: ping | 22:04 |
| vhosakot | sbezverk: which rpm are you trying to pin ? erlang / rabbitmq ? | 22:04 |
| *** ccesario_ has quit IRC | 22:05 | |
| *** ccesario___ has joined #openstack-kolla | 22:05 | |
| *** wuhg has joined #openstack-kolla | 22:07 | |
| *** kragniz has joined #openstack-kolla | 22:10 | |
| sbezverk | vhosakot pong | 22:11 |
| vhosakot | sbezverk: which rpm are you trying to pin ? erlang / rabbitmq ? | 22:11 |
| sbezverk | vhosakot I have a specific version of erlang which is fixing | 22:11 |
| sbezverk | the problem I saw with rabbitmq | 22:12 |
| vhosakot | sbezverk: nice, the rabbitmq crash ? | 22:12 |
| sbezverk | but I have hard time getting it into the image | 22:12 |
| sbezverk | vhosakot not crash, but rabbitmq is unable to write logs as a result requests from openstack components are timing out.. | 22:13 |
| sbezverk | and rabbitmq container generates nasty messages in the log.. | 22:13 |
| vhosakot | sbezverk: are you talking about https://bugs.launchpad.net/kolla/+bug/1562701 ? | 22:13 |
| openstack | Launchpad bug 1562701 in kolla mitaka "rabbitmq crashed on centos binary deploy" [Critical,Confirmed] | 22:13 |
| sbezverk | vhosakot have you ever done adding custom repo? | 22:14 |
| *** cbits has quit IRC | 22:14 | |
| vhosakot | sbezverk: no | 22:14 |
| vhosakot | sbezverk: are you talking about https://bugs.launchpad.net/kolla/+bug/1562701 ? | 22:14 |
| sbezverk | so it would be searchable by yum? | 22:14 |
| vhosakot | sbezverk: I think you'd need to point to your custom .repo file | 22:15 |
| sbezverk | vhosakot do not think so, loggs look different and I use centos source | 22:15 |
| sbezverk | I have repo file | 22:15 |
| vhosakot | sbezverk: ah ok.. | 22:15 |
| sbezverk | I added to config.py | 22:15 |
| sbezverk | where it is supposed to go | 22:16 |
| vhosakot | sbezverk: so, you want the rabbit image to use that rpm when it builds ? | 22:16 |
| sbezverk | but when I install rpm it does not resolve dependecy | 22:16 |
| *** stvnoyes has joined #openstack-kolla | 22:16 | |
| sbezverk | vhosakot that would work too, I was thinking to add it to the base | 22:17 |
| sbezverk | but this is just Proof of Concept | 22:17 |
| vhosakot | sbezverk: your custom rabbit rpm from your custom repo in base, and kolla's rabbit rpm in rabbitmq's Docekrfile.... wouldn't this conflict ? | 22:17 |
| vhosakot | sbezverk: can't you point this to your custom / local path ? https://github.com/openstack/kolla/blob/master/docker/rabbitmq/Dockerfile.j2#L8 | 22:18 |
| *** shakamunyi has joined #openstack-kolla | 22:18 | |
| *** shakamunyi has quit IRC | 22:22 | |
| *** Marga_ has quit IRC | 22:23 | |
| sbezverk | vhosakot the problem it will work for single rpm but if there are dependecies it will not search | 22:24 |
| sbezverk | and resolve | 22:24 |
| *** d_code has joined #openstack-kolla | 22:24 | |
| vhosakot | sbezverk: yep, customized build will need to take care of all the deps... can't you install all the deps as well from custom path | 22:24 |
| sbezverk | man it is about 30-40 rpms | 22:25 |
| vhosakot | wow... | 22:25 |
| *** ccesario___ has quit IRC | 22:25 | |
| vhosakot | sbezverk: yum deplist shows the dep... | 22:25 |
| sbezverk | check this out http://paste.openstack.org/show/493550/ | 22:26 |
| vhosakot | sbezverk: if so many RPMs, yes, a good idea is to create a custom.fix.repo, add it in base, and comment out the broken rabbit RPM in master's Dockerfile... let me see if I find the place where we can replace deps | 22:27 |
| *** kragniz has quit IRC | 22:27 | |
| sbezverk | I have done exactly that | 22:27 |
| sbezverk | in the base I added new repo called erlang | 22:27 |
| sbezverk | it contains info I got for that repo | 22:27 |
| sbezverk | I see that it gets added to the base | 22:28 |
| sbezverk | but yum does not use it for resolving dependecies | 22:28 |
| sbezverk | that is what I cannot undersatnd why | 22:28 |
| vhosakot | sbezverk: I think you need to run createrepo command or soemthing like that to make yum know that it needs to pick a custom rpm... let me find something.. 1 sec | 22:28 |
| sbezverk | ok | 22:28 |
| *** Marga__ has joined #openstack-kolla | 22:28 | |
| *** kragniz has joined #openstack-kolla | 22:29 | |
| vhosakot | sbezverk: did you add a COPY statement in the base Dockerfile to copy it into /etc/yum.repos.d/ inside the container ? | 22:30 |
| vhosakot | sbezverk: https://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L22-L30 | 22:30 |
| sbezverk | yes | 22:31 |
| vhosakot | ok | 22:31 |
| sbezverk | I used different method | 22:32 |
| sbezverk | sorry there is a place in config.py where custom repo can be specified | 22:32 |
| sbezverk | It is just I tried both approaches neither worked | 22:32 |
| *** Marga__ has quit IRC | 22:33 | |
| vhosakot | sbezverk: ok | 22:34 |
| *** Guest80019 is now known as zeus | 22:34 | |
| *** zeus has joined #openstack-kolla | 22:34 | |
| vhosakot | sbezverk: in config.py, I think we rdo trunk, right, let me check | 22:35 |
| sbezverk | yes and I used exactly the same approach | 22:36 |
| sbezverk | let me show | 22:36 |
| *** kragniz has left #openstack-kolla | 22:38 | |
| sbezverk | http://paste.openstack.org/show/493551/ | 22:38 |
| vhosakot | sbezverk: it should be sent to base Dockerfile thru build.py | 22:44 |
| *** ccesario___ has joined #openstack-kolla | 22:44 | |
| vhosakot | sbezverk: I see you added ERLANG here https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L142 | 22:45 |
| vhosakot | sbezverk: build.py uses rpm_setup_config like this --> https://github.com/openstack/kolla/blob/master/kolla/cmd/build.py#L349-L350 | 22:46 |
| vhosakot | sbezverk: and, the base Dockerfile uses rpm_setup as {{cmd}} like this --> https://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L41-L42 | 22:47 |
| sbezverk | if you check what I pasted I added to rpm_setup | 22:48 |
| sbezverk | ERLANG | 22:48 |
| vhosakot | sbezverk: so, you could do echo {{ cmd }} after https://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L42 and see if your custom commands are printed in base DOckerfile | 22:48 |
| vhosakot | sbezverk: you added ERLANG to rpm_setup | 22:48 |
| vhosakot | yep | 22:48 |
| *** kragniz has joined #openstack-kolla | 22:49 | |
| vhosakot | sbezverk: i think the custom repo code hits these three lines in base...https://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L41-L43 | 22:50 |
| vhosakot | sbezverk: or atleast, sholud hit :) | 22:50 |
| sbezverk | will be back in 10 minutes | 22:51 |
| vhosakot | sbezverk: cool | 22:51 |
| sbezverk | vhosakot self.rpm_setup = self.build_rpm_setup(rpm_setup_config) | 22:55 |
| sbezverk | it seems it takes rpm_setup_config I set and then convert it into rpm_setup | 22:56 |
| *** Marga_ has joined #openstack-kolla | 22:56 | |
| sbezverk | vhosakot It does hit these lines I can see it when base gets built | 22:57 |
| *** vhosakot has quit IRC | 22:58 | |
| *** shakamunyi has joined #openstack-kolla | 23:18 | |
| *** ccesario___ has quit IRC | 23:18 | |
| *** ccesario___ has joined #openstack-kolla | 23:21 | |
| *** shakamunyi has quit IRC | 23:23 | |
| *** Marga_ has quit IRC | 23:25 | |
| *** Marga_ has joined #openstack-kolla | 23:26 | |
| *** jtriley has joined #openstack-kolla | 23:29 | |
| *** salv-orl_ has joined #openstack-kolla | 23:40 | |
| *** salv-orlando has quit IRC | 23:42 | |
| *** vhosakot has joined #openstack-kolla | 23:42 | |
| vhosakot | sbezverk: ping | 23:43 |
| *** SiRiuS_ has quit IRC | 23:47 | |
| sbezverk | vhosakot pong | 23:52 |
| *** Jeffrey4l_ has quit IRC | 23:53 | |
| vhosakot | sbezverk: wanted to check if the code is hit... I checked IRC logs and saw your message that the code is hit and seen in the output of kolla-build..... | 23:54 |
| sbezverk | that is correct, maybe it is not picking it up because it does not have the security key?!? | 23:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!