Tuesday, 2017-02-28

*** alanmeadows has quit IRC00:00
*** SaMnCo has quit IRC00:00
jemcevoywhich is from /etc/kolla/config/neutron00:00
*** alanmeadows has joined #openstack-kolla00:00
*** qwang has quit IRC00:00
*** sindhu has quit IRC00:00
jemcevoydid you want it from one of the containers?00:00
*** peterjenkins__ has quit IRC00:01
*** kklimonda has quit IRC00:01
*** NachoDuck has quit IRC00:02
*** berendt has quit IRC00:02
*** akwasnie has quit IRC00:02
*** zeus has quit IRC00:02
*** SaMnCo has joined #openstack-kolla00:02
*** classicsnail has quit IRC00:02
SamYaplejemcevoy: try setting '[ml2]path_mtu=9000' as well00:03
*** qwang has joined #openstack-kolla00:03
SamYaplebut i think this is because of your br-ex patch thing00:03
jascott1sbezverk: im back now00:03
SamYaplejemcevoy: i dont think youll be able to do that00:03
*** sindhu has joined #openstack-kolla00:04
*** peterjenkins__ has joined #openstack-kolla00:04
*** akwasnie has joined #openstack-kolla00:04
sp_SamYaple: jascott1  hi..... can i get the kolla logo link used for Atlanta ?00:06
sdakesp_ indeed its on the mailing list00:06
*** yingjun has joined #openstack-kolla00:06
*** NachoDuck has joined #openstack-kolla00:06
*** berendt has joined #openstack-kolla00:06
sdakesp_ look for [kolla] and search for mascot00:06
jascott1sp_ https://www.dropbox.com/sh/94pukquiw425ji2/AAAl2wVmm72KHPLCAzR6Uboha?dl=000:06
sp_sdake: thanks00:06
sp_jascott1: thanx00:07
*** eaguilar has joined #openstack-kolla00:07
*** zeus has joined #openstack-kolla00:07
financeMannwhere are the logs, var/log/kolla?00:07
*** zeus is now known as Guest6872400:08
*** kklimonda has joined #openstack-kolla00:11
*** classicsnail has joined #openstack-kolla00:11
Mannheomyou have to attach to the heka container to inspect logs00:12
Mannheomat least for me there is nothing in /var/log/kolla on host00:12
financeMannwhere do i tail00:13
jemcevoySamYaple: This is what it looks like to me... The additions that I did to the /etc/kolla/config/neutron/ml2_conf.ini was distributed by ansible-kolla but the global_physnet_mtu = 9000 in neutron.conf did not get distributed out to the cluster...00:13
sdakesp_ np dude glad jascott1 could short circuit my laziness :)00:16
jascott1haha00:16
sp_sdake:  :D :D00:17
sdakefinding things is srs biz in my household ;)00:17
*** lamt has quit IRC00:17
sp_jascott1: nice job :)00:17
*** jtriley has joined #openstack-kolla00:20
*** TxGirlGeek has quit IRC00:20
financeMannI found the logs00:20
financeMannif my image is failing to upload even though it looks like everything was good whose fault would that be, glance?00:20
financeMannim trying to figure out what logs to check00:20
sdakeuse --debug to the openstack cli tool00:21
*** m1dev has quit IRC00:22
alopfinanceMann: how did you try to add the image to glance?00:22
financeMannalop through the gui00:22
alopok00:22
inc0right, I'm back, alive and relatively well00:23
sdakeinc0 good to hear00:23
alopfinanceMann: you might want to verify that glance allows both 'file' and 'http' stores00:23
inc0anethetics is wearing off and I'm not in pain so I'm hopeful00:23
alopif you're pointing to a remote URL for your image source00:24
financeMannalop a centos image worked just fine the other day00:25
financeMannthe difference is that this one is like 50gbs00:25
financeMannwhereas the centos was like 1.500:25
financeMannfirst of all i need to find the logs00:25
financeMannso i can see what the logs say00:25
alopthat might exceed your image store00:26
financeMannhow do i verify it’s size?00:26
financeManni just added 800 gbs00:26
financeMannto root00:26
alopI know you were asking about your SSD in #openstack - I assume you mounted your SSD under /var/lib/glance/images ?00:27
financeMannno to root directly00:27
financeMannalop i followed http://rgheck.blogspot.com/2012/07/how-to-add-to-lvm-drive.html00:27
alophow big is your root lv?00:27
financeMannhow do i tell00:27
aloplets hop on over to #openstack00:28
financeMannomg alop it just worked it hink00:28
financeManni think00:28
financeMannalop i uploaded it without specifying a minimum container size00:28
*** yuanying has quit IRC00:28
financeMannseemed to work00:28
*** bigjools has joined #openstack-kolla00:29
inc0kfox1111: ping00:29
*** gfhellma_ has quit IRC00:31
*** gfhellma has joined #openstack-kolla00:32
*** jrobinson is now known as jrobinson-afk00:33
financeMannA deploy of it fails saying “No valid host was found. There are not enough hosts available."00:34
inc0financeMann: call nova service list00:35
inc0service-list00:35
financeMannwhats the command00:35
alop`nova service-list`00:35
financeManncommand not found00:35
inc0ahh00:35
inc0well00:35
inc0go to admin tab in horizon00:36
inc0and check hypervisors tab00:36
inc0to see if you have any working00:36
alopfinanceMann: are you deploying to real systems or is this a virtual deployment?00:39
financeMannreal system00:39
financeMannsingle host00:39
*** gfhellma_ has joined #openstack-kolla00:39
financeManninc0 http://imgur.com/a/0sYTL00:40
alopand since you're in this channel, I'm assuming you did a containerized deployment00:40
*** gfhellma has quit IRC00:40
inc0financeMann: looks good00:40
inc0check nova logs00:40
inc0you can find them in /var/lib/docker/volumes/kolla_logs/_data/nova00:41
inc0you're looking for nova-scheduler.log and see what is real error during boot00:41
*** MarginHu has joined #openstack-kolla00:42
sdakeinc0 did you see that thread shardy started re collaboration with other deployment projects so we re not scrambling prior to each major release of OpenStack indvidually?  Instead, we can scramble together :)00:42
inc0sdake: no I have not, but Steve is great and I'm totally for it00:43
*** saneax is now known as saneax-_-|AFK00:43
sdakethere is some question as to which channel we should use00:43
sdakeI think weighing in on that thread would be helpful - seems like there is some consensus around #openstack-deployment but not total agreement00:44
inc0I am actually happy how this tripleo session went, even tho I was arguing all the time, but well, can't help being myself00:44
sdakeinc0 you argue?  unbelievable!00:44
sdakealthough I think the 11am session was argument free00:44
sdakenot sure what happened at the 4pm session00:45
inc0there were arguments00:45
inc0arguments were present00:45
inc0and I merely voiced them00:45
sdakeand the session on the helm wg went well - now we have a neutral ground for helm development integration upstream with  the deis cats00:45
sdakeinc0 if you only read your email :)00:45
inc0session with helm? isn't it tomorrow tho?00:46
inc0meeting I mean00:47
inc0or you mean openstack-sig in ptg?00:47
sdakeinc0 right, although its being folded into the openstack-sig working group rather than having deis have to deal with hosting it00:47
sdakeinc0 email is your friend ;-)00:47
inc0yeah, we already talked with Lachie and Chris00:48
inc0to get actual ball rolling00:48
*** yingjun has quit IRC00:49
*** jrobinson-afk is now known as jrobinson00:49
sdakeinc0 ya - announcement on ml related to that work00:49
*** Mannheom has quit IRC00:51
*** Manheim has quit IRC00:51
*** daidv has joined #openstack-kolla00:52
*** manheim has joined #openstack-kolla00:53
*** manheim has quit IRC00:53
*** manheim has joined #openstack-kolla00:53
*** dave-mccowan has joined #openstack-kolla00:55
*** gfhellma_ has quit IRC00:56
jascott1what is logo license? cc?00:57
sdakejascott1 no idea - ask heidi joy on the ml00:57
sdakejascott1 I'm sure your not the only one with that Q00:57
jascott1surely they published it somewhere00:58
* jascott1 exerts google-fu00:58
openstackgerritMerged openstack/kolla-kubernetes master: Configuring ironic gates (basic tests)  https://review.openstack.org/43750400:59
sdake@financeMann if you want to add your ssd, it needs to go to /var/lib/docker00:59
sdakethat is where persistent data is stored without ceph01:00
sdakenote when you do this it will wipe out your docker data01:00
sdakeso back it up01:00
sdakestop docker01:00
sdakecopy it over01:00
sdakethen start docker01:00
sdakeif you want to get fancy format it with btrfs in the process01:00
sdakeand change the storage driver to btrfs in docker - your life will be better - or alternately btrfs will eat your data01:00
jascott1found the big tent logo doc, doesnt seem to have a common license like cc but is instead restricted to community building events and blogs etc.01:01
jascott1no marks on the kolla, which is nice01:02
*** Guest68724 is now known as zeus`01:02
jascott1er imean koala01:02
*** zeus` is now known as zeus01:03
*** zeus has joined #openstack-kolla01:03
jascott1meaning the sticker doesnt have 'TM C Registered tradmark of blah'01:03
*** alop has quit IRC01:03
jascott1which is weird because I think you have to have a mark to exert control?01:04
*** sambetts|afk has quit IRC01:04
jascott1+1 for boot leg stickers01:04
sbezverkjascott1: can I ask you qq?01:04
jascott1hey01:04
jascott1sure01:05
sbezverkjascott1: I put together a fix for one of issue I found in helm issue queue. What is the procedure to push the change for review in helm realm?01:05
*** zhugaoxiao has quit IRC01:06
*** Administrator_ has quit IRC01:06
sdakejascott1 how did you hook up bootleg stickers01:06
jascott1fork to your repo, follow their commit msg format01:06
jascott1submit pr01:06
jascott1sdake: joking that all the stickers on the table lack marks01:06
sdakejascott1 i asked heidi if she can make legit stickers available for purchase01:06
jascott1so we will have the unmarked ones which is cool01:07
sdakealthough she is probbaly drinking gasoline atm01:07
*** sambetts_ has joined #openstack-kolla01:07
jascott1thats quite a passtime01:07
sdakewhat i mean is its monday and she has a million emails in her inbox from the PTGs01:08
sbezverkjascott1: that is the problem I do not know what it means ;) I cloned their repo, the fix is in the branch, I run unit tests all pass. can you be a bit more specific about further steps please01:08
jascott1oh sorry01:08
jascott1yeah run the tests with `make test`01:08
inc0I'm off guys, talk to you all tomorrow01:08
sbezverkjascott1: done it passes01:08
jascott1rebase on latest, hmm I could make a cheat sheet01:09
jascott1inc0 have a good one01:09
sbezverkjascott1: done too01:09
jascott1yeah thats about it01:09
jascott1fixd a bug?01:09
sbezverkjascott1: yep01:09
jascott1oh i can find format, one sec01:09
jascott1https://github.com/kubernetes/helm/blob/master/docs/developers.md#git-conventions01:10
jascott1sdake I brought up all the logo stuff cause I just got a new 20oz RTIC tumbler (YETI clone at 1/2 price) and was thinking about getting logo engraved at local makerspace01:12
jascott1and if that worked out, might make a bunch for summit01:12
sbezverkjascott1: thanks a lot!01:12
*** Pavo has joined #openstack-kolla01:13
jascott1well i havent made them yet ;)01:13
sdakejascott1 cool let me know how the RTIC thing goes - my son requested a yeti - ask and ye shall receive01:14
sdakeand of course when you ahve two children you have to balance everything, size of the pile, count of the pile, weight of the pile01:14
sdakehuge PITA :)01:14
jascott1yup01:14
*** pramodrj07 has quit IRC01:14
*** MasterOfBugs has quit IRC01:14
jascott1even when they dont complain, dont want hurt feelings01:15
sdakejascott1 pretty sure santa claus was invented to punish parents :)01:15
sdakeor fund the retail economy one or the other ;)01:15
jascott1same for graduation :)01:18
*** Pavo has quit IRC01:18
jascott1haha les claypool and dean ween to be in reality fishing show by south park guys01:21
jascott1maybe those ATL guys were right and there is a god01:21
jascott1http://liveforlivemusic.com/news/matt-stone-and-trey-parker-to-produce-reality-show-starring-dean-ween-and-les-claypool/01:21
sdakedid openstack status bot link this review merged in channel?  I just git pull --rebased and have it merged: https://review.openstack.org/#/c/437504/01:23
sdakehowever don't see anything from the status bot01:23
*** yuanying has joined #openstack-kolla01:24
jascott1not in the past 3 hrs01:24
jascott1oh wait01:25
sdakewierd - its in the eavesdrop logs: http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-02-28.log.html#t2017-02-28T00:59:1101:25
sdakei blame my godawful VPN that I have to deal with01:26
jascott1idk mines not going back that far on lime01:27
jemcevoySamYaple: It seems like there is chicken/egg problem with bootstrapping openvswitch... If I setup the network using openvswitch before installing kolla I end up with two openvswitch databases... NOT GOOD! Is there a way to install the OVS containers in a prep phase?01:27
jascott116:59 openstackgerrit: Merged openstack/kolla-kubernetes master: Configuring ironic gates (basic tests)  https://review.openstack.org/43750401:27
jascott1sdake ^ that one?01:28
sdakeya - my vpn boucning on and off all the time01:28
sdakeannoying01:28
*** sp_ has quit IRC01:29
sbezverkjascott1: what about it? ;)01:29
jascott1its absolutely perfect :)01:29
*** sdake_ has joined #openstack-kolla01:30
*** dave-mccowan has quit IRC01:30
sbezverkjascott1: :) I thought you found some issues01:30
jascott1nah sdake is just missing the message form the bot01:31
*** jrobinson has quit IRC01:32
*** Pavo has joined #openstack-kolla01:32
*** sdake has quit IRC01:33
*** dave-mccowan has joined #openstack-kolla01:37
sdake_bbl dinner time01:40
*** sdake_ has quit IRC01:40
*** cuongnv has joined #openstack-kolla01:43
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Exit with error code 1 when there is failure container in gate  https://review.openstack.org/43822801:46
*** caowei has joined #openstack-kolla01:50
*** adrian_otto has quit IRC01:51
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding ironic inspector microservices  https://review.openstack.org/43855601:53
*** MarginHu has quit IRC01:55
*** eaguilar has quit IRC01:58
financeMannspecs “Disk01:58
financeMann200000GB”01:58
financeMannlol01:58
financeMannno wonder it wont start01:58
*** dave-mccowan has quit IRC01:59
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Fix Creating Panko MongoDB database failed  https://review.openstack.org/43840802:01
*** Pavo has quit IRC02:02
*** goldyfruit has quit IRC02:02
*** goldyfruit has joined #openstack-kolla02:03
*** manheim has quit IRC02:04
jascott1financeMan has all the worlds knowledge on his dev server02:06
*** goldyfruit has quit IRC02:07
*** Margin has joined #openstack-kolla02:10
*** unicell has quit IRC02:14
*** goldyfruit has joined #openstack-kolla02:18
*** goldyfruit has quit IRC02:19
*** goldyfruit has joined #openstack-kolla02:19
sbezverkjascott1: sorry to bug you with this :( how do you add your repo as a remote ?02:28
*** manheim has joined #openstack-kolla02:28
sbezverkjascott1: I forked the repo, now it says I need to add my repo as remote to helm repo02:29
*** goldyfruit has quit IRC02:35
*** eaguilar has joined #openstack-kolla02:35
*** goldyfruit has joined #openstack-kolla02:36
*** jrobinson has joined #openstack-kolla02:41
*** jrobinson has quit IRC02:43
*** jrobinson has joined #openstack-kolla02:44
*** manheim has quit IRC02:53
jascott1sbezverk: fork on github02:59
jascott1goto helm repo and click Fork this repo02:59
jascott1oh sorry i get you now02:59
jascott1if its a real simple fix i would clone your forked and patch it03:00
jascott1(on a new branch) then push to your repo, go back to gituhub and create PR03:00
*** eaguilar has quit IRC03:03
financeMannanyone know why i am getting this error on my windows image deploy -> http://pastebin.com/08XgaRjb03:11
*** eaguilar has joined #openstack-kolla03:16
Marginhttps://bugs.launchpad.net/kolla/+bug/1668059  it has been solved , we can close it.03:21
openstackLaunchpad bug 1668059 in kolla "Failed to Starting kolla-toolbox container" [Critical,Triaged]03:21
*** fooliouno has joined #openstack-kolla03:22
*** zhenguo has joined #openstack-kolla03:37
openstackgerritMerged openstack/kolla-ansible master: Update the panko location of site.yml  https://review.openstack.org/43838003:45
*** eaguilar has quit IRC03:46
*** mdnadeem has joined #openstack-kolla03:46
*** goldyfruit has quit IRC03:50
*** lucasxu has joined #openstack-kolla03:53
financeMannhow the heck do i create a floating ip03:56
openstackgerritzhongshengping proposed openstack/kolla-ansible master: Remove signing_dir option  https://review.openstack.org/41560703:57
openstackgerritzhongshengping proposed openstack/kolla-ansible master: Deprecate use_neutron option  https://review.openstack.org/41347103:57
sbezverkjascott1: that what I was doing but by some reason local changes do not get pushed to my fork at github..04:22
*** saneax-_-|AFK is now known as saneax04:22
*** crushil has quit IRC04:22
sbezverkI do see in git log my change in my local copy04:22
sbezverkjascott1: but the change does not exist at github04:23
Marginhi, I open a bug https://bugs.launchpad.net/kolla/+bug/1668489, can you help me ?04:29
openstackLaunchpad bug 1668489 in kolla "failed to deploy because haproxy Waiting for virtual IP to appear" [Undecided,New]04:29
*** p6_ has quit IRC04:36
*** sudipto has joined #openstack-kolla04:40
*** sudipto_ has joined #openstack-kolla04:40
*** sudipto_ has quit IRC04:41
*** sudipto has quit IRC04:41
*** sudipto has joined #openstack-kolla04:43
*** sudipto_ has joined #openstack-kolla04:43
*** p6 has joined #openstack-kolla04:43
*** Margin has quit IRC04:48
*** Jeffrey4l__ has quit IRC04:49
*** trinaths has joined #openstack-kolla04:50
*** Jeffrey4l__ has joined #openstack-kolla04:52
*** crushil has joined #openstack-kolla04:54
*** lucasxu has quit IRC04:54
*** trinaths1 has joined #openstack-kolla04:57
*** trinaths has quit IRC04:57
*** unicell has joined #openstack-kolla04:58
*** sudipto_ has quit IRC04:58
*** sudipto has quit IRC04:59
*** sdake_ has joined #openstack-kolla05:09
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add default dnsmasq_dns_servers in dhcp_agent.ini file  https://review.openstack.org/43880205:12
sdake_Jeffrey4l__ hae a bit of an emergency with docker-py05:12
Jeffrey4l__sdake_, yep?05:12
jascott1sbezverk: in your local helm folder, do a `git config -l`05:21
jascott1whats the value of remote.origin?05:21
*** daidv_ has joined #openstack-kolla05:22
*** daidv has quit IRC05:22
sdake_Jeffrey4l__ https://bugs.launchpad.net/bugs/166834605:22
openstackLaunchpad bug 1668346 in kolla "docker-py 2.1.0 looks broken with docker 1.13.1 which breaks Kolla stable/ocata" [Critical,Confirmed]05:22
Jeffrey4l__got. let me check it.05:23
sdake_Jeffrey4l__ readbug first05:23
*** skramaja has joined #openstack-kolla05:24
jascott1ah the openstack logo has the R mark05:25
jascott1id bet the mascots are sure to follow05:25
*** trinaths1 has quit IRC05:26
*** trinaths has joined #openstack-kolla05:26
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements  https://review.openstack.org/43830605:40
*** sdake_ has quit IRC05:47
*** sdake has joined #openstack-kolla05:47
*** unicell1 has joined #openstack-kolla05:56
*** unicell has quit IRC05:56
Jeffrey4l__sdake, check this https://bugs.launchpad.net/kolla/+bug/1668346/comments/306:02
openstackLaunchpad bug 1668346 in kolla "docker-py 2.1.0 looks broken with docker 1.13.1 which breaks Kolla stable/ocata" [Critical,Confirmed]06:02
Jeffrey4l__i do not think this matter actually.06:02
Jeffrey4l__the new package name is called "docker" not "docker-py"06:02
sdakeJeffrey4l__ good hunting06:03
sdakekaebigus reported the bug from gentoo emerge iiuc06:03
sdakeperhaps gentoo hasn't renamed the package06:03
*** salv-orlando has joined #openstack-kolla06:03
Jeffrey4l__i guess there are something  wrong in gentoo packages.  it use docker-py 2.1.0, this shouldn't exist.06:04
sdakecan probably be closed as invalid, although egonzalez had done some initial triage and marked it confirmed iIRC.06:04
sdakeI may have marked it confirmed based upon egonzalez's analysis - I don't recall the specifics - that was like 8 hours ago ;-)06:04
Jeffrey4l__on the other hand, i think i can make kolla-ansible work with docker and docker-py with some trick . i am testing this. if we can handle both docker and docker-py, that will be better.06:05
sdakei guess if worse comes to worse and docker releases docker-py as 2.1.0 as docker-py in pip, we can work around the problem06:05
sdakethe question though becomes what happens when someone pip installs docker because that is what their system demands?06:05
sdakeagreed06:05
sdakeI think it should be pretty easy to test the existance of that class06:06
sdakedoes docker-py and docker pip install to the same location?06:07
Jeffrey4l__the issue is: docker and docker-py share the same namespace "docker" in site-packages folder. and you can install docker and docker-py at the same.06:07
Jeffrey4l__WTF.06:07
Jeffrey4l__the files and docker and docker-py are mixed in one folder. who knows what will happen .; (06:08
*** Jeffrey4l__ is now known as Jeffrey4l06:10
*** sdake_ has joined #openstack-kolla06:10
*** sdake has quit IRC06:13
*** sacharya has quit IRC06:13
*** sacharya has joined #openstack-kolla06:14
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix the incompatible of docker-py 2.0.0  https://review.openstack.org/43881306:14
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Remove serial in playbooks  https://review.openstack.org/43837406:15
*** salv-orl_ has joined #openstack-kolla06:18
*** sacharya has quit IRC06:19
*** salv-orl_ has quit IRC06:20
*** salv-orl_ has joined #openstack-kolla06:20
*** salv-orlando has quit IRC06:21
*** jtriley has quit IRC06:22
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [TEST][MASTER][KOLLA-ANSIBLE] TEST CEILOMETER RELATED SERVICE  https://review.openstack.org/41227806:23
sdake_jascott1 still around?  or sleepy time06:25
*** lamt has joined #openstack-kolla06:25
openstackgerritMerged openstack/kolla-ansible master: Add until in restart nova libvirt task  https://review.openstack.org/43823706:26
*** tovin07_ has joined #openstack-kolla06:27
*** masuberu has quit IRC06:27
*** satyar has joined #openstack-kolla06:30
*** Margin has joined #openstack-kolla06:35
jascott1sdake_ whats up?06:36
sdake_jascott1 if your not aware, the OpenStack on Helm wg meeting has changed times: http://lists.openstack.org/pipermail/openstack-dev/2017-February/112980.html06:39
jascott1oh thanks, saw that in chris's post on ML06:44
*** jtriley has joined #openstack-kolla06:46
*** sudipto_ has joined #openstack-kolla06:48
*** sudipto has joined #openstack-kolla06:48
*** jtriley has quit IRC06:51
*** lamt has quit IRC06:52
*** saneax is now known as saneax-_-|AFK06:53
*** saneax-_-|AFK is now known as saneax06:56
sdake_jascott1 my calendar still has the old meeting - which is why i mentioned it06:57
sdake_jascott1 your calendar may as well, which might be confusing :)06:57
jascott1sdake_ btw the RTIC is rocking so far07:07
jascott1going to give it the coffee test tomorrow07:07
sdake_jascott1 right- i looked that up on AMZN and thought you were talking about a mic not a coffee mug :)07:10
jascott1oh07:11
jascott1im talking about 20oz tumbler https://www.rticcoolers.com/shop/drinkware/tumblers07:11
sdake_re bootleg stickers, I think berendt made some (I spoke to him during breakfast on Monday) so perhaps he has an idea of how to make that happen ;-)07:11
sdake_jascott1 yup I saw it on AMZN07:11
jascott1needs a laser etched koala07:12
sdake_that would be sweet ;-)07:12
sdake_no idea how to make that happen07:13
sdake_there is probably someone that will do it for ya07:13
*** sacharya has joined #openstack-kolla07:15
*** bmace has quit IRC07:16
*** bmace has joined #openstack-kolla07:16
Marginhi ,sdake , I opened a bug https://bugs.launchpad.net/kolla/+bug/166848907:17
openstackLaunchpad bug 1668489 in kolla "failed to deploy because haproxy Waiting for virtual IP to appear" [Undecided,New]07:17
*** jrobinson has quit IRC07:21
*** magicboiz has quit IRC07:21
sdake_margin that problem has been submitted hundreds of times - it always comes down to incorrect configuration -try running kolla-ansible prechecks07:23
*** yuanying_ has joined #openstack-kolla07:25
*** sp__ has joined #openstack-kolla07:26
*** yuanying has quit IRC07:27
sp__duonghq:  hi......07:27
duonghqsp__, hi07:28
sp__duonghq: what can be best way to test the ks-rolling-upgrade implementation07:29
sp__?07:29
sp__duonghq: i have shared the entherpad link after ps07:30
duonghqwe need something like functional test, but still cannot form any idea to implement it in efficient way07:30
duonghqI read your etherpad07:30
sp__duonghq: as of my knowledge kolla support nova rolling upgrade feature07:31
sp__duonghq: we get pick the same idea from there07:32
sp__can pick*07:32
duonghqsp__, can you describe this more detail?07:33
sp__duonghq: I am saying that kolla already support nova rolling upgrade feature07:36
*** crushil has quit IRC07:36
duonghqbut gating in Kolla is not good enough07:36
duonghqI want to build something better for our potion07:36
Marginhi,sdake, about bug 1668489, I have prechecked , its result is ok.07:37
openstackbug 1668489 in kolla "failed to deploy because haproxy Waiting for virtual IP to appear" [Undecided,New] https://launchpad.net/bugs/166848907:37
sp__duonghq: may be we can discuss it in upcoming kolla-meeting with sdake_ and inc0 for better way of implementation07:38
duonghqsp__, sure07:38
duonghqgreat idea07:38
sdake_Margin no idea - however I need to hit the rack - 1am here07:38
sdake_Margin europe comes online in about 1-2 hours - might try boucning that q off europe :)07:39
sdake_night folks07:39
duonghqnight sdake_07:39
sp__duonghq: so we have to add this in agenda too beforehand07:39
duonghqsp__, pleas07:39
sp__night sdake_07:39
sdake_night sp__ duonghq07:39
Marginnight sdake , ok07:39
*** jtriley has joined #openstack-kolla07:40
*** manheim has joined #openstack-kolla07:42
MarginJeffrey4l, can you help me ?07:43
sp__duonghq: still around ??07:44
duonghqya07:45
* duonghq fixing Ceilometer bug 07:45
*** manheim has quit IRC07:46
Jeffrey4lsup Margin07:47
Marginhi07:47
sp__please check and update if required   https://wiki.openstack.org/wiki/Meetings/Kolla#Agenda_for_next_meeting_.28Mar_1st_2017.2907:47
Margincan you take a minute to see https://bugs.launchpad.net/kolla/+bug/1668489 ?07:48
openstackLaunchpad bug 1668489 in kolla "failed to deploy because haproxy Waiting for virtual IP to appear" [Undecided,New]07:48
openstackgerritNguyen Van Trung proposed openstack/kolla-ansible master: Fix 'is' to 'are'  https://review.openstack.org/43883207:48
*** matrohon has joined #openstack-kolla07:49
Jeffrey4lis the vip 192.168.102.254 correct? can you ping that ip?07:50
*** jtriley has quit IRC07:51
Marginyes ,102.254 ,the vip hasn't been assigned07:52
Marginwhere is  keepalived's log07:52
Jeffrey4lshow me the "ip addr" and "ip route"07:52
Jeffrey4lMargin, docker logs keepalived07:53
Marginhttp://pastebin.com/WRyzs3id07:54
*** shardy has joined #openstack-kolla07:56
Marginthanks07:56
*** sdake_ is now known as sdake07:56
MarginI find a clue in the log07:57
*** tovin07_ has quit IRC07:57
Jeffrey4lMargin, try to find which node is the vip?07:59
MarginI found I haven't config external network ip address on eth2 ,but precheck is ok08:00
Jeffrey4lthere should be a bug, then.08:01
Jeffrey4lMargin, then if you fix your issue, please close/mark invaid for your bug. Or change it to more specific about pre-checks.08:02
Marginok, thanks, but I dont' know where I can change the bug's status08:03
berendtsdake: i just ordered a few booklets at moo.com08:03
berendtsdake: will be delivered next week08:04
Jeffrey4lhrm guess normally user may haven't such permission. anyway, tell me, and i can change it.08:04
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Fix Ceilometer cannot create MySQL database  https://review.openstack.org/43883808:08
*** Mannheom has joined #openstack-kolla08:12
*** Manheim has joined #openstack-kolla08:12
*** pcaruana has joined #openstack-kolla08:17
*** athomas has joined #openstack-kolla08:21
sp__duonghq: sorry to bother you again08:23
sp__duonghq: did you checked https://wiki.openstack.org/wiki/Meetings/Kolla#Agenda_for_next_meeting_.28Mar_1st_2017.29 ?08:24
*** masber has joined #openstack-kolla08:24
*** Margin has quit IRC08:24
*** shardy has quit IRC08:25
duonghqsp__, nice, but can you join the meeting?08:25
sp__duonghq: yes, I will.08:26
sdakehogepodge disaster averted ;-)08:28
duonghqsp__, nice, so I'll put your name at the agenda too, I'll join the meeting but nobody can make sure anything08:28
*** jmccarthy has joined #openstack-kolla08:29
*** jmccarthy has quit IRC08:32
sp__duonghq: sure08:32
sp__duonghq: have you attended PTG in atlanta ?08:33
*** yingjun has joined #openstack-kolla08:33
*** jmccarthy has joined #openstack-kolla08:34
duonghqsp__, no08:35
sp__duonghq: me too08:35
sp__duonghq: is there any discussion related to this happened in PTG ?08:35
sp__duonghq: any idea ?08:36
duonghqiirc, no discussion08:36
sp__duonghq: ok08:37
hogepodgeGot both the time and link wrong. Ugh.08:37
sp__duonghq:  then certainly we should discuss about this in upcoming meeting08:38
duonghqsure08:38
sp__duonghq: thanks for your help08:38
duonghqsp__, thank you08:38
*** jtriley has joined #openstack-kolla08:40
*** zioproto has joined #openstack-kolla08:40
*** jtriley has quit IRC08:45
*** jtriley has joined #openstack-kolla08:45
*** egonzalez has joined #openstack-kolla08:46
egonzalezmorning08:46
jascott1good morning08:48
sp__egonzalez: good morning08:49
duonghqmorning egonzalez08:52
*** yuanying_ has quit IRC08:53
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Fix Creating Ceilometer MongoDB database failed  https://review.openstack.org/43781908:54
duonghqJeffrey4l, ping08:58
duonghqJeffrey4l, it's seem that the ubuntu gate in kolla-ansible is busted?08:59
*** Jeffrey4l has quit IRC09:00
*** Jeffrey4l has joined #openstack-kolla09:00
egonzalezduonghq, yep09:01
*** manheim_ has joined #openstack-kolla09:01
*** manheim_ has quit IRC09:02
jascott1have a good day folks09:02
*** manheim_ has joined #openstack-kolla09:02
sp__egonzalez: hi.... me and duonghq expecting you too in tomorrow meeting for ks-rolling-upgrade discussion.09:06
egonzalezsp__, kolla meeting?09:06
egonzalezor other?09:06
sp__egonzalez: kolla meeting09:06
sp__yes09:06
egonzalezi'll  be there09:06
sp__egonzalez: thanks09:07
egonzalezany special topic what to discuss?09:07
*** jtriley has quit IRC09:07
sp__egonzalez: duonghq and me want to discuss about the ks-rolling-upgrade feature's better way of implementation.09:08
sp__egonzalez: and many more thing which needs to be taken care of for that09:08
egonzalezsp__, sure09:08
*** retr0h has quit IRC09:09
*** vcn[m] has quit IRC09:09
*** bachp has quit IRC09:09
sdakeegonzalez Jeffrey4l is looking at that bug from yessterday09:13
sdakeand i am off to bed09:13
sdakenight for real thi time09:13
*** sdake has quit IRC09:13
openstackgerritNguyen Van Trung proposed openstack/kolla-ansible master: change amoung to among  https://review.openstack.org/43887109:19
*** trinaths has left #openstack-kolla09:20
berendthas anybody a working ldap read only configuration using the domains directory?09:22
berendti want to add a second domain with ldap (read only), but it is not really working09:22
*** gfidente has joined #openstack-kolla09:27
*** gfidente has quit IRC09:27
*** gfidente has joined #openstack-kolla09:27
*** Serlex has joined #openstack-kolla09:28
*** magicboiz has joined #openstack-kolla09:29
satyarHi09:33
satyaris there any way i can enable monasca with kolla?09:33
egonzalezsatyar, not for now09:34
satyarok currently we dont support monasca?09:35
egonzaleznope, there is a blueprint for it09:36
*** manheim__ has joined #openstack-kolla09:36
egonzalezsatyar, https://blueprints.launchpad.net/kolla/+spec/monasca-containers https://blueprints.launchpad.net/kolla-ansible/+spec/monasca-roles09:37
*** skramaja has quit IRC09:39
*** manheim_ has quit IRC09:39
*** hrw has quit IRC09:39
*** skramaja has joined #openstack-kolla09:41
*** jtriley has joined #openstack-kolla09:41
*** hrw has joined #openstack-kolla09:44
*** jtriley has quit IRC09:49
*** mgoddard has joined #openstack-kolla09:50
*** manheim__ has quit IRC09:51
jmccarthyAny Ironic folks here ?  James McCarthy proposed openstack/bifrost master: Add '-E' options to pass proxy env for sudo  https://review.openstack.org/43888409:55
openstackgerritDuong Ha-Quang proposed openstack/kolla master: Fix missing pecan module in Ceilometer  https://review.openstack.org/43888609:56
*** skramaja has quit IRC09:57
*** duonghq has quit IRC09:59
*** cuongnv has quit IRC10:02
*** pbourke has quit IRC10:05
*** sdake has joined #openstack-kolla10:05
Jeffrey4lduonghq, yep. deploy ubuntu gate is busted.10:06
*** skramaja has joined #openstack-kolla10:06
*** pbourke has joined #openstack-kolla10:07
egonzalezJeffrey4l, i'm trying to figure out what happens in gate but cannot find any clue in logs :/10:08
Jeffrey4legonzalez, i tried but still no idea. seems libvirt trying create console.log file in /var/lib/nova/isntance/xxx/console.log, but failed. guess the user/group permission issue.10:09
Jeffrey4lstill no idea how to fix this.10:09
egonzalezJeffrey4l, when gates build the images, use the same images to deploy?10:11
Jeffrey4legonzalez, yep. in kolla's gate, the image is built rather than pull from tarballs.o.o.10:12
*** manheim_ has joined #openstack-kolla10:12
*** daidv_ has quit IRC10:14
*** sambetts_ is now known as sambetts10:17
egonzalezJeffrey4l, might be this the issue https://review.openstack.org/#/c/412231/33/docker/nova/nova-libvirt/Dockerfile.j210:17
egonzalezuser is not in libvirtd group guess10:17
*** wirehead_ has joined #openstack-kolla10:18
Jeffrey4lmaybe. please try to push a patch ;)10:18
egonzalezyup10:18
Jeffrey4legonzalez, currently, the libvirt group is: "libvirt" not libvirtd10:19
Jeffrey4lcat /etc/group | grep libv10:19
Jeffrey4llibvirt:x:111:10:19
Jeffrey4llibvirt-qemu:x:64055:libvirt-qemu10:19
egonzalezyeah, was a typo, anyway nova user is not in libvirt group, i'm going to try revert all that part and debug what happen after that10:21
Jeffrey4lcool. thanks.10:21
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Test fix ubuntu libvirt gate  https://review.openstack.org/43889710:26
*** yingjun has quit IRC10:28
*** sacharya has quit IRC10:29
*** vcn[m] has joined #openstack-kolla10:31
egonzalezJeffrey4l, btw, nova 14.0.4 was merged10:32
egonzalezrdo build packages when a tag is created or with each commit?10:33
*** manheim_ has quit IRC10:34
*** Serlex has quit IRC10:35
*** magicboiz has quit IRC10:36
*** satyar has quit IRC10:36
*** manheim_ has joined #openstack-kolla10:38
Jeffrey4legonzalez, got. thanks.  rdo master repo: yes. but we use master-passed-ci repo in kolla master. and use centos-release-openstack-xxx repo in stable branch.10:39
*** yuanying has joined #openstack-kolla10:40
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Test fix ubuntu libvirt gate  https://review.openstack.org/43889710:41
egonzalezJeffrey4l, thanks10:42
*** sdake has quit IRC10:42
Jeffrey4lnp ;)10:42
openstackgerritJeffrey Zhang proposed openstack/kolla stable/newton: Bump nova release to 14.0.4 tag  https://review.openstack.org/43890310:44
*** Serlex has joined #openstack-kolla10:45
*** shardy has joined #openstack-kolla10:45
*** tvignaud has quit IRC10:51
*** salv-orl_ has quit IRC10:57
*** tvignaud has joined #openstack-kolla11:03
*** salv-orlando has joined #openstack-kolla11:05
*** sp_ has joined #openstack-kolla11:10
*** mnasiadka has joined #openstack-kolla11:16
*** retr0h has joined #openstack-kolla11:18
*** bachp has joined #openstack-kolla11:18
egonzalezgates still failing :/11:30
*** sudipto has quit IRC11:32
*** sudipto_ has quit IRC11:32
*** jmccarthy has quit IRC11:52
*** zeus has quit IRC11:52
*** athomas has quit IRC11:53
hrwhi11:53
*** zeus has joined #openstack-kolla11:54
hrwdoes someone had situation when run of kolla hangs somewhere and do nothing for hours?11:54
*** zeus is now known as Guest9124911:54
manheim_hello, let's say i want to integrate kolla with hyper-v compute, this requires networking-hyperv to be installed on neutron-server11:58
manheim_how can i do it, if i simply run pip install networking-hyperv on neutron-server container i get permission denied11:58
manheim_how can I achive this?11:59
egonzalezManheim, source or binary?11:59
manheim_with pip12:00
*** athomas has joined #openstack-kolla12:00
manheim_binary12:00
egonzalezis not a package for that?12:00
egonzalezhttps://github.com/openstack/kolla/blob/master/doc/image-building.rst#package-customisation12:00
egonzalezcan use package-customization12:01
*** salv-orlando has quit IRC12:01
egonzalezor https://github.com/openstack/kolla/blob/master/doc/image-building.rst#additions-functionality12:02
manheim_cool, i will look into it, thanks12:03
*** jmccarthy has joined #openstack-kolla12:11
*** magicboiz has joined #openstack-kolla12:12
manheim_ok, i've managed to do it, now how do i promote the new image as active neutron-server container, run kolla deploy again?12:16
*** Guest91249 has quit IRC12:17
*** Margin has joined #openstack-kolla12:17
MarginI open a bug , https://bugs.launchpad.net/kolla/+bug/166854612:18
openstackLaunchpad bug 1668546 in kolla "failed to deploy because can't access mariadb service through vip " [Undecided,New]12:18
*** bachp has quit IRC12:19
Marginwho can help me ?12:20
*** rhallisey has joined #openstack-kolla12:23
*** zeus- has joined #openstack-kolla12:24
*** jtriley has joined #openstack-kolla12:26
*** alanmeadows has quit IRC12:27
*** bachp has joined #openstack-kolla12:29
*** jtriley has quit IRC12:30
*** skramaja is now known as skramaja|afk12:31
*** rwallner has joined #openstack-kolla12:32
*** rwallner has quit IRC12:33
*** rwellum has joined #openstack-kolla12:34
*** kong has quit IRC12:39
*** saneax is now known as saneax-_-|AFK12:39
*** zeus- is now known as zeus12:41
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Improve Swift docs  https://review.openstack.org/43894912:41
*** rwallner has joined #openstack-kolla12:41
*** zeus is now known as Guest8399112:41
*** salv-orlando has joined #openstack-kolla12:42
*** retr0h has quit IRC12:53
*** manheim_ has quit IRC12:53
*** markmcclain has quit IRC12:53
*** retr0h has joined #openstack-kolla12:55
*** afranc has quit IRC12:56
sp_Margin: hi..12:56
Marginhi sp12:57
*** manheim_ has joined #openstack-kolla12:59
Marginwhat's up ? friend13:00
*** markmcclain has joined #openstack-kolla13:00
*** afranc has joined #openstack-kolla13:01
*** manheim_ has quit IRC13:01
egonzalezMargin, can you telnet to VIP 3306 ?13:01
Marginno13:02
egonzalezcannot ssh because haproxy is not binding/allowing ssh connections13:02
*** yingjun has joined #openstack-kolla13:02
egonzalezmariadb containers are running?13:02
egonzalez*in control node13:03
Marginmariadb is in control node , yes they are running ok13:04
*** satyar has joined #openstack-kolla13:04
egonzalezsaw that VIP is in a different range than IPs in network node *.102.* vs *.101.*13:04
Margincontrol kode0 node, I can see  192.168.102.20:3306 by netstat command13:05
-openstackstatus- NOTICE: restarting gerrit to address performance problems13:06
*** ChanServ changes topic to "restarting gerrit to address performance problems"13:06
egonzalezat first glance I think the issue is that net *.102 is not reachable from control node, don't know if because a typo in globals.yml or because missing routes in infra network13:08
egonzalezprobably telnet from network node to VIP 3306 works, and telnet from control to control_api_ip 3306 works too13:08
Marginyou are right,  telnet from network node to VIP 3306 works13:14
Marginbut from control node to VIP 3306  doesn't  work13:15
*** sudipto_ has joined #openstack-kolla13:17
egonzalezthen is a infra network issue with ranges13:17
*** sudipto has joined #openstack-kolla13:17
*** manheim_ has joined #openstack-kolla13:18
egonzalezMargin, is a test environment or is under a network topology managed by switches and firewalls?13:18
Margintest environment , all nodes are  kvm virtual machine.13:20
Marginping from control to VIP  is ok13:20
Marginhttps://bugs.launchpad.net/kolla/+bug/1668546/comments/113:23
openstackLaunchpad bug 1668546 in kolla "failed to deploy because can't access mariadb service through vip " [Undecided,New]13:23
Marginit is caused by firewall on network node.13:24
egonzalezMargin, cool :D13:25
*** dave-mccowan has joined #openstack-kolla13:26
Margindo you mean  we need to disable firewall  when using kolla deploy?13:28
egonzalezyou may leave firewall enabled but at least open all rules for required ports. Dunno how will be the behaviour with docker and neutron security group pushing rules into iptables13:30
*** bachp has quit IRC13:31
*** bachp has joined #openstack-kolla13:33
MarginI see, so it's better settle it in kolla ansible code.13:33
-openstackstatus- NOTICE: ok gerrit is back to normal13:35
*** ChanServ changes topic to "ok gerrit is back to normal"13:35
*** sudipto_ has quit IRC13:36
*** sudipto has quit IRC13:36
manheim_so i had to regenerate the neutron image (had to include a newtoking driver) on an existing kolla deployment, how can i redeploy only the neutron-server container?13:38
manheim_or what can i do in this case13:38
*** sbezverk has quit IRC13:39
*** salv-orlando has quit IRC13:42
*** ChanServ changes topic to "support: ask.openstack.org | New to Kolla: docs @ http://docs.openstack.org/developer/kolla/ | | Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"13:42
-openstackstatus- NOTICE: gerrit is back to normal and I don't know how to use the openstackstaus bot13:42
Marginmanheim, only remain network in /etc/kolla/global.yml13:45
*** lamt has joined #openstack-kolla13:45
*** shardy has quit IRC13:46
*** shardy has joined #openstack-kolla13:47
*** jtriley has joined #openstack-kolla13:47
*** wirehead_ has quit IRC13:48
*** dciabrin has quit IRC13:49
*** rwellum has quit IRC13:49
*** dciabrin has joined #openstack-kolla13:50
*** rwellum has joined #openstack-kolla13:51
*** zhurong has joined #openstack-kolla13:52
*** wirehead_ has joined #openstack-kolla13:52
*** rwallner has quit IRC13:54
*** goldyfruit has joined #openstack-kolla13:56
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements  https://review.openstack.org/43830613:58
*** retr0h has quit IRC14:00
*** retr0h has joined #openstack-kolla14:01
*** rwellum has quit IRC14:01
*** openstackgerrit has quit IRC14:03
*** smekel_ has quit IRC14:04
*** smekel_ has joined #openstack-kolla14:06
*** rwellum has joined #openstack-kolla14:10
*** jtriley has quit IRC14:12
*** crushil has joined #openstack-kolla14:18
*** rwallner has joined #openstack-kolla14:19
*** manheim_ has quit IRC14:19
*** sudipto_ has joined #openstack-kolla14:22
*** sudipto has joined #openstack-kolla14:22
*** sdake has joined #openstack-kolla14:23
*** manheim_ has joined #openstack-kolla14:23
*** mdnadeem has quit IRC14:24
*** goldyfruit has quit IRC14:24
*** goldyfruit has joined #openstack-kolla14:25
*** sacharya has joined #openstack-kolla14:26
*** rhallisey has quit IRC14:26
financeMannAfter creating a network wtih a subnet and then I go to http://192.168.1.113/admin/floating_ips/, I can’t create a floating ip because there are no pools defined to select from, any idea why?14:28
egonzalezfinanceMann, is the network marked as external?14:28
financeMann‘No'14:29
egonzalezthere is the issue14:29
egonzalezfloatingip only can be created on external networks14:30
financeMannah gotcha, last time i tried to do deploys they only worked if i could assign a floating ip to the instance before deploy14:30
*** sacharya has quit IRC14:30
financeMannotherwise it’d complain that there arent any hosts available14:30
financeMannIt keeps saying “Failed to create network” if I select extdernal14:32
financeMannexternal14:32
*** jtriley has joined #openstack-kolla14:33
egonzalezfinanceMann, failed while creating the network, the instance or the floatingip?14:33
financeMannnetwork14:33
financeMannspecifically14:33
egonzalezany error log more that failed to create network message?14:34
*** srwilkers has joined #openstack-kolla14:36
*** alanmeadows has joined #openstack-kolla14:36
srwilkersgood morning everyone14:36
*** lamt has quit IRC14:37
financeMannIt was an issue with the GUI egonzalez I had to create the network in the non admin section14:37
financeMannwas a little wonky but got it14:38
financeMannegonzalez is there any log tool I can setup to easily view all the logs of everything all the time?14:38
financeMannhard digging around14:38
egonzalezexternal networks can only be created with admin privs in admin tab14:38
egonzalezenable central_login: yes14:38
*** Guest83991 is now known as zeus14:42
*** zeus has quit IRC14:42
*** zeus has joined #openstack-kolla14:42
financeMannegonzalez if I enable central logging would that show-up in the GUI then14:43
financeMannall logs for all services14:43
*** yingjun has quit IRC14:43
financeMannI keep getting “No valid host was found. There are not enough hosts available.” when I deploy and I deployed this instance the other day just fine and i deleted all instances to make sure it wasnt a resource issue14:43
egonzalezwill deploy a kibana instance, not in horizon https://github.com/openstack/kolla-ansible/blob/master/doc/kibana-guide.rst14:44
*** openstackgerrit has joined #openstack-kolla14:46
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Disable unneeded fact gathering  https://review.openstack.org/43788114:46
financeMannegonzalez this is what i am seeing http://pastebin.com/VFZXF5E114:47
*** yingjun has joined #openstack-kolla14:50
pbourkefinanceMann: https://docs.openstack.org/developer/kolla-ansible/kibana-guide.html#example-using-kibana-to-diagnose-a-common-failure14:50
*** lamt has joined #openstack-kolla14:52
*** srwilkers has quit IRC14:52
*** retr0h has quit IRC14:54
*** srwilkers has joined #openstack-kolla14:55
*** retr0h has joined #openstack-kolla14:56
sdakemorning peeps15:02
sdakehey srwilkers15:02
sdakehe pbourke15:02
sdakehey egonzalez15:02
srwilkershey sdake15:03
egonzalezsup sdake15:03
sdakeatleast its nto monday morning is hwat is up :)15:03
sdakewhenver my day starts out with "You don't have any meetings this morning do you" from my wife it usually ends up poorly :)15:03
financeMannyou work from home sdake?15:04
sdakefinanceMann indeed15:04
egonzalezJeffrey4l, nova ubuntu issue is present also outside gates too, though was only a gate issue15:05
*** sp_ has quit IRC15:06
financeMannsdake for openstack-kolla?15:06
Jeffrey4li tried it locally. yes.15:06
sdakefinanceMann actually I am employed by Cisco Systems15:06
*** TxGirlGeek has joined #openstack-kolla15:06
sdakefinanceMann I work on a variety of things including openstack-kolla15:06
egonzalezno logs, nothing, hate .deb distros :/15:07
sdakefinanceMann also on openstack BOD and manage to look after my two teenagers with my wife as well :)15:07
sdakestill haven't managed to get the chirstmas lights off the house and its almost march15:07
sdakeso somewhere in there, I'm failing at something :)15:07
financeMannhttp://imgur.com/a/dyWqi15:10
financeManntrying to set up kibana15:10
financeManninitial go at the website15:10
financeMannfollowing the guide15:10
*** zhurong has quit IRC15:10
openstackgerritRonan-Alexandre Cherrueau proposed openstack/kolla-ansible master: Multi-regions Support  https://review.openstack.org/43165815:13
pbourkefinanceMann: make sure elasticsearch is running correctly15:13
financeMannpbourke have a guide on that?15:13
pbourkefinanceMann: it should be deployed along side kibana15:14
pbourkefinanceMann: but maybe the container is giving errors?15:14
pbourkefinanceMann: you also may need to reconfigure your existing deploy if you deployed without central logging and then turned it on after the fact15:14
financeMannI turned that on15:15
*** lucasxu has joined #openstack-kolla15:15
financeMannand then did a deploy15:15
financeManndo i need to do a redeploy or some other command15:15
pbourketry 'kolla-ansible reconfigure'15:16
sdakefinanceMann good to hear your reading the docs now :)15:16
pbourkefinanceMann: I think that will fix it15:16
pbourkefinanceMann: haproxy needs to know about it15:16
sdakefinanceMann such as they are - we are working continuously to improve them15:17
financeMannya theres just a lot and finding the right guide is usually the hard part but i’m getting better at understand the global architecture so it’s easier15:18
*** Margin has quit IRC15:19
financeManni keep having weird no available host errors on deploy so i figured centralized logging would make it easier and not spend so much time tryin to figure it out15:20
*** rhallisey has joined #openstack-kolla15:21
*** tovin07_ has joined #openstack-kolla15:24
*** fooliouno has quit IRC15:25
*** lamt has quit IRC15:27
*** sacharya has joined #openstack-kolla15:27
*** sacharya has quit IRC15:27
*** sacharya has joined #openstack-kolla15:28
*** nathharp has joined #openstack-kolla15:28
financeMannwhen i try to deploy an instance now I am getting these errors only http://imgur.com/a/iDqSV15:29
financeMannand I cant seem to find anything more descriptive15:29
*** lamt has joined #openstack-kolla15:30
hrwguys: how to debug why kolla hangs?15:30
*** denaitre has joined #openstack-kolla15:32
*** satyar has quit IRC15:35
*** rwallner has quit IRC15:37
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: WIP Use versionless keystone endpoints  https://review.openstack.org/43902415:38
*** jgriffith has quit IRC15:39
*** jgriffith has joined #openstack-kolla15:39
*** skramaja|afk is now known as skramaja15:39
*** retr0h has quit IRC15:41
openstackgerritMerged openstack/kolla stable/newton: Bump nova release to 14.0.4 tag  https://review.openstack.org/43890315:42
*** adrian_otto has joined #openstack-kolla15:42
*** rwallner has joined #openstack-kolla15:43
*** ediardo has quit IRC15:44
*** bachp has quit IRC15:45
*** ediardo has joined #openstack-kolla15:46
*** bachp has joined #openstack-kolla15:49
*** tonanhngo has joined #openstack-kolla15:49
*** retr0h has joined #openstack-kolla15:49
openstackgerritMerged openstack/kolla-ansible master: Remove Heka yaml file  https://review.openstack.org/43834615:52
*** sayantan_ has joined #openstack-kolla15:53
*** mnasiadka has quit IRC15:53
*** unicell has joined #openstack-kolla15:54
*** unicell1 has quit IRC15:55
*** unicell has quit IRC15:56
*** sdake has quit IRC15:58
*** wirehead_ has quit IRC15:59
*** tvignaud has quit IRC16:00
*** salv-orlando has joined #openstack-kolla16:01
*** retr0h has quit IRC16:03
*** wirehead_ has joined #openstack-kolla16:05
manheim_i am running is some issues when trying to install a pip package in neutrons Dockerfile16:06
manheim_it keeps saying diretory not empty upon install16:06
*** vhosakot has joined #openstack-kolla16:07
manheim_it's a docker issue but maybe you've runned into it16:07
*** sdake has joined #openstack-kolla16:08
*** tvignaud has joined #openstack-kolla16:12
*** yingjun has quit IRC16:14
*** srwilkers has quit IRC16:16
*** srwilkers has joined #openstack-kolla16:20
openstackgerritMerged openstack/kolla-ansible master: Add the missing "ironic_inspector" into precheks  https://review.openstack.org/43823316:20
*** retr0h has joined #openstack-kolla16:21
*** vhosakot has quit IRC16:23
*** vhosakot has joined #openstack-kolla16:24
*** fooliouno has joined #openstack-kolla16:24
kfox1111morning16:25
*** tovin07_ has left #openstack-kolla16:28
*** salv-orlando has quit IRC16:32
*** bachp has quit IRC16:32
*** ipsecguy_ has joined #openstack-kolla16:33
*** ipsecguy has quit IRC16:33
manheim_lol, acutally looks like kolla-build --template-override does nothing in my case16:33
*** pcaruana has quit IRC16:34
*** vhosakot has quit IRC16:36
*** vhosakot has joined #openstack-kolla16:37
pbourkemanheim_: you need to supply a file16:38
manheim_i did16:38
*** bachp has joined #openstack-kolla16:38
manheim_ kolla-build --template-override template-overrides.j2 neutron-server16:39
manheim_and ofc template-overrides.js contained what i need16:39
manheim_after i saw it was not installing what i needed, i tested by writing garbage into that file so i can see it how it fails16:40
manheim_but it does not fail16:40
manheim_which tells me template override does nothing16:40
*** Jeffrey4l has quit IRC16:41
*** Jeffrey4l has joined #openstack-kolla16:42
*** vhosakot has quit IRC16:43
*** salv-orlando has joined #openstack-kolla16:44
des_consoladohttps://docs.openstack.org/developer/kolla-ansible/bifrost.html#create-servers-yml - anyone know where this uuid is supposed to come from?16:44
*** alop has joined #openstack-kolla16:47
sdakejascott1 ping re kubernetes 1.5 changes as it relates to helm16:48
sdakekfox1111 perhaps you can answer too - good morning as well :)16:49
kfox1111sure. whats up?16:49
sdakeI know there was a big change in 1.5 related to somethign going out of beta and into non-beta, requiring kubernetes 1.5 in the helm charts16:49
kfox1111oh, probably.16:50
sdakeit was one of the fundamental kubernetes objects16:50
sdakethe one that was renamed16:50
kfox1111oh. statefulsets.16:50
sdakethanks!16:50
kfox1111used to be called petsets.16:50
kfox1111np.16:50
*** janem has joined #openstack-kolla16:52
*** sbezverk has joined #openstack-kolla16:52
sdakekfox1111 yup i recall it was called petsets16:53
sdakeI just didn't know what it was renamed to :)16:53
kfox1111I kind of liked the term petset. but, whatever. :)16:53
*** igordcar1 has quit IRC16:54
*** igordcard has joined #openstack-kolla16:54
sbezverkgood morning16:56
kfox1111morning16:57
nathharpHi - hoping to get some help.   Getting to grips with Kolla-ansible, and deployed an all-in-one16:58
sbezverksdake: after all at what time we have openstack-helm meeting?16:58
sdakesbezverk 18:3016:58
sdakeUTC16:58
nathharpattempting to create an instance, I get “libvirtError: Activation of org.freedesktop.machine1 timed out” buried in the nova compute logs16:58
nathharpany ideas?16:59
egonzaleznathharp, selinux?16:59
*** mgoddard has quit IRC16:59
nathharpdisabled on host17:00
sbezverksdake: in 2.5 hours right?17:00
*** zioproto has quit IRC17:00
*** sayanta__ has joined #openstack-kolla17:00
*** sayantan_ has quit IRC17:00
sdake1.5 hours17:01
*** gfhellma has joined #openstack-kolla17:01
sdakei sent an update on the thread relating to the mailing list17:01
sbezverksdake: thanks a lot for confirming17:01
sdakeI'd recommend folks filter on [deployment] as that is how communication will be handled in the future related to general cross-project efforts related to deployment17:02
*** gfhellma has quit IRC17:02
*** gfhellma has joined #openstack-kolla17:02
*** janem has quit IRC17:03
*** sdake_ has joined #openstack-kolla17:03
*** crushil has quit IRC17:05
*** sdake has quit IRC17:07
nathharpegonzalez, any other ideas?17:07
openstackgerritSayantani Goswami proposed openstack/kolla-ansible master: Update Multinode deployment of kolla guide  https://review.openstack.org/42685217:07
*** rwallner has quit IRC17:08
sdake_portdirect pingola17:09
*** janem has joined #openstack-kolla17:10
*** vcn[m] has quit IRC17:11
*** bachp has quit IRC17:11
*** retr0h has quit IRC17:11
*** jtriley has quit IRC17:14
portdirectpong-a-tron17:14
*** egonzalez has quit IRC17:15
*** manheim_ has quit IRC17:19
*** _trey has joined #openstack-kolla17:21
openstackgerritMerged openstack/kolla master: Improve kolla image build script for finding parent images  https://review.openstack.org/43678117:23
*** manheim_ has joined #openstack-kolla17:23
*** sdake_ is now known as sdake17:24
*** Serlex has quit IRC17:26
*** caowei has quit IRC17:30
*** caowei has joined #openstack-kolla17:30
*** manheim_ has quit IRC17:31
*** bpattewar has joined #openstack-kolla17:32
financeMannno valid hosts found, could that be a firewall issue and if so would systemctl stop firewalld fix it as a test17:33
qwangsdake: The meeting time is 19:30 UTC from ML. Is it changed again?17:37
*** mgoddard has joined #openstack-kolla17:38
mgoddardpbourke, Jeffrey4l: when you get a chance could you refresh your +2s on https://review.openstack.org/#/c/435348/ as it was rebased? thanks!17:38
mgoddarddes_consolado: that uuid is for the ironic node. Ironic should autogenerate one if omitted: https://github.com/openstack/bifrost/blob/master/playbooks/roles/ironic-enroll-dynamic/tasks/main.yml#L4517:39
*** sudipto_ has quit IRC17:39
*** sudipto has quit IRC17:39
*** strigazi has quit IRC17:39
*** strigazi has joined #openstack-kolla17:40
sdakeqwang indeed, chris made an error in his original post and corrected it on the ml17:40
sdakemoment i'll find the link17:40
*** nathharp has quit IRC17:41
sdakehttp://lists.openstack.org/pipermail/openstack-dev/2017-February/113019.html17:41
qwangsdake: thanks a lot17:41
sdakeqwang np ;)17:44
*** rwallner has joined #openstack-kolla17:44
*** vcn[m] has joined #openstack-kolla17:46
*** matrohon has quit IRC17:47
*** sdake_ has joined #openstack-kolla17:48
*** rwallner has quit IRC17:49
*** sdake has quit IRC17:50
*** jtriley has joined #openstack-kolla17:52
*** crushil has joined #openstack-kolla17:54
*** rwallner has joined #openstack-kolla17:56
*** jtriley has quit IRC17:56
*** financeMann has quit IRC17:57
*** rwallner_ has joined #openstack-kolla17:57
*** rwallner has quit IRC18:01
*** sdake_ has quit IRC18:02
*** rwallner_ has quit IRC18:02
inc0good morning18:05
*** sdake has joined #openstack-kolla18:05
inc0how is my favorite community doing?18:06
*** jtriley has joined #openstack-kolla18:08
des_consoladoah ty mgoddard18:08
mgoddarddes_consolado: np18:08
*** rwellum has quit IRC18:10
inc0how are we doing in release departament?18:11
inc0any lingering bugs we need to fix before next week?18:11
inc0(release is next week)18:11
sdakeinc0 Jeffrey4l working on the docker-py and docker workaroudn thing18:13
sdakethat seems esseential although i'd let jeffrey4l comment on that as i am only peripherally involved18:13
inc0ah good that you reminded me, I'll test it out after helm meeting18:16
*** jtriley has quit IRC18:17
*** salv-orl_ has joined #openstack-kolla18:17
*** financeMann has joined #openstack-kolla18:19
financeMannDo anyone know what “No valid host was found. There are not enough hosts available.” could elude to? Everything resource wise looks good, my little computer in my bathroom hosting this has 256 GB of ram, 1TB SSD, and 64 CORES18:19
financeMannIs it just a non-descript generic error that could mean anything?18:19
*** salv-orlando has quit IRC18:20
*** sdake has quit IRC18:21
*** mgoddard has quit IRC18:23
*** sdake has joined #openstack-kolla18:23
*** gfidente is now known as gfidente|afk18:24
*** unicell has joined #openstack-kolla18:25
sdakeinc0 can you login to the helm on openstack meeting?  getting an invalid meeting id msg18:30
sdakeor no meeting at all18:30
sdakeoropenstack on helm18:30
sdakeor whatever this shindig is called18:30
inc0link seems broken18:30
sdakeboth links seem broken18:30
inc0but if you copy meeting id and input it into zoom app18:31
inc0that works18:31
inc0hogepodge: ^18:31
*** jtriley has joined #openstack-kolla18:31
*** krtaylor has quit IRC18:32
sdakeinc0 where abouts do you put it into the application?18:32
*** gfhellma_ has joined #openstack-kolla18:33
inc0sdake: when I ran app without any link it just asked me for it18:33
*** sambetts is now known as sambetts|afk18:33
*** lucasxu has quit IRC18:34
sdakeinc0 which meeting id shoudl I enter -the one on the ml is invalid18:35
*** gfhellma has quit IRC18:35
inc0just copy numbers18:35
inc0and use just numbers18:35
inc0668276583 this one works18:35
sdakeInvalid meeting ID.18:35
sdake668 276 58318:35
sdaketoosl ftw ;)18:36
sdakeok got it :)18:36
*** retr0h has joined #openstack-kolla18:37
*** bachp has joined #openstack-kolla18:37
*** eaguilar has joined #openstack-kolla18:37
*** jtriley has quit IRC18:38
*** thumpba has quit IRC18:40
*** salv-orl_ has quit IRC18:45
*** jtriley has joined #openstack-kolla18:52
financeMannim convinced my issue with with floating ips18:52
*** eaguilar has quit IRC18:53
des_consoladohmmm I've turned threads down to 1 in kolla-build.conf and I've stuck import pdb; pdb.set_trace() in the middle of build.py18:53
des_consoladothis doesn't work when running the thing though? :S18:53
des_consoladoI'm running it like: sudo python tools/build.py bifrost-deploy --push18:54
*** gfhellma__ has joined #openstack-kolla18:54
des_consoladothe kolla-build.conf is definitely being used, as I set debug in there and that's working...18:54
inc0des_consolado: are you saying someone merged import pdb?;)18:54
des_consoladono I just added that myself18:54
inc0ahh you got me worried18:55
*** manheim_ has joined #openstack-kolla18:55
des_consoladothe kolla-build binary that I've got installed with pip works fine, the tools/build.py hangs at trying to base itself on ubuntu:16.04 right now though18:56
*** manheim_ has quit IRC18:56
*** papacz has quit IRC18:56
des_consoladowhy the frig doesn't pdb work though gah18:58
*** gfhellma_ has quit IRC18:58
des_consoladoopenstack is like an abusive husband who keeps beating the shit out of me18:59
*** gfhellma__ has quit IRC19:00
inc0des_consolado: that is pretty accurate19:01
des_consoladoeven if I put import pdb and pdb.set_trace() in the actual tools/build.py it doesn't work19:02
des_consoladohow is this not just like... how even does it just fucking ignore the fucking code I'm putting in the fucking code lollll19:02
inc0check which build.py is actually uised19:02
des_consoladoI'm literally running 'python tools/build.py'19:02
inc0hmm19:03
des_consoladomust be something with my import? if I do python -m pdb tools/build.py that works19:03
inc0python can be moody19:04
inc0hard to tell rly19:04
financeMannif I want to run “openstack network create net1” I am getting openstack command not found19:05
financeMannwhere is that located19:05
financeManni need to create these networks manually i think19:05
*** jtriley has quit IRC19:06
*** lucasxu has joined #openstack-kolla19:06
*** sayanta__ has quit IRC19:07
*** jtriley has joined #openstack-kolla19:07
*** lucasxu has quit IRC19:08
des_consoladooh wtf19:09
des_consoladorunning tools/build.py still ends up calling like: /usr/local/lib/python2.7/dist-packages/kolla/image/build.py(986)run_build()19:10
inc0yeah I figured19:10
des_consoladoyeah because it imports kolla... guess I need to do pip install -U . or something19:10
*** thumpba has joined #openstack-kolla19:10
inc0check yo pythonpath19:10
*** thumpba has quit IRC19:10
*** salv-orlando has joined #openstack-kolla19:12
*** fooliouno has quit IRC19:12
*** krtaylor has joined #openstack-kolla19:15
*** krtaylor has quit IRC19:18
*** fooliouno has joined #openstack-kolla19:18
*** krtaylor has joined #openstack-kolla19:19
*** sdake has quit IRC19:19
*** krtaylor has quit IRC19:23
*** sdake has joined #openstack-kolla19:28
*** krtaylor has joined #openstack-kolla19:36
*** shardy has quit IRC19:38
*** krtaylor has quit IRC19:39
*** krtaylor has joined #openstack-kolla19:40
jemcevoySamYaple: Reinstalled from kickstart and flat VLAN network for the VMs which get the VM an MTU of 1500... I must be missing something basic with my configs... I need to get an MTU of 9000 to the VMs.  This is my first foray with neutron... We have Folsom with nova network in production...19:40
jemcevoymy ml2_config.ini http://paste.openstack.org/show/600687/19:41
SamYaplejemcevoy: do you still have that wierd patch config in ovs?19:42
*** nathharp has joined #openstack-kolla19:42
*** krtaylor has quit IRC19:42
SamYaplebecause the way the neutron code works, it will detect that with its hackish PMTU code (the way i understand it)19:42
*** krtaylor has joined #openstack-kolla19:44
jemcevoyHow to I change neutron to use my my bridge rather than br-ex?19:44
*** lucasxu has joined #openstack-kolla19:45
*** skramaja has quit IRC19:46
*** krtaylor has quit IRC19:46
*** krtaylor has joined #openstack-kolla19:48
*** _trey has quit IRC19:49
*** krtaylor has quit IRC19:50
*** looking4help has joined #openstack-kolla19:51
looking4helpis there a way to create an NFS filesystem for Kolla containers?19:54
financeManni am trying to deploy an instance and keep getting error saying “No hosts available” I dig into it and I see the following error “Netlink: filter function error”19:54
financeMannany ideas SamYaple19:54
jemcevoySamYaple:  Do I just need to change this entry [ovs]19:54
jemcevoybridge_mappings = physnet1:br-ex to be [ovs]19:54
jemcevoybridge_mappings = physnet1:br-ex,physnet2:br-uplink19:54
jemcevoyand remove the ovs patches19:55
*** manheim_ has joined #openstack-kolla19:58
jemcevoyThen I can use br-ex bridge for access to the 1 gig network at 150019:59
SamYaplejemcevoy: based on your previous config, physnet1:br-10g (whatever that was)19:59
SamYaplebr-ex need not exist19:59
jemcevoyThat config has been kickstarted to oblivion19:59
jemcevoyHow do I stop kolla from creating it?19:59
*** rwellum has joined #openstack-kolla20:05
jemcevoySamYaple: new ml2_conf.ini  ->  http://paste.openstack.org/show/600837/20:06
*** sdake has quit IRC20:07
*** sdake has joined #openstack-kolla20:08
*** janem has quit IRC20:10
*** sayantan_ has joined #openstack-kolla20:11
SamYaplejemcevoy: neutron_bridge_name20:12
SamYapleset it to br-10g20:12
SamYapleit defaults to br-ex20:12
jemcevoyThanks SamYaple20:12
*** sdake has quit IRC20:13
*** mrhillsman has quit IRC20:13
*** sdake has joined #openstack-kolla20:14
*** athomas has quit IRC20:15
*** harlowja has quit IRC20:15
*** harlowja has joined #openstack-kolla20:16
*** adrian_otto has quit IRC20:16
openstackgerritBalaji G Pattewar proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725220:18
financeMannyo SamYaple any idea why i might be getting the follow on trying to get an instance running.  “Netlink: filter function error”20:18
*** jtriley_ has joined #openstack-kolla20:19
SamYaplefinanceMann: kernel issues? off the top of my head20:19
financeMannwhat would cause that? I was jsut deploying instances the other day just fine20:19
Mannheomis there an issue with the docker images from dockerhub? i cannot build from centos720:19
SamYaplefinanceMann: what distro and kernel are you using?20:20
financeManncentos 720:20
financeMannit was workin the other day20:20
*** mrhillsman has joined #openstack-kolla20:21
*** jtriley has quit IRC20:21
SamYaplei blame centos20:21
SamYaplealso, i do not know because i dont use ancient kernels and backport new things just to retain the kernel version number20:22
SamYaplealso, sdake ^^ Netlink issue in centos again20:22
*** manheim_ has quit IRC20:23
fooliounoHello .. I am trying to bring up kolla-kubernetes for the first time and am seeing the following error. Could someone please suggest how to resolve these errors:20:23
fooliounoroot@server6612:~/kolla-kubernetes/tools# ./kolla-kubernetes bootstrap mariadb Error from server (AlreadyExists): configmaps "mariadb" already exists Error from server (AlreadyExists): error when creating "STDIN": persistentvolumes "mariadb" already exists Error from server (AlreadyExists): error when creating "STDIN": persistentvolumeclaims "mariadb" already exists Error from server (AlreadyExists): error when creating "ST20:23
sdakeSamYaple no idea either - best bet is to contact the centos irc channel20:26
sdakeSamYaple although I do know that tripleo has run into this problem20:26
sdakeSamYaple if we all got together in #openstack-deployment we could solve these issues together20:26
SamYaplei think this is docker specific like all the other netlink issues have been20:26
sdakethe same error happens in tripleo so i disagree ;)20:27
sdakehere:20:30
sdakehttps://bugs.launchpad.net/tripleo/+bug/136774220:30
openstackLaunchpad bug 1367742 in tripleo "keepalived brings the VIP up in multiple places on a controller reboot" [High,Expired]20:30
sdakeand here:20:30
sdakehttps://sourceforge.net/p/keepalived/mailman/message/25658273/20:30
sdakefinanceMann so if you are rebooting, essentially keepalived is broken20:32
*** jmontleon has quit IRC20:32
sdakefinanceMann are you rebooting?20:32
financeMannwhat do you mean rebooting, i just rebooted to see if that’d help :)20:32
*** salv-orlando has quit IRC20:32
sdakeright - that breaks keepalived - read the bug log from tripleo20:32
sdakeI think there are two problems here. One's the keepalived race (if you SIGHUP keepalived on the rebooted host, it recovers correctly). The second is that keepalived should probably be configured with a quorum greater than the default 1 for this situation, which would prevent a split-brain keepalive from hosing the working half of the cluster.20:33
sdakeright from the bug log20:33
*** jmontleon has joined #openstack-kolla20:33
financeMannjust reload sdake SIGHUP ?20:34
sdakeno idea what the workaround is20:35
sdakeno idea what the root cause is20:35
SamYaplewell the root cause is that centos kernel!20:35
sdakethe tripleo cats seemed to analyze it to some problem with keeplaived20:35
SamYaple:P20:35
financeMannit says to reload SIGHUP20:35
sdakeSamYaple it also fails on 14.04.0120:35
financeMann“You don't have to restart Keepalived to recover in any case.  A reload20:35
financeMann(SIGHUP) will do.”20:35
financeMannWhat is that even, just a service?20:36
SamYaplenot with the proper kernl (ie, not the 3.13 kernel)20:36
financeMannsudo service SIGHUP restart?20:36
sdakefinanceMann did you yum update your server?20:36
financeManni do not believe so but maybe something else did perhaps20:36
financeManni could have20:36
sdakefinanceMann run yum -y update20:36
sdakefrom the cli20:36
financeMannupdating20:37
sdakesee if that changes things for the better20:37
sdakeor if it does nothing20:37
sdakefinanceMann you will need to yet again reboot after the yum update20:37
*** Mannheom has quit IRC20:37
*** Manheim has quit IRC20:37
sdakethat should go in our docs and is implemented in the host playbook iirc20:38
*** Manheim has joined #openstack-kolla20:38
*** Mannheom has joined #openstack-kolla20:38
*** tonanhngo has quit IRC20:42
*** jmontleon has quit IRC20:42
sdakehey peeps20:43
sdakethe internet is on fire today because S3 is down20:43
sdakeeven amazon's status page reports everything in good working order because s3 reports s3 is in good working order20:43
sdakewhen in fact, it is not20:43
sdakethe one thing that does work in openstack infra ;-)20:44
sdakealthough kolla-kubernetes depends on dockerhub, which depends on s3, which is busted20:45
*** gfidente|afk is now known as gfidente20:45
bmacemaybe some day people will actually learn the whole all eggs in one basket thing is bad, but probably not.. there have been other similar cloud failure in the past that bring sites down but they stick with aws only.20:46
*** sdake_ has joined #openstack-kolla20:46
*** jmontleon has joined #openstack-kolla20:47
financeMannsdake i updated, rebooted, now deploying20:47
financeMannand crossing fingers20:47
*** lamt has quit IRC20:47
*** athomas has joined #openstack-kolla20:47
bmacei bet the crossing fingers thing will make the difference :)20:48
*** lamt has joined #openstack-kolla20:48
*** sdake has quit IRC20:50
*** nathharp has quit IRC20:50
financeMannsdake_ new error [req-84a22816-3c0f-4719-afb1-d3ca715ce3aa 62d2ea98dead48a29a9c7c60ee54829b 1cb44e090edf4a4dbe19dd4545a923a7 - - -] Could not load neutron.agent.linux.interface.OVSInterfaceDriver20:55
sdake_financeMann - s3 is down: http://venturebeat.com/2017/02/28/aws-is-investigating-s3-issues-affecting-quora-slack-trello/20:55
financeMann“libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.”20:55
sdake_financeMann you need see to be up for any of this stuff to work20:55
sdake_see/s320:56
financeMannserious20:56
financeManns320:56
financeMannso go make a sandwich or something?20:56
sdake_kolla does work airgapped20:56
sdake_it does require some knowledge to get it to work in that mode which is not documented20:56
financeMannany eta?20:56
financeMannwe talkin hour, hours?20:56
financeManndays20:56
financeMannmonths years20:56
sdake_financeMann I am not amzn20:56
sdake_if it is more then a week, all society will collapse ;)20:57
financeMannis that why i couldnt view my orders on amazon eariler?20:57
sdake_my guess is yes, although I am not amzn tech support ;-)20:57
sdake_read the article20:57
*** kong has joined #openstack-kolla20:58
sdake_the lesson here is everyone depends on AWS for all kinds of stuff20:58
sdake_and that web of depndencies creates problems if aws fails20:58
sdake_this is why dependency selection is such an important choice20:58
sdake_and really highlights the value of openstack ;-)20:58
sdake_on that note, i'm going to bed, since I can't get any work done at all20:59
financeMannk20:59
inc0from the looks of it it's getting better21:00
*** TxGirlGeek has quit IRC21:00
*** TxGirlGeek has joined #openstack-kolla21:01
* portdirect reads scrollback21:02
* portdirect is taking bets on the number of blogs written about the aws status page using s3 to help monitor s3...21:03
*** TxGirlGeek has quit IRC21:09
*** TxGirlGeek has joined #openstack-kolla21:09
*** TxGirlGeek has quit IRC21:11
*** adrian_otto has joined #openstack-kolla21:13
*** dave-mccowan has quit IRC21:13
financeMannso your tellin me inc0 that i cant deploy anything now21:15
financeMannuntil amazon fixes their computers21:15
SamYaplefinanceMann: you can build it yourself im sure?21:18
*** TxGirlGeek has joined #openstack-kolla21:18
financeMannI dont need to build nothin, Im running it21:18
financeMannand just tryin to spin up an instance from the GUI21:19
financeMannSamYaple -> “Could not load neutron.agent.linux.interface.OVSInterfaceDriver”21:19
financeMann“libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."21:19
*** rwallner has joined #openstack-kolla21:21
*** TxGirlGeek has quit IRC21:22
*** crushil has quit IRC21:24
*** janem has joined #openstack-kolla21:27
Manheimimage building works now :D21:27
financeManndeploy works?21:29
financeManninstance creation that is21:30
*** TxGirlGeek has joined #openstack-kolla21:31
inc0image building works for sure21:32
inc0(just built)21:32
financeManncan you spin up an instance or is amazon break that too21:33
*** TxGirlGeek has quit IRC21:34
*** jtriley_ has quit IRC21:34
inc0financeMann: spinning up instance shouldnt be affected by amazon...21:35
*** TxGirlGeek has joined #openstack-kolla21:35
*** Jeffrey4l has quit IRC21:35
*** srwilkers has quit IRC21:35
*** crushil has joined #openstack-kolla21:38
*** salv-orlando has joined #openstack-kolla21:39
*** jrobinson has joined #openstack-kolla21:39
*** TxGirlGeek has quit IRC21:40
Manheimi always get ":OSError: [Errno 39] Directory not empty" when i try to install a package in from dockefile, kolla-build with override does not work21:41
*** sdake_ has quit IRC21:45
*** gfidente is now known as gfidente|afk21:46
*** rwellum has quit IRC21:46
*** lucasxu has quit IRC21:46
financeManninc0 then any idea whats causing my errors21:47
financeMann“Could not load neutron.agent.linux.interface.OVSInterfaceDriver”21:47
financeMann“libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."21:47
*** rwallner has quit IRC21:47
*** Jeffrey4l has joined #openstack-kolla21:48
*** unicell has quit IRC21:49
*** lucasxu has joined #openstack-kolla21:49
*** rwallner has joined #openstack-kolla21:51
*** unicell has joined #openstack-kolla21:53
*** athomas has quit IRC21:54
*** rwallner has quit IRC21:55
inc0financeMann: running centos?21:56
*** tonanhngo has joined #openstack-kolla22:01
*** TxGirlGeek has joined #openstack-kolla22:02
*** krtaylor has joined #openstack-kolla22:08
*** tonanhngo has quit IRC22:13
inc0kfox1111: ping22:15
kfox1111hi22:17
kfox1111meeting next in 13 min.22:17
kfox1111whats up?22:18
inc0so any chances I could borrow some of your time later this week?22:18
kfox1111oh, probably.22:18
inc0I have k8s and ceph installed, let's work together to document getting this to kolla-k8s22:18
kfox1111+122:18
inc0which day would be best for you?22:19
inc0I'd get jascott1 and qwang in a room too22:19
kfox1111thursday afternoon?22:19
inc0yeah that works22:19
kfox1111k.22:20
inc0thanks man22:20
kfox1111np. :)22:20
inc0also I tried to clean up our images22:20
SamYaplejascott1 is best at writing docuementation. he told me to tell you to assign it to him. he wants to do documentation and not coding22:20
SamYapleits true he said it22:20
kfox1111I think SamYaple just volunteered to write the docs. :)22:20
SamYapleoh you dont want me writing them22:20
SamYaplehave you seen the docs i wrote for openstakc before?22:21
SamYaplethats right. you havent. because i didnt22:21
inc0so Sam I ran your keystone yaodu image;)22:21
SamYapledid it blow up like it should?22:21
inc0it's same size as kolla image if you remove cleanups22:21
inc0so what we should do to lower size of images is to introduce cleanups like yours22:22
SamYapleexactly! but you can't with multilayer22:22
inc0well, it's same shit22:22
inc0if you remove apt cache in every layer22:22
SamYapleif you say so!22:22
inc0however I have few ideas how to mitigate it22:22
inc0we'll see22:23
SamYaplei wrote them to solve cicd, not to make them smaller. smaller was a by product22:23
inc0yeah I get that22:23
SamYapleim not _as_ concerned with image size in kolla as I am with layering22:23
inc0and it's not really solved by not layering anyway, just cleanups are much easier22:23
inc0what I was thinking is to make use of macros and stuff like that to install things just in xx-base22:24
inc0like nova-base22:24
inc0we could solve layering-ish things with jinja22:25
inc0so we can have both single layer and dry22:25
*** TxGirlGeek has quit IRC22:26
inc0in fact, if we do this we could also keep full backwards compatibility22:26
SamYaplehey man. if you can get there all the better22:26
SamYaplei think kolla is a year away from anything resembling yaodu due to compatibility22:26
inc0instead of FROM openstack-base do {% include ../openstack-base %}22:26
inc0all the {% block %} stuff keeps working22:27
inc0and we can do proper cleanups at the end of each image22:27
*** jbryce has joined #openstack-kolla22:27
inc0I wonder if docker will understand that RUN-style layers are also shared by hash22:27
inc0to deduplicate in that front22:28
*** sayantan_ has quit IRC22:28
inc0then we're in really good place22:28
SamYaplefrankly. i don't think kolla can get to single layer22:28
inc0why?22:28
SamYapleyou trade one list of pros and cons for another22:28
portdirectyou could use docker --squash, but I agree with SamYaple getting to a single layer would be virtually impossible with the current layout of kolla22:28
SamYapleif you get single layer the way you are talking, well now keystone goes from 20m to build, to 15m. and totaly time goes from 45m to build.... to 3 hours22:29
portdirectit would be easier to migrate to somthing like ansible-container and build with that22:29
inc0I disagree really, question is how big layer will be22:29
inc0again, just replace FROM with jinja2 {% include %}22:29
SamYaplei hear your disagreement. i accept it. and I would love to be proved wrong22:29
inc0and you have one huge dockerfile, but single22:29
SamYaplebut for as long as i have been working on this issue I have yet to come up with a solution that is compatible with Kolla22:30
inc0well you can do really crazy shit with jinja222:30
SamYapleyou say that like a good thing :P22:31
portdirect^^22:31
inc0hey, all you're doing is shellscript22:31
SamYaplei have no doubt you can make this work inc022:31
SamYaplebut im not sure its the right path for kolla. and im not sure its worth it22:31
portdirectbut its far from crazy ;) but echoing sam above if we could get kolla comperable that would be fantasic22:31
inc0it's good to experiment and see where it gets us22:31
harlowjasomething i would like, drop the stupid dockerfile format entirely imho, lol22:32
SamYaplehow are you solving the build-tools across layers issue inc0?22:32
SamYapleharlowja: ansible-container is for you!22:32
portdirectharlowja: thats one of the things yaodu does22:32
inc0harlowja: and replace it with what? I mean dockerfile is just that - a glorified shellscript22:32
harlowjayoda?22:32
SamYapleportdirect: no it doesnt22:32
portdirectunless you mean dockerfiles in their entirety22:32
inc0portdirect: it's RUN xx && yy && ....22:32
harlowjai dislike the format, and it seems like the hybrid that kolla has is pretty far away from a dockerfile anyway :)22:32
*** eaguilar has joined #openstack-kolla22:32
inc0that's not droping the format;)22:33
SamYapleharlowja: nah yaodu is just a side project i did to attempt to make the images more cicd compatible22:33
harlowjau should of named it yoda22:33
harlowjalol22:33
harlowjawhere is the code for that?22:33
inc0yet another openstack deployment utuluty22:33
SamYapleharlowja: its not dropping dockerfiles. this conversation is just about how to bring those improvements to kolla22:33
portdirectsorry i thought harlowja was mening the half-dockerfile templates that kolla uses22:33
SamYapleinc0: that name is deprecated!22:33
SamYapleinc0: now its just yaodu. it stands for nothing!22:33
financeManninc0 yea centos22:34
financeMann722:34
inc0financeMann: turn off selinux22:34
SamYapleharlowja: https://github.com/yaodu -- if you want to look at it and figure out how to make kolla do a simliar thing, that would be great22:34
inc0selinux is causing this issue22:34
harlowjai mean, one of the thing that's always concerned me, is the crazy ability to templatize those kolla dockerfiles22:34
SamYapleharlowja: so far, i havent figureout out how to do this in a kolla comp way :(22:34
harlowjathe files are so templatized, that it imho approaches fragility (in ability to change things)22:34
inc0harlowja: agree, but that's the cost of multi distro and multi-type we do22:35
inc0not to mention overrides22:35
inc0it's always tradeoff - flexibility vs stability22:35
harlowjaya, no i  get that, nothing is for free22:35
harlowjathe thing though i would personally like though (and it may just be me)22:35
SamYapleyep. and thats what happening here. the yaodu images are rock solid and stable. but they are not flexible enough to do, say, a build with a redhat license key22:36
SamYapleor "binary" distro support22:36
SamYapleand they never will be22:36
harlowjab = DockerFile().author("bob").\22:36
harlowja    run('abc').\22:36
harlowja    run('abc')22:36
harlowjat = b.write()22:36
inc0ansible container is actually neat idea if we add "remove ansible" at the end22:36
harlowjasomething like that kind of interface to docker files, where u don't care about the underlying crap format (imho its a bad format)22:37
inc0well, you're trading bastardized shell for python22:37
inc0which is cool22:37
inc0depends how you look at it22:37
inc0total lack of flexibility really bothers me22:38
inc0so SamYaple if you remember reason we even added jinja, part of it, was to DRY-out containers22:38
SamYapleoh i remember.22:38
harlowjaat some point, i'd expect kolla to coalesce on less flexibility anyway, wouldn't u?22:38
inc0so you won't have x files which are 90% identical22:38
harlowjai mean, at some point u'd expect people to have pretty much the same files22:38
inc0oh totally, like getting rid of infra containers22:39
inc0just docker is still young project in this field, we still have lengths to go22:39
inc0and rhel containers ... I'd expect Red Hat to have more interest in this field22:39
SamYapleyea inc0. ive just shifted a different direction with how to build containers. one that resembles official builds rather than one that needs external tooling22:40
inc0like...enable rhel in container would give them whole new market22:40
SamYapleit has its downsides, but at least i _know_ others are doing it22:40
inc0but it's going to be less and less scalable in terms of big tent and such22:41
inc0scalability of contributions is also important if project is supposed to succeed22:41
SamYaplewhat is less scalable?22:42
inc0let say you find the uid - like issue22:42
inc0which would require you to change *all* of containers22:42
inc0for us it was just base22:42
inc0for you it's every flavor of every service22:43
SamYaplestatic uid is being used for users22:43
SamYaplesame uid22:43
SamYaplethis matches other image builds22:44
inc0but you need to change it in every container22:44
inc0that's what I'm saying22:44
SamYaplewhy would i need to change it?22:44
SamYapleim confused22:44
inc0also really it's less secure than what we do22:44
portdirectthis is my question to?22:44
inc0add the line to every container to pin the pid22:44
SamYapleif you say so22:44
inc0uid*22:44
SamYapleit was an intentional decision for what its worth22:45
portdirectinc0: these are not ment to be an attack on what you are doing at all, just an alternative implementation that addresses some needs that kolla does not22:45
inc0I don't feel attacked22:45
inc0just saying that each have issues, and issues like that might become apparent only if yaodu community grows22:46
portdirectcool - sorry if I misread things. I agree with ^^ 100%.22:46
inc0remember, it's different to keep tidy project and design with 2 ppl than 20022:46
inc0that's scalability I'm referring to22:46
SamYaplethats not scalability....22:47
*** crushil has quit IRC22:48
inc0find me a better word22:48
SamYaplei think this conversation devolved a bit. the original point was to see how to get yaodu things into kolla22:48
harlowjaSamYaple ya, intersting, the yaodu stuff looks like a path i went down before, lol22:48
harlowjawith automated package building22:48
inc0SamYaple: yeah, I'll play around few ideas22:48
SamYaplei would love to see your patchset for making things smaller inc022:48
harlowjaseperated files, vs different ones :)22:48
harlowjathough https://github.com/yaodu/docker-nova/blob/master/dockerfiles/Dockerfile-centos#L11 is always lol to me22:49
SamYapleharlowja: this is modeled after official docker images. there are quite a few things i dont like about it. but im trying to stick to docker upstream22:49
harlowjathat's what i call some crazy file format :-P22:49
portdirectset -x is my baby :P22:49
inc0yeah for me proper image building is still an open question22:49
*** adrian_otto has quit IRC22:50
portdirectand yeah following dockerfiles run commands is scary22:50
SamYapleharlowja: you may want to look at https://github.com/yaodu/docker-keystone/blob/cleanup/dockerfiles/Dockerfile-debian for an alternative format22:50
inc0once OCI lands in it's final form I expect startups to arise22:50
*** salv-orlando has quit IRC22:50
harlowjaportdirect more or less it'd be fixable with a thing like RUN_TRANS, .... commands .... END_TRANS22:50
SamYapleharlowja: but the one you linked is very much "upstream"22:50
inc0just for image building22:50
*** salv-orlando has joined #openstack-kolla22:50
portdirectharlowja: that would be nice actually22:51
SamYapleharlowja: docker said no to that. permanently22:51
inc0commit I dare you!22:51
inc0also frankly? package manager for containers would be a fun thing22:52
harlowjaSamYaple thats dumb22:52
inc0apt could use some changes for that22:52
inc0harlowja: so here's thing about docker.inc22:52
inc0I think docker.inc is worst thing about docker22:52
inc0and really RKT is a thing only because of that imho22:53
SamYapleharlowja: https://github.com/docker/docker/issues/332 - its a fun read. it discusses (and implements) exactly what you are talking about22:53
harlowjaSamYaple nice nice, reading22:53
SamYapleharlowja: my favorite comments from the thread.. https://github.com/docker/docker/issues/332#issuecomment-221903205 -- those two22:55
inc0right... --squash22:55
harlowjalol22:55
portdirectinc0: re package managers, I'd keep an eye on RH's system containers22:56
SamYapleinc0: nah thats the compromise. its better than nothing, but its not the same thing22:56
inc0SamYaple: I agree22:56
portdirectinc0: I expect them to move yum to being an external tool for those soon22:56
SamYapleinc0: kolla can use that to refactor its building signficantly btw22:56
portdirectand build direct from spec file22:56
inc0yeah22:56
SamYaplesave a good amount of space with --squash22:56
SamYaplethough, you will add a decent amount of time to the final build22:57
portdirect^^ I and sam have prototyped with squash22:57
portdirectthough not dockers22:57
harlowjaSamYaple interesting read though22:57
inc0so build time isn't something I'm that concerned with22:57
harlowjaseems like a basic operation, that should just be built-in22:57
portdirectmy POC broke the gate with timeouts :(22:57
harlowjabut meh, who am i to know :-P22:57
inc0if it's less than 1hr or so22:57
inc0right now building whole set took ma ~15min on my vm22:57
* SamYaple mutters 'it concerns ci/cd....'22:57
inc0agree22:58
portdirectyeah - its often about building a sinlge image, not the whole lot22:58
inc0but you can cicd non-squashed and squash if it passes cicd22:58
inc0then you do deploy pipeline of built images22:58
SamYaplei mean i guess. but the layering still matters22:58
inc0layering is different matter22:58
SamYaplebut i feel like we are going around on this subject again. not saying anything new22:58
inc0yeah, I hear ya22:59
inc0one thing I'm also playing with22:59
SamYapleyou point me to a kolla cleanup PR, and I will do my best to help22:59
inc0thanks SamYaple will do22:59
inc0I need to wrap my head around it but I feel I'm close22:59
*** looking4help has quit IRC23:00
inc0I need a glass of laphroaig and a whiteboard23:00
SamYaplelol. sounds like a fun weekend23:00
inc0who said anything about weekend23:00
SamYaplesounds like a fun tuesday!23:00
inc0there ya go23:01
v1k0d3n+1 cheers!23:01
v1k0d3n:)23:01
inc0I was thinking of it yesterday but then I realized that I'm stuffed with anesthetics from dentist and probably not the best of ideas23:01
v1k0d3ninc0: sorry to hear you were in so much pain man.23:01
v1k0d3nhope you're feeling better.23:01
inc0yeah I do, still feel it23:02
inc0but I hope tomorrow it will be all gone23:02
inc0thanks23:02
portdirectlaphroaig FTW23:02
inc0I have cask strength...23:02
inc057% is it?23:03
portdirectoh gawd23:03
v1k0d3ni will have to try this laphroaig23:03
inc0it's an aquired taste, but yeah, Ila whiskeys are the best23:03
portdirectI'll bring some next time (and try not to leave it at home...) inc0 I will deliver one day!23:03
SamYaplenice. ive got a 120-130 proof straight rye whiskey for my 5th anniversary23:03
SamYaplestrong is good23:04
inc0Islay, sorry23:04
inc0portdirect: I am counting on that23:04
portdirect+1 to all of the above!23:04
inc0don't make me go all the way to scotland! My liver failure will be your fault23:05
* SamYaple reads scrollback23:05
inc0oh I still have lots of islay whiskeys to try23:06
inc0Bruichladdich - never had this one23:06
inc0anyway, I'm moving home23:08
inc0gonna hack around these images23:08
inc0and play some torment tides of numenera...23:09
* inc0 was waiting for this game for 3 years23:09
inc0afk23:10
*** janem has quit IRC23:12
*** lucasxu has quit IRC23:15
*** saneax-_-|AFK is now known as saneax23:18
*** fooliouno has quit IRC23:20
*** crushil has joined #openstack-kolla23:20
*** lamt has quit IRC23:27
*** lamt has joined #openstack-kolla23:30
*** financeMann has quit IRC23:38
*** lamt has quit IRC23:40
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding ironic inspector microservices  https://review.openstack.org/43855623:45
*** sacharya has quit IRC23:47
*** sayantan_ has joined #openstack-kolla23:48
*** sp_ has joined #openstack-kolla23:48
*** sayantan_ has quit IRC23:49
*** eaguilar has quit IRC23:52
*** crushil has quit IRC23:53
*** sayantan_ has joined #openstack-kolla23:55
*** dave-mccowan has joined #openstack-kolla23:57
*** financeMann has joined #openstack-kolla23:59
*** Pavo has joined #openstack-kolla23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!