Wednesday, 2016-10-12

*** tonanhngo_ has quit IRC00:02
*** zhurong has quit IRC00:03
*** williamcaban has quit IRC00:04
*** williamcaban has joined #openstack-kolla00:05
*** liyifeng has quit IRC00:10
*** HyperJohnGraham_ has joined #openstack-kolla00:15
*** sdake_ is now known as sdake00:22
*** ediardo has quit IRC00:23
*** ediardo has joined #openstack-kolla00:23
*** yingjun has joined #openstack-kolla00:25
*** zhubingbing has joined #openstack-kolla00:25
*** sogabe has joined #openstack-kolla00:26
*** zhubingbing has quit IRC00:28
*** tonanhngo has joined #openstack-kolla00:30
*** bachp has quit IRC00:30
*** ssurana has quit IRC00:36
*** bachp has joined #openstack-kolla00:37
*** Pavo_ has joined #openstack-kolla00:42
*** sdake has quit IRC00:46
*** Pavo_ has quit IRC00:47
*** tovin07 has joined #openstack-kolla00:47
*** haplo37 has quit IRC00:47
*** g3ek has quit IRC00:48
*** mgoddard has joined #openstack-kolla00:54
*** haplo37 has joined #openstack-kolla00:54
*** g3ek has joined #openstack-kolla00:55
*** zhubingbing has joined #openstack-kolla01:00
*** sdake has joined #openstack-kolla01:01
*** phuongnh has joined #openstack-kolla01:03
*** duonghq has joined #openstack-kolla01:11
*** zhurong has joined #openstack-kolla01:12
*** mgoddard has quit IRC01:17
*** duonghq has quit IRC01:25
*** hieulq_ has quit IRC01:25
*** phuongnh has quit IRC01:25
*** tovin07 has quit IRC01:25
*** phuongnh has joined #openstack-kolla01:26
*** tovin07 has joined #openstack-kolla01:27
*** zhubingbing has quit IRC01:29
*** duonghq has joined #openstack-kolla01:30
duonghqmorning01:30
duonghqRC2 is coming?01:30
*** hieulq_ has joined #openstack-kolla01:31
*** mtaylor22 has quit IRC01:32
*** bachp has quit IRC01:32
*** kklimonda has quit IRC01:32
*** zhubingbing has joined #openstack-kolla01:34
duonghqhi zhubingbing01:37
zhubingbinghi01:37
duonghqnice to see you01:37
zhubingbingme too01:37
duonghqwhich base distro are you using?01:38
*** kklimonda has joined #openstack-kolla01:38
duonghq*base image01:38
duonghqcentos, ubuntu....?01:38
*** kklimonda is now known as Guest6703201:38
*** MarMat_ has quit IRC01:38
zhubingbingcentos01:39
*** bachp has joined #openstack-kolla01:39
duonghqhave you build image today?01:40
duonghq*built01:40
duonghqzhubingbing01:41
zhubingbingyes01:42
zhubingbingyou can try  koala-build kolla-toolbox01:42
duonghqdid you get 404 not found?01:42
duonghqe.g:01:43
duonghqINFO:kolla.image.build.base:http://mirror01.idc.hinet.net/EPEL/7/x86_64/repodata/1152a964d852a5213e19396a9ba49a9aaea9821a88a64a3fb2f21ba6308a1d92-primary.sqlite.xz: [Errno 14] HTTP Error 404 - Not Found01:43
*** mtaylor22 has joined #openstack-kolla01:43
zhubingbingem01:46
zhubingbingme too01:46
zhubingbingNot Found 40401:46
*** Guest50630 has quit IRC02:03
*** zigo has joined #openstack-kolla02:05
*** zigo is now known as Guest5942702:06
*** JianqingJiang has joined #openstack-kolla02:07
*** MarMat_ has joined #openstack-kolla02:12
*** unicell has quit IRC02:12
*** jtriley has quit IRC02:19
duonghqzhubingbing: seem that epel is updated yesterday02:20
duonghqmaybe mirrors is not fully synchronized02:20
zhubingbingyes02:21
*** jtriley has joined #openstack-kolla02:23
*** JianqingJiang has quit IRC02:25
openstackgerritJeffrey Zhang proposed openstack/kolla: DO NOT MERGE: test eth1  https://review.openstack.org/38523802:29
*** huikang has joined #openstack-kolla02:38
*** yuanying has quit IRC02:48
*** sdake has quit IRC02:57
*** yuanying has joined #openstack-kolla02:58
*** ssurana has joined #openstack-kolla02:58
*** yuanying has quit IRC03:02
*** jrich523 has quit IRC03:05
*** jrich523 has joined #openstack-kolla03:07
*** sdake has joined #openstack-kolla03:18
*** williamcaban has quit IRC03:24
openstackgerritrenminmin proposed openstack/kolla: Hyperlink is expired  https://review.openstack.org/38524603:29
*** unicell has joined #openstack-kolla03:29
*** yuanying has joined #openstack-kolla03:30
*** unicell has joined #openstack-kolla03:30
*** yuanying has quit IRC03:33
*** yuanying has joined #openstack-kolla03:36
sdakehey peeps03:39
sdakeJeffrey4l_ did you try kolla without epel?03:39
sdakeJeffrey4l_ if not i guess i'll give that a go now03:39
*** ayoung has quit IRC03:40
Jeffrey4l_sdake, yes. https://review.openstack.org/38510403:43
Jeffrey4l_but got some error. some container need epel.03:43
Jeffrey4l_like tgtd  http://logs.openstack.org/04/385104/1/check/gate-kolla-dsvm-build-centos-source-centos-7-nv/f6cc1d8/console.html#_2016-10-11_19_34_40_03897203:44
sdakeya was concerned of that03:44
sdakeanything else beyond tgtd?03:44
*** yuanying has quit IRC03:45
sdakeput epel in the containers that need them03:45
Jeffrey4l_sdake, http://logs.openstack.org/04/385104/1/check/gate-kolla-dsvm-build-centos-source-centos-7-nv/f6cc1d8/console.html#_2016-10-11_19_34_39_03679003:45
sdakesurprised barbiaacn-base failed03:46
sdakeoh this is centos source03:46
sdakewhat is story on cinder-volume failure?03:46
sdakei would expect all those containers to fail to build without epel but cinder-volume03:46
sdakehttp://logs.openstack.org/04/385104/1/check/gate-kolla-dsvm-build-centos-source-centos-7-nv/f6cc1d8/console.html#_2016-10-11_19_34_40_65025803:47
Jeffrey4l_sdake, http://logs.openstack.org/04/385104/1/check/gate-kolla-dsvm-build-centos-source-centos-7-nv/f6cc1d8/console.html#_2016-10-11_19_34_40_66292803:48
sdaketgtd-daemon03:48
Jeffrey4l_it is scsi-target-utils.03:48
sdakeright03:48
sdakeis there a way to keep epel in the repo but install the proper kvm package?03:48
sdakeI got impression from apevec there was03:48
Jeffrey4l_if scsi is needed in cinder ( no other better alternative ) it should be added into SIG03:49
*** yuanying has joined #openstack-kolla03:49
sdakeyuanying what you up to these days03:49
sdakeJeffrey4l_ i'll speak with apevec about that tomorrow03:50
Jeffrey4l_cool03:51
sdakeJeffrey4l_ however that isn't going to happen for newton i suspect03:51
*** haplo37 has quit IRC03:51
sdakeso we will have to live with what we have now availabale to us03:51
*** g3ek has quit IRC03:51
Jeffrey4l_yes.03:51
sdakeso lets figure that out - seems like two options03:52
sdakesort out how to instal lqemu-kvm (the proper one)03:52
sdakeor sort out how to put epel into the containers that depend on it03:52
Jeffrey4l_sdake, i have installed the proper one. it is called qemu-kvm not qemu-system-x86 ( this come from epel )03:53
sdakewhat installed qemu-system-x86?03:53
sdakewas that hardcoded in the dockerfile?03:53
Jeffrey4l_sdake, this PS have fix this issue.  and i have already tested it . it works.03:53
Jeffrey4l_https://review.openstack.org/38510203:53
Jeffrey4l_sdake, check this ^^03:53
sdakeok lets just go with that for now Jeffrey4l_03:53
Jeffrey4l_ya03:54
*** haplo37 has joined #openstack-kolla03:57
*** g3ek has joined #openstack-kolla03:58
*** zhubingbing has quit IRC04:03
*** Jeffrey4l_ has quit IRC04:05
*** huikang has quit IRC04:06
*** MarMat_ has quit IRC04:07
duonghqsdake: we use keepalived as VIP failover solution on its own?04:18
*** Jeffrey4l has joined #openstack-kolla04:26
*** zhubingbing has joined #openstack-kolla04:33
*** tovin07 has quit IRC04:36
*** jtriley has quit IRC04:40
*** bachp has quit IRC04:44
*** unicell has joined #openstack-kolla04:51
*** bachp has joined #openstack-kolla04:51
sdakeduonghq right04:54
sdakeJeffrey4l boy we get alot of invalid bugs04:55
sdakeJeffrey4l i think when things settle down weneed to take a hard look at those invlaid bugs and ssee if there are any patterns that can be documented04:56
sdakeJeffrey4l you sort out the patterns - I'll document em - after usmmit :)04:56
Jeffrey4lsdake, good idea.04:56
Jeffrey4lsdake, ok. np.04:56
sdakewhat is the story on this one: https://bugs.launchpad.net/kolla/+bug/156477304:57
openstackLaunchpad bug 1564773 in kolla mitaka "Continously restarting rabbitmq container for CentOS" [High,Confirmed]04:57
sdakeand why is it not critical04:57
Jeffrey4lsdake, no idea now. i think it should be wrong configuration. but still  not get the point.04:58
sdakeJeffrey4l iddy ou fix ubuntu upgrade?05:01
sdakeand did centos upgrade bug merge?05:01
*** salv-orlando has joined #openstack-kolla05:01
Jeffrey4lsdake, no https://review.openstack.org/38459805:02
sdakeJeffrey4l pbourke just wants the commit log to document the deicsion05:03
sdakethat seems easy enough05:03
Jeffrey4lok.05:04
*** Jeffrey4l has quit IRC05:09
*** salv-orlando has quit IRC05:11
*** Jeffrey4l has joined #openstack-kolla05:11
sdakeJeffrey4l 19 bugs fixed so far in rc2 ;)05:12
sdakeman your on fire05:12
sdakethats mor ethen 1 bug a day!05:12
*** Jeffrey4l has quit IRC05:12
*** Jeffrey4l has joined #openstack-kolla05:13
*** Jeffrey4l has quit IRC05:13
*** Jeffrey4l_ has joined #openstack-kolla05:13
* Jeffrey4l_ is back 05:14
*** MarMat has joined #openstack-kolla05:15
*** jtriley has joined #openstack-kolla05:19
*** bachp has quit IRC05:21
*** salv-orlando has joined #openstack-kolla05:25
*** bachp has joined #openstack-kolla05:27
*** jtriley has quit IRC05:27
sdakeJeffrey4l_ any thoughts on htis problem: https://bugs.launchpad.net/kolla/+bug/158365505:29
openstackLaunchpad bug 1583655 in kolla "rabbitmq setup from secondary IP" [Critical,Confirmed]05:29
sdakeman rabbitmq is a pain in the ass05:29
* Jeffrey4l_ is checking.05:29
Jeffrey4l_yes..05:29
*** msimonin has joined #openstack-kolla05:31
Jeffrey4l_sdake, leave a comment. seem a misconfigured in his env.05:35
*** tonanhngo has quit IRC05:36
*** tonanhngo has joined #openstack-kolla05:36
*** skramaja has joined #openstack-kolla05:37
sdakeJeffrey4l_ thanks05:39
*** Jeffrey4l_ has quit IRC05:41
*** Jeffrey4l_ has joined #openstack-kolla05:52
*** Jeffrey4l_ has quit IRC05:57
*** Jeffrey4l_ has joined #openstack-kolla05:58
*** zhubingbing has quit IRC06:01
*** unicell1 has joined #openstack-kolla06:02
*** unicell has quit IRC06:02
*** zhubingbing has joined #openstack-kolla06:03
*** zhubingbing has quit IRC06:05
*** Jeffrey4l_ has quit IRC06:06
sdakeyo cat06:07
sdakes06:07
sdakeso - our deadline is 12th for rc206:07
sdakeafter rc2, only critical bugs should be fixed06:08
sdakeour deadline for rc3 is the 18th06:08
sdakethats a total drop dead hard deadline06:08
sdakeso if your not working on a critical bug at this point, recommend getting on it :)06:08
sdakeif your working on something inprogress, thats good too06:08
sdakeall high and lower bugs will be moved to ocata on 18th06:08
*** ssurana has quit IRC06:17
*** jtriley has joined #openstack-kolla06:17
*** Jeffrey4l_ has joined #openstack-kolla06:19
*** Jeffrey4l_ has quit IRC06:23
*** Jeffrey4l_ has joined #openstack-kolla06:23
*** msimonin has quit IRC06:23
*** jtriley has quit IRC06:27
duonghqroger06:27
*** rkrum has quit IRC06:33
*** mtaylor22 has quit IRC06:34
*** haplo37 has quit IRC06:34
*** cristicalin has joined #openstack-kolla06:38
bjolo__sdake, Jeffrey4l_ please see my thoughts on https://bugs.launchpad.net/kolla/+bug/158365506:39
openstackLaunchpad bug 1583655 in kolla "rabbitmq setup from secondary IP" [Critical,Confirmed]06:39
Jeffrey4l_ok06:40
*** zhubingbing has joined #openstack-kolla06:41
bjolo__morning btw :)06:42
bjolo__rc2 tag day06:42
*** zhubingbing has quit IRC06:43
*** haplo37 has joined #openstack-kolla06:44
Jeffrey4l_yes. tag should be make at 23:50 utc+006:44
sdakeJeffrey4l_ that is when tag will be submitted06:51
sdakei can't guarnatee when it will be made06:51
sdakeand it will be thereabouts06:51
sdakei got kids and dinner and stuff to sort out along the day :)06:51
Jeffrey4l_roger.06:52
sdakealso not sure when the branch willbe made06:52
sdakebut i will find out once rc2 is tagged06:52
sdakerc2 has to be tagged before a stable branch can e made06:52
*** tonanhngo has quit IRC06:54
*** unicell1 has quit IRC06:55
*** zhurong_ has joined #openstack-kolla06:55
*** unicell has joined #openstack-kolla06:55
*** zhurong has quit IRC06:57
*** zhubingbing has joined #openstack-kolla06:58
*** tonanhngo has joined #openstack-kolla07:01
*** jmccarthy has quit IRC07:01
*** tonanhngo has quit IRC07:02
*** jmccarthy has joined #openstack-kolla07:04
Jeffrey4l_bjolo__, sound a good idea. Could u push a PS for this?07:07
sdakebjolo__ i am pretty sure /etc/hosts file in container comes from /etc/hosts bindmount of os07:10
sdakeJeffrey4l_ ^^07:10
Jeffrey4l_sdake, no. try this `docker run -it --add-host a:8.8.8.8 --net host centos:7 bash`07:10
*** cristicalin has quit IRC07:11
sdake```tooo tired07:11
sdakesorry Jeffrey4l_07:11
Jeffrey4l_if we specified --add-host, the hosts file will be not the same with host OS07:12
*** msimonin has joined #openstack-kolla07:12
Jeffrey4l_without --add-host, /etc/hosts in container will be the same with host OS.07:12
*** matrohon has joined #openstack-kolla07:12
Jeffrey4l_never minder. this is the test result ^^07:12
sdakei see - wasn't aware of htis option07:12
bjolo__not me either07:12
sdakethat may not be in 10.0.z07:13
bjolo__working with computers one thing is for sure. you learn something new every day :)07:13
Jeffrey4l_but with --add-host, it will override the host OS hosts file. no idea whether this will cause any issue.07:13
*** b_bezak has joined #openstack-kolla07:13
*** coolsvap has joined #openstack-kolla07:13
sdakenot sure but i've hit the wall07:14
sdakekids running me ragged07:14
sdakesolo dad for 3 weeks = rough07:14
sdakenight all07:14
Jeffrey4l_how old are they?07:15
bjolo__night sdake07:15
Jeffrey4l_night ;)07:15
bjolo__cya in my afternoon :)07:15
bjolo__so Jeffrey4l_ you suggest that we should only do a special start for the rabbitmq containers?07:15
Jeffrey4l_bjolo__, yes. it will solve the issue i think.07:16
bjolo__and not generate a hosts file that we copy into the container using the config.json07:16
bjolo__ok, im just thinking about copy_always and copy_once07:17
Jeffrey4l_bjolo__, copy it is not enough. because when host os hosts file change, the container will not be updated.07:17
*** jtriley has joined #openstack-kolla07:17
bjolo__forget the hosts /etc/hosts file07:17
*** tonanhngo has joined #openstack-kolla07:17
Jeffrey4l_i am thinking, whether there is a parameter, in which we can append some item in /etc/hosts/07:18
Jeffrey4l_bjolo__, how about --add-host?07:18
Jeffrey4l_btw, we can also specify the hostname in container --hostname07:19
*** tonanhngo has quit IRC07:19
bjolo__the add-host means that we need to generate new containers when something changes since the start parameters will change right07:19
bjolo__no matter what the copy policy is07:20
*** MarMat has quit IRC07:20
Jeffrey4l_bjolo__, should we need add all items in /etc/hosts to --add-host?07:20
bjolo__for rabbit, does the cluster hostname have to match the "real" hostname07:20
Jeffrey4l_bjolo__, i do not think it is matter.07:21
bjolo__so anything will work in hosts file and rabbitmq cluster setup as long as it resolves to right IP07:22
Jeffrey4l_use a virtual hostname should be fine. like rabbit1 rabbit2 rabbit307:22
bjolo__cool07:22
*** sdake has quit IRC07:22
Jeffrey4l_oh wait.07:22
bjolo__ok so a few options on how to get that into the containers /etc/hosts file07:22
Jeffrey4l_hostname matter if we do not change the container's hostname.07:22
bjolo__?07:23
Jeffrey4l_hostname matter if we do not change the container's hostname through --hostname parameter.07:23
Jeffrey4l_so we need `docker run --add-host {{ ansible_hostname }}:{{ api_interface_address }}  --add-host <node2>07:24
*** msimonin1 has joined #openstack-kolla07:24
*** msimonin has quit IRC07:25
*** msimonin1 has left #openstack-kolla07:26
*** awebber has joined #openstack-kolla07:26
*** jtriley has quit IRC07:27
*** shardy has joined #openstack-kolla07:27
*** williamcaban has joined #openstack-kolla07:28
bjolo__ok07:29
bjolo__but the drawback of this approach is that we break the copy_always policy in the sense that a container restart should load the right config. If a new rabbit container is added, we need to regenerate all rabbit containers right07:30
Jeffrey4l_yes. another approach is copying hosts though set_config.py feature.07:32
*** Guest59427 is now known as zigo07:33
bjolo__hmm let me tinker with it a bit07:33
Jeffrey4l_it will solution your concern.07:35
*** awebber has quit IRC07:36
Jeffrey4l_but both of these two solution have a drawback: it blocked the host OS's hosts file.  no idea wheter it will cause any other issue.07:36
*** mnasiadka has joined #openstack-kolla07:39
*** mkoderer has joined #openstack-kolla07:41
*** tonanhngo has joined #openstack-kolla07:42
*** awebber has joined #openstack-kolla07:42
*** rmart04 has joined #openstack-kolla07:43
*** tonanhngo has quit IRC07:44
*** williamcaban has quit IRC07:45
awebberwhen does inc0 normally come online :)07:47
openstackgerritzhubingbing proposed openstack/kolla: change source to . in gnocchi-base dockerfile  https://review.openstack.org/38532807:54
openstackgerritzhubingbing proposed openstack/kolla: change source to . in gnocchi-base dockerfile  https://review.openstack.org/38532807:56
*** salv-orlando has quit IRC07:59
*** ankush has joined #openstack-kolla07:59
*** Serlex has joined #openstack-kolla08:02
*** awebber has quit IRC08:03
bjolo__hmmm you are right08:10
*** zhubingbing has quit IRC08:11
bjolo__Jeffrey4l_, but the /etc/hosts in the container is only a copy of the hosts /etc/hosts right. It does not look like a bind mount08:15
*** athomas has joined #openstack-kolla08:16
*** jtriley has joined #openstack-kolla08:17
bjolo__so if we generate the hosts entries needed for rabbitmq and append those to the containers /etc/hosts08:19
*** jtriley has quit IRC08:22
openstackgerritZeyu Zhu proposed openstack/kolla: Visit target nodes without password.  https://review.openstack.org/36852408:27
*** bjolo__ is now known as bjolo08:27
openstackgerritZeyu Zhu proposed openstack/kolla: Visit target nodes without password.  https://review.openstack.org/36852408:30
*** berendt has joined #openstack-kolla08:31
*** williamcaban has joined #openstack-kolla08:42
*** williamcaban has quit IRC08:46
*** mkoderer has quit IRC08:49
*** mkoderer has joined #openstack-kolla08:50
*** janem has joined #openstack-kolla08:55
janemIm trying to build centos images on my Ubuntu host.. there are issues with aufs?08:56
janemhaving problems building some of the base images.08:56
pbourkejanem: http://docs.openstack.org/developer/kolla/image-building.html#known-issues08:59
*** zhubingbing has joined #openstack-kolla09:01
*** shardy is now known as shardy_mtg09:04
*** hieulq_ has quit IRC09:05
*** hieulq_ has joined #openstack-kolla09:06
*** jtriley has joined #openstack-kolla09:11
janempbourke: thanks09:12
*** awebber has joined #openstack-kolla09:13
*** gfidente has joined #openstack-kolla09:15
*** gfidente has quit IRC09:15
*** gfidente has joined #openstack-kolla09:15
*** tovin07 has joined #openstack-kolla09:15
*** jtriley has quit IRC09:15
*** matrohon has quit IRC09:23
*** shardy_mtg is now known as shardy09:27
*** salv-orlando has joined #openstack-kolla09:30
*** salv-orlando has quit IRC09:34
*** haplo37 has quit IRC09:37
*** g3ek has quit IRC09:37
*** yingjun has quit IRC09:42
*** g3ek has joined #openstack-kolla09:47
*** haplo37 has joined #openstack-kolla09:47
*** zhubingbing has quit IRC09:48
*** duonghq has quit IRC10:00
*** tovin07 has quit IRC10:00
*** zhurong_ has quit IRC10:01
*** tonanhngo has joined #openstack-kolla10:03
*** jtriley has joined #openstack-kolla10:05
*** tonanhngo has quit IRC10:05
*** amaged__ has quit IRC10:06
openstackgerritVladislav Belogrudov proposed openstack/kolla: Prechecks fail if using separate api interface  https://review.openstack.org/38537510:07
*** amaged__ has joined #openstack-kolla10:09
*** bachp has quit IRC10:11
*** msimonin has joined #openstack-kolla10:12
*** bachp has joined #openstack-kolla10:12
*** salv-orlando has joined #openstack-kolla10:15
*** jtriley has quit IRC10:15
*** mgoddard has joined #openstack-kolla10:16
openstackgerritMerged openstack/kolla: Fix the failure to launch instance when enabled Ceph  https://review.openstack.org/38510210:22
*** tonanhngo has joined #openstack-kolla10:23
*** tonanhngo has quit IRC10:26
*** Serlex has quit IRC10:33
*** tonanhngo has joined #openstack-kolla10:44
*** tonanhngo has quit IRC10:45
*** haplo37_ has quit IRC10:59
*** haplo37_ has joined #openstack-kolla11:01
*** jtriley has joined #openstack-kolla11:04
*** jtriley has quit IRC11:09
*** jtriley has joined #openstack-kolla11:10
*** shardy is now known as shardy_lunch11:12
*** jtriley has quit IRC11:15
*** mkoderer has quit IRC11:19
*** msimonin has quit IRC11:20
*** mkoderer has joined #openstack-kolla11:20
*** dwalsh_ has joined #openstack-kolla11:23
*** salv-orlando has quit IRC11:23
awidders_so I noticed when I install Kolla using pip using git as the source it doesn't fully install into the /usr/share/kolla directory11:31
awidders_git+https://github.com/openstack/kolla.git@3.0.0.0rc1#egg=kolla11:31
awidders_for example11:31
awidders_am I missing something11:32
awidders_have to go then run setup.py install manually11:32
awidders_thought that was just supposed to happen11:32
*** phuongnh has quit IRC11:33
awidders_ah nm11:37
awidders_figured it out11:37
*** jtriley has joined #openstack-kolla11:40
*** eaguilar has joined #openstack-kolla11:42
*** kproskurin has joined #openstack-kolla11:43
*** Serlex has joined #openstack-kolla11:47
*** salv-orlando has joined #openstack-kolla11:51
*** narasimha_SV has joined #openstack-kolla11:54
narasimha_SVhttps://bugs.launchpad.net/kolla/+bug/163267711:54
openstackLaunchpad bug 1632677 in kolla "ironic-conductor container missing iscsci_tcp module" [Undecided,New]11:54
narasimha_SVgetting this issue while trying to pxe boot of a server using pxe_ipmitool11:54
*** matrohon has joined #openstack-kolla12:07
*** awidders_ has quit IRC12:07
*** awiddersheim has joined #openstack-kolla12:07
*** jtriley has quit IRC12:09
*** awiddersheim has quit IRC12:11
*** awiddersheim has joined #openstack-kolla12:11
*** zhurong has joined #openstack-kolla12:13
*** zhubingbing has joined #openstack-kolla12:14
*** awidders_ has joined #openstack-kolla12:15
*** awiddersheim has quit IRC12:15
*** zhubingbing has quit IRC12:16
*** rhallisey has joined #openstack-kolla12:20
*** sdake has joined #openstack-kolla12:21
*** sdake has quit IRC12:22
*** zhubingbing has joined #openstack-kolla12:22
*** sdake has joined #openstack-kolla12:23
sdakemorning peeps12:24
sdaketag day!!12:24
sdakehows the release looking12:24
*** shardy_lunch is now known as shardy12:28
*** williamcaban has joined #openstack-kolla12:30
*** mgoddard has quit IRC12:31
sbezverk_sdake: good morning12:31
zhubingbingguys12:32
zhubingbingtag day!!12:32
zhubingbingfeel so excited12:33
*** tonanhngo has joined #openstack-kolla12:33
*** prithiv has joined #openstack-kolla12:34
*** tonanhngo has quit IRC12:35
*** lamt has quit IRC12:37
awebberawidders_ i use the following container to build kolla - https://gist.github.com/andrewwebber/abcb2f07e27c8227a3bd2aaf1f8c51bf12:46
*** jheroux has joined #openstack-kolla12:46
*** quasar__ is now known as quasar12:48
*** haplo37 has quit IRC12:49
*** zhubingbing has quit IRC12:50
*** g3ek has quit IRC12:51
*** g3ek has joined #openstack-kolla12:51
*** haplo37 has joined #openstack-kolla12:52
sean-k-mooneyon issue that may be out of our hands it it appears that http://download.ceph.com is down12:52
sean-k-mooneyas a result you cannot build the base image12:52
*** salv-orlando has quit IRC12:53
sean-k-mooneyactully it looks like ceph.com and all its subdomains are down12:54
sdakemorning sbezverk_12:54
sdakesean-k-mooney GRAN12:55
sdakeGROAN12:55
sdakesean-k-mooney this is just ubuntu right?12:56
sdakeour ceph images don'tcome from ceph.com12:56
sean-k-mooneyits been down for a while. they were down yesterday evening when we were talking about my rabbitmq issues. i assumed they would be back up today but im just getting operat timed out issues so maybe they are under a ddos12:56
sdakeon centos12:57
sdakeare you usingcentos?12:57
*** dave-mccowan has joined #openstack-kolla12:57
*** bachp has quit IRC12:57
sean-k-mooneyam good question. dose centos package jewel in an rpm12:57
*** bachp has joined #openstack-kolla12:57
sdakeya centos packages jewel in rpms12:59
sdakein the storage sig12:59
sdakewhich we install12:59
sdakeif we are installing a ceph repo at the same time - that is wrong12:59
sean-k-mooneyno looking at the base dockerfile downloads.ceph.com is only used on ubuntu13:00
sean-k-mooneythe storage sig is beeing used on centos as you said https://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L8413:00
sean-k-mooneyi wonder if there is a mirror list that we can use on ubuntu13:01
sean-k-mooneyactully now that ubunut 16.04 uses systemd centos containers work correct13:02
sean-k-mooneyyes the regional mirros are up got to love google cache fro viewing things when they are down.13:04
awebbereu.ceph.com is up - "deb http://eu.ceph.com/debian-jewel/ xenial main" was my work around to build from source13:05
*** zhurong has quit IRC13:06
sean-k-mooneyyep just checked that. am is there documetation for how to overdide the sources list in the kolla build?13:06
sean-k-mooneyi know the variable is there but have never seen someone use it13:06
*** zhubingbing has joined #openstack-kolla13:07
awebberi am currently building master but with patches to build for ceph hammer. That source was a life saver13:08
*** prithiv has quit IRC13:09
*** zhubingbing has quit IRC13:09
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Workflow Gate Testing  https://review.openstack.org/38202113:10
*** zhubingbing has joined #openstack-kolla13:10
*** lamt has joined #openstack-kolla13:11
*** mnasiadka has quit IRC13:11
openstackgerritMick Thompson proposed openstack/kolla: Add register to upgrade.yml to create correct endpoint  https://review.openstack.org/38546513:11
sdakesean-k-mooney our friends in #rdo have said ceph may be part of the dreamhost outage which is here:13:14
sdakehttp://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/13:14
awebbersdake ironic that ceph.com is down because the backend ceph is down :)13:24
*** kproskurin has quit IRC13:26
*** mewald has joined #openstack-kolla13:26
openstackgerritChristian Berendt proposed openstack/kolla: Install MySQL-python with pip in openstack-base container (type source)  https://review.openstack.org/38493813:26
mewaldHey guys, it looks like keepalived is not working well … I look at the VIPs every once in a while and usually see that two of my hosts have the same VIP. Anyone with similar problems?13:27
*** inc0 has joined #openstack-kolla13:30
*** sogabe has quit IRC13:31
openstackgerritMerged openstack/kolla: Fix 'Address already in use' for barbican-api  https://review.openstack.org/38477113:31
inc0good morning13:31
*** ankush has quit IRC13:31
inc0how's my favorite project team doing?13:31
openstackgerritMerged openstack/kolla: Enable keystone authentication for Barbican  https://review.openstack.org/38416113:31
sdakeawebber lol13:32
*** zhurong has joined #openstack-kolla13:34
awebberinc0 in relation to getting kolla working with ceph-hammer these are the changes i have currently made - https://github.com/openstack/kolla/compare/master...andrewwebber:ceph-hammer?expand=113:34
awebberinc0 i am testing external ceph and would be grateful if you could tell me if im going in the right direction13:35
inc0awebber, you can make these changes locally (without affecting kolla code) by using this mechanism: http://docs.openstack.org/developer/kolla/image-building.html#dockerfile-customisation13:36
*** zhubingbing has quit IRC13:36
inc0once we can get this up and running, that would actually be great thing to have in docs13:36
awebberinc0 yes this is probably the elegant approach but i am just spiking if this integration with hammer is even possible13:36
inc0so I dont think we can make hammer and jewel coexist in trunk13:37
inc0if clients aren't backward compatible (which is duh)13:37
awebberinc0 i am not looking to submit a merge request13:38
sdakeinc0 i'm pretty sure we dont know if there is backwards copmatibillity or not in the protocol clients13:38
awebberinc0 would this be A correct approach to test using an older client version?13:38
awebberinc0 if so i could imagine the templates being extended to allow the admin to define a ceph version13:40
inc0awebber, so we have a rule to not configure building images based on deploy13:44
inc0I think best solution for situation like that is to use template override mechanism13:44
inc0and document it in kolla docs13:44
inc0at least until we figure out something better13:45
awebberinc0 sure13:45
inc0(in fact, I might have an idea already;))13:45
awebberinc0 i just thought with the templates one could specify the ceph version - kolla-build --ceph-version 0.94.4. This would propergate to the Dockerfile templates13:46
inc0so we can't possibly do something like this for every service we have13:47
inc0it would be impossible to maintain13:47
awebberinc0 this seems only applicable to external ceph...13:47
*** ankush has joined #openstack-kolla13:47
inc0exactly, but code would have to be there nonetheless13:47
awebberinc0 ceph the first external service candidate?13:47
inc0today its ceph, tomorrow rabbitmq13:47
inc0no, we have more13:47
awebberinc0 how are they versioned13:47
inc0in fact, with careful config generation you can make pretty much every service external13:48
inc0we'll help with few, but it's possible13:48
inc0but what I was thinking about is to have contrib dir in kolla with our own override files for different situation13:49
inc0like kolla-build --template-override contrib/ceph-version-hammer.j213:49
inc0I think having stuff like this would be helpful, we'd first need to support multiple override files13:50
awebberinc0 yes13:51
awebberinc0 that would be awesome, currently i am merging a folked repro post fetching the kolla repro13:51
inc0something to think about in Ocata, awebber ceph hammer would be great start, would you be willing to share override file if you make it work?13:52
inc0well you can put it all into override file - no merging required13:52
awebberinc0 no problem, i am spiking with my patches for now, then i will refactor them (if they work) into a template override13:52
inc0for us that's only matter to gather few of these files into one dir13:52
inc0wonderful, thank you good sir13:53
bjolomewald, https://bugs.launchpad.net/kolla/+bug/162727513:53
openstackLaunchpad bug 1627275 in kolla "docker stop keepalived does not release VIPs" [High,Triaged]13:53
*** jtriley has joined #openstack-kolla13:53
inc0bjolo, I don't think it's possible13:53
inc0to fix13:53
*** MarMat has joined #openstack-kolla13:53
bjolook13:54
inc0if you kill -9 keepalived (no docker involved) it wont remove ip as well13:54
inc0because..there will be no process to do it13:54
bjolobut i did a docker stop keepalived13:54
inc0well I guess keepalived doesn't handle SIGKILL in a way that removes IP:)13:55
bjolomewald, whats your use case?13:55
inc0arguably it shouldn't13:55
bjoloi did google it a bit, and it seemed doable, but it was a race condition on shutdown that caused. Need to find the URLs again13:56
narasimha_SVironic-condcutor is unable to talk to iscsid container build13:56
*** bjolo is now known as bjolo-afk13:56
narasimha_SVbecause of that it is unable to proceed the pxe booting after deploying deploy_ramdisk13:56
*** mgoddard has joined #openstack-kolla13:56
narasimha_SVare there any specific configurations I need to keep for these so that it talks to iscsid container ???13:57
bjolo-afkinc0, it is a serious issue for ops tho. having same IP on two hosts will cause a lot of issues13:57
inc0yes I know13:57
inc0there are more situations like that13:58
inc0when keepalived will cause problems13:58
inc0again, that's one of main issues I'd like to move on HA session13:58
bjolo-afkHA session?13:58
*** papacz has joined #openstack-kolla13:59
*** liyifeng has joined #openstack-kolla13:59
*** ankush has quit IRC14:00
*** tonanhngo has joined #openstack-kolla14:08
*** tonanhngo has quit IRC14:12
sdakenarasimha_SV - sean-k-mooney is working on bmaas in Ocata14:15
sdakenarasimha_SV pretty sure its not operational now14:15
sdakenarasimha_SV Jeffrey4l_ indicated ironic bmaas is operational for pxe+ssh14:15
sdakenarasimha_SV BMAAS not to e confused with our integration with bifrost for openstack deployment14:16
sdakenarasimha_SV if you have a problem there, please reach out to sean-k-mooney - we need that working reliably for everyone14:16
sean-k-mooneynarasimha_SV: actully if its on the ironic wznoinsk  is looking at that more then i.14:19
sean-k-mooneynarasimha_SV: if its birfrost then agent_ipmitool and agent_ssh were the two drivers i was testing14:21
narasimha_SVok14:22
sean-k-mooneywznoinsk: https://bugs.launchpad.net/kolla/+bug/1632677 this is cause by the fact the kernel module is not loaded correct?14:22
openstackLaunchpad bug 1632677 in kolla "ironic-conductor container missing iscsci_tcp module" [Undecided,Invalid]14:22
narasimha_SVsean-k-mooney: when I am using agent_ipmitool  nova-scheduler itself unable to select the host14:22
narasimha_SVwhich is working with pxe_ipmitool14:23
*** mark-casey1 has joined #openstack-kolla14:23
sean-k-mooneynarasimha_SV: interesting. i know that we have the pxe_ipmitool driver working for ironic as a service in our lab14:23
narasimha_SVyes but in kolla now what I am facing is it is unable to talk to iscsid container because the command is executing from ironic-condutor14:23
narasimha_SVi think that you lab is with normal deployment right not over containers ?14:24
*** salv-orlando has joined #openstack-kolla14:24
narasimha_SVthat launchpad bug which I raised is invalid as I can see the iscsi_tcp module in iscsi contianer deployed14:26
narasimha_SVso now as of my environment I can see that ironic-conductor is trying to execute the command from it but when I try the same command form iscsi container it is working14:27
*** salv-orlando has quit IRC14:29
sdakenarasimha_SV - ironic-conductor needs ipc shared if it is to use iscsi tgtd14:31
sdakenarasimha_SV i'm pretty sure when that work was implemented ipc sharing wasn't available in kolla14:31
sdakenarasimha_SV so - that needs to be added14:31
narasimha_SVsdake: i am new to this could you please explain more14:31
narasimha_SVok how to add ipc sharing now ?14:31
sdakenarasimha_SV file bug i'll add commentary there14:31
narasimha_SVok sure14:31
sdakenarasimha_SV with bmaas, you are going to continue to run into problems :)14:32
sdakeit hasn't been shaken out yet14:32
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla: Unmount Ceph OSD disks as part of destroy  https://review.openstack.org/38002814:32
sdakepbourke why the -2 on https://review.openstack.org/#/c/380028/3/tools/cleanup-host14:33
sdakeseems like a high severity bug to me14:33
pbourkesdake: hmm I didnt think so14:34
pbourkewe can always get better at cleanup14:34
inc0Jeffrey4l_, sdake where are we in upgrade permission crysis?14:34
sdakeinc0 everything is blocked14:34
sdakeinc0 ceph has major outage14:35
inc0how so?14:35
inc0what?14:35
sdakeopen a webbrowser to ceph.com14:35
sdakehttp://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/14:35
inc0ahh...we can't build shit then14:35
sdakeright14:35
sdakewe are tagging today anyway14:36
sdakewe are releasing rc3 on 18th14:36
sdakeand branching todayish (when release team can do so)14:36
inc0yes, we'll backport all these bugfixes14:36
sdakerc3 is critical bugso only14:36
inc0to stable14:36
*** jtriley has quit IRC14:36
sdake3.0.0 will be tagged from rc3 on 18th14:36
inc0but apart from not able to build shit, do we have reviews up?14:36
sdakeinc0 we can build centos14:37
sdakewe can't build ubuntu14:37
sdakehard to test without something to test against...14:37
sdakeso no14:37
*** snoyes_ has joined #openstack-kolla14:37
sdakei have jerked around with getting a vm of ubuntu 16.04 going tho14:37
*** jtriley has joined #openstack-kolla14:38
sdakeinc0 i need to be able to see inside the containers to understand how to change permissions for ubuntu14:38
sdakeand not being able to build them is a problem :)14:38
inc0yeah14:38
inc0well I have prebuilt containers14:39
inc0so I'll try to work on it14:39
inc0I have built 3.0 and 2.0 can be pulled from dockerhub14:40
*** b_bezak has quit IRC14:41
narasimha_SVsdake: https://bugs.launchpad.net/kolla/+bug/163273814:46
openstackLaunchpad bug 1632738 in kolla "ironic-conductor container unable to talk to iscsid container" [Undecided,New]14:46
*** steve-noyes has joined #openstack-kolla14:47
*** mark-casey1 has quit IRC14:47
*** liyifeng has quit IRC14:48
openstackgerritMerged openstack/kolla: Add register to upgrade.yml to create correct endpoint  https://review.openstack.org/38546514:48
*** mark-casey1 has joined #openstack-kolla14:48
*** liyifeng has joined #openstack-kolla14:50
NachoDuckIs there a strategy for adding new hosts to a Kolla deployment, preferably with minimal downtime?14:50
*** athomas has quit IRC14:50
*** bachp has quit IRC14:50
SerlexHi, what does enable_tempest actually enable/install?14:52
*** snoyes_ has quit IRC14:53
*** tonanhngo has joined #openstack-kolla14:53
mark-casey1fyi there is a k8s 1.4 Ask Me Anything webcast today with team members from Google, Deis, CoreOS, and a few others. looks like new features rundown plus audience questions https://www.brighttalk.com/webcast/14601/22790514:54
mewaldbjolo-afk: I just let all keepaliveds running and after a while I have duplicate IPs14:54
sdakenarasimha_SV triaged thanks for filing a bug :)14:54
narasimha_SVthanks sdake. I was struggling on this from past 2 days14:55
*** bachp has joined #openstack-kolla14:56
narasimha_SVsdake: in my kolla ipc_mode: "host" this is already there14:57
sdakenarasimha_SV for the play for ironic?14:57
*** zhubingbing has joined #openstack-kolla14:59
narasimha_SVyes in iscsi start.yml file i have ipc_mode: "host"15:00
rhalliseyNachoDuck, I believe you can add hosts to your inventory file and re run15:02
sdakeis that what play is failing?15:02
rhalliseyNachoDuck, I'll have to test that strategy though.  Otherwise you can have your inventory file with only the new hosts15:02
rhalliseythat will scale up15:03
narasimha_SVno the container is running15:03
narasimha_SVwhen i run the command which is failing in conductor the same command is working from that container15:04
NachoDuckrhallisey: Ok thanks, I tried it with a reconfigure yesterday but it failed when it couldn't find heka on the new host, which makes sense15:04
narasimha_SVi think it didn't put the confs related to iscsi on the host15:04
NachoDuckI guess rerunning the deploy is the correct move15:05
*** sdake has quit IRC15:05
*** zhubingbing has quit IRC15:05
*** sdake has joined #openstack-kolla15:06
narasimha_SVsdake: i am rebuilding the iscsi contianer again15:07
narasimha_SVlet me check15:07
*** mark-casey1 has quit IRC15:07
*** mark-casey1 has joined #openstack-kolla15:08
rhalliseyNachoDuck, ya re configure won't do it15:09
rhalliseyNachoDuck, a re deploy with new hosts will land those containers there15:09
NachoDuckOk cool, I'll give it a go now15:09
rhalliseyNachoDuck, I don't think there is a doc for this, but if you have success with it the community would love for you to share :)15:09
rhalliseycould right a doc or something15:10
sdakenarasimha_SV if the ironic container is using iscsi internally (which i htink itis0 it needs that ipc sharing15:10
sdakethe grep in there was an exmaple of how its done15:10
NachoDuckrhallisey, hehe sure wouldn't mind doing that at all15:10
narasimha_SVok15:11
rhalliseyNachoDuck, awesome!15:11
*** mewald has quit IRC15:13
*** bachp has quit IRC15:15
narasimha_SVsdake: how to identify that iscsi is runing on ipc-mode : host ?15:15
*** msimonin has joined #openstack-kolla15:16
wznoinsknarasimha_SV: sean-k-mooney sorry, busy atm, I can't say for sure about iscsi_tcp as my xenial had iscsid on the host installed and accidentally started on host startup15:20
*** bachp has joined #openstack-kolla15:21
*** janem has quit IRC15:21
*** salv-orlando has joined #openstack-kolla15:21
narasimha_SVwznoinsk: i think this is issue with iscsi container15:21
narasimha_SVas per sdake mentioned this iscsi contianer should have run on host mode15:22
sdakenarasimha_SV docker insepct will tell you15:23
narasimha_SVok15:23
*** bachp has quit IRC15:26
*** rmart04 has quit IRC15:27
*** zhurong has quit IRC15:28
*** hieulq has joined #openstack-kolla15:30
*** jtriley has quit IRC15:34
*** duonghq has joined #openstack-kolla15:35
*** lrensing has joined #openstack-kolla15:37
*** bachp has joined #openstack-kolla15:37
*** mewald has joined #openstack-kolla15:38
*** fguillot has quit IRC15:40
*** hieulq has quit IRC15:43
*** janem has joined #openstack-kolla15:43
*** matrohon has quit IRC15:44
*** hieulq has joined #openstack-kolla15:44
narasimha_SVsdake: http://paste.openstack.org/show/585482/ after redeploying th iscsi container15:46
*** david-lyle has joined #openstack-kolla15:46
narasimha_SVgetting this issue15:46
sdakenarasimha_SV not sure how many timems I have to say ironic is busted in master and it will be addressed in ocata15:46
sdakebut i'll say it once more :)15:46
narasimha_SVyes but the problem is my task in org is this I have enable ironic with kolla15:47
sdakenarasimha_SV oh your a dev15:48
sdakei see15:48
narasimha_SVyes15:48
sdakeok - well why didn't you say so - i thought you were trying to deploy ironic for operatonal purposes :)15:49
narasimha_SVno i am doing for the org where i work15:49
sdakeok - in that case, wznoinsk (I htink) is handling that work - can you sync up with him and join forces?15:49
narasimha_SVyou cant even think I am working on this from more than a month15:49
narasimha_SVsure15:49
sdakenarasimha_SV working together is a big part of kolla - we split work up all the time15:49
narasimha_SVyes that's a good part15:50
narasimha_SVbecause of that I am getting help from you all to get this into this stage atleast now15:50
Jeffrey4l_sdake, check this https://bugs.launchpad.net/kolla/+bug/1616268/comments/4115:51
openstackLaunchpad bug 1616268 in kolla mitaka "Stale namespace removal causing "RTNETLINK answers: Invalid argument" errors" [Critical,Confirmed]15:51
sdake41 comments15:51
sdakeugh15:51
Jeffrey4l_i think the namespace issue can be fixed now.15:51
*** david-lyle has quit IRC15:52
sdakeJeffrey4l_ so you are planning to ust mount /run?15:52
inc0Jeffrey4l_, thats good news15:52
*** mkoderer has quit IRC15:52
sdakei htink /run/netns is a seprate filesystem15:52
sdakewhich is why it is mounted separately15:52
Jeffrey4l_sdake, no use proper /run/xxx file rather than /run direct.15:52
sdakeJeffrey4l_ the nyou miss out on all the stuff that is needed in /run15:53
sdakeis this for the nova container?15:53
Jeffrey4l_for example in neutron_dhcp_agent, we use -v /run/openvswitch:/ru/openvswitch  -v /run/netns:/run/netns:shared15:53
sdakedo you mean mount the run systemd files directly?15:53
*** vhosakot has joined #openstack-kolla15:53
Jeffrey4l_sdake, we only need one of folder in /run in most container.15:54
Jeffrey4l_like neutron_dhcp_agent need access openvswitch socks in /run/openvswitch/ folder.15:54
sdakeJeffrey4l_ yup agree - although be careful with that change (test upgrades)15:54
Jeffrey4l_sdake, yep.15:54
sdakebecause some things need sockets directly in /run15:54
inc05min till meeting15:54
sdakeJeffrey4l_ nice job sorting that out :)15:54
Jeffrey4l_;)15:55
*** kproskurin has joined #openstack-kolla15:56
*** kproskurin_ has joined #openstack-kolla15:57
rhallisey15:58
sdakesup rhallisey15:58
rhalliseysuper15:58
rhalliseysdake, hey15:58
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix wrong mongodb address variable in ceilomter bootstrap  https://review.openstack.org/38553315:58
*** jtriley has joined #openstack-kolla15:59
*** egonzalez90 has joined #openstack-kolla15:59
mewaldfolks my keystone.token table if growing to infinity. Is it possible we're not using memcache for keystone tokens?15:59
*** hrito has joined #openstack-kolla16:01
*** kproskurin has quit IRC16:01
*** athomas has joined #openstack-kolla16:01
*** zhubingbing has joined #openstack-kolla16:02
*** david-lyle has joined #openstack-kolla16:06
wznoinsknarasimha_SV: I got the kolla ironic couple on mitaka working, I'll try to help you there16:06
wznoinsknarasimha_SV: long show is /dev maybe mounted from host to ironic-conductor container so conductor sees the iscsi devices discovered16:06
wznoinsks/show/shot16:06
*** steve-noyes has quit IRC16:15
*** lrensing has quit IRC16:17
*** eaguilar has quit IRC16:18
*** neilus has joined #openstack-kolla16:25
openstackgerritOpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements  https://review.openstack.org/38512716:26
openstackgerritMark Duggan proposed openstack/kolla: Parameterising iscsi_helper and setting default value  https://review.openstack.org/38555316:29
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix keystone fernet file exchange via ssh  https://review.openstack.org/36941816:29
*** lrensing has joined #openstack-kolla16:30
*** lrensing has quit IRC16:32
*** HyperJohnGraham_ has quit IRC16:38
*** tonanhngo has quit IRC16:39
sdakepbourke you were fast on that -2 :)16:39
*** tonanhngo has joined #openstack-kolla16:39
awebbersdake inc0 based upon our conversation yesterday about integration with ceph hammer i can confirm that external ceph integration works with the master branch. I can successfully launch instances from CEPH and upload images to glance (stored in CEPH). I will test cinder tomorrow. I had to of course make some changes. They can be found here http://pastebin.com/raw/Ut8Mmcq016:48
openstackgerritJeffrey Zhang proposed openstack/kolla: [wip]Fix Stale namespace removal issue  https://review.openstack.org/38555916:48
sdakeawebber external ceph hammer that is?16:49
awebbersdake ceph version 0.94.3 (95cefea9fd9ab740263bf8bb4796fd864d9afe2b)16:50
*** tonanhngo has quit IRC16:51
*** ssurana has joined #openstack-kolla16:52
*** jtriley has quit IRC16:52
*** msimonin has quit IRC16:53
*** unicell1 has joined #openstack-kolla16:54
*** unicell has quit IRC16:56
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix Stale namespace removal issue  https://review.openstack.org/38555916:57
Jeffrey4l_hey guys, please review this PS: https://review.openstack.org/#/c/385559/16:58
*** ssurana has quit IRC16:58
Jeffrey4l_i removed lots of /run in some containers. ( i really do not know which specified folder should be use )16:58
Jeffrey4l_please review it and add some comments.16:59
*** ssurana has joined #openstack-kolla16:59
Jeffrey4l_sdake, inc0 pbourke berendt ^^16:59
*** egonzalez90 has quit IRC17:00
*** zhubingbing has quit IRC17:00
*** haplo37_ has quit IRC17:02
*** unicell1 has quit IRC17:04
inc0rhallisey, so whats up with summit?17:04
*** haplo37_ has joined #openstack-kolla17:04
*** unicell has joined #openstack-kolla17:04
*** unicell has quit IRC17:05
rhalliseyinc0, in the etherpad I'll order them in the times based on what how I think they should be orderd17:05
sdakeci at 9am for one thing17:05
*** shardy has quit IRC17:05
inc0already switched with osic review17:06
sdakenice17:06
sdakereset looks good to me17:06
sdakeinc0 looking at the most recnt schedule, etherpad links are missing for all of the sessions17:07
inc0ah, I forgot to add them, will do it no17:08
inc0w17:08
berendti have to leave for today, see you tomorrow17:09
rhalliseyinc0, https://etherpad.openstack.org/p/kolla-o-summit-schedule17:09
rhalliseysomething like that17:09
sdakeinc0 re ta at friday 11:30 - have you synced up with hyakuhei to make sure that time works for the security team?17:09
rhalliseythat's rough estimate I need to 2x my conflicts17:09
inc0you want documentation as fishbowl?17:09
sdakerhallisey thats not how fishbowls work ;)17:10
inc0and roadmap planning for ocata as last session in friday?;)17:10
*** papacz has quit IRC17:10
rhalliseyfirst on frida17:10
inc0first17:10
*** awebber has quit IRC17:10
inc0but I think it should be fb17:10
rhalliseywhich of those slots of FB17:11
rhalliseyand design session17:11
inc0all 3 in Wed17:11
inc0are FB17:11
rhalliseyoh17:11
inc0rest are design17:11
rhallisey:/17:11
rhalliseywell then O road map well have to be on wed17:11
inc0yup17:11
*** hieulq has quit IRC17:12
rhalliseythursday will be the loaded day17:13
*** Serlex has quit IRC17:13
inc0that it will17:13
rhalliseythat's perfect17:13
rhalliseyfirday everyone is tired17:13
inc0hence relatively easy going sessions there17:14
rhalliseyya I live that17:14
rhalliseyHA at 1:5017:14
rhalliseyright after lunch17:14
rhalliseyinc0, what do you think17:15
inc0rhallisey, double check my changes plz https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads#Kolla17:15
inc03rd party plugins will be equally ahrd17:15
rhalliseyya I haven't updated that bit yet17:15
inc0I just did17:15
rhalliseyinc0, ya that will follow HA17:15
rhalliseyok17:15
inc0CI is less technical more "stuff we need to do"17:16
inc0and Distro requirements is pure discussion17:16
rhalliseyinc0, I don't see your changes17:16
inc0I switched sessions in Thu - osic with ci17:16
rhalliseyinc0, right, 9am is perfect for that17:16
inc0added fishbowl - goals for ocata17:17
rhalliseyI was thinking about that too, but that session is a big woot17:17
inc0and comunity roadmap planning17:17
inc0good to get us going17:17
inc0a little brag;)17:17
inc0and we need to discuss publishing it17:17
inc0and work with results17:18
inc0we need to parse data before thoi17:18
inc0I'll do it but I would ask others to review results too17:18
inc0I might miss something important17:18
rhalliseybrb need to grab lunch17:19
rhalliseyinc0, switch around what you want in the etherpad and we can discuss when I'm back17:19
inc0so this is what I came up with - look at actual sched and wiki17:19
*** NachoDuck has quit IRC17:22
*** neilus has quit IRC17:23
*** hrito has quit IRC17:23
*** NachoDuck has joined #openstack-kolla17:24
*** karlamrhein has quit IRC17:24
*** karlamrhein has joined #openstack-kolla17:27
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Adds Fluentd docker image required for kolla-kube  https://review.openstack.org/37561517:28
*** jtriley has joined #openstack-kolla17:28
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix rabbitmq upgrade permission issue  https://review.openstack.org/38459817:28
*** dwalsh_ has quit IRC17:28
openstackgerritJeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MASTER BRANCH  https://review.openstack.org/32630717:31
sean-k-mooneyon a different topic i was trying to use the source_list overide in the kolla-build.conf17:31
*** neilus has joined #openstack-kolla17:31
sean-k-mooneylooking at the build.py https://github.com/openstack/kolla/blob/master/kolla/image/build.py#L615 it is coping the file to sources.list17:32
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Adds Fluentd docker image required for kolla-kube  https://review.openstack.org/37561517:32
sean-k-mooneybut looking at the docker file it is expecting sources.list.{{base_distro}}17:32
sean-k-mooneyhttps://github.com/openstack/kolla/blob/master/docker/base/Dockerfile.j2#L21717:32
*** unicell has joined #openstack-kolla17:32
sean-k-mooneyim going to quickly hack it to confim but i belive the current behavor is broken but i could be doing something wrong17:33
inc0yeah, seems like a bug17:33
*** papacz has joined #openstack-kolla17:33
inc0so I was thinking of moking sources.list as j2 tpl17:34
inc0so we can use our override mechanism there too17:34
openstackgerritJeffrey Zhang proposed openstack/kolla: DO NOT MERGE: test eth1  https://review.openstack.org/38523817:34
openstackgerritJeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MITAKA BRANCH  https://review.openstack.org/29918217:35
sean-k-mooneywell that would be resonably easy to do. should i submit a patch to correct the current mismatch in filenames or wait for the template change17:35
inc0submit it plz, this is bug today17:35
inc0we can still merge it17:36
inc0j2 will be ocata thingy17:36
*** lrensing has joined #openstack-kolla17:36
*** jtriley has quit IRC17:36
sean-k-mooneyok ill test locally and push soon. will i mark with trival fix or do you want a bug to tack it?17:36
inc0trivial is ok imho17:37
sdakeyup ceph still MIA17:37
sean-k-mooneysdake: http://eu.ceph.com/ is working17:37
sean-k-mooneybut ya since you cant overdie the sources.list  in the build that dose not help much unless you use the docker custimiations to modify the default sources.list17:38
*** mag009 has quit IRC17:39
sdakehence why we need mirrors for this stuff in openstack infra ;)17:42
*** eaguilar has joined #openstack-kolla17:42
sean-k-mooneyi was wondering that in the back of my mind so i take it we dont have a ceph mirror in infra17:43
*** duonghq has quit IRC17:43
rhalliseyinc0, where did you make changes?17:44
rhalliseydon't see them in the etherpad or on wiki17:44
inc0wiki is updated..17:45
mewaldGuys, can you please check your keystone.token tables? How big are they? If there is anything more than 0 rows in it, please tell me why we are using sql backend for tokens.17:56
inc0sbezverk_, rhallisey wirehead_ kfox1111 summit k8s talk meeting:)18:00
*** dwalsh_ has joined #openstack-kolla18:02
openstackgerritJeffrey Zhang proposed openstack/kolla: Choose node with largest seqno number for mariadb recovery  https://review.openstack.org/38424718:02
sean-k-mooneyinc0: the issue with the sources list is a little bigger then i taught.18:09
sean-k-mooneyinc0: it looks like none of the config options that are part of https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L143-L152 are being parsed18:09
openstackgerritBartĹ‚omiej Daca proposed openstack/kolla: Add external rabbitmq  https://review.openstack.org/37452518:12
*** eaguilar has quit IRC18:13
*** satya_ has joined #openstack-kolla18:16
openstackgerritJeffrey Zhang proposed openstack/kolla: Move graphviz dependency to test stage  https://review.openstack.org/38558118:16
*** neilus has quit IRC18:18
satya_I am using template_override method to integrate third party plugin with latest trunk neutron code. I am getting the following error while deploying neutro-server container18:18
satya_ImportError: No module named neutron-master.setup18:18
satya_Did any one get this error?18:18
sean-k-mooneydoes i havent tried that but are you installing neutron as part of your template overide?18:19
satya_yes i am installing third party plugin in neutron-server container18:20
*** fguillot has joined #openstack-kolla18:20
sean-k-mooneyyes but are you installing neurtron as part of the code or jsut the thirdparty plugin. there is no module called neutron-master so im wondering is that coming form your code.18:23
*** jtriley has joined #openstack-kolla18:24
*** lrensing has quit IRC18:27
*** msimonin has joined #openstack-kolla18:28
satya_the third party plugin code is added to the existing neutron-server service18:28
mewaldis something wrong with gerrit? "git review" just hangs18:29
satya_it can be an issue with third party plugin code also18:29
*** eaguilar has joined #openstack-kolla18:29
openstackgerritMathias proposed openstack/kolla: Use memcache for keystone token persistence  https://review.openstack.org/38558618:31
*** msimonin has quit IRC18:32
*** narasimha_SV has quit IRC18:38
sean-k-mooneyJeffrey4l_: is the kolla-build.conf intended to still be used  as https://github.com/openstack/kolla/commit/620d610eaaf03217f3a038c031fe28ce43132166#diff-6c850feeb9eb952c01303c97d6efb343L735 seams to have stoped it working.18:39
*** neilus has joined #openstack-kolla18:39
sean-k-mooneyas far as i can tell on master the kolla-build.conf is not read but i am using it for seting some values such as source or distro type https://github.com/openstack/kolla/blob/078b08acfa0f15aecfbc0619efa1b2f4855c8b40/kolla/image/build.py#L94118:40
*** Guest67032 has quit IRC18:40
*** kklimonda has joined #openstack-kolla18:41
*** HyperJohnGraham_ has joined #openstack-kolla18:42
*** lrensing has joined #openstack-kolla18:42
*** neilus has quit IRC18:44
*** msimonin has joined #openstack-kolla18:44
*** tonanhngo has joined #openstack-kolla18:46
*** HyperJohnGraham_ has quit IRC18:47
*** mewald has quit IRC18:48
*** salv-orlando has quit IRC18:51
*** mewald has joined #openstack-kolla18:51
*** satya_ has quit IRC18:56
*** williamcaban has quit IRC19:01
*** neilus has joined #openstack-kolla19:05
*** msimonin has quit IRC19:09
*** neilus has quit IRC19:10
*** liyifeng has quit IRC19:10
*** liyifeng has joined #openstack-kolla19:11
*** pbourke has quit IRC19:11
*** pbourke has joined #openstack-kolla19:11
*** msimonin has joined #openstack-kolla19:18
*** lrensing has quit IRC19:19
*** eaguilar has quit IRC19:24
openstackgerritsean mooney proposed openstack/kolla: read kolla-build.conf by default  https://review.openstack.org/38560119:24
sean-k-mooneyinc0: Jeffrey4l_ ^ that should fix the sources list issue and also fix reading form the kolla-build.conf in general.19:25
*** eaguilar has joined #openstack-kolla19:26
sdakesean-k-mooney groan...19:29
sdakehow important is this change?19:30
sean-k-mooneysdake: ill leave that up to us to decide but  without it the kolla-build.conf seams to be ignored.19:31
sean-k-mooneysdake: as a result if the paramter you want to change is not a commandline option to build.py it cannot be changed.19:31
sean-k-mooneysdake: that is basically just https://github.com/openstack/kolla/blob/078b08acfa0f15aecfbc0619efa1b2f4855c8b40/kolla/common/config.py#L143-L15219:31
sean-k-mooneysdake: i belive you can use the template overide to change these now but that would not be an option for mitaka19:32
sdakesean-k-mooney i'd feel more comfortable with this change after 3.0.0 in a zstream backport19:33
sdakeafter it has some soak time19:33
sean-k-mooneyok though looking at https://github.com/openstack/kolla/blob/078b08acfa0f15aecfbc0619efa1b2f4855c8b40/docker/base/Dockerfile.j2#L212-L218 it would not be that easy to trivaly replace the sources.list with a template overide19:34
sdakewell we aren't merging in rc2 - i can tell ya that :)19:35
sdakeif the cores wnt to merge in master and backport we can do that19:35
sdakei really don't want to touch build.py :)19:35
sean-k-mooneyok that makes sense but i am basically reverting some of Jeffrey4l_ changes he made in https://github.com/openstack/kolla/commit/620d610eaaf03217f3a038c031fe28ce43132166#diff-6c850feeb9eb952c01303c97d6efb343L73519:36
sean-k-mooneysdake: im not really i know i must be missing somethig because i dont know how this could be broken since january and no on noticed.19:37
sean-k-mooneys/im not really//19:38
sdakeok well if jeffrey acks it that wfm :)19:38
sdakebut not for rc219:38
sdakebut it does need a bug id to be backported19:38
sdakeso i'd add that19:38
sdakeand it has to be critical to be backported19:38
sdakethis may be critical - not sure19:38
sean-k-mooneyok will do. are we still supporting kolla-build.conf by they way?19:39
*** eaguilar has quit IRC19:39
sean-k-mooneyi alway configure my build via /etc/kolla/kolla-build.conf but i cant see how the current code on master reads from it19:40
sean-k-mooneyat least without the change i proposed.19:40
*** lrensing has joined #openstack-kolla19:40
sean-k-mooneysdake: ill open a bug for this then in anycase and ill let ye decide if it warrents a backport19:41
sean-k-mooneysdake: on the pluse side now that i have this locally i have been able to rebuild my rabbitmq container and at least single node works for me now19:42
sdakesean-k-mooney nice!19:42
*** eaguilar has joined #openstack-kolla19:46
*** salv-orlando has joined #openstack-kolla19:47
*** jtriley has quit IRC19:50
sdakesean-k-mooney yes kolla-build.conf is a feature19:54
sdakesean-k-mooney that is why this may be critical19:54
sdakesean-k-mooney i want to give folks an opportunity to test out our rc2 without regressions19:54
sdakeand any changes to build.py make me super nervous ;)19:54
sean-k-mooneyyep i understand. i was really happy to c that ctrl-c now kills build.py again19:58
sean-k-mooneyor rather ctrl-c twice kills it now19:58
*** msimonin has quit IRC20:03
*** msimonin has joined #openstack-kolla20:06
*** msimonin has quit IRC20:07
*** MarMat_ has joined #openstack-kolla20:08
*** MarMat has quit IRC20:08
*** sean-k-mooney is now known as sean-k-mooneyAFK20:10
wznoinskhi, anyone seen this https://bugs.launchpad.net/kolla/+bug/1632840 ? Jeffrey4l_ ?20:11
openstackLaunchpad bug 1632840 in kolla "ldlinux.c32 File not found in ironic_pxe" [Undecided,New] - Assigned to Waldemar Znoinski (wznoinsk)20:11
*** msimonin has joined #openstack-kolla20:11
*** jtriley has joined #openstack-kolla20:12
*** msimonin has quit IRC20:16
*** msimonin has joined #openstack-kolla20:16
*** eaguilar has quit IRC20:18
openstackgerritWaldemar Znoinski proposed openstack/kolla: copy ldlinux.c32 to /tftpboot in ironic_pxe  https://review.openstack.org/38561220:22
*** eaguilar has joined #openstack-kolla20:28
*** eaguilar has quit IRC20:32
*** lrensing has left #openstack-kolla20:32
openstackgerritWaldemar Znoinski proposed openstack/kolla: setting basic structures for iscsi container  https://review.openstack.org/38194220:38
*** salv-orl_ has joined #openstack-kolla20:43
*** salv-orlando has quit IRC20:46
*** awebber has joined #openstack-kolla20:57
*** rhallisey has quit IRC20:57
*** jtriley has quit IRC20:58
*** jtriley has joined #openstack-kolla21:06
*** salv-orl_ has quit IRC21:07
*** jtriley has quit IRC21:10
*** mark-casey11 has joined #openstack-kolla21:12
*** janem_ has joined #openstack-kolla21:13
*** salv-orlando has joined #openstack-kolla21:13
*** rbbratta has quit IRC21:15
*** rbbratta has joined #openstack-kolla21:15
*** zigo_ has joined #openstack-kolla21:17
*** irtermit- has joined #openstack-kolla21:18
*** haplo37- has joined #openstack-kolla21:18
*** hieulq__ has joined #openstack-kolla21:18
*** dwalsh_ has quit IRC21:19
*** athomas has quit IRC21:19
*** janem has quit IRC21:19
*** bachp has quit IRC21:19
*** mark-casey1 has quit IRC21:19
*** haplo37 has quit IRC21:19
*** jheroux has quit IRC21:19
*** zigo has quit IRC21:19
*** irtermite has quit IRC21:19
*** hyakuhei has quit IRC21:19
*** haplo37- is now known as haplo3721:19
*** jmccarthy1 has joined #openstack-kolla21:19
*** awebber_ has joined #openstack-kolla21:20
*** hyakuhei- has joined #openstack-kolla21:20
*** mark-casey11 has quit IRC21:20
*** janem_ has quit IRC21:20
*** hieulq_ has quit IRC21:20
*** jmccarthy has quit IRC21:20
*** awebber has quit IRC21:20
*** liyifeng has quit IRC21:21
*** liyifeng has joined #openstack-kolla21:21
*** gfidente has quit IRC21:21
MarMat_sean-k-mooney not sure whether i'm following what you mean but there is --config-file PATH param for build.py, doesn't this solve what you need?21:22
*** hyakuhei- has quit IRC21:23
*** wirehead_ has quit IRC21:23
*** macsz has quit IRC21:23
*** jidar has quit IRC21:23
*** awebber_ has quit IRC21:23
*** dave-mccowan has quit IRC21:24
*** athomas has joined #openstack-kolla21:25
*** bachp has joined #openstack-kolla21:25
*** hyakuhei- has joined #openstack-kolla21:28
*** wirehead_ has joined #openstack-kolla21:28
*** macsz has joined #openstack-kolla21:28
*** jidar has joined #openstack-kolla21:28
*** macsz has quit IRC21:28
*** zigo_ is now known as zigo21:28
*** macsz has joined #openstack-kolla21:33
*** Jeffrey4l__ has joined #openstack-kolla21:34
*** Jeffrey4l_ has quit IRC21:35
*** macsz is now known as macsz_21:37
*** portdirect has joined #openstack-kolla21:39
*** inc0 has quit IRC21:46
*** neilus has joined #openstack-kolla21:55
*** neilus has quit IRC22:00
*** fguillot has quit IRC22:05
*** mtaylor22 has joined #openstack-kolla22:10
mtaylor22Jeffrey4l__: good find man22:10
mtaylor22re: RTNETLINK isue22:10
mtaylor22:)22:11
*** vhosakot has quit IRC22:17
*** eaguilar has joined #openstack-kolla22:18
*** unicell has quit IRC22:20
*** lamt has quit IRC22:22
*** papacz has quit IRC22:23
*** mgoddard has quit IRC22:25
*** HyperJohnGraham_ has joined #openstack-kolla22:27
*** mewald has quit IRC22:28
*** msimonin has quit IRC22:30
*** salv-orlando has quit IRC22:30
sdakewhere is the summit etherpad schedule22:32
*** HyperJohnGraham_ has quit IRC22:34
*** HyperJohnGraham_ has joined #openstack-kolla22:40
Pavoevening peeps22:54
Pavowanna see my new toy I got today?22:54
harlowjaadult-safe?22:55
harlowjasfw?22:55
harlowjalol22:55
Pavoofc22:55
Pavohttps://www.dropbox.com/s/l3fow3upj53qg85/49800443060__3B9B4073-C938-40E6-9616-1DBBD7739AA5.JPG?dl=022:55
*** athomas has quit IRC22:55
harlowjalol22:55
harlowjanice22:55
Pavoyeah 2 HP Proliant DL560 with 4x Intel® Xeon® E5-4657L v2, 32Tb storage, 384Gb RAM, Cisco 3650 switch and Cisco 4300 ISR Router and Gigaview tap22:56
Pavothis is what I brought home from work to work on Kolla with22:58
*** kproskurin_ has quit IRC23:01
*** HyperJohnGraham_ has quit IRC23:08
mtaylor22harlowja: looks tasty.23:13
mtaylor22i mean, Pavo23:13
Pavoyeah its a beast23:13
sdakewznoinsk triaged23:17
sdakewznoinsk nobody has really got ironic working yet23:17
sdakeharlowja could you do me a favor (code review)23:17
harlowjasureee23:18
harlowjai do all the favors23:18
sdakeharlowja be careful with that, i've got a smuggling  gig going on too :)23:18
harlowjaonly 1 smuggle per day23:18
sdakehttps://review.openstack.org/#/c/385601/23:19
sdakeharlowja if you could follow it to conclusion (merge) would be appreciated :)23:19
sdakereally don't want to change anything in build.py23:19
sdakepavo did your gig pay for that or did you?23:20
Pavoofc gig, I can't afford this shit23:21
Pavolol23:21
Pavoopps mybad23:21
Pavosry language23:21
sdakeany idea what the sticker price was (if thats not confidential)23:21
sdakepavo it happens23:21
Pavoclose to $250K23:21
sdakeyikes23:21
Pavoyeah I am stoked23:22
Pavo192 cpus, 32Tb SAS SSDs, 386GB ram23:22
Pavocombined ofc23:22
sdakei've got a 10 node cluster at work with 256 ram per and 24 disks and whatnot23:23
Pavogonna use it to continue our development with packstack and also get kolla for prime time23:24
Pavonice23:24
sdaketwo ask9k switches in redundancy mode23:24
*** zhurong has joined #openstack-kolla23:24
jascott1hmm kolla-ansible deploy is crashing out during 'starting heka container' and I can see that its trying to hit my local registry Get https://10.64.142.93:4000/v1/_ping23:26
jascott1but im not running https on my registry23:27
*** HyperJohnGraham_ has joined #openstack-kolla23:28
jascott1is there a place to tell kolla-ansible deploy to use http?23:29
jascott1or is https registry required?23:29
jascott1idk if it matters but I am running registry v2 which I assume is backwards compatible23:30
Pavohttps registry is for runing a secure registry which uses certs23:32
Pavothe certs need to be on all nodes23:32
jascott1Pavo so running local registry w https is required you are saying?23:34
Pavoafaik it is with kolla but I could be wrong23:34
jascott1or rather 'if you run a local registry it must use https'23:34
jascott1i get all the cert requirements (I was trying to avoid it all for what I am doing)23:34
Pavoyou can run a unsecure registry23:34
sbezverk_jascott1: you should probably check you docker configuration23:35
Pavobut I don't know if kolla will use a unsecure registry23:35
sbezverk_for registry23:35
*** HyperJohnGraham_ has quit IRC23:35
jascott1i have the insecure flag for the daemon23:35
sbezverk_other_args="--insecure-registry 192.168.80.240:4000"23:35
jascott1i have pushed images alreay so I know docker and local registry are jiving23:36
jascott1pushed images via kolla-build23:36
*** zhurong has quit IRC23:37
jascott1its seems like kolla-ansible deploy is either hard coded to https or I missed a config option23:37
*** HyperJohnGraham_ has joined #openstack-kolla23:38
Pavokolla-build only builds images, didn't think it pushed them23:38
jascott1it does with --push and --registry flag23:38
sbezverk_jascott1: I do not think deploy directly pulls images, everything is done via docker interface23:38
sbezverk_can you paste the error you are seeing?23:38
sbezverk_also when you start deploy please use -vvvv23:39
sbezverk_for more logs23:39
jascott1TASK [common : Starting heka container]23:39
jascott1APIError: 500 Server Error: Internal Server Error (\"{\"message\":\"Get https://10.64.142.93:4000/v1/_ping: EOF23:39
jascott1there is no https on my registry23:40
sbezverk_jascott1: use paste bin23:40
jascott1oops sorry23:40
jascott1that was it tho23:40
sdakejascott1 a tls enabled registry is not required23:40
jascott1sdake ok thanks23:40
sdakepavo kolla-ansible --push ;-)23:41
Pavoyeah thats what I thought, not kolla-build23:41
sdakejascott1 please run ps -ef  |grep kolla23:41
sdakepavo sorry kolla-build --push --registry will getyou there23:41
sdakenot kolla-ansible...23:41
Pavoyeah but not by default23:41
Pavoyou have to push the images yourself23:42
sdakejascott1 please run ps -ef | grep docker23:42
sdakepavo kollab-uild will build all images it pushes if you specify those two flags with correct options23:42
sdakee.g. kolla-build --registry 192.168.1.103:4000 --push is what I use for centos rdo23:42
jascott1sdake /usr/bin/dockerd --dns 8.8.8.8 --dns 8.8.4.4 --insecure-registry 10.64.142.93:4000 --raw-logs23:42
sdakejascott1 kolla-build and kolla itself use different mechanisms for communicating with docker registry23:43
sdakejascott1 looksgood23:43
sdakejascott1 - ok paste globals.yml pls23:43
Pavoyeah I understand that, I was just saying to jascott1 that it doesn't push images to your local registry on building unless you tell it to23:43
sdakepavo good point - jascott1 did you use the --push option?23:43
sdakejascott1 run docker ps -a | grep registry pls23:43
jascott1i did use the push option23:44
jascott1globals http://paste.openstack.org/show/585521/23:44
sdakeceph.com is back23:44
jascott1yay!23:44
jascott1i changed a couple params to eu.ceph.com earlier and it worked but glad to hear23:45
jascott1rough days at dreamhost :/23:45
sdakejascott1 run docker --version pls23:45
sdakeand docker ps -a | grep registry23:46
jascott1sdake registry line: 3d7fdc60fa3d        registry:2          "/entrypoint.sh /etc/"   2 days ago          Up 23 minutes       0.0.0.0:4000->5000/tcp   registry23:46
sdakejascott1 that looks good23:46
jascott1Docker version 1.12.1, build 23cf63823:46
jascott1:D23:46
sdakeok - 1.12.123:46
sdakethis is bleeding edge23:46
sdakecan you downgrade your version of docker?23:46
jascott1its stable release afaik but yes I can try that23:47
jascott1what ver?23:47
sdakemoment before we go down that rabbit hole23:47
sdakepip show jinja223:47
sdakeand pip show kolla23:47
sdakeand pip show ansible23:47
jascott1jinja2 is 2.823:47
jascott1kolla is 3.0.0.0rc2.dev30023:48
sdakegood23:48
jascott1ansible is 2.1.2.023:48
sdakejascott1 when you work with master you should not pip install23:48
sdakelatest version we have tested with ansible is 2.1.1.023:48
jascott1ooohh23:48
sdakelets solve one problem at a time23:48
sdakefirst pip remove kolla23:48
Pavo4000:5000 sure thats tls?23:49
jascott1it is not tls23:49
jascott1Im not running tls23:49
sdakepavo that maps port 4000 outside to 5000 in the registry container23:49
Pavoah yeah thats right you using non secure registry mybad23:49
Pavolittle lit right now23:49
sdakeya i'm beat myself :)23:49
Pavonot beat, getting drunk lol23:49
Pavoand setting servers up at the same time23:50
Pavoprobably bad idea23:50
Pavolol23:50
sdakei work from home so dont drink where i work ;)23:50
jascott1oh noes!23:50
jascott1no drinking at home?23:50
Pavooh btw my surgery went well23:50
jascott1sorry23:50
jascott1glad to hear23:50
sdakei make up for it on the road ;-)23:50
Pavoalmost fully recovered23:50
jascott1haha23:50
jascott1pip uninstall kolla - no files were found to uninstall23:51
Pavodid you install kolla from git?23:51
sdakehow did you pip install kolla23:51
jascott1dev isntall from the docs23:51
Pavooh btw wanted to ask, anyone here a jenkins expert?23:52
jascott1looking23:52
jascott1sudo pip install --editable .23:53
jascott1iirc thats what I did from here http://docs.openstack.org/developer/kolla-kubernetes/host-setup.html23:53
jascott1"Development Install"23:53
Pavo--editable is an option not a package23:54
jascott1. is local dir23:54
jascott1sdake did you want me to downgrade something?23:55
sdakejascott1 not yet23:56
sdakejascott1 are you trying to install kolla-kubernetes?23:56
sdakeor kolla23:56
jascott1kolla-kubernetes23:56
jascott1my ultimate goal is to help with the ansible workflows23:57
jascott1but I am still trying to get multinode k8s with kolla-kuberenetes going23:57
sdakejascott1 i can't help wit hthe -e option on the pip install23:58
sdakejascott1 that doesn't jive with my understanding of how things should be working23:58
sdakejascott1 fwiw kfox1111 who wrote those directions deploys 2.0.2 images from the docker hub23:58
sdakerhallisey sbezverk_ or kfox1111 can help you get going though23:58
jascott1i really appreciate the pointers23:58
sdakejascott1 what i would recommend for now is not using a registry at all23:59
sdakeand use the 2.0.2 images23:59
sdakeand follow the rest of the kolla-kubernetes directions23:59
sdake(from docker hub)23:59
sdaketo do that, comment out the registry line23:59
sdakethen the docker images will automatically come from the hub23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!