Thursday, 2017-06-01

masberI would like to add dhcp_domain parameter to my nova.conf file, however kolla installs multiple of these files for different containers. How do I know which file to edit?00:00
*** yingjun has joined #openstack-kolla00:01
*** bmaltais has quit IRC00:04
*** powerd has quit IRC00:05
*** MarginHu has joined #openstack-kolla00:06
*** caoyuan has joined #openstack-kolla00:09
dimssdake : the current review is a overall policy statement, so i don't think a specific exception works in there, we don't even mention kolla even once. So you should probably follow up with ttx on that one00:11
sdakedims i left a comment in the review00:12
dimsy responding there00:12
sdakedims the probelm is the policy statement locks out releasing ocata zstreams via the automted release process00:12
*** ducttape_ has joined #openstack-kolla00:12
sdakeas we can't backport manifest code00:12
dimsit does not say anything about releases00:12
*** caoyuan has quit IRC00:12
dimsor the release process change00:13
sdakedims perhaps i misunderstodo the language - i readi t earlyl this mronign and injured myself last night - so not feeling weel00:13
dimsone sec sdake00:13
dimssdake : fresh read from the rendered doc would help i think - http://docs-draft.openstack.org/65/469265/2/check/gate-governance-docs-ubuntu-xenial/85c307b//doc/build/html/resolutions/20170530-binary-artifacts.html00:14
sdakebut i dont really want ot discuss it atm, as i'm heading to bed00:14
sdakethink i broke my hand or injuredit badly00:14
dimssdake : ouch! no worries, when you get to it00:14
*** mnasiadka has joined #openstack-kolla00:15
*** mnasiadka has quit IRC00:19
*** salv-orlando has joined #openstack-kolla00:21
*** Pavo has quit IRC00:24
*** krtaylor has joined #openstack-kolla00:28
*** salv-orlando has quit IRC00:30
*** manheim has joined #openstack-kolla00:34
*** manheim has quit IRC00:37
kfox1111sdake: I don't think the manifest generating code needs to live inside the containers. that avoids the issue alltogether.00:38
kfox1111see: https://review.openstack.org/#/c/458174/5/tools/gate_run.sh does it all from within the container without changing it.00:39
kfox1111gotta head out. bbl.00:39
*** manheim has joined #openstack-kolla00:39
*** rwsu has quit IRC00:41
*** hongbin has quit IRC00:44
*** tonanhngo has quit IRC00:47
*** ducttape_ has quit IRC00:50
*** jtriley has quit IRC00:51
*** cuongnv has joined #openstack-kolla00:54
*** tonanhngo has joined #openstack-kolla00:54
*** daidv has joined #openstack-kolla00:55
*** tonanhngo has quit IRC00:58
*** tovin07_ has joined #openstack-kolla01:02
*** Pavo has joined #openstack-kolla01:11
*** Pavo has quit IRC01:13
*** duonghq has joined #openstack-kolla01:23
duonghqmorning guys01:25
*** zhubingbing_ has joined #openstack-kolla01:25
zhubingbing_hi guys01:26
duonghqhi zhubingbing_01:26
duonghqhow are you01:26
*** ducttape_ has joined #openstack-kolla01:27
*** hrw has quit IRC01:32
*** hrw has joined #openstack-kolla01:34
*** kbyrne has quit IRC01:39
*** chas has joined #openstack-kolla01:40
*** kbyrne has joined #openstack-kolla01:42
*** chas has quit IRC01:45
*** manheim has quit IRC01:51
*** manheim has joined #openstack-kolla01:51
zhubingbing_hi duonghq01:52
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Apply neutron database migration  https://review.openstack.org/40792201:56
*** MarginHu has quit IRC01:59
*** MarginHu has joined #openstack-kolla01:59
*** ducttape_ has quit IRC01:59
*** mnasiadka has joined #openstack-kolla02:00
*** MarginHu has quit IRC02:00
*** kbaegis has joined #openstack-kolla02:04
*** mnasiadka has quit IRC02:05
*** salv-orlando has joined #openstack-kolla02:05
*** unicell has quit IRC02:08
*** caowei has joined #openstack-kolla02:08
*** manheim has quit IRC02:14
*** caoyuan has joined #openstack-kolla02:14
*** caoyuan has quit IRC02:15
*** rwsu has joined #openstack-kolla02:16
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600702:18
*** goldyfruit has joined #openstack-kolla02:18
*** manheim has joined #openstack-kolla02:18
*** caoyuan has joined #openstack-kolla02:18
*** salv-orlando has quit IRC02:20
*** manheim has quit IRC02:39
*** manheim has joined #openstack-kolla02:39
*** manheim has quit IRC02:39
*** manheim has joined #openstack-kolla02:40
*** goldyfruit has quit IRC02:41
*** sayantan_ has joined #openstack-kolla02:43
*** manheim has quit IRC02:44
*** masber has quit IRC02:46
*** sayantan_ has quit IRC02:48
*** caoyuan has quit IRC02:50
*** caoyuan has joined #openstack-kolla02:51
*** caoyuan has quit IRC02:59
*** yingjun has quit IRC02:59
spsuryamorning all03:04
spsuryasup zhubingbing_03:04
spsuryasup duonghq03:04
*** yingjun has joined #openstack-kolla03:15
duonghqsup spsurya03:15
*** dave-mccowan has quit IRC03:16
*** salv-orlando has joined #openstack-kolla03:17
*** xinliang has joined #openstack-kolla03:21
*** xinliang has quit IRC03:21
*** xinliang has joined #openstack-kolla03:21
*** salv-orl_ has joined #openstack-kolla03:24
*** salv-orlando has quit IRC03:24
*** janki has joined #openstack-kolla03:26
*** zhurong has joined #openstack-kolla03:27
*** jamesPR has joined #openstack-kolla03:27
*** salv-orl_ has quit IRC03:29
*** caowei has quit IRC03:39
*** caoyuan has joined #openstack-kolla03:39
*** chas has joined #openstack-kolla03:41
*** yingjun has quit IRC03:44
*** chas has quit IRC03:46
*** yingjun has joined #openstack-kolla03:53
*** zhubingbing_ has quit IRC03:55
*** zhubingbing has joined #openstack-kolla03:56
openstackgerritSteve Baker proposed openstack/kolla master: kolla-build --list-dependencies use json, not pprint  https://review.openstack.org/46974104:03
*** sayantan_ has joined #openstack-kolla04:06
*** rwsu has quit IRC04:07
*** zhubingbing has quit IRC04:11
*** caoyuan has quit IRC04:14
*** eanylin has quit IRC04:18
*** tovin07_ has quit IRC04:22
*** Serlex has joined #openstack-kolla04:24
*** salv-orlando has joined #openstack-kolla04:26
*** zhurong has quit IRC04:27
*** jamesPR has quit IRC04:28
*** sgrasley has joined #openstack-kolla04:28
*** sgrasley has quit IRC04:33
*** zhurong has joined #openstack-kolla04:36
*** unicell has joined #openstack-kolla04:39
*** salv-orlando has quit IRC04:40
*** manheim has joined #openstack-kolla04:42
*** pcaruana has joined #openstack-kolla04:43
*** eanylin has joined #openstack-kolla04:51
*** caoyuan has joined #openstack-kolla04:53
*** salv-orlando has joined #openstack-kolla04:53
*** zhubingbing has joined #openstack-kolla04:53
*** skramaja has joined #openstack-kolla05:08
*** caoyuan has quit IRC05:09
*** caoyuan has joined #openstack-kolla05:10
*** MarginHu has joined #openstack-kolla05:10
*** manheim has quit IRC05:14
*** kfir has quit IRC05:15
*** zhurong has quit IRC05:16
*** zhurong has joined #openstack-kolla05:17
*** jaosorior_away is now known as jaosorior05:24
*** rwsu has joined #openstack-kolla05:38
*** chas has joined #openstack-kolla05:42
*** unicell has quit IRC05:44
*** unicell has joined #openstack-kolla05:45
*** pcaruana has quit IRC05:49
*** chas has quit IRC05:49
*** masber has joined #openstack-kolla06:00
*** pcaruana has joined #openstack-kolla06:02
*** chas has joined #openstack-kolla06:07
*** salv-orlando has quit IRC06:11
*** manheim has joined #openstack-kolla06:11
*** salv-orlando has joined #openstack-kolla06:11
*** salv-orlando has quit IRC06:16
*** manheim has quit IRC06:24
*** caowei has joined #openstack-kolla06:27
*** korzen has joined #openstack-kolla06:27
*** yingjun has quit IRC06:29
openstackgerritHanxi Liu proposed openstack/kolla-ansible master: Stop using old gnocchi reference  https://review.openstack.org/46977106:47
openstackgerritHanxi Liu proposed openstack/kolla master: Stop using old gnocchi reference  https://review.openstack.org/46977206:47
*** mnasiadka has joined #openstack-kolla06:49
*** sayantan_ has quit IRC06:50
*** caoyuan has quit IRC06:52
*** chas has quit IRC06:55
*** salv-orlando has joined #openstack-kolla06:58
*** tovin07_ has joined #openstack-kolla06:59
*** jmccarthy has joined #openstack-kolla06:59
*** zhubingbing has quit IRC07:04
*** zhubingbing has joined #openstack-kolla07:06
*** n0isyn0ise has quit IRC07:12
*** slagle has quit IRC07:16
*** shardy has joined #openstack-kolla07:16
*** mgoddard has joined #openstack-kolla07:19
*** caoyuan has joined #openstack-kolla07:19
*** caoyuan_ has joined #openstack-kolla07:19
*** caoyuan has quit IRC07:23
*** manheim has joined #openstack-kolla07:25
*** salv-orl_ has joined #openstack-kolla07:28
hrwmorning07:28
*** manheim has quit IRC07:30
*** salv-orlando has quit IRC07:30
Serlexhello07:32
hrwhttps://www.redhat.com/archives/rdo-list/2017-June/msg00000.html - setuptools v36 (released today) fails to work07:37
*** cuongnv has quit IRC07:37
*** cuongnv has joined #openstack-kolla07:38
openstackgerritJuan Badia Payno proposed openstack/kolla master: Modified fluentd Image  https://review.openstack.org/46721907:45
*** tvignaud has quit IRC08:03
*** egonzalez has joined #openstack-kolla08:16
*** tvignaud has joined #openstack-kolla08:16
*** mgoddard has quit IRC08:16
hrwhttps://github.com/pypa/setuptools/issues/1042 as well08:16
*** zhurong has quit IRC08:29
*** athomas has joined #openstack-kolla08:30
reidracegonzalez: thanks for the review! \o/08:33
egonzalezyw08:33
*** manheim has joined #openstack-kolla08:42
*** chas has joined #openstack-kolla08:46
openstackgerritHanxi Liu proposed openstack/kolla master: Stop using old gnocchi reference  https://review.openstack.org/46977208:46
*** chas has quit IRC08:51
*** mgoddard has joined #openstack-kolla08:52
*** pbourke_ has quit IRC08:53
*** pbourke_ has joined #openstack-kolla08:53
*** zhubingbing has quit IRC08:56
*** zhubingbing has joined #openstack-kolla08:57
*** manheim has quit IRC09:00
openstackgerritHanxi Liu proposed openstack/kolla master: Stop using old gnocchi reference  https://review.openstack.org/46977209:00
*** manheim has joined #openstack-kolla09:00
*** powerd has joined #openstack-kolla09:06
*** chas has joined #openstack-kolla09:07
*** zhubingbing_ has joined #openstack-kolla09:11
*** chas has quit IRC09:12
*** zhubingbing has quit IRC09:12
*** caoyuan_ has quit IRC09:12
*** sambetts|afk is now known as sambetts09:13
egonzalezHi guys, regarding gnocchi not being an openstack project anymore09:17
egonzalezshould we still supporting gnocchi source installs or move source images to consume binary packages?09:17
openstackgerritHanxi Liu proposed openstack/kolla master: Stop using old gnocchi reference  https://review.openstack.org/46977209:18
masbergood afternoon all, I would like to edit nova.conf file, I see that kolla creates multiple ones, does it matters which one to use? I want to add a dhcp_domain entry to change my FQDN through cloud-init http://paste.openstack.org/raw/611180/09:19
openstackgerritHanxi Liu proposed openstack/kolla-ansible master: Stop using old gnocchi reference  https://review.openstack.org/46977109:19
egonzalezmasber, all depends on which service what to change the config09:21
egonzalezif is for all nova services, /etc/kolla/config/nova.conf09:22
egonzalezie: nova-compute /etc/kolla/config/nova/nova-compute.conf09:22
masberI just want to change the FQDN for my vms09:23
masberI am not sure which nova service is managing that, I assume nova-api?09:23
masberor should I just change it for all (/etc/kolla/config/nova.conf)?09:23
egonzalezactually is not nova who manages fqdn, is neutron https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/neutron/templates/neutron.conf.j2#L5409:25
egonzalezbut I opened a but in nova 2 days ago because also need to enforce use_neutron: true in nova.conf09:26
masberis this outdated then http://lists.openstack.org/pipermail/openstack/2017-March/019000.html???09:26
*** caoyuan has joined #openstack-kolla09:27
egonzalezmasber, sth like this http://paste.openstack.org/show/611182/09:28
*** chas has joined #openstack-kolla09:28
masberegonzalez, I assume I have to create /etc/kolla/config/nova.conf in my network nodes? or just openstack controller?09:31
masberbecause I cana;t find it09:31
egonzalezmasber, in the deployer node (ansible)09:31
egonzalezthose are custom files that are merged with nova or neutron conf files09:31
reidraclooks like there's an issue with reno 3.2.1; it uses six but fails to include it in the requirements09:31
*** caoyuan has quit IRC09:31
masberdo I need to reconfigure afterwards?09:31
reidraceh, 2.3.109:31
egonzalezmasber, this way you can customize your services09:32
SerlexDitto reidrac09:32
*** chas has quit IRC09:32
egonzalezmasber, it depends of what changed, you can do it afterwards if is just a config option change, if need a db sync something else will be needed09:32
egonzalezmasber, ie, here is done after deploy to include a config option https://github.com/openstack/kolla-ansible/blob/master/doc/designate-guide.rst#neutron-and-nova-integration09:33
reidraclooks like is a regression: https://bugs.launchpad.net/reno/+bug/159673109:34
openstackLaunchpad bug 1596731 in reno "Undeclared dependency on six" [Undecided,Fix released] - Assigned to Jay Faulkner (jason-oldos)09:34
egonzalezreidrac, is a setuptools bug https://github.com/pypa/setuptools/issues/104209:34
reidracegonzalez: good one, thanks09:35
*** openstackstatus has quit IRC09:36
egonzalezthanks to hrw ;)09:37
*** openstackstatus has joined #openstack-kolla09:38
*** ChanServ sets mode: +v openstackstatus09:38
masberegonzalez, yes I did that already new vms' got the domain from designate, however it reverts back to novalocal after reboot09:39
masberI also have use_neutron = True on mostly all nova.conf files --> http://paste.openstack.org/raw/611184/09:42
-openstackstatus- NOTICE: There is a known issue with setuptools 36.0.0 and errors about the "six" package. For current details see https://github.com/pypa/setuptools/issues/1042 and monitor #openstack-infra09:44
egonzalezmasber, thats right, the only missing is the cron file09:46
egonzalezmasber, are you using master or ocata?09:47
*** shardy is now known as shardy_afk09:47
*** chas has joined #openstack-kolla09:49
*** zhubingbing_ has quit IRC09:52
*** chas has quit IRC09:53
*** yangyapeng has joined #openstack-kolla09:56
*** yangyapeng has quit IRC09:56
*** tovin07_ has quit IRC09:57
*** salv-orl_ has quit IRC10:04
*** cuongnv has quit IRC10:04
*** shardy_afk is now known as shardy10:05
*** caowei has quit IRC10:05
*** chas has joined #openstack-kolla10:10
*** iniazi has joined #openstack-kolla10:10
*** iniazi_ has quit IRC10:13
*** chas has quit IRC10:14
*** chas has joined #openstack-kolla10:21
*** kolla-slack has quit IRC10:24
*** kolla-slack has joined #openstack-kolla10:24
*** chas has quit IRC10:26
bjolohowdy10:28
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-ansible master: [WIP] Add dragonflow ansible role  https://review.openstack.org/46586210:31
bjolognocchi not openstack project anymore?10:33
*** zhurong has joined #openstack-kolla10:34
bjolowhat has replaced it?10:34
masberegonzalez, not sure centos binary 4.0.010:49
egonzalezbjolo, they decided go outside openstack https://julien.danjou.info/blog/2017/gnocchi-independence10:50
*** caoyuan has joined #openstack-kolla10:52
*** powerd has quit IRC10:55
*** caoyuan has quit IRC10:56
*** mnasiadka has quit IRC10:57
hrwegonzalez: gnocchi? still allow source10:58
*** powerd has joined #openstack-kolla10:59
*** mnasiadka has joined #openstack-kolla10:59
hrwegonzalez: it is important module and debian lacks up-to-date packages10:59
egonzalezhrw, thats why asked, we dont use to support non openstack or infrastructure services as databases in the source install, only binary packages11:00
hrwegonzalez: how devstack devs went?11:02
*** matrohon has joined #openstack-kolla11:07
*** dave-mccowan has joined #openstack-kolla11:08
*** kbaegis has quit IRC11:09
*** Pavo has joined #openstack-kolla11:19
*** rwallner has joined #openstack-kolla11:22
*** rwallner has quit IRC11:24
*** rwallner has joined #openstack-kolla11:30
*** rhallisey has joined #openstack-kolla11:31
*** skramaja_ has joined #openstack-kolla11:33
*** skramaja has quit IRC11:33
*** janki has quit IRC11:35
*** shardy has quit IRC11:36
*** pradk has joined #openstack-kolla11:45
*** rwallner has quit IRC11:48
openstackgerritMerged openstack/kolla master: Stop using old gnocchi reference  https://review.openstack.org/46977211:53
*** shardy has joined #openstack-kolla11:58
*** skramaja_ has quit IRC11:59
*** skramaja has joined #openstack-kolla11:59
*** goldyfruit has joined #openstack-kolla11:59
*** goldyfruit has quit IRC12:00
*** rwallner has joined #openstack-kolla12:02
*** manheim has quit IRC12:03
*** slagle has joined #openstack-kolla12:09
*** rwallner has quit IRC12:15
*** zhurong has quit IRC12:16
*** krtaylor has quit IRC12:19
*** chas has joined #openstack-kolla12:22
*** rwallner has joined #openstack-kolla12:23
*** rmart04 has joined #openstack-kolla12:25
*** chas has quit IRC12:26
*** ducttape_ has joined #openstack-kolla12:34
*** salv-orlando has joined #openstack-kolla12:39
*** goldyfruit has joined #openstack-kolla12:46
*** ducttape_ has quit IRC12:46
*** eanylin has quit IRC12:47
*** ducttape_ has joined #openstack-kolla12:47
*** salv-orlando has quit IRC12:47
*** janki has joined #openstack-kolla12:49
*** ducttape_ has quit IRC12:52
*** salv-orlando has joined #openstack-kolla12:52
*** xinliang has quit IRC12:57
*** xinliang has joined #openstack-kolla12:58
*** xinliang has quit IRC12:58
*** xinliang has joined #openstack-kolla12:58
*** ansiwen is now known as ansiwen|lunch12:59
*** rwallner has quit IRC13:03
*** rwallner has joined #openstack-kolla13:03
*** ducttape_ has joined #openstack-kolla13:04
*** eanylin has joined #openstack-kolla13:05
*** mewald has joined #openstack-kolla13:06
*** GonZo2000 has joined #openstack-kolla13:08
*** GonZo2000 has quit IRC13:08
*** GonZo2000 has joined #openstack-kolla13:08
*** lucasxu has joined #openstack-kolla13:08
*** ducttape_ has quit IRC13:09
*** ducttape_ has joined #openstack-kolla13:12
*** ducttape_ has quit IRC13:14
*** powerd has quit IRC13:15
Reepicheepqq from ansible newbe.  how intergrade is the kolla_docker comand .. can I call it from my own playbook?13:15
*** ducttap__ has joined #openstack-kolla13:16
Reepicheephere is kinda what I'm trying to do: http://paste.openstack.org/show/611198/13:16
Reepicheepmy goal is to deploy a new cert quickly so I can call it from my acme client and have it verify the cert13:17
*** skramaja has quit IRC13:17
*** bjolo has quit IRC13:18
ReepicheepI started to do it with ad-hoc commands from a bash script but I figured if I half way there I may as well try to do it more proper13:18
egonzalezReepicheep, why not copy the file to /etc/kolla/haproxy/ in haproxy nodes and restart or reconfigure haproxy containers?13:19
egonzalezwhen the container is restarted, config.json will replace old .pem inside containers13:19
Reepicheepoh.. so I can skip the "Copy haproxy.pem to existing container"?13:20
* Reepicheep admits this is the first time I have actually attempted to write any ansible13:21
egonzalezReepicheep, have 2 options, copy the pem into /etc/kolla/certificates and do a reconfigure (all this in deploy node)13:21
egonzalezor copy pem to /etc/kolla/haproxy on each haproxy node and manually restart containers13:21
Reepicheepegonzalez: that second option is what I was attempting to do13:22
Reepicheepcalling kolla-ansible reconfigure or deploy take way to long even with the --tag haproxy13:22
ReepicheepI could replace the kolla_docker: command with "command: docker restart haproxy"13:23
Reepicheepbut I thought eventually if I make this more generic with ansible parameters someone else may want to use it to do something similar13:25
*** salv-orl_ has joined #openstack-kolla13:28
egonzalezReepicheep, estrange, haproxy reconfigures not take too long in my env with --tags haproxy13:29
egonzalezReepicheep, where hangs?13:29
*** rhallisey has quit IRC13:30
*** rhallisey has joined #openstack-kolla13:30
Reepicheephmm. mine always seems to be slow. I timed a deploy --tags haproxy last night and it took 15 minutes .. the full deploy was 35minutes13:30
*** salv-orlando has quit IRC13:31
Reepicheepfor my needs I need it down to preferably less then 30 seconds :|13:31
*** ducttap__ has quit IRC13:32
Reepicheepmaybe my kolla machine needs more resources.. I'm not sure what the bottleneck is13:32
egonzalezReepicheep, in which task hangs?13:33
Reepicheepegonzalez: I'm trying to figure that out now13:34
*** ducttape_ has joined #openstack-kolla13:35
ReepicheepSo I'm watching a redeploy and it has been about a minute before it even gets trough prechecks gathering facts13:37
Reepicheepsorry s/redeploy/reconfigure/ ^ ^13:37
egonzalezReepicheep, how many nodes you have?13:37
egonzalezmaybe take too long to stablish ssh connection13:38
Reepicheepcurrently I'm at 2013:38
ReepicheepI still have more to add13:38
Reepicheeplet me do some timing of ssh connections13:38
sdakemorning13:39
*** daidv_ has joined #openstack-kolla13:40
egonzalezmorning sdake13:40
Reepicheepegonzalez: I'm not thinking it's ssh: http://paste.openstack.org/show/611201/13:41
openstackgerritAndy Smith proposed openstack/kolla-ansible master: Add qdrouterd role for messaging infrastructure component  https://review.openstack.org/46946213:41
Reepicheepthat is a random sample of my nodes13:42
egonzalezReepicheep, a reconfigure --tags haproxy (no load in the env)13:42
egonzalezreal0m36.390s13:42
egonzalezuser0m26.812s13:42
egonzalezsys0m8.804s13:42
Reepicheephow many nodes?13:43
*** rwellum has joined #openstack-kolla13:43
egonzalez3, im not rich ;)13:43
*** rwallner has quit IRC13:43
*** GonZo2000 has quit IRC13:44
sdakeReepicheep which part of the world are you in13:44
Reepicheepinterestingly my reconfigure failed after 4 minutes :(13:44
Reepicheepfatal: [luigi]: FAILED! => {"failed": true, "reason": "the file_name '/usr/share/kolla-ansible/ansible/start.yml' does not exist, or is not readable"}13:44
ReepicheepI'm in central US13:44
egonzalezbut sometimes some VMs hangs in a weird state where take like 1m to stablish ssh connection13:45
sdakeReepicheep intenet speed and storage speed are the two factors13:45
sdaketwo performance factors13:45
*** ducttape_ has quit IRC13:45
sdakeif your running in a vm dont' expect things to be speedy13:45
Reepicheepwell my nodes are all in two racks right next to each other13:45
*** ducttape_ has joined #openstack-kolla13:45
Reepicheepthe kolla machine is the only VM13:46
sdakei see13:46
*** eaguilar has joined #openstack-kolla13:46
Reepicheepand my internet probably isn't the issue I have multilple multi gig links leaving this building13:46
Reepicheepwith pretty good latancy considering I'm in the middle of Nebraska ;)13:47
sdakessdstorage for /var/lib/docker?13:47
sdakeand using a proper backing store filesystem?13:47
Reepicheepyes on all but a few nodes13:47
Reepicheepthe only nodes that do not have all SSD are the storage nodes and those are only for the ceph osd13:48
sdakecool13:48
rwellumMorning sdake13:48
sdakenot using lvm loopback i assuem?13:48
sdakemorning rwellum13:48
sdakeReepicheep kolla takes 20 minutes to eploy on 120 nodes13:49
openstackgerritAndy Smith proposed openstack/kolla-ansible master: Add support for hybrid messaging backends  https://review.openstack.org/46896613:49
Reepicheepnope my /var/lib/docker is all on btrfs with at  least it's own partition if not it's own drive (all ssds)13:49
*** jtriley has joined #openstack-kolla13:49
sdakeReepicheep sounds like your doing everything right :)13:49
rwellumsdake: I didn't get to backing out that patch, sbezverk had some questions and the fact that it was more than one fix in the patch stopped me - then I had to leave for the evening. Can we discuss if/when sbezverk is available?13:50
*** ansmith has joined #openstack-kolla13:50
sdakerwellum sure13:50
Reepicheepwell I have another issue now. my reconfigure did fail.. do you see the error I posted?13:50
sdakerwellum the patch looked like one change to me13:50
sdakeReepicheep dont know why that occured sorry13:50
rwellumsdake: https://review.openstack.org/#/c/466536/ - the description has three things:13:52
rwellumhttps://www.irccloud.com/pastebin/kSPzcP1h/13:52
*** powerd has joined #openstack-kolla13:52
Reepicheephmm. it was telegraf it failed on .. let me disable that13:52
*** janki has quit IRC13:54
*** rwallner has joined #openstack-kolla13:54
Reepicheepshould it run the telegraf playbooks when I used --tag haproxy?13:54
*** janki has joined #openstack-kolla13:54
sbezverkrwellum :  I am working on additional check which will check nova-api for registered compute resource before starting cell job13:55
egonzalezReepicheep, there is a known issue with telegraf reconfiguration13:56
egonzalezlet me find the review13:56
*** sgrasley has joined #openstack-kolla13:56
blallauhttps://review.openstack.org/#/c/464039/13:57
Reepicheepegonzalez: thanks.. I'm going to just disable it for now.. actually I'm disabling everything but the bare min until I get my slowness figured out13:57
egonzalezReepicheep, was merged yesterday the backport https://review.openstack.org/#/c/450846/213:57
Reepicheepok.. I my move to stable/ocata instead of the kolla-ansible 4.0.1 that pip installed13:59
*** caoyuan has joined #openstack-kolla14:00
*** ducttape_ has quit IRC14:01
*** emccormick has joined #openstack-kolla14:01
*** ducttape_ has joined #openstack-kolla14:01
*** janki has quit IRC14:02
*** ducttape_ has quit IRC14:04
*** manheim has joined #openstack-kolla14:04
*** mnasiadka has quit IRC14:05
*** ducttape_ has joined #openstack-kolla14:06
*** korzen has quit IRC14:07
*** ducttape_ has quit IRC14:08
*** manheim has quit IRC14:08
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Use labels in pv/pvc  https://review.openstack.org/46987014:10
*** sgrasley has quit IRC14:12
*** sgrasley has joined #openstack-kolla14:12
egonzalezcan I get eyes on cinder binary ubuntu fix https://review.openstack.org/#/c/457993/  zun deploy fix https://review.openstack.org/#/c/456256/14:14
mewaldIs ist possible cinder-api container is broken?14:16
*** sgrasley has quit IRC14:16
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Use labels in pv/pvc  https://review.openstack.org/46987014:16
mewaldI build on stable ocata and get "/usr/local/bin/kolla_start: line 24: exec: cinder-api: not found"14:16
*** sgrasley has joined #openstack-kolla14:17
egonzalezmewald, yep only ubuntu binary is broken14:17
egonzalezmewald, ubuntu packages removed support for cinder-api command and only works under apache now :/14:17
mewaldDo we have a bug report for that? I couldnt finde any that would match14:19
egonzalezmewald, https://bugs.launchpad.net/kolla/+bug/167675114:20
openstackLaunchpad bug 1676751 in kolla-ansible pike "cinder-api keeps restarting after the deployment" [Critical,In progress] - Assigned to Eduardo Gonzalez (egonzalez90)14:20
egonzalezand many marked as duplicated of this14:20
openstackgerritMerged openstack/kolla-ansible master: Avoid facts gathered twice when deploying with --limit  https://review.openstack.org/46496314:21
mewaldegonzalez: your patch definitely has enough +2 :D What's blocking?14:22
egonzalezmewald, only have +1 atm https://review.openstack.org/#/c/457993/14:22
mewaldah I was looking at the container only14:23
mewaldwe have to get this out fast, broken cinder is really really bad :D14:23
hrw206 debian/source images built in 51 minutes14:24
hrwnice14:24
egonzalezhrw, nice14:24
hrwegonzalez: on aarch64 machine ;D14:25
hrwwithout ssd ;D14:25
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix zun deployment  https://review.openstack.org/45625614:26
Reepicheepit seems to be the "Gathering Facts" that is slowest14:33
Reepicheepon almost all roles14:33
-openstackstatus- NOTICE: python-setuptools 36.0.1 has been released and now making its way into jobs. Feel free to 'recheck' your failures. If you have any problems, please join #openstack-infra14:33
mewaldegonzalez: Also your Docker container patch was not backported into ocata ...14:34
*** openstackgerrit has quit IRC14:34
egonzalezmewald, yep, i dont want to backport both changes until master are merged14:35
Reepicheepdo the facts get writen to the kolla machine?  could that be the bottle neck?14:35
mewaldyeah makes sense14:35
mewaldI am testing your patches right now in my lab14:36
egonzalezmewald, thanks, let me know any issue14:36
*** jaosorior has quit IRC14:36
*** openstackgerrit has joined #openstack-kolla14:38
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/newton: [DNM]: test kolla newton branch  https://review.openstack.org/46987914:38
hrwam I the only one who thinks that hub.docker.com documentation suxx?14:38
*** kbaegis has joined #openstack-kolla14:41
*** jamesbenson has joined #openstack-kolla14:44
*** ducttape_ has joined #openstack-kolla14:45
jamesbensonGood morning everyone14:45
hrwhi jamesbenson14:45
*** sean-k-m1 is now known as sean-k-mooney14:47
jamesbensoninc0, are you free?14:47
hrwhttps://hub.docker.com/r/haerwu/kolla-aarch64-debian-source-base/ - does such naming has sense?14:47
kolla-slack<inc0> Barely awake:)14:47
*** hongbin has joined #openstack-kolla14:47
jamesbenson ^_^14:48
rwellumsbezverk: sounds good - let me know if i can help?14:48
hrw~curse docker cli14:52
mewaldegonzalez: the patches work like a charm, I will +1 although it's worth nothing :D14:52
egonzalezmewald, every review are good14:53
*** chas has joined #openstack-kolla14:53
hrwwhat you guys use to mass tag/push images to registry?14:54
hrwmewald: every review matters14:54
mewaldreminds me of the "black lives matter" thing :D14:55
jamesbensonhrw: kolla-ansible pull -i multinode14:55
hrwjamesbenson: push not pull14:56
jamesbensonah sorry, I thought you wanted to pull the images to the machines....14:56
hrwjamesbenson: I built 206 images and now have to tag them and push14:56
jamesbensonhrw: wow, just a few....14:56
hrwjamesbenson: current solution was "docker images|grep kolla-aarch64|sort|vim -, then make a script for tag and next one for push14:57
jamesbensonprobably a good approach....14:57
*** chas has quit IRC14:58
hrwjamesbenson: imho 'docker push kolla-aarch64/*:4.0.0' would be better ;D14:59
*** mewald has quit IRC14:59
openstackgerritMerged openstack/kolla stable/ocata: Remove sudo operations that are no longer necessary  https://review.openstack.org/46865515:00
*** manheim has joined #openstack-kolla15:02
*** reidrac_ has joined #openstack-kolla15:02
*** reidrac has quit IRC15:02
*** reidrac has joined #openstack-kolla15:05
*** manheim has quit IRC15:06
jamesbensonSo I'm having an issue with Fernet keys failing in HA environment.... it's yelling at me for a rsync protocol version mismatch, is your shell clean?15:09
jamesbensonIt's under the TASK [keystone : Initialise fernet key authentication]15:10
*** zhubingbing has joined #openstack-kolla15:12
zhubingbinghi guys15:12
egonzalezsup zhubingbing15:13
*** Serlex has left #openstack-kolla15:14
*** Serlex has quit IRC15:14
*** kbaegis has quit IRC15:14
*** sgrasley has left #openstack-kolla15:15
zhubingbinghi egonzalez15:15
zhubingbing;)15:15
jamesbensonhi zhubingbing15:15
zhubingbinghi jamesbenson15:15
zhubingbing;)15:15
*** manheim has joined #openstack-kolla15:17
*** klindgren__ has joined #openstack-kolla15:18
*** klindgren_ has quit IRC15:21
jamesbensonI need to manually run a command, what's keystone's groupname?  keystone?15:21
zhubingbingkolla-ansible post-deploy15:22
zhubingbingsource /etc/kolla/admin-openrc15:22
kfox1111morning15:24
*** reidrac_ has quit IRC15:29
*** reidrac has quit IRC15:29
*** ansiwen|lunch is now known as ansiwen15:29
*** reidrac has joined #openstack-kolla15:29
*** rmart04 has quit IRC15:30
*** openstackgerrit has quit IRC15:33
inc0ok, now I'm back15:35
jamesbensonmorning inc015:35
jamesbensonDo you have time to continue helping to debug that fernet issue from yesterday?15:36
inc0hrw repushing images takes some "bash-fu"15:36
inc0jamesbenson: happily, but I'm not good at fernet15:36
inc0but let's try15:36
sbezverkinc0: do you have by any chance a working example of curl requesting keystone for a token?15:36
jamesbensonwell currently the docker image is constantly rebooting and I can't find anything in the logs besides that "rsync protocol version mismatch, is your shell clean?"15:36
inc0sbezverk: no15:37
sbezverkkfox1111: good morning15:37
sbezverkinc0: ok will keep looking then15:37
inc0jamesbenson: in docker logs?15:37
jamesbensonyeah15:37
inc0sbezverk:  if you do nova --debug list it will print out curls15:38
sdakesbezverk if that works, that sounds like a wonderful solution!15:38
inc0including keystone15:38
sdakesbezverk re your msg about 1 hr go15:38
sbezverkinc0: thanks for the idea!!15:38
sdakesup kfox111115:38
sdakesup inc015:39
sdakesup sbezverk  :)15:39
jamesbensonhttp://paste.openstack.org/show/611219/15:39
jamesbensoninc0: and remember, this was a completely clean environment, just redeployed the OS's....15:41
*** manheim has quit IRC15:41
inc0jamesbenson: try googling this error, first time I see it15:42
inc0brb, food15:42
*** manheim has joined #openstack-kolla15:42
*** chas has joined #openstack-kolla15:43
*** Mannheim has joined #openstack-kolla15:43
*** unicell1 has joined #openstack-kolla15:44
*** unicell has quit IRC15:44
jamesbensoninc0:  I might have found a solution... it's been a while since I've contributed to openstack, and never to kolla.  I'd love to help build/add to the pre-tests to help produce more successful deployments out of the box, any suggestions?  Or should I just do bug fixes, etc?15:45
*** manheim has quit IRC15:46
*** openstackgerrit has joined #openstack-kolla15:47
openstackgerritJuan J. Martínez proposed openstack/kolla master: Remove dead code related to the Vagrant dev environment  https://review.openstack.org/46993015:47
*** matrohon has quit IRC15:49
*** zhubingbing has quit IRC15:58
*** ntpttr has quit IRC15:59
*** iniazi_ has joined #openstack-kolla16:00
Reepicheephow important is it to configure ansible with piplining=True and forks?16:01
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix outdated keystone-paste.ini file  https://review.openstack.org/46993416:02
*** bmaltais has joined #openstack-kolla16:03
*** iniazi has quit IRC16:03
bmaltaisHi. I am trying to figure out how to apply kvm/qemu config on a per node basis in kolla-ansible.16:05
bmaltaisI am putting the needed changes in file /etc/kolla/config/nova/{node name}/nova-compute.conf16:05
egonzalezbmaltais, per node or per service?16:06
bmaltaisIs it where it is supposed to go?16:06
egonzalezper node is /etc/kolla/config/nova/{node name}/nova.conf16:06
bmaltaisPer node. Some need to be set as qemu16:06
bmaltaisok... so not nova-compute.conf16:06
bmaltaisI tried to use the same file name as the "global" way of setting it16:07
*** egonzalez has quit IRC16:09
*** caoyuan has quit IRC16:09
*** lucasxu has quit IRC16:10
*** zhubingbing has joined #openstack-kolla16:10
bmaltaisegonzalez: Working. I had two issues. file name and was using "node1" instead as "node1.fqdn.name" for the folder...16:12
*** mgoddard has quit IRC16:12
*** ntpttr has joined #openstack-kolla16:13
jamesbensoninc0: no luck...I tried enabled ssh between the nodes; didn't fix it. rsync versions are the same as well...16:17
*** chas has quit IRC16:18
*** powerd has quit IRC16:22
*** mnasiadka has joined #openstack-kolla16:23
*** sambetts is now known as sambetts|afk16:23
*** mnasiadka has quit IRC16:27
inc0kfox1111: around?16:28
*** jascott1 has joined #openstack-kolla16:28
inc0jamesbenson: we use ssh in container16:28
inc0there is keystone-ssh container16:28
*** lucasxu has joined #openstack-kolla16:29
kfox1111inc0: meeting in 5 min. back at about 10:00.16:29
kfox1111it quick? :)16:29
inc0kk I was hoping you could help jamesbenson debug the thing16:29
kfox1111ah. k.16:30
inc0I don't know anything about keystone fernet16:30
*** sayantan_ has joined #openstack-kolla16:30
kfox1111kk. I know a thing or two about it. but not how kolla-ansible distributes the keys.16:30
kfox1111what is the problem?16:30
*** lucasxu has quit IRC16:30
inc0I'll help on that front16:30
inc0jamesbenson: ^16:31
inc0kfox1111: http://paste.openstack.org/show/611219/16:31
kfox1111that does not apear to be a fernet issue, but a distribution issue?16:32
kfox1111rsync not working.16:32
inc0yeah, my question would be, have you seen anything like that?16:32
inc0does kolla-k8s solve distribution already?16:32
kfox1111usually when rsync versions are two far out of sync.16:32
kfox1111it says whta t he server is. what about the client version?16:33
kfox1111k8s solves it a completely different way, using k8s natively for shipping.16:33
kfox1111cant reuse it I'm afraid.16:33
*** tonanhngo has joined #openstack-kolla16:33
kfox1111it does a k8s job to do the rotation that updates a k8s secret, and the updated secrets automaticaly get pushed by k8s to the pods.16:34
*** blallau has quit IRC16:35
inc0right, we can use configmaps for distribution16:35
inc0or secret rather16:35
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix outdated barbican-api-paste.ini file  https://review.openstack.org/46993916:36
*** rwallner has quit IRC16:39
*** mnasiadka has joined #openstack-kolla16:40
*** mnasiadka has quit IRC16:40
inc0can I have more reviews please? https://review.openstack.org/#/c/466007/16:41
inc0this is really really important to me...16:41
*** rwallner has joined #openstack-kolla16:41
jamesbensonkfox1111: the clients are all the same version..16:41
*** hyakuhei has quit IRC16:41
*** hyakuhei has joined #openstack-kolla16:41
*** hyakuhei has quit IRC16:41
*** hyakuhei has joined #openstack-kolla16:41
inc0jamesbenson: can you check if ssh from inside keystone-ssh works?16:41
*** rmart04 has joined #openstack-kolla16:42
*** rwallner_ has joined #openstack-kolla16:42
jamesbensonr2-710-6:~# docker exec keystone_ssh "ssh r2-710-7"16:43
jamesbensonrpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: container_linux.go:247: starting container process caused \"exec: \\\"ssh r2-710-7\\\": executable file not found in $PATH\"\n"16:43
jamesbensoninc0: something like: ?16:43
*** eaguilar has quit IRC16:43
*** rwallne__ has joined #openstack-kolla16:43
inc0hmm16:43
inc0that doesn't look good16:44
inc0hold on, let me deploy setup with fernet16:44
jamesbensonoutside of the containers, I can ssh without issue..16:44
inc0yeah, fernet will use ssh in container16:44
jamesbensonI've got 3 controllers if that helps16:44
jamesbensoninc0 Do you want my multinode?16:45
jamesbensonglobals?16:45
*** rwallner has quit IRC16:45
inc0show me globals please16:46
inc0I'll try exactlyu same env16:46
*** unicell1 has quit IRC16:46
jamesbensonhttps://github.com/JamesOBenson/Kolla-misc/blob/master/globals.yml16:47
*** rwallner_ has quit IRC16:47
jamesbensonhttps://github.com/JamesOBenson/Kolla-misc/blob/master/multinode16:48
kfox1111back. meeting was short.16:49
jamesbensoninc0: Separating the components like that is okay, in the multinode16:49
inc0yeah looks good16:50
inc0hold on let me setup something very similar16:50
*** rmart04 has quit IRC16:51
*** rmart04 has joined #openstack-kolla16:52
*** MasterOfBugs has joined #openstack-kolla16:52
*** shardy has quit IRC16:53
sbezverkkfox1111: can you check this reply from keystone:  paste.openstack.org/show/61122716:53
inc0jamesbenson: what images did you use?16:53
jamesbensoninc0 and kfox1111: in the fernet-node-synch.sh file, the rsync commands there don't seem to work manually on the system or keystone containers16:53
jamesbenson4.0.016:53
sbezverkkfox1111: by some reason I do not get token id16:53
inc0built by yourself of pulled from dockerhub?16:53
jamesbensonpulled from dockerhub16:54
kfox1111sbezverk: looking16:54
inc0mhm so you probably want to build them yourself;)16:54
inc0dockerhub 4.0.0 are very old16:54
jamesbenson:-(16:54
inc0we're working on clean pipeline to provide fresh images16:55
jamesbensoneta?16:55
inc0in fact this is my personal #1 priority now16:55
jamesbenson*high five*16:55
kfox1111sbezverk: do a -v on the curl and look at the return code?16:55
inc0well, lots of moving pieces that involve more than just Kolla community16:55
inc0so hard to tell16:55
kfox1111did it succeed?16:55
inc0for now, if you want fresh images you can use small tool I made16:56
inc0https://github.com/inc0/kolla_dockerhub_pusher16:56
sbezverk201 created16:56
kfox1111hmm... weird.16:57
kfox1111any error in the keystone logs?16:57
*** sayantan_ has quit IRC16:57
inc0jamesbenson: install this thing and do kolla-dockerhub-pushed download --base ubuntu --type source --release ocata16:57
sbezverkkfox1111: can you check this reply from keystone:  paste.openstack.org/show/61122916:57
*** sayantan_ has joined #openstack-kolla16:57
kfox1111sbezverk: is that the token in the response headers?16:58
sbezverkkfox1111: maybe it has something with fernet?16:58
kfox1111X-Subject-Token?16:58
kfox1111it looks about right to me.16:58
jamesbensoninc0: how do I install it?  I've cloned it down16:59
sbezverkkfox1111: I see it, it is just different from documented api16:59
kfox1111yeah.16:59
kfox1111also looks like they may have a bug.16:59
inc0jamesbenson: pip install -e .16:59
inc0from dir16:59
kfox1111tehy have the Vary: X-Auth-Token set but return X-Subject-Token.16:59
kfox1111I think those shoudl match up.17:00
kfox1111though doesn't llook like they turn on any caching anyway, so may be ignored all together.17:01
jamesbensoninc0: kolla-dockerhub-pushed command not found after the pip install17:01
inc0pusher17:01
sbezverkkfox1111: I guess then I need to parse header and not returning json17:01
inc0try kolla-doc<<tab>>;)17:01
jamesbensonah got it17:01
inc0what it will do is it'll pull latest ocata images built in gates17:02
kfox1111sbezverk: this related to the previous topic we had about deps?17:02
inc0so last week17:02
Reepicheepso it looks like my system takes about 5-8 seconds for each role when it is "Gathering Facts",  everything else flys by17:02
sbezverkkfox1111: yes17:02
inc0oh sorry, today;)17:02
kfox1111sbezverk: you trying to avoid using the cli tools?17:03
sbezverkkfox1111: I did not wanted to depend on openstack tools17:03
jamesbensonyeah, downloading the tarball now...17:03
kfox1111sbezverk: can be done but a bit hard to get all the bits right.17:03
Reepicheepthat seems to be at least where my stuff is takin so long.. now I need to figure out why17:03
inc0after that you can do kolla-dockerhub-pusher ls to see what do you have17:04
inc0I think tag will be 4.0.217:04
jamesbensonok17:04
inc0so mind the tag17:04
sbezverkkfox1111: yeah, it might be problematic as they do not follow their own documentation17:04
inc0make sure to put it into openstack_release in globals17:04
kfox1111sbezverk: indeed.17:04
kfox1111openstack does not make it easy on users to use. :/17:05
sbezverkI guess relying on client tool moves responsibilityy ;)17:05
kfox1111yup. :)17:05
*** rmart04 has quit IRC17:06
kfox1111less code to maintain ourselves.17:06
kfox1111and gets more testing then we would give it.17:06
jamesbensoninc0 if I find bugs in the code for that, how do you want me to let you know?  through github? here?17:07
sbezverkkfox1111: sound good, will do it with cli17:09
inc0jamesbenson: https://bugs.launchpad.net/kolla-ansible17:09
*** lucasxu has joined #openstack-kolla17:12
*** salv-orl_ has quit IRC17:14
*** unicell has joined #openstack-kolla17:15
jamesbensoninc0: thanks, finding a few small ones..17:15
*** jmccarthy has quit IRC17:15
*** dmsimard is now known as dmsimard|off17:16
inc0jamesbenson: there is good chance they're already fixed in master17:16
*** chas has joined #openstack-kolla17:18
inc0ahh jamesbenson you were asking for bugs in dockerhub pusher?;)17:18
kfox1111sbezverk: did you get an answer to your fernet question?17:18
inc0github then please17:18
jamesbensonyeah17:19
jamesbensonsorry17:19
kfox1111sbezverk: looking at it, it looks like k8s is trying to authenticate the client with a client ssl cert.17:19
kfox1111sbezverk: which it shouldn't be doing if its using a service account token.17:19
jamesbensoninc0: thought that was an odd place for errors in the pusher17:19
kfox1111seems like it may be rejecting the service account token for some reason.17:19
kfox1111guessing rbac related?17:19
inc0jamesbenson: so few things I forgot to mention17:20
inc0it needs python317:20
sbezverkkfox1111: if you refer to my email, then strangely reboot cluster helped :(17:20
openstackgerritMerged openstack/kolla stable/newton: Fix Creating Ceilometer MongoDB database failed  https://review.openstack.org/45651317:20
inc0tarball lib in python2.7 is no bueno:()17:20
kfox1111sbezverk: weird... selinux get enabled by and update? or maybe token rotation failed.17:21
*** daidv_ has quit IRC17:21
*** Slower has quit IRC17:23
jamesbensonpython3 is installed by default in ubuntu16.. Got 3.5.1-317:24
jamesbensoninc0: is it smart enough to use 3 vs 2 thought? python -V pulled python217:24
inc0negative17:25
*** eaguilar has joined #openstack-kolla17:25
jamesbensonok, updated an alias...17:26
jamesbensononce it's complete with the pull, and issue an ls what happens?17:26
kfox1111inc0: all the more reason to containerize stuff. :)17:28
kfox1111include the right python in the container and no more fighting with versions. :)17:28
inc0that actually makes sense17:29
inc0I'll make Dockerfile for this thing:)17:29
kfox1111hehe. :)17:29
kfox1111containers turn everythign inside out. :)17:29
jamesbensonyeah, sort of fighting small bugs in this, but I love the idea/concept.. a docker might help a bunch17:29
inc0jamesbenson: it's quite literally 2 days of me hacking around17:29
jamesbensonand thank you so much inc0 and kfox1111 for all of the help!17:30
jamesbensondamn impressive for 2 days.  :-)17:30
*** eaguilar has quit IRC17:30
inc0it's not that complex17:30
jamesbensonI consider myself a programmer, but would be able to hack this in 2 days.17:30
inc0it really downloads a file, untars it17:30
jamesbensonwell, looks more impressive then :-p17:30
inc0and runs docker registry with -b /path/to/untar:/var/lib/registry17:30
inc0jamesbenson: all because of progress bars17:31
inc0everything is better with progress bar17:31
jamesbensonyeppers17:31
jamesbensoninc0:17:31
jamesbensonConflict. The container name "/kolla-registry" is already in use by container17:31
inc0you can override this I think17:32
jamesbenson?17:32
jamesbensonrm the container?17:32
inc0that will definetly help;)17:32
jamesbensoninc0:  Want me to assist with patches on this on your github page?17:33
jamesbensoninc0: after all of this is done, do you think this will work?  Did the deploy succeed on your side17:34
inc0if you want, I'll be happy to:)17:34
inc0downloading registry now, I found few bugs in this tool myself;)17:35
*** rwallne__ has quit IRC17:35
jamesbensonin python2 at least, /tmp/kolla dir needs to be verified exists; and numeric_owner=True in tarballs it didn't like...17:36
jamesbensonissuing the ls command it doesn't like because the _manifests/tags doesn't exist17:36
*** rwallner has joined #openstack-kolla17:37
jamesbensoninc0: ls giving me a IndexError: list index out of range17:38
inc0jamesbenson: that's exactly why you need python3:)17:39
jamesbensonI'm using 3 now. :-). still giving me it17:39
inc0alias won't do I think17:39
inc0but you can always manually untar it17:39
inc0go to /tmp/kolla17:39
inc0and untar tarball17:39
*** rwallner has quit IRC17:42
*** mgoddard has joined #openstack-kolla17:42
jamesbensonthe first command works now...., it's the ls that fails17:44
*** kelv_ has joined #openstack-kolla17:44
jamesbensonI did the alias in my bashrc and re sourced it17:44
*** Manheim has joined #openstack-kolla17:46
*** eaguilar has joined #openstack-kolla17:46
jamesbensoninc0: i know why it's yelling, it's because of the lokolla directory17:48
jamesbensonthat's where it stops the tagging in the ls function.17:48
inc0lokolla is our "gate" registry17:48
*** Mannheim has quit IRC17:48
inc0mv lokolla kolla17:48
kfox1111inc0: any reason to not fix that at the gate?17:49
kfox1111have it do kolla instead of lokolla?17:49
inc0there is reason we did it in the gate in the first place17:49
kfox1111yeah?17:50
inc0to ensure that images are pulled from local registry not dockerhub, can't remember exact scenario tho17:50
kfox1111ah. could that be resolved by just double tagging?17:50
*** salv-orlando has joined #openstack-kolla17:50
kfox1111tag the images both kolla and lokolla?17:51
kfox1111meeting. bbiab.17:51
inc0well, this can be resolved while retagging to push17:51
inc0it's really internal17:51
jamesbensonrunning into a bunch of issues, do you have a manual way of doing it?17:52
jamesbensonor guide?17:52
openstackgerritOpenStack Proposal Bot proposed openstack/kolla master: Updated from global requirements  https://review.openstack.org/46956717:52
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements  https://review.openstack.org/46956817:52
inc0so you untarred it?17:52
*** zhubingbing has quit IRC17:53
inc0but yeah, I see issues with fernet myself17:53
inc0different issues tho17:53
jamesbensoninc0: ?17:54
jamesbensonyes, untarred17:55
inc06docker run -d --name registry --restart=always -p 4000:5000 -v /tmp/kolla/registry/:/var/lib/registry registry:217:56
jamesbensonk17:57
*** ducttap__ has joined #openstack-kolla18:00
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding extra condition for simple cell0 create job  https://review.openstack.org/46996318:01
*** ducttape_ has quit IRC18:04
*** rwallner has joined #openstack-kolla18:07
rwellumsbezverk: if the cell0 issue doesn't show up in the gates anyway - then the gate tests for #469963 won't reveal whether this is fixed?18:11
*** eaguilar has joined #openstack-kolla18:12
*** mgoddard has quit IRC18:13
*** eaguilar has quit IRC18:17
hrwinc0: noticed ;D18:18
hrwinc0: building 206 images: 51 minutes. pushing them: 164 minutes (+5 for scripting)18:18
*** ducttape_ has joined #openstack-kolla18:19
*** ducttap__ has quit IRC18:22
*** rwallner has quit IRC18:23
*** salv-orlando has quit IRC18:25
jamesbensoninc0:  I verified my registry is there (curl -X GET http://10.245.122.31:5000/v2/kolla/kolla/ubuntu-source-keystone/tags/list) and looked at the tag of keystone, 4.0.2....18:26
jamesbensontrying to figure out what I need to do next for the local registry... modify the docker.server, /etc/default/docker, and the restart the service?18:27
*** rwallner has joined #openstack-kolla18:29
sbezverkrwellum: the condition will be bullet proof if it passes it will guarantee that you will not hit it..18:29
sdakekfox1111 i dont know how a storage provider works, and i've read numerous pv/pvc docs and blogs etc18:30
sdakekfox1111 mind giving me a quick explination?18:30
*** eaguilar has joined #openstack-kolla18:30
kfox1111ok. but not sure which part your confused about. what are you wanting to know?18:31
sdakeyou said use a storage provider instead of a label?18:31
sdakei understand how labels select things18:31
kfox1111oh.18:31
sdakenot storage providers18:31
kfox1111not 100% sure on this one, but pv's are assigned to storage providers. Some are dynamic,18:32
kfox1111where there is something watching on that for creation requests and deleting them when they are done.18:32
kfox1111I think there is a static verision too.18:32
trozetmgkwill: hi18:32
kfox1111where an admin can create pv's that users can ask for a particular type.18:32
kfox1111like you may create some 'fast' and some 'slow' pv's, that users can request from.18:33
kfox1111I'm thinking, for the use case your trying to cover, maybe we make a 'kolla' type.18:33
sdakeyes - the use csae i want is to make sure mariadb maps to mariadb every time18:33
kfox1111so a random pvc request won't ask for the 'kolla' pool.18:33
kfox1111just the ones from kolla.18:34
kfox1111that shoudl allow the names to always match up I think.18:34
sbezverksdake: I suspect main issue will happen when we start having multiple mariadb or rabbitmq, your labeling system will fail..18:34
kfox1111sbezverk: true. element name will need to go along with it.18:34
kfox1111the mariadb service chart would do pv/pvc/svc/statefulset all with the same element name18:35
sdakeat pressent with the serach path, isn't there only one mariadb that may be deployed?18:36
kfox1111each instantiation of the chart would change the elment name to ensure uniqueness.18:36
sbezverkkfox1111: right, but I am not sure adding complexity to pv naming is really worth it, anonymous PV sounds better to me ;)18:36
sdakeanonymous pv is not good - as if i delte the pvc, and then later recreate it, it connects to hte wrong thing in some circumstances18:36
* kfox1111 shrugs.18:36
kfox1111sdake: you should never delete a pvc unless your done done with it.18:37
kfox1111there is not a garantee you can ever get the data back after that event.18:37
kfox1111same as with cinder.18:37
sbezverksdake: right and then we need to recycle if this service is really gone..18:37
kfox1111sbezverk: right now we're in a bad state I think.18:39
kfox1111we half implement pv creation.18:39
kfox1111if we want to do it right by your way, we should drop pv microservices entirely.18:39
kfox1111or we should fix them so they work right, which is what sdake's trying to do.18:40
kfox1111I'm kind of good either way, as long as its optional.18:40
trozetinc0, mgkwill : there is some issue where for me, opendaylight doesnt start with the dumb-init bash shell when the exec runs in kolla start.  It will run if i just use a regular bash shell though18:40
sbezverkkfox1111: I would vote for excluding PV from microservices18:41
sbezverkas this most likely the optiom will be used in REAL production18:41
kfox1111I'm in that boat, so I don't disagree with that.18:42
mgkwilltrozet: hmm. weird. I don't know if I have any helpful info there.18:45
trozetmgkwill: well how do you start it in kolla-ansible?18:45
mgkwilltrozet: I wonder what the difference is between your start and kolla-ansible start.18:45
mgkwilltrozet: I'll have to look but I have to head to offsite. Will get back to you later.18:46
trozetmgkwill: exactly18:46
trozetmgkwill: ok np18:46
*** emccormick has quit IRC18:48
*** chas has quit IRC18:55
*** chas has joined #openstack-kolla18:55
*** egonzalez has joined #openstack-kolla18:57
trozetdoes anyone else know what dumb-init does in kolla start? egonzalez ?18:58
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding extra condition for simple cell0 create job  https://review.openstack.org/46996319:00
*** chas has quit IRC19:03
egonzaleztrozet good explanation https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html19:05
trozetegonzalez: yeah i found this one and was just reading it https://github.com/Yelp/dumb-init19:06
trozetegonzalez: but the opendaylight start script launches a java process, and for some reason this doesnt work with the dumb-init19:06
egonzaleztrozet odl server starts with mgkwill's ansible change19:07
trozetegonzalez: it tries to run another exec inside of the odl start script, i wonder if that is the issue19:08
trozetegonzalez: yeah i was having trouble finding where in his patch the opendaylight start command is19:08
egonzaleztrozet line 2 https://review.openstack.org/#/c/416367/57/ansible/roles/opendaylight/templates/opendaylight.json.j219:09
trozetegonzalez: oh he doesnt use start, he just starts karaf directly...let me try that19:10
egonzaleztrozet yeah, we use to put the command  to execute the process directly19:10
openstackgerritArtom Lifshitz proposed openstack/kolla master: DNM: testing Nova fix for 1694666  https://review.openstack.org/46998019:11
egonzalezsame as other services19:11
trozetegonzalez: ok cool, it started for me19:11
*** Pavo has quit IRC19:11
trozetegonzalez: let me try it with full openstack deployment now, thanks for your help!19:12
*** sayantan_ has quit IRC19:20
*** sayantan_ has joined #openstack-kolla19:27
*** sayantan_ has quit IRC19:32
*** dave-mccowan has quit IRC19:32
*** sayantan_ has joined #openstack-kolla19:40
*** ducttape_ has quit IRC19:42
*** sayantan_ has quit IRC19:44
*** dave-mccowan has joined #openstack-kolla19:46
*** randylev has joined #openstack-kolla19:47
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding extra condition for simple cell0 create job  https://review.openstack.org/46996319:52
*** bmaltais has quit IRC19:54
*** ducttape_ has joined #openstack-kolla19:54
*** salv-orlando has joined #openstack-kolla19:55
*** rwallner has quit IRC19:58
*** sayantan_ has joined #openstack-kolla19:59
*** klowse1 has left #openstack-kolla19:59
*** rwallner has joined #openstack-kolla20:02
*** spsurya has quit IRC20:03
jamesbensonvery odd fail in the prechecks, TASK [rabbitmq : fail] : Hostname has to resolve to IP address of api_interface20:06
*** rwallner has quit IRC20:07
*** sayantan_ has quit IRC20:15
*** sayantan_ has joined #openstack-kolla20:16
*** mgoddard has joined #openstack-kolla20:16
Manheimthey to put the IP in /etc/hosts20:22
Manheimtry*20:23
Manheimi remember I had this issue in older releases20:23
*** dave-mccowan has quit IRC20:32
jamesbensonyeah, debugging it now, they IP's are all in the /etc/hosts... the IP's seem to be there in the rabbitmq_hostnames.results, the ansible function "{{ hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] }}" seems to have an error in it....20:38
*** madhukar has joined #openstack-kolla20:44
egonzalezjamesbenson verify  if pinging to the hostname pings the correct IP20:44
*** jrist has quit IRC20:45
egonzalezsometimes resolv to localhost20:45
jamesbensonecho $HOSTNAME is good in each machine...ping is good20:46
madhukarHi All, IS there a document which tells which docker hub container tag maps to which openstack-release?20:46
madhukarFor example docker.io/kolla/ubuntu-source-neutron-server:3.0.3 ==> newton20:47
egonzalezmadhukar nope afik, 4.0.* ocata, 3.0.* newton, etc20:47
*** jtriley has quit IRC20:47
jemcevoyHas anyone seen this error "Could not resolve hostname none hostname none" when kolla is generating ceph keys?   http://paste.openstack.org/show/611248/20:48
madhukarThanks @egonzalez! It also means mitaka ==> 2.0.*20:48
madhukarThats what I was looking for :)20:48
*** eaguilar has quit IRC20:48
*** jrist has joined #openstack-kolla20:49
jamesbensonjemcevoy: I haven't seen that one before, but have had issues with ceph as well, which is a bummer...20:50
egonzalezjamesbenson check if the api interface  ip appears at getent ahostsv4 <hostname>20:51
jamesbenson192.168.2.5     STREAM r2-710-620:53
jamesbensonegonzalez: This fails on the three controllers.... http://paste.openstack.org/show/611249/20:54
jamesbensonegonzalez:  Is that what is expected?20:54
egonzalezjamesbenson if those are the IPs in api_interface yes20:55
jamesbensonapi_interface is just eno120:55
*** manheim_ has joined #openstack-kolla20:56
egonzalezjamesbenson eno1 in r2-710-6 have IP 192.168.2.5 ?20:57
jamesbenson192.168.2. CIDR is for eno120:57
jamesbensonyep20:57
jamesbensonegonzales:  I have a 10. for br020:59
*** manheim_ has quit IRC21:01
*** manheim_ has joined #openstack-kolla21:02
jamesbensonegonzalez: sorry for the typo.21:02
*** chas has joined #openstack-kolla21:03
inc0hey sorry, I'm back21:04
inc0lots of meetings ad hoc21:04
jamesbensonno worries inc021:06
jamesbenson:-)21:06
inc0this day is killing me, I can't think straight:/ did you get your issue solved jamesbenson?21:06
*** manheim_ has quit IRC21:06
*** jamesPR has joined #openstack-kolla21:06
jamesPRsup everyone21:06
*** lucasxu has quit IRC21:06
egonzalezjamesbenson share prechecks with -vvv in the command21:07
jamesbensonnope, blew it out and trying again... lol, now failing in prechecks!  ...  rabbit can't resolve all hostnames ?!?21:07
egonzalezjamesbenson should be sth like this http://paste.openstack.org/show/611251/21:07
inc0jamesbenson: riiiiight21:07
inc0did you run kolla-ansible bootstrap servers?21:07
inc0kolla-ansible -i multinode bootstrap_servers21:07
randylevWe are running into some issue with Kolla and Magnum.  I'm curious to know if anyone has or had Kolla-ansible / Magnum / Newton working.21:08
jamesPRppl every deployment I have been testing deploy without issues but when I stop all container and try to deploy again mariadb : Waiting for MariaDB service to be ready and it fails21:08
inc0hey Randy, so, what kind of issues?21:08
*** chas has quit IRC21:09
egonzalezjamesPR stop or  destroy?21:09
inc0also I know we've put quite a bit of work into it during ocata timeframe21:09
jamesbensoninc0: I saw something about ssh not working between servers in a forum, which failed for me too, but corrected it, passing root ssh key to everyone, but still failed...21:09
jamesPRegonzalez, stop21:09
jamesbensonegonzalez: here's the output... http://paste.openstack.org/show/611252/21:10
egonzalezjamesPR to recover mariadb cluster need to follow this  steps (doc under review yet) https://review.openstack.org/#/c/466881/14/doc/operating-kolla.rst21:10
*** manheim_ has joined #openstack-kolla21:11
randylevinc0: I'm trying to get details from the dev team.  Before we spend too  much time on it, I was just wondering if other have gotten it up and running.  The ppl working on this feature are in India.21:11
inc0randylev: yes, we did, including keynote in Barcelona:)21:11
inc0but that was ocata21:11
jamesbensoninc0: I did, but going to run it again just for giggles...21:11
inc0or was it? can't remember really21:11
inc0jamesbenson: cool, what was the issue?21:12
randylevinc0: ok.  I was wondering if we would have to move to Octa.21:12
*** manheim_ has quit IRC21:12
jamesbensonvery odd fail in the prechecks, TASK [rabbitmq : fail] : Hostname has to resolve to IP address of api_interface21:12
egonzalezwas newton,  ocata came in atlanta21:13
jamesPRegonzalez, ok trying21:13
egonzalezjamesPR cannot see the execution of that task, only the ssh connection stuff21:14
egonzalez^^ jamesbenson21:14
inc0randylev: lots of good things happened in ocata, but if you can't let's try to fix it21:14
randylevinc0:  Thanks.  I'll get more information from the dev team and have them ask again.  They must asked on this channel when folks were not paying attention.21:15
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding extra condition for simple cell0 create job  https://review.openstack.org/46996321:16
inc0well we have quite a lot of contributors in Asia21:16
inc0just ask them to be persistent;)21:16
randylevWe are trying to wrap up some final features on Newton now, and then jump to Pike later on.21:17
randylevinc0: I will...many thanks for your help.21:17
inc0oh, that's gonna be quite a ride randylev21:17
jamesbensonegonzalez: rabbitmq : Check if all rabbit hostnames are resolvable task reports roughly the same.... the fail test next, actually fails instead of skips21:17
inc0be very careful with upgrading21:17
inc0as N->O was not a straightforward upgrade21:18
*** Mannheim has joined #openstack-kolla21:18
inc0if you'll be running Kolla-ansible, I strongly encourage to do upgrade to Ocata and Ocata to Pike right after that21:18
inc0but don't skip Ocata21:18
egonzalezjemcevoy the previous task failed and keys was not generated, may find more info doing docker logs bootstrap_ceph_mon21:19
*** sayantan_ has quit IRC21:19
randylevinc0: Fortunately, I can re-install in my lab and don't have to do an upgrade :D21:19
inc0right, that makes it easier;)21:20
*** sayantan_ has joined #openstack-kolla21:20
*** Manheim has quit IRC21:20
jamesbensonegonzalez: http://paste.openstack.org/show/611254/21:21
*** manheim has joined #openstack-kolla21:22
jamesbensoninc0: never could get the docker repo working right though....21:23
jamesbensonstill on 4.0.021:23
*** krtaylor has joined #openstack-kolla21:23
inc0jamesbenson: meh, well, bear with us, soon21:24
jamesbenson:-)21:24
jamesbensonI tried once before and had issues too :-(21:24
inc0speaking of which, I really really need reviews on this https://review.openstack.org/#/c/466007/21:24
egonzalezjamesbenson seems the hostname is right, what jinja version have installed?21:25
*** pramodrj07 has joined #openstack-kolla21:26
*** manheim has quit IRC21:26
*** sayantan_ has quit IRC21:26
jamesbensonJinja2 (2.9.6)21:26
*** manheim has joined #openstack-kolla21:27
*** manheim has quit IRC21:28
jamesbensonegonzalez: ^^21:28
*** manheim has joined #openstack-kolla21:28
egonzalezhrm, ansible doesn't like 2.9.6 iirc,  try with 2.8.121:28
*** MasterOfBugs has quit IRC21:29
*** manheim has quit IRC21:32
jamesPRegonzalez, doesn't work21:32
*** powerd has joined #openstack-kolla21:33
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multi-scenerio gates  https://review.openstack.org/46681521:33
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600721:33
jamesbensonegonzalez: fixed version, still fails... could the {{ }} that are required for ansible be the issue for jinja?21:34
jamesbensongot to run... debug some more...21:34
*** jamesbenson has quit IRC21:35
jemcevoyWill check docker logs egonzalez Have not had a problem here after kickstarting cluster but this tile I just tried a cleanup.  Made some temp network changes that I did want to code in my ansible prep for this one case...21:35
*** myoung is now known as myoung|gone21:36
egonzalezjamesbenson maybe something changed in ansible, i use 2.3.0.0 with jinja 2.8.1 and prechecks dont complain about syntax (only warnings)21:36
egonzalezjamesPR what  are the erors or where is failing?21:38
egonzalezjamesPR when do stop, are you trying to deploy a new env or recover the stopped?21:38
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600721:39
*** goldyfruit has quit IRC21:39
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding extra condition for simple cell0 create job  https://review.openstack.org/46996321:40
*** salv-orlando has quit IRC21:40
sbezverkkfox1111: rwellum: it is still WIP but appreciate your review. ^^^21:41
sbezverkI tested it locally and it seems to be working..21:41
jamesPRegonzalez, I'm just testing kolla for future production environment, I just simply deploy the cluster without issues then I test stoping the cluster with kolla-ansible -i inventory stop and then deploy again, then mysql is not starting21:41
*** jamesbenson has joined #openstack-kolla21:41
inc0jamesPR: my question is, you sure you want to stop cluster?21:42
inc0it's not great for mariadb21:42
egonzalezjamesPR to stop databases need a clean stop, following the previous link guide you thorough the process to clean stop databases21:42
inc0it might happen, but shouldn't be a rule21:42
*** emccormick has joined #openstack-kolla21:44
*** jascott1 has quit IRC21:45
jamesPRinc0, if happends what can I do? ;) that's the question21:45
jamesPRstress test ;)21:45
*** jamesbenson has quit IRC21:46
inc0jamesPR: well, just follow regular galera maintenance then;)21:46
*** jascott1 has joined #openstack-kolla21:47
jamesPRinc0, good point ;)21:49
jemcevoyegonzalez:  old config left around. Looks like I did not do a good cleanup between deploys.  I deleted all docker instances, images and volumes but what else?  rm -rf /etc/kolla?   http://paste.openstack.org/show/611258/21:50
jemcevoyany other places to clean?21:50
egonzalezjemcevoy if using ceph clean up  osd disks21:50
egonzalezkolla-ansible destroy  do most of the cleanups except the disk21:51
jemcevoyI did that,,, Entries in the fstab, umount and relabel disks...21:51
jemcevoyShould I wipe /etc/kolla/... across the cluster?21:52
jemcevoyI will try the destroy///21:52
egonzalezjust use destroy command, will remove containers, config files21:52
*** ipsecguy_ has joined #openstack-kolla21:56
*** ipsecguy has quit IRC21:59
*** jascott1 has quit IRC22:02
*** jascott1 has joined #openstack-kolla22:03
*** jascott1 has quit IRC22:04
*** jascott1 has joined #openstack-kolla22:06
*** rwallner has joined #openstack-kolla22:06
*** powerd has quit IRC22:07
*** jascott1 has quit IRC22:10
*** rwallner has quit IRC22:11
*** egonzalez has quit IRC22:12
openstackgerritOpenStack Proposal Bot proposed openstack/kolla master: Updated from global requirements  https://review.openstack.org/46956722:12
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements  https://review.openstack.org/46956822:12
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Fix fernet deployment on certain configurations  https://review.openstack.org/47004022:20
*** powerd has joined #openstack-kolla22:28
jemcevoyegonzalez:  Thanks for the pointers on ceph... After running the kolla destroy... I had to my docker cleanup which found a few images and 2 volumes to delete on the controllers.  Then I relabeled all the osd drives and got further... All the OSDs are rebuilding :-)22:39
jemcevoyinc0 and sean-k-mooney:  I ended up having to manually edit the /etc/kolla/nova-scheduler/nova.conf to have the two separate pci_alias entries to allow both GPU types to work with the scheduler.  Attempts to configure pci_aliases as a single list entry failed when the nova scheduler tried to read the entry.22:44
sdakesbezverk in production, we are using pv_enabled: false and modifying the PVCs to have labels associated with them - indepdnent of what upstream does - woudl be nice if upstream matched what was going into production22:46
*** Mannheim has quit IRC22:46
*** manheim has joined #openstack-kolla22:54
*** powerd has quit IRC22:56
*** jascott1 has joined #openstack-kolla22:58
*** madhukar has quit IRC22:59
*** rhallisey has quit IRC23:00
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Fix fernet deployment on certain configurations  https://review.openstack.org/47004023:01
*** jamesbenson has joined #openstack-kolla23:07
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600723:08
*** manheim_ has joined #openstack-kolla23:09
*** jamesbenson has quit IRC23:11
*** manheim has quit IRC23:12
*** rwellum has quit IRC23:13
*** emccormick has quit IRC23:13
inc0beep beep red alert, prechecks are broken23:21
inc0https://review.openstack.org/#/c/470040/ seems like something bad happened today23:22
jascott1:|23:23
kfox1111jascott1: hey. :)23:25
jascott1hey kfox111123:25
kfox1111will you have time soon to work on the --node-selector -o json thing?23:25
inc0ansible 2.3.1 broken23:25
inc0fun life23:25
jascott1yeah hopefully I can start on it tomorrow23:25
kfox1111ok. cool.23:25
jascott1working helm issue 2331 atm23:25
inc0yeah, released 7hrs ago23:26
jascott1bug in importValues23:26
kfox1111if you don't have time and don't mind, I can take a quick stab at a first pass too.23:26
jascott1inc0 hows it borken?23:26
kfox1111the windows folks are chomping at the bit for a fix,23:26
kfox1111and I'd rather it be a proper one then a hack.23:26
jascott1i definitely dont mind23:26
inc0http://logs.openstack.org/40/470040/2/check/gate-kolla-ansible-dsvm-deploy-centos-source-centos-7-nv/c459bc6/console.html#_2017-06-01_23_09_15_39728323:27
kfox1111ok. no promices I can get to it, but we'll see if any time frees up.23:27
kfox1111I'll ping you before I start just in case.23:27
jascott1kfox1111 ill check with ye if I can start23:27
*** jamesbenson has joined #openstack-kolla23:27
kfox1111ok. cool. thanks. :)23:27
inc0https://bugs.launchpad.net/kolla-ansible/+bug/1695111 I have to run, be back later23:28
openstackLaunchpad bug 1695111 in kolla-ansible "Ansible 2.3.1 broke preckecks" [Critical,Triaged]23:28
inc0in the meantime if someone could take a look, it will be greatly appreciated23:28
*** jamesbenson has quit IRC23:31
*** athomas has quit IRC23:38
*** ducttape_ has quit IRC23:41
*** jamesbenson has joined #openstack-kolla23:48
*** tonanhngo has quit IRC23:52
*** jamesbenson has quit IRC23:52
*** Pavo has joined #openstack-kolla23:52
*** manheim_ has quit IRC23:53
*** dklyle has joined #openstack-kolla23:55
*** hongbin has quit IRC23:56
*** tonanhngo has joined #openstack-kolla23:58
*** david-lyle has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!