Thursday, 2019-12-05

*** strobert1 has quit IRC00:09
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Custom CA certificates deployment  https://review.opendev.org/68602400:30
*** goldyfruit_ has joined #openstack-kolla01:20
*** k_mouza has joined #openstack-kolla01:33
*** k_mouza has quit IRC01:37
openstackgerritChason Chan proposed openstack/kolla-ansible master: Optimize reconfigure action for swift  https://review.opendev.org/69652501:46
*** igordc has quit IRC02:54
*** oncall-pokemon has quit IRC03:06
*** zhanglong has joined #openstack-kolla03:33
*** gfidente|afk is now known as gfidente03:43
*** zhanglong has quit IRC03:44
*** watersj has joined #openstack-kolla03:48
*** zhanglong has joined #openstack-kolla03:49
watersjhello, is it a know issue for preflight "Checking docker SDK version" fail on localhost for deployment. deployment is not baremetal so why it checking for docker?03:50
*** gfidente has quit IRC03:53
*** schwicht has joined #openstack-kolla04:19
*** schwicht has quit IRC04:19
*** pcaruana has joined #openstack-kolla05:38
*** pcaruana has quit IRC05:42
*** raukadah is now known as chkumar|ruck05:45
*** sri_ has joined #openstack-kolla05:46
sri_morning05:47
*** vmixor has quit IRC05:48
*** Luzi has joined #openstack-kolla06:03
*** jaicaa has quit IRC06:14
*** jaicaa has joined #openstack-kolla06:16
*** dciabrin_ has joined #openstack-kolla06:48
*** dciabrin has quit IRC06:49
mnasiadkamorning06:53
*** lennyb has joined #openstack-kolla07:09
*** pcaruana has joined #openstack-kolla07:17
*** jonaspaulo has quit IRC07:31
*** jbadiapa has joined #openstack-kolla07:31
*** sm806 has quit IRC07:33
*** sm806 has joined #openstack-kolla07:34
yoctozeptomorning07:50
hrwmorning07:53
*** zhanglong has quit IRC07:56
hrw'So we use Ansible to install Ansible so we are able to use Ansible'08:05
*** tonythomas has joined #openstack-kolla08:18
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: OVN Support  https://review.opendev.org/69684108:21
chkumar|ruckhrw: yoctozepto morning08:24
chkumar|ruckplease have a look on this https://review.opendev.org/#/c/697236/ when free !08:24
patchbotpatch 697236 - kolla - Added python3 packages for mellanox agent container - 6 patch sets08:24
*** sm806 has quit IRC08:26
*** sm806 has joined #openstack-kolla08:27
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: OVN Support  https://review.opendev.org/69684108:31
*** rpittau|afk is now known as rpittau08:32
*** bodgix has quit IRC08:34
*** niedbalski has quit IRC08:34
*** gfidente has joined #openstack-kolla08:37
*** bodgix has joined #openstack-kolla08:38
*** niedbalski has joined #openstack-kolla08:38
*** gfidente has quit IRC08:41
mnasiadkayoctozepto: https://review.opendev.org/#/c/697262/ - apache logging stuff08:44
patchbotpatch 697262 - kolla-ansible - Improve Apache logging - 4 patch sets08:44
*** cgrosjean has joined #openstack-kolla08:48
hrwchkumar|ruck: +208:49
yoctozeptochkumar|ruck: w+109:11
yoctozeptomnasiadka: checking09:11
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: OVN Support  https://review.opendev.org/69684109:12
yoctozeptomnasiadka: cr+209:15
mnasiadkayoctozepto: awesome09:16
openstackgerritYang Youseok proposed openstack/kolla-ansible master: Configure proxy settings for bootstrap-servers  https://review.opendev.org/69200609:22
chkumar|ruckhrw: yoctozepto Thanks!09:23
openstackgerritRadosław Piliszek proposed openstack/kolla stable/train: Install etcd3gw to fix Ubuntu binary tooz coordination  https://review.opendev.org/69742609:30
openstackgerritRadosław Piliszek proposed openstack/kolla stable/stein: Install etcd3gw to fix Ubuntu binary tooz coordination  https://review.opendev.org/69742709:30
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Use python 3 for local kolla-ansible execution  https://review.opendev.org/69606109:34
*** dr_feelgood has joined #openstack-kolla09:36
mgoddardmorning09:42
*** bodgix has quit IRC09:54
*** niedbalski has quit IRC09:54
*** sri_ has quit IRC09:54
*** goldyfruit_ has quit IRC09:54
*** Statscowski has quit IRC09:54
*** spiette has quit IRC09:54
*** dpawlik has quit IRC09:54
*** hjensas has quit IRC09:54
*** hamzy has quit IRC09:54
*** jschlueter has quit IRC09:54
*** dmellado has quit IRC09:54
*** tonythomas has quit IRC09:54
*** jaicaa has quit IRC09:54
*** andrein has quit IRC09:54
*** cloudnull has quit IRC09:54
*** owalsh has quit IRC09:54
*** TheJulia has quit IRC09:54
*** mnasiadka has quit IRC09:54
*** rpittau has quit IRC09:54
*** EmilienM has quit IRC09:54
*** r3ap3r has quit IRC09:54
*** emccormick has quit IRC09:54
*** mrunge has quit IRC09:54
*** Satarel has quit IRC09:54
*** openstackgerrit has quit IRC09:54
*** antonym has quit IRC09:54
*** brtknr has quit IRC09:54
*** Reepicheep has quit IRC09:54
*** evrardjp has quit IRC09:54
*** dr_feelgood has quit IRC09:54
*** jcmdln has quit IRC09:54
*** dasp has quit IRC09:54
*** mgoddard has quit IRC09:54
*** iniazi_ has quit IRC09:54
*** _Cyclone_ has quit IRC09:54
*** logan- has quit IRC09:54
*** mloza has quit IRC09:54
*** stingrayza has quit IRC09:54
*** kevinz has quit IRC09:54
*** markmcclain has quit IRC09:54
*** yankcrime has quit IRC09:54
*** stackedsax has quit IRC09:54
*** spotz has quit IRC09:54
*** dabukalam has quit IRC09:54
*** pcaruana has quit IRC09:54
*** Luzi has quit IRC09:54
*** lennyb has quit IRC09:54
*** factor has quit IRC09:54
*** Jeffrey4l has quit IRC09:54
*** numans has quit IRC09:54
*** cosmicsound has quit IRC09:54
*** baha has quit IRC09:55
*** irclogbot_3 has quit IRC09:55
*** benj_ has quit IRC09:55
*** klindgren_ has quit IRC09:55
*** ianw has quit IRC09:55
*** bjoernt has quit IRC09:55
*** dosaboy has quit IRC09:55
*** heikkine has quit IRC09:55
*** obre has quit IRC09:55
*** larsks has quit IRC09:55
*** mjturek has quit IRC09:55
*** strigazi has quit IRC09:55
*** Tengu has quit IRC09:55
*** rouk has quit IRC09:55
*** jroll has quit IRC09:55
*** gouthamr has quit IRC09:55
*** LaurentDumont has quit IRC09:55
*** egonzalez has quit IRC09:55
*** dirtwash has quit IRC09:55
*** crindi has quit IRC09:55
*** jrist has quit IRC09:55
*** alanmeadows has quit IRC09:55
*** gmann has quit IRC09:55
*** cz2 has quit IRC09:55
*** cgrosjean has quit IRC09:55
*** sm806 has quit IRC09:55
*** jbadiapa has quit IRC09:55
*** dciabrin_ has quit IRC09:55
*** watersj has quit IRC09:55
*** osmanlicilegi has quit IRC09:55
*** negronjl has quit IRC09:55
*** yoctozepto has quit IRC09:55
*** skramaja has quit IRC09:55
*** calbers has quit IRC09:55
*** Blinkiz has quit IRC09:55
*** jistr has quit IRC09:55
*** patchbot has quit IRC09:55
*** sean-k-mooney has quit IRC09:55
*** tonyb has quit IRC09:55
*** hrw has quit IRC09:55
*** kukacz_ has quit IRC09:55
*** chkumar|ruck has quit IRC09:55
*** billp has quit IRC09:55
*** noxoid has quit IRC09:55
*** gary_perkins has quit IRC09:55
*** timburke has quit IRC09:55
*** portdirect has quit IRC09:55
*** csmart has quit IRC09:55
*** zbr_ has quit IRC09:55
*** vesper11 has quit IRC09:55
*** rphillips has quit IRC09:55
*** verdurin has quit IRC09:55
*** zzzeek has quit IRC09:55
*** mgkwill has quit IRC09:55
*** mwhahaha has quit IRC09:55
*** rwellum_ has quit IRC09:55
*** ironfoot has quit IRC09:55
*** generalfuzz has quit IRC09:55
*** rm_work has quit IRC09:55
*** ChanServ has quit IRC09:55
*** niedbalski has joined #openstack-kolla10:04
*** bodgix has joined #openstack-kolla10:04
*** dmellado has joined #openstack-kolla10:04
*** jschlueter has joined #openstack-kolla10:04
*** hamzy has joined #openstack-kolla10:04
*** hjensas has joined #openstack-kolla10:04
*** dpawlik has joined #openstack-kolla10:04
*** spiette has joined #openstack-kolla10:04
*** Statscowski has joined #openstack-kolla10:04
*** goldyfruit_ has joined #openstack-kolla10:04
*** sri_ has joined #openstack-kolla10:04
*** emccormick has joined #openstack-kolla10:04
*** EmilienM has joined #openstack-kolla10:04
*** r3ap3r has joined #openstack-kolla10:04
*** rpittau has joined #openstack-kolla10:04
*** mnasiadka has joined #openstack-kolla10:04
*** TheJulia has joined #openstack-kolla10:04
*** owalsh has joined #openstack-kolla10:04
*** cloudnull has joined #openstack-kolla10:04
*** andrein has joined #openstack-kolla10:04
*** jaicaa has joined #openstack-kolla10:04
*** tonythomas has joined #openstack-kolla10:04
*** evrardjp has joined #openstack-kolla10:04
*** Reepicheep has joined #openstack-kolla10:04
*** brtknr has joined #openstack-kolla10:04
*** antonym has joined #openstack-kolla10:04
*** openstackgerrit has joined #openstack-kolla10:04
*** mrunge has joined #openstack-kolla10:04
*** Satarel has joined #openstack-kolla10:04
*** cgrosjean has joined #openstack-kolla10:04
*** sm806 has joined #openstack-kolla10:04
*** jbadiapa has joined #openstack-kolla10:04
*** pcaruana has joined #openstack-kolla10:04
*** lennyb has joined #openstack-kolla10:04
*** dciabrin_ has joined #openstack-kolla10:04
*** Luzi has joined #openstack-kolla10:04
*** watersj has joined #openstack-kolla10:04
*** osmanlicilegi has joined #openstack-kolla10:04
*** negronjl has joined #openstack-kolla10:04
*** heikkine has joined #openstack-kolla10:04
*** spotz has joined #openstack-kolla10:04
*** dabukalam has joined #openstack-kolla10:04
*** zbr_ has joined #openstack-kolla10:04
*** chkumar|ruck has joined #openstack-kolla10:04
*** yoctozepto has joined #openstack-kolla10:04
*** skramaja has joined #openstack-kolla10:04
*** Tengu has joined #openstack-kolla10:04
*** calbers has joined #openstack-kolla10:04
*** vesper11 has joined #openstack-kolla10:04
*** Blinkiz has joined #openstack-kolla10:04
*** jistr has joined #openstack-kolla10:04
*** factor has joined #openstack-kolla10:04
*** Jeffrey4l has joined #openstack-kolla10:04
*** rouk has joined #openstack-kolla10:04
*** rphillips has joined #openstack-kolla10:04
*** jroll has joined #openstack-kolla10:04
*** numans has joined #openstack-kolla10:04
*** cosmicsound has joined #openstack-kolla10:04
*** dirtwash has joined #openstack-kolla10:04
*** csmart has joined #openstack-kolla10:04
*** gouthamr has joined #openstack-kolla10:04
*** verdurin has joined #openstack-kolla10:04
*** patchbot has joined #openstack-kolla10:04
*** sean-k-mooney has joined #openstack-kolla10:04
*** tonyb has joined #openstack-kolla10:04
*** baha has joined #openstack-kolla10:04
*** hrw has joined #openstack-kolla10:04
*** irclogbot_3 has joined #openstack-kolla10:04
*** benj_ has joined #openstack-kolla10:04
*** klindgren_ has joined #openstack-kolla10:04
*** LaurentDumont has joined #openstack-kolla10:04
*** egonzalez has joined #openstack-kolla10:04
*** kukacz_ has joined #openstack-kolla10:04
*** ianw has joined #openstack-kolla10:04
*** bjoernt has joined #openstack-kolla10:04
*** zzzeek has joined #openstack-kolla10:04
*** mgkwill has joined #openstack-kolla10:04
*** mwhahaha has joined #openstack-kolla10:04
*** rwellum_ has joined #openstack-kolla10:04
*** crindi has joined #openstack-kolla10:04
*** timburke has joined #openstack-kolla10:04
*** dosaboy has joined #openstack-kolla10:04
*** jrist has joined #openstack-kolla10:04
*** billp has joined #openstack-kolla10:04
*** noxoid has joined #openstack-kolla10:04
*** obre has joined #openstack-kolla10:04
*** larsks has joined #openstack-kolla10:04
*** mjturek has joined #openstack-kolla10:04
*** alanmeadows has joined #openstack-kolla10:04
*** strigazi has joined #openstack-kolla10:04
*** gary_perkins has joined #openstack-kolla10:04
*** cz2 has joined #openstack-kolla10:04
*** gmann has joined #openstack-kolla10:04
*** portdirect has joined #openstack-kolla10:04
*** ironfoot has joined #openstack-kolla10:04
*** rm_work has joined #openstack-kolla10:04
*** generalfuzz has joined #openstack-kolla10:04
*** ChanServ has joined #openstack-kolla10:04
*** orwell.freenode.net sets mode: +o ChanServ10:04
*** dr_feelgood has joined #openstack-kolla10:04
*** jcmdln has joined #openstack-kolla10:04
*** dasp has joined #openstack-kolla10:04
*** mgoddard has joined #openstack-kolla10:04
*** iniazi_ has joined #openstack-kolla10:04
*** _Cyclone_ has joined #openstack-kolla10:04
*** logan- has joined #openstack-kolla10:04
*** mloza has joined #openstack-kolla10:04
*** stingrayza has joined #openstack-kolla10:04
*** kevinz has joined #openstack-kolla10:04
*** markmcclain has joined #openstack-kolla10:04
*** yankcrime has joined #openstack-kolla10:04
*** stackedsax has joined #openstack-kolla10:04
*** gfidente has joined #openstack-kolla10:05
*** logan- has quit IRC10:05
*** logan- has joined #openstack-kolla10:06
*** osmanlicilegi has quit IRC10:18
*** osmanlicilegi has joined #openstack-kolla10:19
*** dswebb has joined #openstack-kolla10:20
*** osmanlicilegi has quit IRC10:22
*** numans has quit IRC10:23
*** sri_ has quit IRC10:25
*** numans has joined #openstack-kolla10:26
*** osmanlicilegi has joined #openstack-kolla10:27
*** skramaja_ has joined #openstack-kolla10:32
*** skramaja has quit IRC10:33
*** skramaja_ is now known as skramaja10:33
*** zhanglong has joined #openstack-kolla10:37
*** pcaruana has quit IRC10:42
hrwmorning mgoddard10:46
mgoddardhi hrw10:48
mgoddardnext py2 drop to break us: networking-bgpvpn10:48
hrwmgoddard: thanks for reviews. going through them10:48
mgoddardhttps://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_548/697236/6/gate/kolla-build-centos-source/548ab52/kolla/build/000_FAILED_neutron-server-opendaylight.txt.gz10:48
mgoddardnp10:48
yoctozeptosad, sad10:49
mgoddarddo we still need the neutron-server-opendaylight image?10:51
mgoddardweird that bgpvpn plugin is only in the odl image, not neutron-server10:52
yoctozeptomgoddard: I believe tripleo uses it10:52
yoctozeptothere is a bit of weirdness all around10:52
yoctozeptomgoddard: quick one - re: debian and ubuntu for swift - I added debian due to kernel 4.19 xfs fix10:53
yoctozeptoso essentially debian has the latest kernel10:53
yoctozeptobrings us closest to ubuntu 20.0410:53
mgoddardgood to have tested it, but I don't think we can afford 3 jobs for each scenario10:53
yoctozeptono problem with removal10:53
yoctozeptome neither10:53
yoctozeptoon weekends it flows, but during the week...10:54
yoctozeptowaiting half a day for all the checks is not nice :D10:54
mgoddardI'll look at the odl issue10:54
yoctozeptopity c8 will have an older kernel10:54
yoctozepto(actually "has" but "will" for us)10:55
hrwyoctozepto: do not look at rhel kernels as x.y version.10:56
hrwyoctozepto: they have lot of stuff backported10:56
yoctozeptohrw: but they don't backport breakages like the xfs one :D10:56
hrwsure10:56
yoctozeptoespecially since they coordinate xfs10:57
hrwmgoddard: "echo ; echo ; echo" vs "echo && echo && echo" does not change much imho10:57
mgoddardhrw: it means we don't ignore errors10:58
mgoddardif the first one fails we ignore it10:58
hrwok10:58
yoctozeptomgoddard, hrw: https://review.opendev.org/697426 - and propose kolla rc211:00
patchbotpatch 697426 - kolla (stable/train) - Install etcd3gw to fix Ubuntu binary tooz coordina... - 1 patch set11:00
yoctozeptomgoddard: https://review.opendev.org/694476 <- any ideas how to please you yet fix the issue? :D deprecation does not solve your problem either :/11:02
patchbotpatch 694476 - kolla-ansible - Default to etcd3gw driver for etcd-based coordination - 4 patch sets11:02
hrwmgoddard: I do wonder what to do with that 'is None' check. in normal kolla use those variables will have values so checking them does not have sense even. They are checked in case someone writes something like enable_repos_empty() test does.11:03
mgoddardhrw: just raise an exception?11:03
hrwok, right11:04
openstackgerritMark Goddard proposed openstack/kolla-ansible master: DNM: CI: Enable etcd  https://review.opendev.org/69744111:10
*** k_mouza has joined #openstack-kolla11:10
mgoddardyoctozepto: trying out an upgrade with etcd enabled11:11
mgoddard^11:11
openstackgerritMark Goddard proposed openstack/kolla-ansible master: DNM: CI: Enable etcd  https://review.opendev.org/69744111:11
*** Statscowski has quit IRC11:17
chkumar|ruckmgoddard: on this review https://review.opendev.org/#/c/697236/ https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_548/697236/6/gate/kolla-build-centos-source/548ab52/job-output.txt neutron-server-opendaylight Failed with status: error is it a legit error?11:18
patchbotpatch 697236 - kolla - Added python3 packages for mellanox agent container - 6 patch sets11:18
chkumar|ruckERROR:kolla.common.utils.neutron-server-opendaylight:The command '/bin/sh -c if [ "$(ls /plugins)" ]; then         sed -i 's|^ceilometer===.*$||g' requirements/upper-constraints.txt         && pip --no-cache-dir install --upgrade -c /requirements/upper-constraints.txt /plugins/*;     fi' returned a non-zero code: 111:18
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: add more tests for enable_repos template method  https://review.opendev.org/69735411:18
chkumar|ruckor recheck will do the job?11:19
cosmicsoundgood day11:19
cosmicsoundanyone can help me trace this RUNNING HANDLER [mariadb : remove restart policy from master mariadb]11:20
cosmicsound    "stderr": "Error response from daemon: No such container: mariadb",11:20
*** k_mouza has quit IRC11:20
* hrw goes on bgpvpn error. similar fix as for horizon11:22
hrwmgoddard: or maybe we should have 'sorry_centos7_is_on_py2(plugins_to_remove)' macro?11:23
openstackgerritDincer Celik proposed openstack/kolla master: Disable building binary images for senlin-conductor and senlin-health-manager on debian/ubuntu  https://review.opendev.org/69744511:26
mnasiadkaneutron build broken in master... ugh11:27
mnasiadkahttps://zuul.opendev.org/t/openstack/build/d480b6453d314cc58c57e7281d37aabf/log/kolla/build/000_FAILED_neutron-server-opendaylight.txt.gz#64011:27
mnasiadka(python3 required)11:27
hrwmnasiadka: 12:22  * hrw goes on bgpvpn error. similar fix as for horizon11:27
mnasiadkahrw: I feel in 3 months we'll have all services like that :D11:27
hrw{% macro remove_plugins_because_centos7_is_py2(plugins_to_remove) -%}11:31
*** jbadiapa has quit IRC11:34
hrwany opinions on macro name?11:36
osmanlicilegilike a poem11:39
*** watersj has quit IRC11:39
hrw+    {{ macros.remove_plugins_because_centos7_is_py2(['searchlight-ui', 'neutron-vpnaas-dashboard']) }} \11:39
hrw+        {{ macros.remove_plugins_because_centos7_is_py2(['networking-bgpvpn']) }} \11:39
hrwfirst is horizon, second is neutron-server-opendaylight11:39
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Introduce macro to remove Py3 only plugins on CentOS 7  https://review.opendev.org/69744911:41
hrwRP+211:42
hrwmnasiadka, mgoddard, yoctozepto: ^^11:42
cosmicsoundanyone know why kolla 8 fails with stein on mariadb prechecks when mariadb is disabled.11:45
mgoddardhrw: I was looking at odl :)11:46
hrwmgoddard: ;D11:47
cosmicsoundhttps://pastebin.com/uSycXJCA11:48
mgoddardhrw: I like your approach though11:48
hrwmgoddard: thanks.11:49
hrwmgoddard: one use can be simple. second can be simple too. but more are coming so third+ one can not be simple one11:49
*** zhanglong has quit IRC11:49
mgoddardhrw: I think we also need this:11:50
mgoddard         {% set neutron_server_opendaylight_packages = [11:50
mgoddard-            'python-networking-bgpvpn',11:50
mgoddard-            'python-networking-bgpvpn-heat',11:50
mgoddard             'python-networking-l2gw',11:50
hrwmgoddard: as long as there are packages it is fine imho11:51
mgoddardhrw: assuming they just stop being updated11:51
hrwmgoddard: following patch to do?11:53
mgoddardsure11:53
osmanlicilegicosmicsound: did you configure external mariadb?11:53
mgoddardosmanlicilegi: hi, are you looking at the senlin issue?11:54
osmanlicilegimgoddard: yep https://review.opendev.org/#/c/697445/11:55
patchbotpatch 697445 - kolla - Disable building binary images for senlin-conducto... - 1 patch set11:55
mgoddardhrw: did you test the patch? I think the whitespace is borked11:55
mgoddardosmanlicilegi: cool, thanks11:55
osmanlicilegiI had to reinstall my laptop yesterday, sorry for the delay11:56
hrwmgoddard: INFO:kolla.common.utils.neutron-server-opendaylight:Step 5/6 : RUN if [ "$(ls /plugins)" ]; then         sed -i 's|^ceilometer===.*$||g' requirements/upper-constraints.txt         && rm -rf /plugins/networking-bgpvpn*         && pip --no-cache-dir install --upgrade -c /requirements/upper-constraints.txt /plugins/*;     fi11:56
mgoddardhrw: http://paste.openstack.org/show/787156/11:57
hrwwill fix11:58
hrwthanks11:58
mgoddardhrw: I'm just testing a fix11:59
hrwmgoddard: cool ;)12:00
hrwmgoddard: feel free to send for review12:00
*** sri_ has joined #openstack-kolla12:00
mgoddardwill do12:01
cosmicsoundSeems more people hit this12:02
cosmicsoundShould I try opening a bug for this12:02
cosmicsoundhttps://stackoverflow.com/questions/57985093/while-running-deploy-it-is-showing-problem-in-maria-db12:02
osmanlicilegicosmicsound: please raise a bug for your issue12:03
*** k_mouza has joined #openstack-kolla12:04
openstackgerritMark Goddard proposed openstack/kolla master: Introduce macro to remove Py3 only plugins on CentOS 7  https://review.opendev.org/69744912:08
openstackgerritMark Goddard proposed openstack/kolla master: Introduce macro to remove Py3 only plugins on CentOS 7  https://review.opendev.org/69744912:10
cosmicsoundosmanlicilegi my issue is same issue as in stackoverflow12:11
*** zbr_ is now known as zbr|out12:13
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Use python 3 for local kolla-ansible execution  https://review.opendev.org/69606112:21
*** yan0s has joined #openstack-kolla12:25
openstackgerritMichal Nasiadka proposed openstack/kolla master: Build images with healthcheck scripts for core services  https://review.opendev.org/67612812:27
*** k_mouza has quit IRC12:28
*** k_mouza has joined #openstack-kolla12:29
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services  https://review.opendev.org/67638912:29
dirtwashwhat could be the reason deployment fails at nova : Running Nova bootstrap container]12:31
dirtwashi destroyed the old deployment, didnt change condifs, maybe cleanup is not perfect causing some issue?12:31
roukwell, there is a bunch of bind mounts etc that could still have data. im sure your docker logs would tell you whats up.12:32
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637612:37
*** schwicht has joined #openstack-kolla12:37
*** k_mouza has quit IRC12:51
*** schwicht has quit IRC12:54
*** pcaruana has joined #openstack-kolla12:54
mnasiadkahrw: you were looking for somebody that governs the tripleo ci job that we use in Kolla, right?12:59
hrwyep12:59
chkumar|ruckhrw: mnasiadka What happened to tripleo ci job? How can we help?13:00
mnasiadkachkumar|ruck: I think the main issue we see, is that it fetches the master branch code - not the tested gerrit change.13:01
hrwchkumar|ruck: where I can look at it's code?13:01
hrwchkumar|ruck: would like to see how it is done and maybe suggest changes/improvements13:01
chkumar|ruckhrw: mnasiadka https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers13:06
hrwthanks13:06
cosmicsoundopened it here: https://bugs.launchpad.net/kolla-ansible/+bug/185526813:06
openstackLaunchpad bug 1855268 in kolla-ansible "Error response from daemon: No such container: mariadb" [Undecided,New]13:06
chkumar|rucktripleo-build-containers-rhel-8-master  use above code13:07
hrwchkumar|ruck: will look13:09
chkumar|ruckhrw: if you are not able to find any job look for http://codesearch.openstack.org/ and https://codesearch.rdoproject.org/ if still not able, ping us on #tripleo, we are happy to help13:12
chkumar|ruckmnasiadka: is it about rhel8 job?13:13
mnasiadkachkumar|ruck: yeah, I see in the code that you always check out the branch - it would be really beneficial for tripleo if the job would check the current change, not the code from master branch. It would make our life easier - instead of blaming each other that Kolla change broke the build for tripleo.13:14
hrwchkumar|ruck: thanks13:15
chkumar|ruckmnasiadka: Do we have patch example where it broke so that I can fix it ?13:15
*** goldyfruit_ has quit IRC13:20
mnasiadkachkumar|ruck: it's not about this particular breakage, but the fact rhel8 job fetches zuul.branch (which is master), not the actual change we propose.13:24
mnasiadkahttps://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/main.yaml#L413:24
chkumar|ruckmnasiadka: let me take a llok13:24
hrwmgoddard: fix for macro on a way13:25
mgoddardhrw: I proposed one. Did it fail?13:26
*** schwicht has joined #openstack-kolla13:26
hrwmgoddard: yes.13:26
hrwmgoddard: horizon has 2 plugins so "&& rm -rf path \&& rm -rf path \" happened13:27
mgoddardhmm13:27
hrw~curse whitespaces in jinja213:33
hrw    && rm -rf /plugins/searchlight-ui*&& rm -rf /plugins/neutron-vpnaas-dashboard*13:36
hrwcause space after * was removed13:36
*** schwicht has quit IRC13:38
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637613:41
hrw    && rm -rf /plugins/searchlight-ui* && rm -rf /plugins/neutron-vpnaas-dashboard*  \13:41
hrwuf.13:41
hrw        {% for plugin in plugins_to_remove %}&& rm -rf /plugins/{{ plugin }}* {% endfor %} \13:41
hrwwhitespaces suxx13:42
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Introduce macro to remove Py3 only plugins on CentOS 7  https://review.opendev.org/69744913:42
mgoddardhrw: now has an empty line on ubuntu13:44
hrwgrteuky4e ot8k 53q 76[09a53786t a3-9e 67u24p0986ty72083chkumar|ruck: [81y7!@$13:45
hrwfscking jinja213:45
mgoddardhrw: did the 'else true' cause a problem?13:46
mgoddardthat is what it was for13:46
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Introduce macro to remove Py3 only plugins on CentOS 7  https://review.opendev.org/69744913:48
hrwif not py2.7 then \13:48
hrwmgoddard: sorry, dropped that else at some moment and looked at centos13:48
mgoddardk13:48
*** schwicht has joined #openstack-kolla13:49
openstackgerritMerged openstack/kolla stable/train: Install etcd3gw to fix Ubuntu binary tooz coordination  https://review.opendev.org/69742613:58
mgoddardyoctozepto: I tried etcd3 -> etcd3gw upgrade: https://555931d863fd8c0101b2-58d93d7929b8e264fef7aa823e8ca608.ssl.cf2.rackcdn.com/697441/2/check/kolla-ansible-ubuntu-source-upgrade-ceph/d145f7e/primary/logs/kolla/cinder/cinder-volume.txt.gz14:00
mgoddardcinder cannot connect to etcd using etcd3 driver :/14:01
* hrw out14:01
*** schwicht has quit IRC14:02
mnasiadkamgoddard: why not use redis or zookeeper instead? it seems tooz docs recommend those two over anything else...14:04
mgoddardmnasiadka: etcd is a base service14:05
mnasiadkamgoddard: well, if there is no driver that works as it should, maybe we need to change the solution :)14:05
mnasiadkastill there is the mysql driver for tooz, wonder if that's sufficient14:06
mgoddardmnasiadka: etcd3gw looks promising, I'm just checking the upgrade path14:06
mnasiadkamgoddard: I think I read in some review that etcd3gw works only with single etcd endpoint?14:07
mgoddardmnasiadka: yes, same as etcd314:07
mgoddardmnasiadka: but in the case of etcd3gw we could use haproxy14:07
mgoddard(I assume)14:08
mnasiadkamgoddard: well, I think etcd can work as a reverse proxy - so if we would deploy it locally on each node we need, it will direct you to the active etcd cluster member14:08
mnasiadkamgoddard: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/gateway.md14:10
*** gfidente has quit IRC14:12
*** gfidente has joined #openstack-kolla14:12
mnasiadkamgoddard: but I see we would need to add the gateway setup to the etcd role14:13
*** jbadiapa has joined #openstack-kolla14:15
*** jbadiapa has quit IRC14:16
yoctozeptome on mobile14:20
yoctozeptomysql is bad, not galera-friendly14:20
yoctozeptono touchinf14:21
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Don't fail on exepected critical log messages  https://review.opendev.org/69747814:23
mgoddardmnasiadka: I don't think we need the gateway - haproxy would work14:25
*** Luzi has quit IRC14:28
*** jbadiapa has joined #openstack-kolla14:33
*** zbr|out has quit IRC14:33
*** schwicht has joined #openstack-kolla14:33
*** zbr has joined #openstack-kolla14:35
*** pcaruana has quit IRC14:38
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Enable etcd in ceph and cinder-lvm scenarios  https://review.opendev.org/69748514:39
mnasiadkamgoddard: probably, but I always prefer native solutions :)14:40
mgoddardit's just a tcp proxy afaict14:41
*** zbr_ has joined #openstack-kolla14:42
*** zbr has quit IRC14:44
*** sri_ has quit IRC14:48
*** goldyfruit_ has joined #openstack-kolla14:48
*** zbr_ has quit IRC14:49
mnasiadkaYup, probably with some discovery capabilities14:50
mnasiadkamgoddard: you’re right, let’s go with haproxy14:50
yoctozeptoyou can have haproxy with http I guess - then health checks can also be included14:54
*** zbr has joined #openstack-kolla14:55
*** goldyfruit___ has joined #openstack-kolla15:00
*** goldyfruit_ has quit IRC15:02
*** rpittau is now known as rpittau|afk15:03
yoctozeptomgoddard, mnasiadka: regarding etcd3 driver - it does not seem as popular as etcd3gw15:15
yoctozeptomgoddard: if you can't get etcd3 to work then it just proves nobody uses this combination15:15
mgoddardyoctozepto: perhaps, although you were testing it with zun before right?15:16
yoctozeptomgoddard: that's why I proposed the off toggle15:16
*** dpawlik has quit IRC15:16
yoctozeptothought it was me being lame and I did not need it either way due to ceph15:16
yoctozepto:D15:16
mgoddardyou said zun was unstable with etcd3?15:17
yoctozeptomgoddard: enabling zun requires enabling etcd3 - this caused cinder to pick it up and volumes stopped working15:18
yoctozeptofor clarity, zun does not use etcd315:18
yoctozeptoneither kuryr15:19
yoctozeptobut kuryr needs docker with distributed key-value database15:19
yoctozeptoto support inter-connectivity15:19
yoctozeptoand docker uses neither tooz nor python-etcd3 driver15:19
roukoh hey zun, i recently started testing zun, found a bunch of bugs in playbooks/templates, will make a PR eventually.15:20
roukbut... never had cinder fail yet15:20
yoctozeptorouk: report and propose15:21
roukyep, i gotta clean up my hacks first, i deployed zun on nodes which are not "compute" nodes, and theres needless cross-dep in the playbooks, and theres no ssl websocket  support when using ssl external, breaking exec --interactive, few other things.15:23
roukill make a pr that fixes them all15:23
yoctozeptorouk: ah, yeah, I saw it could break trying to move compute away from zun-compute (or vice versa :-) )15:26
yoctozeptomgoddard, mnasiadka: maybe time to also make nova-compute group properly ^15:27
mgoddardsure15:27
yoctozeptofeels like Ussuri will boil down to rhel8,py3&clean15:27
mgoddardI'm sure we'll squeeze some more in15:27
yoctozeptolike ovn from mnasiadka, yeah15:27
yoctozeptoand cells v2^215:28
chkumar|ruckmgoddard: yoctozepto hrw by the way people started working on centos8 https://review.opendev.org/#/c/688614/ it is coming soon may be 2-3 weeks from rdo side15:29
patchbotpatch 688614 - devstack - [dnm] testing centos-8 bringup - 12 patch sets15:29
yoctozeptochkumar|ruck: very good!15:29
chkumar|ruckOSA people has also started working https://review.opendev.org/#/c/689629/15:30
patchbotpatch 689629 - openstack-ansible - [WIP] Centos-8 support - 13 patch sets15:30
yoctozeptochkumar|ruck: oh noez, we have to hurry to be first :O15:31
chkumar|ruck:-)15:34
hrwchkumar|ruck: https://review.opendev.org/69236815:34
patchbotpatch 692368 - kolla - WIP: move to CentOS 8 in containers - 22 patch sets15:34
chkumar|ruckawesome15:35
*** gfidente has quit IRC15:37
hrwand we moved to centos8 on host for centos7 builds15:37
yoctozeptomgoddard: in these situations I wonder whether it really worked for the original proponent of this (etcd3)15:37
hrwchkumar|ruck: still, even with rdo test repo we lack ceph and few other things15:38
*** ricolin has joined #openstack-kolla15:38
hrwchkumar|ruck: would be good if someone experienced with c8 look at package lists and told which things to drop.15:38
hrwchkumar|ruck: bridge-utils is gone, btrfs stuff is gone, scsi-utils-target is gone. and probably countless other15:39
mgoddardyoctozepto: I'm trying out etcd3 locally. If it's busted we're +215:39
mgoddardbut this stuff matters :)15:39
chkumar|ruckhrw: https://review.rdoproject.org/etherpad/p/moving-rdo-to-centos8 will give an idea what is the status15:39
hrwchkumar|ruck: thanks15:40
chkumar|ruckhrw: we have rdo meeting each wed on 8:30 IST feel free to chim in15:40
hrwchkumar|ruck: 14:00 UTC. right before Kolla one iirc15:40
yoctozeptomgoddard: no doubt15:41
chkumar|ruckhrw: we added the reviews to etherpad, if any questions, please add, it will help to speed up the things15:52
*** chkumar|ruck is now known as raukadah15:57
*** yan0s has quit IRC16:10
*** vmixor has joined #openstack-kolla16:11
*** skramaja has quit IRC16:27
*** igordc has joined #openstack-kolla16:27
*** k_mouza has joined #openstack-kolla16:30
mgoddardyoctozepto: from the initial commit of etcd3+http:16:38
mgoddardUsing grpc to directly access etcd3 does not work well with16:38
mgoddard    eventlet based services16:38
yoctozeptomgoddard: well, it looks stable16:39
mgoddardyoctozepto: etcd3 uses grpc, etcd3+http does not (REST)16:39
mgoddardso it's the right move16:40
yoctozeptomgoddard: ah, misread16:40
yoctozeptosuperb :D16:40
yoctozeptolet me add it to the commit message16:40
mgoddardI couldn't get etcd3 to work16:40
yoctozeptocould you link it?16:40
yoctozeptome neither16:40
mgoddardweirdly I can import tooz directly and do the same stuff cinder is doing and it works16:40
yoctozeptoprobably this eventlet stuff is what makes it break16:41
mgoddardI'll blame eventlet :)16:41
yoctozeptoin my opinion it looks like it can't do anything else16:41
yoctozeptoas it starts timing out on rmq and mdb16:41
mgoddardI have a couple of comments on the PR, can you wait before updating?16:41
yoctozeptomgoddard: sure16:41
*** openstackgerrit has quit IRC16:42
*** jcmdln has quit IRC16:43
yoctozeptowaiting for you anyway16:43
*** goldyfruit_ has joined #openstack-kolla16:43
*** born2bake has joined #openstack-kolla16:43
mgoddardyoctozepto: reviewed16:45
*** goldyfruit___ has quit IRC16:46
*** openstackgerrit has joined #openstack-kolla16:49
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Don't fail on exepected critical log messages  https://review.opendev.org/69747816:49
mgoddardyoctozepto, mnasiadka: looking for a review to unblock CI: https://review.opendev.org/#/c/697449/516:51
patchbotpatch 697449 - kolla - Introduce macro to remove Py3 only plugins on Cent... - 5 patch sets16:51
* hrw off16:52
*** ricolin has quit IRC16:54
*** dave-mccowan has joined #openstack-kolla17:00
yoctozeptomgoddard, hrw: https://review.opendev.org/69744917:07
patchbotpatch 697449 - kolla - Introduce macro to remove Py3 only plugins on Cent... - 5 patch sets17:07
yoctozeptowhat about binary here? why remove?17:07
*** dave-mccowan has quit IRC17:10
raukadahyoctozepto: what is the source of patchbot, I want to bring it to tripleo or rdo channel?17:10
*** gfidente has joined #openstack-kolla17:10
yoctozeptomgoddard: did you check designate as well?17:10
mgoddardyoctozepto: it's not failing, but could do at any time if the py2 package is removed. We'll need to remove that branch of the if eventually anyway17:10
yoctozepto(assume not)17:10
mgoddardyoctozepto: nope17:10
yoctozeptook17:10
yoctozeptolet me check for that eventlet stuff17:11
yoctozeptoraukadah: hah, I brought it here with the powers of magic :D17:11
raukadahwhat is science behind that magic?17:11
yoctozeptoI believe timburke is the sourcerer here :D17:12
*** goldyfruit___ has joined #openstack-kolla17:12
yoctozeptoI saw it first on #openstack-ironic17:13
yoctozeptoand asked around17:13
yoctozeptoturned out to be a bot running on private server using sopel17:13
yoctozeptoraukadah: ^17:13
raukadahah, great, will dig the source out but it is quite useful :-)17:14
raukadahthanks!17:14
yoctozeptoraukadah: yw17:15
*** goldyfruit_ has quit IRC17:15
yoctozeptomgoddard: designate is eventlet-based as well17:15
mgoddardk17:16
yoctozeptoso it's a bye-bye to etcd317:16
*** graysonh has joined #openstack-kolla17:16
yoctozeptomaybe they converted to eventlet in the meantime17:17
yoctozeptowhatever17:17
mnasiadkaback17:19
mnasiadkayoctozepto: there's your last comment in this unblock change - sure we should +w it? :)17:20
*** lennyb has quit IRC17:20
yoctozeptomnasiadka: yeah, now convinced by mgoddard17:21
mnasiadkaok, good - because all my changes fail on this crap :-)17:24
yoctozeptomgoddard, mnasiadka: is the senlin removal for ubuntu/debian binary proposed already?17:26
*** lennyb has joined #openstack-kolla17:26
yoctozeptobecause it broke their jobs recently17:26
mnasiadkabinary is non-voting, what do I know :)17:26
mgoddardyoctozepto: yeah, proposed17:26
yoctozeptook17:26
yoctozeptoah, Isee17:26
mgoddardhttps://review.opendev.org/69744517:27
patchbotpatch 697445 - kolla - Disable building binary images for senlin-conducto... - 1 patch set17:27
mgoddardneed to fix CI first17:27
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637617:27
mnasiadkayoctozepto: coming back to features - when the CI will be unblocked, the docker healthchecks for core services should pass17:28
*** lennyb has quit IRC17:32
graysonhCan someone help me understand what the designate_ns_record key in the globals.yaml is supposed to do? I know it sets dns_domain in neutron-dhcp-agent's neutron.conf which effectively sets the search domain for all VMs, but it also does something in the pools.yaml.j2 template that I don't understand. I've inherited a designate setup that someone else configured, and the current value doesn't resolve and probably never will in17:35
graysonhour setup (which is causing problems since its in the default search path for every VM created)17:35
noxoidi believe its the default NS record for new zones that are created17:42
noxoidi had to override it, it didnt make much sense to me17:42
yoctozeptomnasiadka: great :D17:43
mnasiadkayoctozepto: but that's only the beginning of the path for healthchecks :D17:44
yoctozeptomnasiadka: thought so, still better than nuffin'17:44
*** cgrosjean has quit IRC17:49
*** k_mouza has quit IRC17:52
*** cgrosjean has joined #openstack-kolla17:53
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Improve Apache logging  https://review.opendev.org/69726218:03
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Default to etcd3gw driver for etcd-based coordination  https://review.opendev.org/69447618:06
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637618:07
yoctozeptomgoddard, mnasiadka: ^^18:08
yoctozeptoalso https://bugs.launchpad.net/kolla-ansible/+bug/185531618:15
openstackLaunchpad bug 1855316 in kolla-ansible ussuri "etcd coordinator is not configured HA" [Medium,Triaged]18:15
openstackgerritRadosław Piliszek proposed openstack/kolla master: Disable building binary images for senlin-conductor and senlin-health-manager on debian/ubuntu  https://review.opendev.org/69744518:19
*** schwicht has quit IRC18:19
openstackgerritRadosław Piliszek proposed openstack/kolla master: Disable building binary images for senlin-conductor and senlin-health-manager on debian/ubuntu  https://review.opendev.org/69744518:20
yoctozepto^ had to change that - should only disable binary, not all18:21
yoctozeptoalso failed on spaces18:21
*** gfidente is now known as gfidente|afk18:38
*** dave-mccowan has joined #openstack-kolla18:38
*** tonythomas has quit IRC18:38
*** schwicht has joined #openstack-kolla18:39
*** schwicht has quit IRC18:50
openstackgerritMerged openstack/kolla master: Introduce macro to remove Py3 only plugins on CentOS 7  https://review.opendev.org/69744918:52
*** jonaspaulo has joined #openstack-kolla18:54
*** graysonh has quit IRC19:01
*** graysonh has joined #openstack-kolla19:01
*** k_mouza_ has joined #openstack-kolla19:11
*** schwicht has joined #openstack-kolla19:16
*** k_mouza_ has quit IRC19:17
timburkeraukadah, fwiw, the source is at https://github.com/notmyname/Patches2/blob/master/patchbot.py but i'd be happy to have the existing bot join another channel19:36
*** dave-mccowan has quit IRC19:36
*** dosaboy has quit IRC20:04
*** igordc has quit IRC20:05
*** schwicht has quit IRC20:07
*** schwicht has joined #openstack-kolla20:09
*** schwicht has quit IRC20:14
*** cgrosjean has quit IRC20:19
*** cgrosjean has joined #openstack-kolla20:30
*** cgrosjean has quit IRC20:58
*** dosaboy has joined #openstack-kolla21:09
*** dave-mccowan has joined #openstack-kolla21:14
born2bakeHi, I am wondering if they were any fixes done to octavia images? https://bugs.launchpad.net/kolla-ansible/+bug/1847905 ; If yes, is there any possibility how I can update images?21:17
openstackLaunchpad bug 1847905 in kolla-ansible "Octavia loadbalancer pending create state" [Undecided,New]21:17
*** schwicht has joined #openstack-kolla21:21
*** graysonh has quit IRC21:35
*** dasp_ has joined #openstack-kolla21:52
*** dasp has quit IRC21:55
*** goldyfruit___ has quit IRC22:05
*** vmixor has quit IRC22:06
*** dr_feelgood has quit IRC22:25
*** jonaspaulo has quit IRC23:03
openstackgerritMerged openstack/kayobe master: Task to generate nginx.conf in inspection-store missing `become: True`  https://review.opendev.org/69727623:07
*** born2bake has quit IRC23:10
*** k_mouza has joined #openstack-kolla23:21
*** k_mouza has quit IRC23:48
*** k_mouza has joined #openstack-kolla23:48
*** k_mouza has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!