Wednesday, 2019-11-27

« Tuesday, 2019-11-26 Index Thursday, 2019-11-28 »
*** ivve has quit IRC00:04
*** strobert1 has quit IRC00:32
*** igordc has quit IRC00:42
*** dpawlik has joined #openstack-kolla00:57
*** dpawlik has quit IRC01:02
*** strobert1 has joined #openstack-kolla01:10
*** k_mouza has quit IRC01:16
*** duksoo has joined #openstack-kolla01:21
*** spsurya has joined #openstack-kolla01:40
*** lizheming has joined #openstack-kolla01:50
lizheminghi,all01:51
*** lizheming has quit IRC02:32
*** lizheming has joined #openstack-kolla02:45
*** dpawlik has joined #openstack-kolla02:58
*** goldyfruit_ has joined #openstack-kolla03:02
*** dpawlik has quit IRC03:03
*** k_mouza has joined #openstack-kolla03:17
*** k_mouza has quit IRC03:21
*** goldyfruit_ has quit IRC04:01
*** lizheming has quit IRC04:02
openstackgerritChason Chan proposed openstack/kolla-ansible master: Ensure hostname resolved by kolla configured /etc/hosts file  https://review.opendev.org/69604704:54
*** dpawlik has joined #openstack-kolla04:59
*** dpawlik has quit IRC05:04
*** skramaja has joined #openstack-kolla05:14
*** jcmdln has joined #openstack-kolla05:15
*** lizheming has joined #openstack-kolla05:37
*** threestrands has quit IRC05:51
*** sri_ has joined #openstack-kolla05:57
*** lizheming has quit IRC06:04
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Split out OVS/Linuxbridge agent configs from ml2_conf.ini  https://review.opendev.org/69536706:06
*** Luzi has joined #openstack-kolla06:11
yoctozeptomorning06:27
*** numans has joined #openstack-kolla06:31
*** verdurin has quit IRC06:34
*** pcaruana has joined #openstack-kolla06:34
*** jbadiapa has quit IRC06:43
*** jbadiapa has joined #openstack-kolla06:44
*** dpawlik has joined #openstack-kolla06:45
*** verdurin has joined #openstack-kolla06:47
*** xinliang has joined #openstack-kolla06:50
*** cah_link has joined #openstack-kolla06:57
*** hjensas has quit IRC07:00
*** hamdyk has joined #openstack-kolla07:08
*** cah_link has quit IRC07:43
mnasiadkamorning07:54
mnasiadkayoctozepto: Slawek confirmed in the review that we don't need the security groups part in ml2_conf.ini ;)07:54
yoctozeptomnasiadka: how glad I am! Miód na moje uszy!07:56
yoctozeptomnasiadka: I will rephrase your releasenote if you don't mind08:01
mnasiadkayoctozepto: I don't, have a go ;)08:01
hrwmorning08:05
*** lizheming has joined #openstack-kolla08:20
*** lizheming has quit IRC08:23
*** cah_link has joined #openstack-kolla08:39
*** hjensas has joined #openstack-kolla08:46
*** lizheming has joined #openstack-kolla08:46
*** lizheming has left #openstack-kolla08:49
*** rgogunskiy has joined #openstack-kolla08:50
*** rpittau|afk is now known as rpittau08:57
*** ivve has joined #openstack-kolla08:58
hrwfun fact: ubuntu 19.10 has too old erlang (22.0 instead of 22.1+) for rabbitmq needs. poor 20.04 lts users08:59
*** xinliang has quit IRC09:10
*** dougsz has joined #openstack-kolla09:12
*** k_mouza has joined #openstack-kolla09:15
*** csmart has quit IRC09:27
*** klippo has quit IRC09:27
*** openstackstatus has quit IRC09:30
*** dpawlik has quit IRC09:38
*** sri_ has quit IRC09:41
*** jovial[m] has quit IRC09:42
yoctozeptohrw: lol, then it's upstream forever ;p09:42
*** msalo has joined #openstack-kolla09:42
mnasiadkaI see distro maintainers are getting better and better in their job09:50
Blinkizhrw: Erlang "1:22.1.8+dfsg-1" is in Debian Sid (testing) and Ubuntu 20.04 (proposed).09:50
*** csmart has joined #openstack-kolla10:02
hrwBlinkiz: Debian testing yes. good to know that it got to proposed in ubuntu10:06
openstackgerritMerged openstack/kolla-ansible master: Fix kolla_ansible python module check  https://review.opendev.org/69607710:07
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Split out OVS/Linuxbridge agent configs from ml2_conf.ini  https://review.opendev.org/69536710:10
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Split out OVS/Linuxbridge agent configs from ml2_conf.ini  https://review.opendev.org/69536710:11
*** msalo has quit IRC10:14
*** msalo has joined #openstack-kolla10:15
*** msalo has quit IRC10:15
*** gfidente has joined #openstack-kolla10:16
*** skramaja has quit IRC10:27
hrwyoctozepto, mnasiadka: take a look at https://review.opendev.org/#/c/695465/ - probably last thing to do there is removal of 'WIP'10:29
patchbotpatch 695465 - kolla - WIP: Enable repos only when needed - 14 patch sets10:29
*** kevinz has joined #openstack-kolla10:40
*** msalo has joined #openstack-kolla10:50
*** msalo has quit IRC10:55
*** msalo has joined #openstack-kolla10:56
mgoddardmorning10:58
*** msalo has quit IRC11:00
*** duksoo has quit IRC11:03
*** msalo has joined #openstack-kolla11:05
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/train: Fix kolla_ansible python module check  https://review.opendev.org/69628111:06
*** rgogunskiy has quit IRC11:06
*** msalo has quit IRC11:08
*** sri_ has joined #openstack-kolla11:09
*** msalo has joined #openstack-kolla11:09
*** rgogunskiy has joined #openstack-kolla11:10
*** msalo_ has joined #openstack-kolla11:12
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Support python 3 in kolla-ansible script  https://review.opendev.org/69614311:16
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Use python 3 for local kolla-ansible execution  https://review.opendev.org/69606111:16
*** msalo has quit IRC11:16
*** msalo_ has quit IRC11:26
sri_mgoddard, if you have min, glance failing when i tried to create image from volume "FailedToDropPrivileges: privsep helper command exited non-zero" http://paste.openstack.org/show/786791/11:37
*** msalo has joined #openstack-kolla11:37
sri_mgoddard, I'm not sure is this that right place to ask this question :(11:38
yoctozeptosri_, mgoddard: sudo asked for password ;D11:41
sri_yoctozepto, is it inside container ?11:42
yoctozeptosri_: yah11:42
*** msalo has quit IRC11:42
yoctozeptowe might be missing some config for sudo11:42
yoctozeptowhich release is it?11:43
sri_yoctozepto, stain11:43
sri_centos base, this bug also looks very similar https://bugzilla.redhat.com/show_bug.cgi?id=165836711:44
openstackbugzilla.redhat.com bug 1658367 in openstack-glance "Glance store for cinder fails on PrivSep" [Medium,Closed: notabug] - Assigned to akekane11:44
*** msalo has joined #openstack-kolla11:54
yoctozeptosri_: for completeness - source or binary?11:58
sri_yoctozepto, source11:58
*** shyamb has joined #openstack-kolla11:58
*** rgogunskiy has quit IRC12:06
*** rgogunskiy has joined #openstack-kolla12:07
*** rgogunskiy has quit IRC12:07
*** shyamb has quit IRC12:09
*** cz2 is now known as swagnemite12:16
*** rgogunskiy has joined #openstack-kolla12:18
*** rgogunskiy has quit IRC12:18
*** rgogunskiy has joined #openstack-kolla12:18
*** shyamb has joined #openstack-kolla12:24
*** hamdyk has quit IRC12:26
*** msalo has quit IRC12:29
*** msalo has joined #openstack-kolla12:30
*** msalo has quit IRC12:35
mnasiadkayoctozepto: can you look in https://review.opendev.org/#/c/695170/ ?12:37
patchbotpatch 695170 - kolla - Update skydive version to 0.26.0 - 1 patch set12:37
yoctozeptomnasiadka: done12:39
mnasiadkayoctozepto: thx12:43
*** sri_ has quit IRC12:44
hrwok, back to erlang12:45
*** vmixor has joined #openstack-kolla12:46
yoctozeptosri_: if you read this later, let us know what backend you are using for glance12:56
yoctozepto(or any other specific config)12:56
yoctozeptoweird we would not get this issue earlier12:56
*** schwicht has quit IRC13:00
*** msalo has joined #openstack-kolla13:04
*** shyamb has quit IRC13:11
yoctozeptosri_: and report a bug, surely there is some13:12
*** sri_ has joined #openstack-kolla13:15
*** shyamb has joined #openstack-kolla13:15
*** dpawlik has joined #openstack-kolla13:15
sri_yoctozepto, glance backend is ceph, http://paste.openstack.org/show/786798/13:16
yoctozeptosri_: weeeird, I have glance stein centos source working with ceph13:17
yoctozeptodoes it happen with regular glance usage or only via karbor?13:17
sri_yoctozepto, only with karbor13:18
yoctozepto(or could it be that enabling swift misconfigured glance?)13:18
yoctozeptoah, understood13:18
yoctozeptowonder what it does that makes glance use its privsep helper13:18
yoctozeptosri_: post a bug on kolla-ansible, we'll handled it from there13:19
sri_Sure, thank you: yoctozepto13:20
*** shyamb has quit IRC13:28
*** sri_ has quit IRC13:28
*** schwicht has joined #openstack-kolla13:32
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Split out OVS/Linuxbridge agent configs from ml2_conf.ini  https://review.opendev.org/69536713:35
mnasiadkayoctozepto: ^^ this meant to be simple, now it's getting complicated :D13:37
yoctozeptomnasiadka: what can we do?13:37
mnasiadkayoctozepto: as always, deprecate13:38
* yoctozepto and mnasiadka - the openstack deprecators13:39
yoctozeptomnasiadka: thanks for addressing this13:40
yoctozeptoone minor and one optional test asked for13:41
yoctozeptoand you are free to work on something else :D13:41
mnasiadkayoctozepto: that is healthy for us, especially in the light of linux bridge not being deprecated, and plan to deprecate ml2/ovs in some time13:41
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Split out OVS/Linuxbridge agent configs from ml2_conf.ini  https://review.opendev.org/69536713:42
mnasiadkayoctozepto: around testing fake nova/neutron - I'll do it later in the afternoon, to make sure it works13:43
yoctozeptomnasiadka: yeah, that's why I proposed to split and refresh our view on this13:43
yoctozeptomnasiadka: great, thanks13:43
mnasiadkabut for now all my VMs are burning cycles on healthchecks and OVN ;)13:43
yoctozeptomnasiadka: employ CI :D13:43
*** Luzi has quit IRC13:47
openstackgerritMerged openstack/kolla master: CI: Move to CentOS 8 nodepool images  https://review.opendev.org/69279413:47
mgoddardyoctozepto, sri_: following that tripleo bug, looks like they add some volumes to glance-api and run in the container in privileged mode when cinder is the glance backend13:52
mgoddardyoctozepto: I imagine the privilege hides the privsep issue13:52
*** hamdyk has joined #openstack-kolla13:52
*** cah_link has quit IRC13:54
hrwyay, first part of getting rid of centos7 done13:57
*** schwicht has quit IRC13:59
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Use python3 for openstack clients  https://review.opendev.org/69606214:01
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Get rid of Python 2 support  https://review.opendev.org/69516814:01
openstackgerritMerged openstack/kolla master: Update skydive version to 0.26.0  https://review.opendev.org/69517014:16
*** lizheming has joined #openstack-kolla14:21
*** schwicht has joined #openstack-kolla14:25
yoctozeptomgoddard: not rushing with a fix unless I understand the issue or at least persuaded myself I do ;-)14:39
yoctozeptoI can't see how karbor breaks glance with ceph14:40
yoctozeptoas glance with ceph works by itseld14:40
yoctozeptothere is no mention of cinder in this interaction so far14:40
mgoddardyoctozepto: L109, glance_store/_drivers/cinder.py14:42
mgoddard'sudo', 'glance-rootwrap', '/etc/glance/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/glance/glance-api.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpWWVpyU/privsep.sock']14:43
mgoddardyoctozepto: we have no sudoers config for glance. I imagine we need something for privsep14:44
mgoddardyoctozepto: if it was working I'd expect to be able to run privsep with an invalid command and have it complain about the command rather than sudo14:44
*** hamdyk has quit IRC14:46
mgoddardyoctozepto: in train centos/binary I can run this:14:47
mgoddardsudo docker exec glance_api sudo glance-rootwrap /etc/glance/rootwrap.conf privsep-helper14:47
mgoddardyoctozepto: I don't expect source images have the sudoers config though14:48
mgoddardglance ALL = (root) NOPASSWD: /usr/bin/glance-rootwrap /etc/glance/rootwrap.conf *14:48
mgoddardyoctozepto: no sudoers config for centos/source stein14:49
*** priteau has joined #openstack-kolla14:50
*** stingrayza has quit IRC14:52
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Support configuration of Docker client timeout  https://review.opendev.org/69631614:54
*** goldyfruit_ has joined #openstack-kolla14:55
*** swagnemite is now known as cz214:57
*** scottsol has joined #openstack-kolla14:58
yoctozeptomgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne14:58
yoctozeptomeeting in 2 minutes14:58
* hrw -> coffee then14:58
*** stingrayza has joined #openstack-kolla14:59
yoctozeptomgoddard: the "no sudoers" is pretty straightforward, I am more interested as to why it suddenly requires one14:59
mgoddard#startmeeting kolla15:01
openstackMeeting started Wed Nov 27 15:01:28 2019 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
*** openstack changes topic to " (Meeting topic: kolla)"15:01
openstackThe meeting name has been set to 'kolla'15:01
mgoddard#topic rollcall15:01
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:01
mgoddard\o15:01
yoctozeptoo/15:01
hrwo/15:01
scottsolo/15:01
mgoddard#topic agenda15:04
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:04
mgoddard* Roll-call15:04
mgoddard* Announcements15:04
mgoddard** Ussuri priorities now listed on whiteboard https://etherpad.openstack.org/p/KollaWhiteBoard15:04
mgoddard** PTG summary on openstack-discuss: http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011084.html15:04
mgoddard** Infra images discussion on openstack-discuss: http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011120.html15:04
mgoddard** CentOS 8 host migration discussion on openstack-discuss: http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011130.html15:04
mgoddard* Review action items from last meeting15:04
mgoddard* CI status15:04
mgoddard* Train release planning15:04
mgoddard* Ussuri release planning15:04
mgoddard* Review priorities15:04
mgoddard** Ceph orchestrator15:04
mgoddard#topic announcements15:04
*** openstack changes topic to "announcements (Meeting topic: kolla)"15:04
mgoddard#info Ussuri priorities now listed on whiteboard15:04
mgoddard#link https://etherpad.openstack.org/p/KollaWhiteBoard15:04
mgoddard#info PTG summary on openstack-discuss15:05
mgoddard#link http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011084.html15:05
mgoddard#info Infra images discussion on openstack-discuss15:05
mgoddard#link http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011120.html15:05
mgoddard#info CentOS 8 host migration discussion on openstack-discuss15:05
mgoddard#link http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011130.html15:05
mgoddardPlease join in these discussions as we try to firm up some of the more squishy features15:06
mgoddardAny others?15:06
mgoddard#topic Review action items from last meeting15:06
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)"15:06
mgoddardmgoddard to tidy up ussuri priorities and add to whiteboard15:07
mgoddardcheck15:07
mgoddard#topic CI status15:07
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:07
mgoddard"kolla-toolbox fails to build on Ubuntu due to libssl older than libssl-dev"15:07
mgoddardthat one is 'fixed' now right?15:07
hrwCI passed through it15:07
mgoddardok, reverted whiteboard to green15:08
mgoddard#topic Train release planning15:08
*** openstack changes topic to "Train release planning (Meeting topic: kolla)"15:08
mgoddardI think we're getting close15:09
mgoddardhttps://launchpad.net/kolla/+milestone/9.0.015:09
yoctozeptopending is the rmq issue15:09
yoctozeptoosmanlicilegi to work on that but not finished yet15:09
mgoddardyeah15:09
mgoddardis it release blocking?15:09
hrwI would release with 3.7.1015:09
yoctozeptoother one is https://review.opendev.org/69580415:09
patchbotpatch 695804 - kolla-ansible - [Train] Fix Tacker config for the local Glance store - 3 patch sets15:09
yoctozeptojust waiting for you15:10
yoctozeptomgoddard, hrw: if we don't mind mid-cycle upgrade15:10
yoctozeptocould work, not breaking completely obviously15:10
hrwyoctozepto: at least it would be tested in ussuri.15:10
mgoddardyoctozepto: I +2 tacker15:10
yoctozeptomgoddard: I know, and send my thanks15:10
hrwyoctozepto: now it is 'we have train built/tested, lets add fresh rmq and release'15:10
mgoddardhrw: we're not talking about an rmq upgrade, "only" erlang15:11
hrwyoctozepto: we completely lack !x86 support for rmq/erlang15:11
yoctozeptohrw: I don't mind too much15:11
mgoddardeitehr way it's still risky15:11
yoctozeptomgoddard: yeah, only erlang and rmq is patch level15:11
yoctozeptoyeah, because it is erlang dribing most of networking there15:12
yoctozeptodriving*15:12
mgoddardhmm, actually it is a rmq patch upgrade, missed that15:14
yoctozeptomgoddard: no point in leaving bugs since they were blocked by erlang15:14
yoctozeptothough still it's erlang more worrying15:14
yoctozeptothe plus fact is that it's a configuration "tested" by rmq upstream15:15
yoctozeptoso we are possibly getting them to consider our bug reports as worthy ;p15:15
mgoddardscottsol: I think this is your issue: https://bugs.launchpad.net/kolla/+bug/181423315:16
openstackLaunchpad bug 1814233 in kolla "rabbitmq image fails to build" [Critical,Fix released]15:16
*** rgogunskiy has quit IRC15:16
scottsolyeh still stuck with that15:16
scottsolerlang-hipe and the pinning of rmq wont work15:17
yoctozeptowell, not really stuck, it remains to patch tripleo and provide aarch64 repos to kolla15:17
mgoddardhow close are we to aarch64 repos?15:17
yoctozeptoI think hrw has finished them15:18
hrwnot quite15:18
yoctozeptoat least for some15:18
yoctozeptoah ok15:18
yoctozeptostill, not in patch15:18
yoctozeptoosmanlicilegi not around hmm15:18
mgoddardok. Let's see how it progresses over the next few days15:19
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Use valid_interfaces instead of os_interface for placement  https://review.opendev.org/69632315:19
mgoddardhttps://launchpad.net/kolla-ansible/+milestone/9.0.015:19
mgoddardthe open bug list does seem to be going down15:20
yoctozeptoosmanlicilegi: ping me by tomorrow - otherwise I assume you are not able to make it so I can pick it up15:20
yoctozeptothanks15:20
mgoddardare we planning to get enforce_scope to work for train?15:21
mgoddardseems like a big change15:21
yoctozeptoI think he was around during the day15:21
yoctozeptomgoddard: yeah, I'm afraid it could break in funny ways15:22
yoctozeptoI would like to but scared15:22
mgoddardI'm tempted to remove the 9.0.0 milestone for that one15:22
*** hjensas has quit IRC15:22
mgoddardit's a big enough patch that it looks like a feature15:22
yoctozeptoit's kind of to support a feature in train15:22
yoctozepto"fix kolla-ansible to enable that feature" thing15:23
yoctozeptoborderline15:23
mgoddardshould we try to get https://bugs.launchpad.net/bugs/1844349 in ?15:23
openstackLaunchpad bug 1844349 in kolla-ansible ussuri "Neighbour table overflow problem with Neutron" [Medium,In progress] - Assigned to Pierre Riteau (priteau)15:23
priteauThat would be great. I am OK with removing the conntrack bits if that helps15:24
mgoddardprobably not urgent but osmanlicilegi said it helped15:24
priteauI think I only took them from the charm code, it wasn't actually required on the deployment from where this fix originated15:25
yoctozeptomgoddard, priteau: I am fine with either way, I just missed that there was any response on this, sorry ;-/15:25
mgoddardthe patch is W-1 currently priteau15:25
yoctozeptotaking unnecessary bits off is better as always15:25
* priteau checking why it is like this15:25
mgoddardW-1 Need to investigate if this is really relevant only for neutron-l3-agent.15:25
yoctozeptoif we all agree that less is more15:25
yoctozepto(man less)15:25
priteauRight, I wanted to check if it was needed only for neutron-l3-agent hosts15:25
yoctozeptoat least most relevant15:26
priteauThe problem is that the only environment where I can easily reproduce this is a production cloud :D15:26
yoctozeptomaybe get mnasiadka to ask neutron friends15:26
yoctozeptopriteau: :D15:26
*** lizheming has quit IRC15:27
hrwpriteau: perfect testing environment15:27
yoctozeptoclosest to real world scenario15:27
hrwpriteau: at least when it fails lot of people tell you about it15:27
hrwnot that you will miss it etc15:27
yoctozeptoonly pluses15:28
mgoddardok, if priteau can tidy up, we can try to get it into train15:28
mgoddardI'll do another check of backports15:29
priteauI am on it15:29
mgoddardthanks15:29
mgoddardI think rmq/erlang is the only thing holding the release at this point. Let's try to get an answer for that and get our GA out15:30
mgoddardgiven the CentOS 8 migration we'll probably need some big changes to train down the line anyway15:30
hrwonce I sort out building erlang packages I will update patch with repos15:30
mgoddardthanks gr15:30
mgoddardhrw:15:30
mgoddardlet's move on to kayobe train release15:31
mgoddardjovial has offered to do the release handle cranking15:31
mgoddardwe're hoping to be not too far behind kolla here15:32
mgoddardwe have a bunch of features listed on the whiteboard15:32
mgoddardwe don't really have time to do most of them though, so it's probably more of a list for Ussuri at this point15:32
openstackgerritPierre Riteau proposed openstack/kolla-ansible master: [neutron] Adjust neighbour table thresholds  https://review.opendev.org/68266415:33
mgoddardwe have a train release etherpad: https://etherpad.openstack.org/p/kayobe-train-release15:33
mgoddard#action mgoddard to update kayobe train release etherpad15:34
mgoddardanything to add priteau?15:34
priteauThere are many patches that we can still merge, but you're right, the priorities will have to switch to Ussuri15:35
yoctozeptoso kayobe is pretty much mgoddard, priteau, jovial and dougsz?15:35
mgoddardmostly. we have contributions from others at times including scottsol & co15:36
yoctozeptoI see15:36
mgoddardthose are the main active cores15:36
mgoddardroom for more if anyone is interested :)15:36
mgoddard#topic Ussuri release planning15:37
*** openstack changes topic to "Ussuri release planning (Meeting topic: kolla)"15:37
mgoddardFeels like we've made a good start on some features now15:37
mgoddardcentos8/py3/drop py2 in particular15:37
hrwmgoddard: thanks for sorting out that in k-a15:38
mgoddardno problem15:38
mgoddardany changes in the centos8 container patch?15:38
hrwI plan to submit something like https://review.opendev.org/695796 for k-a.15:38
patchbotpatch 695796 - kolla - test-requirements: bump pyflakes to get f-strings ... (MERGED) - 1 patch set15:38
hrwmgoddard: no extra content yet15:39
hrwmgoddard: centos team busy with 8.1 builds now15:39
mgoddardhrw: what is failing in centos/source currently?15:39
mgoddardthat should be smoother than binary, in theory15:39
hrwmgoddard: no ceph, openvswitch etc.15:40
mgoddardhmm, OVS is an issue15:40
hrwmgoddard: in general: check CI result. it shows nicely what fails15:41
*** dpawlik has quit IRC15:41
mgoddardyes. easier to ask you though, and we all find out :)15:41
hrwmgoddard: last update was to just rebase it and let CI check how it goes15:41
mgoddardI'm just thinking, could we get a single node ansible job to pass?15:42
hrwaio?15:42
mgoddardpossibly with some fudging15:42
mgoddardyes15:42
hrwno idea to be honest15:42
*** mloza has joined #openstack-kolla15:42
mgoddardhttps://review.opendev.org/#/c/692368/15:43
patchbotpatch 692368 - kolla - WIP: move to CentOS 8 in containers - 21 patch sets15:43
yoctozeptosomeone mentioned testing in CI https://review.opendev.org/69519215:43
patchbotpatch 695192 - kolla-ansible - CI: Refactor a lot - 23 patch sets15:43
yoctozeptotesting, testing15:43
mgoddardhttps://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_30c/692368/21/check/kolla-build-centos-source/30c7c33/kolla/build/15:43
yoctozeptoI have the card "test swift"15:44
yoctozeptobecause we have a bug pending on it15:44
hrwyoctozepto: 'a lot' is very wrong commit message15:44
yoctozeptohrwL very accurate though15:44
mgoddardaio blockers: fluentd, glance-base, keystone-base, kolla-toolbox, neutron-base, nova-base, nova-libvirt, openvswitch-base, rabbitmq15:45
hrwyoctozepto: can it get split into single small changes?15:45
hrwyoctozepto: as now it looks like a way to sneak 7 separate changes in one commit15:46
yoctozeptohrw: it is15:46
yoctozeptoit was meant to be testing qinling15:46
yoctozeptobut it was a pain so it got refactors in15:46
yoctozeptonow they are split15:46
hrwyoctozepto: I mean to split that one 'split from qinling' patch15:47
yoctozeptoI know15:47
yoctozeptoextra work for not much gain, what can I say15:48
*** schwicht has quit IRC15:48
mgoddardI would have preferred it to be split, but I have reviewed it now15:48
mgoddardat least qinling is factored out now15:49
mgoddard#topic Review priorities15:50
*** openstack changes topic to "Review priorities (Meeting topic: kolla)"15:50
mgoddardSame as usual really, train targeted bugs first, ussuri priorities second15:50
mgoddard#topic Ceph orchestrator15:50
*** openstack changes topic to "Ceph orchestrator (Meeting topic: kolla)"15:51
mgoddardmnasiadka: you there?15:51
mnasiadkayeah, just got to the keyboard15:51
mnasiadkasorry for being late15:51
yoctozepto(mgoddard, not seeing your review)15:51
mgoddard(I reviewed before, not since your update)15:51
yoctozeptomnasiadka: you had to rest from ml2 conf :-)15:51
yoctozepto(I see)15:51
mgoddardmnasiadka: Ceph orchestrator. Go!15:51
mnasiadkaGolang? ugh15:52
mnasiadkaSo - cutting the story short - ceph-ansible's future is a bit not bright and shiny15:52
mnasiadkaWhat Ceph is going towards - is Ceph Orchestrator15:52
mnasiadkaso an abstraction layer between the operator/deployer and some deployment backends15:52
mnasiadkain Octopus it will be the default deployment tool replacing ceph-deploy (and in future ceph-ansible)15:53
mgoddarddo you know which deployment backend will be recommendeD?15:53
hrwmarch 2020, right?15:53
mnasiadkaSo from my perspective - investing time in doing ceph-ansible integration with Kolla-Ansible is useless15:53
mnasiadkamgoddard: the 'ssh' backend will be recommended - so it streams python scripts through ssh15:53
mnasiadkahrw: march 2020 is the plan to release Octopus, yes15:54
mnasiadkaso it more or less fits into Ussuri timelines15:54
mgoddardis the ssh backend different from ceph-deploy?15:54
mnasiadkamgoddard: yeah, it doesn't require you to do all the pre-req jobs15:54
mnasiadkathe workflow is more or less:15:54
mgoddardok15:54
mnasiadka1) deploy 1 node ceph cluster (mon + mgr)15:55
mnasiadka2) enable ceph orchestrator module in mgr15:55
mnasiadka3) generate ssh keys and distribute them to nodes15:55
mnasiadka4) deploy more mons/mgrs for fully HA cluster15:55
mnasiadka5) scan for disks, deploy osds on them15:55
mnasiadka(and add mds or rgw if you need them)15:55
mgoddard6) profit15:55
*** jovial[m] has joined #openstack-kolla15:55
mnasiadkayeah15:56
mnasiadkaso now the question is - what do we want to achieve from kolla-ansible perspective going forward?15:56
mnasiadkaI would say it would make sense to polish the external ceph15:56
mnasiadkato remove current quirks and weirdness15:56
mgoddard+1, and remain agnostic15:57
mnasiadkaI don't know if it makes any sense to integrate k-a with ceph orchestrator - it's should be up to the user to deploy it as he wants15:57
yoctozepto+1 mgoddard15:57
mgoddardwe don't want to get hit by changing winds again15:57
mnasiadkaso from my perspective - maybe it's time to start moving towards removing kolla-ceph - or extracting it to separate project if there are maintainers interested in continuing15:58
mgoddardwe can always add support down the line if ceph orchestrator looks like it's here to stay, but it's hard to remove it15:58
mgoddardmnasiadka: +1 for starting that conversation at least15:59
mnasiadkamgoddard: I would probably prefer to write good docs instead of going that path... but there's still CI15:59
mnasiadkaI think we want to maintain a ceph-related CI, for all external ceph features15:59
mgoddardyes15:59
mgoddardI was going to ask what state your ceph-ansible patch is in15:59
mgoddardcould it be used for CI?15:59
mnasiadkayeah, it could15:59
mnasiadkathe other option is to write a script leveraging the ceph orchestrator16:00
mnasiadkaor rework the ceph-ansible one - to be more frank16:00
mgoddardwe'd need to modify it to move the playbook to tests/16:00
yoctozeptomgoddard, mnasiadka: regarding splitting - it seems to correlate with the general extensibility idea16:00
mgoddardit would be a good test case for extensibility16:00
yoctozeptoindeed16:00
mgoddardalthough I see the ansible side as being a separate CLI + site.yml rather than hooking in16:01
mnasiadkamgoddard: that's not a problem - but I think I would prefer to remove the playbook (which is just an import from ceph-ansible) and add ceph orchestrator based deployment to the bash script16:01
mgoddardmnasiadka: sounds like moar work16:01
mgoddardit's taken quite a while to get ceph-ansible working, why not use it?16:01
mgoddardand ceph orchestrator is still in development by the sound of it16:02
openstackgerritMerged openstack/kolla-ansible stable/train: Fix kolla_ansible python module check  https://review.opendev.org/69628116:02
openstackgerritMerged openstack/kolla-ansible master: Fix 'nova_cephx_raw_key' is undefined  https://review.opendev.org/69593316:02
mnasiadkamgoddard: well, let's assume we can use it for now, I still have some $job plan to investigate ceph orchestrator - so we can move at feasible time to use it instead of ceph-ansible16:02
mgoddarddoesn't ceph orchestrator require octopus16:03
mgoddard?16:03
mnasiadkamgoddard: so 1) adapt current ceph-ansible change to use as CI only 2) look in future to change it to use ceph-orchestrator16:03
mgoddardok, makes sense16:03
mnasiadkamgoddard: it does, but RDO will build Octopus either way in Ussuri :)16:03
mgoddardeventually yes16:03
mgoddardanyway, we're out of time16:03
mgoddardmnasiadka: can you kick off the conversation with openstack-discuss about ceph removal?16:04
mnasiadkayes, will send a mail tomorrow16:04
mnasiadkaasking for contributors for extracting kolla-ceph16:04
mgoddardmaybe try to find some past committers & known users e.g. wangwei16:04
mnasiadkashould we give ourselves some deadline for answers?16:04
mnasiadkaLike end of December?16:04
mgoddardyeah16:04
mnasiadkaok, sounds good16:05
mgoddardlet's try to be a bit active in seeking people out though. This is a big move16:05
mnasiadkaIt is, but we have sort of announced it in the beginning of Train cycle16:05
mgoddardwell we announced a supported migration path to ceph-ansible. This is not that16:05
mnasiadkayeah16:06
mnasiadkaI'll rework the BP to deliver a CI-only implementation of ceph-ansible integration16:06
mgoddardok16:06
*** chrizl has quit IRC16:06
mnasiadkawe could raise a BP around ceph orchestrator and taking over a kolla-ceph deployment16:06
mnasiadkabut I doubt I will have time to work on it in a reasonable velocity, given my $job priorities16:07
mnasiadkaI can ask in the same mail to ML if there are volunteers :)16:07
mgoddardthat would be good.16:07
mgoddardlet's call it a day16:07
mgoddardthanks all16:07
mgoddard#endmeeting16:07
*** openstack changes topic to "Kolla IRC meetings on Wednesdays @ 15:00 UTC - see agenda @ https://goo.gl/OXB0DL | Whiteboard: https://etherpad.openstack.org/p/KollaWhiteBoard | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"16:07
yoctozeptothanks16:07
openstackMeeting ended Wed Nov 27 16:07:54 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:07
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-11-27-15.01.html16:07
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-11-27-15.01.txt16:07
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-11-27-15.01.log.html16:07
*** ivve has quit IRC16:08
mnasiadkayoctozepto: what do you want to use me (k-a neutron liaison) for? :D16:08
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Fix 'nova_cephx_raw_key' is undefined  https://review.opendev.org/69634116:09
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix 'nova_cephx_raw_key' is undefined  https://review.opendev.org/69634216:10
yoctozeptomnasiadka: that priteau's change but it's mostly irrelevant now16:11
mnasiadkairrelevant change? I like these :)16:11
yoctozeptopriteau removed the "just-in-case" bits16:11
yoctozeptoirrelevant call for mnasiadka :-)16:11
yoctozeptoa relevant call is /me begging mgoddard and mnasiadka to review https://review.opendev.org/695192 :-)16:12
patchbotpatch 695192 - kolla-ansible - CI: Refactor a lot - 23 patch sets16:12
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Use python3 for openstack clients  https://review.opendev.org/69606216:14
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Fix hard-coded admin project name and username in blazar task  https://review.opendev.org/69634416:17
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix hard-coded admin project name and username in blazar task  https://review.opendev.org/69634516:17
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: Fix hard-coded admin project name and username in blazar task  https://review.opendev.org/69634616:17
*** scottsol has quit IRC16:19
*** mloza has quit IRC16:30
*** mloza has joined #openstack-kolla16:36
* hrw out16:37
*** gfidente is now known as gfidente|afk16:58
openstackgerritMark Goddard proposed openstack/kayobe master: Add command to deploy only containers  https://review.opendev.org/69612117:05
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: rabbitmq: update to latest 3.7.x (+ erlang 22.x)  https://review.opendev.org/69535617:08
hrwosmanlicilegi: for me this commit message is more informative17:10
hrwosmanlicilegi: will update once ubuntu package builds17:11
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Test Swift  https://review.opendev.org/69636417:16
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Test Swift  https://review.opendev.org/69636417:18
*** msalo has quit IRC17:27
*** msalo has joined #openstack-kolla17:29
*** msalo has quit IRC17:34
*** mixor has joined #openstack-kolla17:38
*** vmixor has quit IRC17:38
*** mixor is now known as vmixor17:38
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: base: do not install compiler on !x86-64  https://review.opendev.org/69637017:39
hrw117MB saved17:39
hrw385 contra 264MB17:39
openstackgerritMark Goddard proposed openstack/kayobe master: Support internal API TLS encryption  https://review.opendev.org/69637117:40
hrw381 D:17:40
mgoddardnice work hrw17:41
hrwmgoddard: to be honest? I left that compiler there in february17:42
hrwthis is good backport material for train17:42
hrwstein too iirc17:42
mgoddard+117:42
mgoddardif it builds, we do not need it17:42
hrweven if we do not publish images17:42
mgoddardso pretty safe backport17:42
hrwyep17:43
hrwI hope to move with arm64 ci in 1-2 weeks17:44
openstackgerritMark Goddard proposed openstack/kayobe master: Support internal API TLS encryption  https://review.opendev.org/69637117:45
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: rabbitmq: update to latest 3.7.x (+ erlang 22.x)  https://review.opendev.org/69535617:47
hrwDebian and Ubuntu covered on aarch6417:47
hrwCentOS has to wait as we need to fix our OBS17:47
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: rabbitmq: update to latest 3.7.x (+ erlang 22.x)  https://review.opendev.org/69535617:49
hrwadded link to requirements into commit message17:49
openstackgerritMark Goddard proposed openstack/kayobe master: Support internal API TLS encryption  https://review.opendev.org/69637117:50
*** mloza has quit IRC17:50
* hrw out17:52
*** priteau has quit IRC17:55
openstackgerritMark Goddard proposed openstack/kayobe master: Support internal API TLS encryption  https://review.opendev.org/69637117:56
*** yoctozepto5 has joined #openstack-kolla17:57
*** mixor has joined #openstack-kolla17:57
*** vmixor has quit IRC17:57
*** msalo has joined #openstack-kolla17:58
*** yoctozepto has quit IRC17:58
*** yoctozepto5 is now known as yoctozepto17:58
*** dougsz has quit IRC17:58
*** mloza has joined #openstack-kolla18:00
*** msalo has quit IRC18:02
mnasiadkahrw: 3.7 will be end of life on March 2020, maybe it's better to move to 3.8?18:05
*** jovial[m] has quit IRC18:05
mnasiadkaah, it's Dincer's change18:06
mnasiadkaosmanlicilegi: ^^ - I don't know how much effort is moving to 3.8, but I think we should plan for it in U18:06
*** msalo has joined #openstack-kolla18:09
*** msalo_ has joined #openstack-kolla18:10
*** rpittau is now known as rpittau|afk18:14
*** msalo has quit IRC18:14
openstackgerritMerged openstack/kolla-ansible master: Split out OVS/Linuxbridge agent configs from ml2_conf.ini  https://review.opendev.org/69536718:14
*** k_mouza has quit IRC18:17
*** sri_ has joined #openstack-kolla18:18
*** gouthamr has quit IRC18:20
*** gouthamr has joined #openstack-kolla18:21
*** k_mouza has joined #openstack-kolla18:21
sri_yoctozepto, https://bugs.launchpad.net/kolla-ansible/+bug/185419418:24
openstackLaunchpad bug 1854194 in kolla-ansible "Glance image asks for sudo password inside container " [Undecided,New]18:24
yoctozeptomnasiadka: yeah, it's already proposed as 3.8 in ussuri by osmanlicilegi18:25
yoctozeptosri_: thanks18:25
*** k_mouza has quit IRC18:25
mnasiadkayoctozepto: well updating to latest 3.7 is also proposed in Ussuri, which one do we merge first? :D18:26
yoctozeptomnasiadka: 3.7 should ideally go to train to not mix releasenotes i think18:26
sri_yoctozepto, np :)18:26
mnasiadkayoctozepto: btw, I found the issue of the docker API timeout... have the same problem sometimes on a VM - probably docker API needs some time to answer on low cpu/memory/slow storage situations18:27
mnasiadkaI'll propose a patch to bump up the timeout in DockerClient18:27
yoctozeptomnasiadka: mgoddard already did18:27
mnasiadka(because that helps)18:27
yoctozeptomnasiadka: https://review.opendev.org/69631618:27
patchbotpatch 696316 - kolla-ansible - Support configuration of Docker client timeout - 1 patch set18:27
mnasiadka60 seconds is not enough, and that's what is currently in master :)18:27
yoctozepto(are you reading me?)18:27
mnasiadkaI'm reading, 60 seconds is not enough :)18:29
*** k_mouza has joined #openstack-kolla18:38
*** k_mouza has quit IRC18:39
*** k_mouza has joined #openstack-kolla18:39
yoctozeptomnasiadka: mgoddard bumped to 120 seconds18:58
mnasiadkayoctozepto: in CI, yes :)18:58
mnasiadkabut whatever, let's see if anybody else hits this :)18:58
yoctozeptoall right18:59
*** sri_ has quit IRC19:00
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Test Swift  https://review.opendev.org/69636419:00
yoctozeptomnasiadka: kolla-ansible not too helpful for swift :D19:01
yoctozeptoso our ceph support is really golden in comparison ;p19:02
mnasiadkaswift? the role needs a major face lifting19:07
yoctozeptomnasiadka: yeah, and feature upgrade19:08
yoctozepto(but soul screams: DEPRECATE!)19:08
mnasiadkaI guess somebody uses swift...19:09
*** msalo_ has quit IRC19:09
openstackgerritMerged openstack/kolla-ansible stable/train: Fix 'nova_cephx_raw_key' is undefined  https://review.opendev.org/69634119:11
*** schwicht has joined #openstack-kolla19:13
yoctozeptomnasiadka: yeah, they do and they reported it's broken in train19:13
yoctozeptohence testing...19:13
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Test Swift  https://review.opendev.org/69636419:13
mnasiadkayoctozepto: can't they fix it? :)19:13
yoctozeptomnasiadka: you know how it is :D19:13
yoctozeptoswift is kinda openstack quasi-core19:14
yoctozeptoyou even get tested on swift on the openstack admin exam19:14
mnasiadkaCOA or Red Hat? ;)19:15
yoctozeptomnasiadka: COA :-)19:25
mnasiadkayoctozepto: haven't approached any certification for the last... 3 years at least19:26
mnasiadkanot counting The Open Group19:26
yoctozeptomnasiadka: exams are fun, I like these practical ones19:26
mnasiadkamaybe19:28
*** spsurya has quit IRC19:30
*** schwicht has quit IRC19:30
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Test Swift  https://review.opendev.org/69636419:31
yoctozeptolet it fail, let it fail...19:32
*** schwicht has joined #openstack-kolla19:33
mnasiadkaI hate Jinja...19:35
mnasiadkaif you set a variable including an int variable, you get a string... and | int filter does not help at all19:36
mnasiadkagrr19:36
*** mixor has quit IRC19:45
*** schwicht has quit IRC19:55
*** k_mouza has quit IRC19:55
*** schwicht has joined #openstack-kolla19:59
*** k_mouza has joined #openstack-kolla20:02
*** k_mouza has quit IRC20:04
yoctozeptoodd20:10
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Test Swift  https://review.opendev.org/69636420:30
*** rgogunskiy has joined #openstack-kolla21:18
*** schwicht has quit IRC21:20
*** rgogunskiy has quit IRC21:22
*** msalo has joined #openstack-kolla21:36
*** schwicht has joined #openstack-kolla21:53
*** jbadiapa has quit IRC22:01
*** k_mouza has joined #openstack-kolla22:04
*** pcaruana has quit IRC22:06
*** k_mouza has quit IRC22:08
cosmicsoundwhen deploying magnum with openstack-kolla-ansible and making a new cluster22:13
cosmicsoundwe hit this22:13
cosmicsoundERROR: The Parameter (octavia_ingress_controller_tag) was not defined in template22:13
*** schwicht has quit IRC22:17
*** schwicht has joined #openstack-kolla22:18
*** msalo has quit IRC22:21
*** k_mouza has joined #openstack-kolla22:47
*** schwicht has quit IRC22:51
*** k_mouza has quit IRC22:53
*** k_mouza has joined #openstack-kolla23:10
*** msalo has joined #openstack-kolla23:19
*** msalo has quit IRC23:23
*** k_mouza has quit IRC23:31
*** schwicht has joined #openstack-kolla23:34
*** goldyfruit_ has quit IRC23:59
« Tuesday, 2019-11-26 Index Thursday, 2019-11-28 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!