Wednesday, 2019-09-18

*** goldyfruit_ has joined #openstack-kolla00:06
*** k_mouza has joined #openstack-kolla00:30
*** yingjun has joined #openstack-kolla00:31
*** k_mouza has quit IRC00:35
*** oneswig has quit IRC01:11
*** BjoernT has joined #openstack-kolla02:26
*** zhangfei has joined #openstack-kolla02:57
*** BjoernT has quit IRC03:01
*** BjoernT has joined #openstack-kolla03:02
*** yingjun has quit IRC03:24
*** dave-mccowan has quit IRC03:36
*** zhangfei has quit IRC03:49
*** zhangfei has joined #openstack-kolla04:07
*** janki has joined #openstack-kolla04:34
*** pcaruana has joined #openstack-kolla04:46
*** BjoernT has quit IRC04:48
*** Luzi has joined #openstack-kolla04:59
*** radeks has joined #openstack-kolla05:04
*** hjensas|afk has quit IRC06:02
*** cah_link has joined #openstack-kolla06:07
*** igordc has joined #openstack-kolla06:17
*** igordc has quit IRC06:19
*** zhangfei has quit IRC06:32
openstackgerritQitao proposed openstack/kolla-ansible master: Update stackalytics link for README  https://review.opendev.org/68280606:40
yoctozeptomorning06:41
*** k_mouza has joined #openstack-kolla06:41
*** cah_link has quit IRC06:43
*** k_mouza has quit IRC06:46
osmanlicilegimorning06:49
*** zhangfei has joined #openstack-kolla06:57
*** tonythomas has joined #openstack-kolla07:01
*** cah_link has joined #openstack-kolla07:08
*** luksky has joined #openstack-kolla07:13
*** hjensas|afk has joined #openstack-kolla07:17
mnasiadkamorning07:29
yoctozeptomorning mnasiadka07:30
yoctozeptomnasiadka: https://review.opendev.org/682600 - this is for staters, OL is broken for dessert: https://bugs.launchpad.net/kolla/+bug/184448407:31
openstackLaunchpad bug 1844484 in kolla train "oraclelinux builds always fail (package gone)" [Critical,Triaged]07:31
yoctozeptowelcome to centos 7.7 8-)07:31
*** ivve has joined #openstack-kolla07:32
*** rpittau|afk is now known as rpittau07:32
mnasiadkayoctozepto: well, based on discussion on centos-devel, it seems we might need epel-rpm-macros to work around this python36 vs python3 crap ;)07:38
*** cah_link has quit IRC07:38
yoctozeptomnasiadka: so there are also python3 issues? :O07:39
yoctozeptothe OL issue is far from python atm ;p07:39
mnasiadkaOL? I thought we deprecated that ;)07:40
yoctozeptomnasiadka: we don't build that for train yeah07:40
yoctozeptobut for stein- we do07:40
mnasiadkaoh crap07:41
yoctozeptomnasiadka: but in train we still support building it07:42
mnasiadkathat's why I said oh crap07:42
yoctozeptojust that we don't offer the releases07:42
yoctozeptoyeah, I'm just giving more details07:42
yoctozepto;p07:43
mnasiadkacan't we just pin OL to 7.6 and be gone? :D07:43
yoctozeptomnasiadka: you mean playing with access to vault? xD07:43
yoctozeptogl&hf :D07:43
yoctozeptoI think we should just centosify ol more07:44
yoctozeptoand treat them equally07:44
mnasiadkaso then it's no more OL07:44
mnasiadka:)07:44
yoctozeptono, it has the whole base of OL07:44
yoctozeptowhich should be binary compatible07:44
yoctozeptowith centos anyways07:44
yoctozeptojust slap the extras so that they work the same07:45
yoctozeptoand not play with direct url07:45
mnasiadkaso go for it and be gone ;)07:49
hrwjust drop OL07:50
*** gfidente has joined #openstack-kolla07:50
hrw;d07:50
yoctozeptohrw: +307:50
yoctozeptomorning you joker07:50
hrwbut as Red Hat employee I probably can not do it myself07:51
*** gfidente has quit IRC07:51
*** cah_link has joined #openstack-kolla07:51
yoctozeptomnasiadka: yeah, after finishing local higher prio stuff ;-)07:51
hrwok, I returned ppc64le machine07:51
hrwin case of need will grab another one. but that's October07:52
yoctozeptoyou two better review those RP+2 changes I and Mark have07:52
yoctozeptoso that we get modernized sooner07:52
yoctozeptothan later07:52
*** radeks has quit IRC07:55
*** radeks has joined #openstack-kolla07:55
hrwmoment, let me read apache arrow changes first07:56
hrwupstream developer took my changes and got them nicely integrated07:56
hrwyoctozepto: https://review.opendev.org/682600 https://review.opendev.org/682437 and?07:59
*** Reepicheep has quit IRC08:00
yoctozeptohrw: thanks, 2nd no, w-1 slapped just now08:00
mnasiadkayoctozepto: can we agree, that we do RP+1/+2 only when Zuul is green? :)08:00
yoctozeptothen one in ansible awaits08:00
yoctozeptohttps://review.opendev.org/67069008:00
yoctozeptomnasiadka: doing that unless not08:01
*** Reepicheep has joined #openstack-kolla08:01
yoctozeptoheh :D08:01
mnasiadkayoctozepto: well, "RP+1 recheck" doesn't sound good for me :D08:01
hrwmnasiadka: some patches are just important enough to get RP+1/+2 right after sending08:01
yoctozeptomnasiadka: when I know the issue is random08:01
yoctozeptothen it's ok ;-)08:01
hrwmnasiadka: it also shows "this is important! if it fail then those CI errors need to be fixed FRIST!"08:02
mnasiadkayoctozepto: let's agree when the issue is sorted out - do it then, I don't like looking in Zuul logs with random issues :D08:02
mnasiadkahrw: this is not the case08:02
yoctozeptoFRIST indeed08:02
mnasiadkahttps://review.opendev.org/#/c/682437/08:02
yoctozeptomnasiadka: I invite you to look at the code08:02
mnasiadkaI don't even know why it is priority?08:02
hrwyoctozepto: I looked at Mark's change and do not understand it. too k-a for me08:03
yoctozeptomnasiadka: that was a bad example, hence rp-1 removed08:03
yoctozepto;-)08:03
mnasiadkayeah, let's agree RP+1 only for those changes that are related to a) CI breakage b) features on the priority list08:04
mnasiadkaI haven't seen Zun on priority list :)08:04
yoctozeptoRP+2 for CI breakage08:04
yoctozeptoRP+1 for priority08:04
mnasiadkagood :)08:04
yoctozeptosure, that was some sloppy mouse click :-)08:04
yoctozeptomnasiadka: anyway - https://review.opendev.org/67069008:05
yoctozeptotoo ansibly for hrw08:06
yoctozeptoco-authored by me and Mark08:06
yoctozeptoso mostly for you to review08:06
yoctozeptoit will have to march back to rocky to unbreak upgrade jobs mind you08:07
mnasiadkayoctozepto: tell me why this happens: https://zuul.opendev.org/t/openstack/build/ccbd5ed4cb8740ab991775116905899f/log/primary/logs/ansible/upgrade#2283808:08
*** dougsz has joined #openstack-kolla08:09
yoctozeptomnasiadka: we don't support moving from non-ha to ha08:09
yoctozeptobut this very much beyond the scope of this change08:09
yoctozeptoremember we ran multinode without ha08:09
mnasiadkaAnd what now? We fail all Ceph upgrade jobs? that's the approach?08:09
yoctozeptomnasiadka: read me above08:09
yoctozeptowe are enabling ha in ci in previous releases08:10
mnasiadkaWell - I don't like it, fix it.08:10
mnasiadka:)08:10
yoctozeptowell, I tried08:10
yoctozeptothe readdressing is the issue08:10
yoctozeptowe would have to support it08:10
mnasiadkaWell, I just don't like the fact, that we add some CI feature and then break something else08:11
mnasiadkahere it's even better: https://zuul.opendev.org/t/openstack/build/7b35fc9c870646e7ae6ba37702b1da7a/console08:12
mnasiadka:)08:12
yoctozeptomnasiadka: it's a chicken and egg problem really08:12
yoctozeptomnasiadka: that is CI for you08:12
yoctozeptowe get this from time to time08:12
yoctozeptoseems you are not acquiainted with CI features :-)08:12
mnasiadkayoctozepto: I understand the technical issue at hand, but if we don't care about Ceph-upgrade jobs - let's just remove them.08:12
yoctozeptomnasiadka: we care, these are only multinode upgrade jobs we have08:13
yoctozepto;p08:13
mnasiadkaWhat is the sense of having a CI that always fails and is non-voting? :)08:13
mgoddardmorning08:13
yoctozeptomnasiadka: you need to eat some snickers08:14
yoctozeptomorning mgoddard08:14
mnasiadkayoctozepto: naah, I just don't like breaking things - and you tend to not think about it :)08:14
yoctozeptomnasiadka: I told you it is fixed by backporting to stein and then for stein to rocky08:14
yoctozeptoI think long distance08:15
*** gfidente has joined #openstack-kolla08:15
yoctozepto;-)08:15
mnasiadkayoctozepto: I don't think it's a feature we should back port - unless we have no other choice.08:15
yoctozeptolet's agree on the general functionality, behavior and code08:16
yoctozeptomnasiadka: why not, it makes CI more realistic08:16
yoctozeptoelse we can change the upgrade jobs if you think no-backport08:16
yoctozeptoso that we modify the pre-upgrade settings08:16
yoctozeptoit might be better if we agree on how to tackle this08:17
yoctozeptoI am fine with either option as long as we reach consensus08:17
mnasiadkawell, I thought the back port policy is not back porting new features, but maybe I'm wrong :)08:17
yoctozeptomgoddard: please join us in our pre-meeting discussion :-)08:17
yoctozeptomnasiadka: surely not backporting Ipv6 support08:17
yoctozeptohell no08:17
yoctozepto:D08:18
yoctozeptobut CI is a gray zone08:18
yoctozeptomore of paying the technical debt08:18
mnasiadkano grey zones please08:18
yoctozeptothan feature adding08:18
yoctozeptoCI is not the product, sir :D08:18
openstackgerritMark Goddard proposed openstack/kayobe stable/stein: Fix kayobe-overcloud-centos CI job flakiness  https://review.opendev.org/68282908:18
mnasiadkayoctozepto: why not? :)08:20
yoctozeptomnasiadka: k-a users don't install it maybe? I feel you are in some strange mood today - but you made me think and for that I thank you - I think what should be done is this:08:21
yoctozeptocurrent branch should control configuring of pre-upgrade env08:22
yoctozeptothis way we fix this thing you don't like now08:22
yoctozeptobut also08:22
yoctozeptothis crosses out the dependency that has bitten us already08:22
yoctozeptoe.g. the comment I left for hrw on his docker change08:22
yoctozeptoabout being careful with those variables08:22
yoctozeptosince they bite later08:23
yoctozeptolet's make later never happen08:23
mnasiadkawell, for some features we need to support both approaches (pre-change and post-change), I know it's additional work - but nothing is easy :)08:23
yoctozeptoindeed, in the same sense we should handle pre-upgrade config locally08:24
yoctozeptorather than from pre-upgrade branch08:24
mnasiadkaIf there is no other way - it would be nice to have some explanation in commit message or a comment in the review itself - I'm not asking for much :)08:24
yoctozeptoit handles its own issues already08:24
hrwyoctozepto: 'backport this that those to keep new CI upgrade jobs working' ends with "why I cannot upgrade from stein to train while it is supported"08:33
hrwmnasiadka hit my common issue08:33
hrwfor me commit message should say what change does so I do not need to look at code to find out08:33
openstackgerritDincer Celik proposed openstack/kolla-ansible master: [prometheus] Added support for extra options  https://review.opendev.org/68283408:34
mgoddardok, finally caught up on the chat08:37
mgoddardthe solution is simples08:37
mgoddardadd a variable to configure enable_haproxy in globals.yml in CI, backport it, set it when available08:38
*** dciabrin_ has quit IRC08:38
mgoddardso first iteration adds VXLAN but keeps haproxy disabled (at least) on upgrades08:38
mgoddardonce the stable branch has the flag, enable haproxy on upgrades too08:39
yoctozeptomgoddard: I think it's cleaner what mnasiadka made me invent - configure pre-upgrade using current (adapted file for pre)08:40
mgoddardI'll take another look at the code08:40
yoctozeptothen we squash both current problems08:40
yoctozeptoand any future one arising from this dependency08:40
mgoddardhmm08:42
yoctozeptothen we avoid "omg, this backport broke master jobs"08:43
yoctozeptowe survived that already08:43
openstackgerritDincer Celik proposed openstack/kolla-ansible master: [prometheus] Added support for extra options  https://review.opendev.org/68283408:43
mgoddardtrue08:44
mgoddardI do quite like the realistic approach of having the initial install decoupled from the upgrad08:44
mgoddard(that we have now)08:45
yoctozeptomgoddard: code is still decoupled08:45
mgoddardsort of08:45
yoctozeptoonly config may differ08:45
mgoddardwe might need to add 'if upgrade else...'08:45
mgoddardin globals.yml08:45
yoctozeptoyeah, we have such a variable08:46
mgoddardwe do08:46
yoctozeptoand we will be using it locally now08:46
yoctozeptono longer "wait for backport to discover the issue in upgrade"08:47
*** k_mouza has joined #openstack-kolla08:47
mgoddardthat can still happen with code :)08:47
yoctozeptomgoddard: true but that's how life goes08:51
yoctozeptoshouldn't if we stick to nice backports08:51
yoctozeptothe same cannot be told about config08:51
yoctozeptobecause those were clean backports08:51
mgoddardthis will add another TODO at the end of a release - remove all the 'if upgrade' bits08:53
mgoddardrecent docker config changes for example08:54
yoctozeptomgoddard: true, on branching off08:54
mgoddardyep08:54
yoctozeptoneed to clean master08:54
*** KeithMnemonic1 has joined #openstack-kolla08:55
mgoddardwe already have a list of things to do, it's not the end of the world08:55
*** KeithMnemonic has quit IRC08:55
yoctozeptoindeed, CI usually breaks then anyway :-)08:55
*** KeithMnemonic1 has quit IRC08:55
yoctozeptolet me try this out then08:56
*** KeithMnemonic1 has joined #openstack-kolla08:56
mnasiadkawell, at least we have a list now :D08:56
yoctozeptodumb approach comes first08:56
*** pcaruana has quit IRC08:57
*** pcaruana has joined #openstack-kolla09:01
*** priteau has joined #openstack-kolla09:07
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: WIP: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284209:09
yoctozepto^ probably gonna fail09:10
yoctozeptobut it's a start09:10
yoctozeptothen we don't need to backport config changes not being a fix to CI operations just to satisfy newer branches09:11
mgoddardyoctozepto: so we're doing it for all configs? could get messy :(09:12
yoctozeptomgoddard: does it make sens to omit some?09:13
yoctozeptoin global picture09:13
mgoddardyoctozepto: inventory can change over time09:13
mgoddardbut normally only additions09:13
yoctozeptoI am worried too but we have control over it in one place now09:13
yoctozeptoso it's a tradeoff09:13
mgoddardwe should stop templating daemon.json, as it gets overwritten now09:14
mgoddardbut still need it for pre-upgrade )09:14
mgoddardI'll make these comments in the review09:15
yoctozeptok thanks mgoddard09:18
yoctozeptowe are failing already as expected09:18
yoctozeptowell, not entirely as expected:09:19
yoctozeptoDestination directory /etc/kolla/config/nova does not exist09:19
yoctozeptoseems our little role did more magic09:19
mgoddardyoctozepto: git diff origin/stable/stein  tests/templates/09:21
mgoddardyou'll need to change the docker config, add back removed groups in inventory09:21
yoctozeptomgoddard: yeah, that would be the next step if only it did not fail the actual templating :D09:22
yoctozeptobrb09:23
hrwmgoddard: daemon.json content is merged09:23
hrwmgoddard: not overwritten09:23
hrwbbl09:24
mgoddardhrw: we didn't do that in the end09:24
mgoddardI'll fix that one up09:25
openstackgerritMerged openstack/kolla master: CI: CentOS - override mirrors ASAP + crmsh mirror override  https://review.opendev.org/68260009:31
mgoddard^ we're back in the game09:32
mgoddardon the OL issue, we should just remove it09:32
mgoddardwe didn't promise to keep it09:32
mgoddardjust haven't removed yet09:32
yoctozeptomgoddard: what about stein- ?09:33
mgoddardyoctozepto: we should fix that09:33
mgoddardassuming it's not too much effort09:34
yoctozeptomgoddard: if my plan works09:34
yoctozeptoit will require only as much as centos09:34
yoctozeptoso gets free09:34
yoctozeptook, back on track with ansible09:34
mgoddardwell let's go for the smallest solution on stable branches09:34
mgoddardI don't fully understand the problem yet09:35
mgoddard(haven't tried)09:35
yoctozeptomgoddard: package is gone :D09:35
yoctozeptowe used absolute paths instead of extras repo09:35
yoctozepto(no idea why though)09:35
yoctozeptoas for the ansible09:35
yoctozeptoI convince myself we should only apply the change to globals.yml09:35
yoctozeptootherwise it gets messy for no gain09:35
yoctozeptoas globals is the only current source of our issues to this point09:36
yoctozeptowdyt09:36
yoctozeptomgoddard,09:37
mgoddardsounds inconsistent :)09:37
yoctozeptoindeed09:38
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Reinstate use of Docker registry mirror  https://review.opendev.org/68284609:38
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637609:38
mnasiadkayoctozepto: maybe it would be good to check what else we don't have mirrored (excluding the GitHub releases we fetch for some Docker images) and add that - I'll look into that later.09:41
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: [designate] Fix admin api configuration  https://review.opendev.org/68284709:45
yoctozeptomnasiadka: yeah, we should add some debug to name resolution to trace this09:46
hrwhttp://paste.debian.net/1101394/ - OL fix09:47
hrwto review or not to?09:47
yoctozeptohrw: check if it's the only one missing09:48
yoctozeptothey did major rebuild there09:48
hrwyoctozepto: in progress09:48
hrwINFO:kolla.common.utils:Successfully built images09:48
hrwINFO:kolla.common.utils:=========================09:48
hrwINFO:kolla.common.utils:base09:48
yoctozeptobut yeah, works fine as the smallest fix for sure09:48
yoctozeptook, so review it09:48
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: base: fix OracleLinux build  https://review.opendev.org/68284909:49
hrwsmall, should be backportable09:49
mnasiadkaand then we can remove OL ;)09:49
yoctozeptomnasiadka: yeah, I don't think it brings any benefits over centos as far as containers are concerned09:50
yoctozeptohrw: we like them small09:50
mnasiadkayoctozepto: and I don't think OL has any future09:50
yoctozeptomnasiadka: their UEK was nice09:51
yoctozeptois*09:51
yoctozeptosee, you already made me doubt about OL future09:51
yoctozepto;p09:51
hrwspeaking of OL... my (personal) opinion is: no one in core team uses it, no one likes it so drop it. if someone care then they will complain and can bring it back09:52
hrwfor br in stein rocky queens pike;do git checkout origin/stable/$br -b test/$br;git am 0001-base-fix-OracleLinux-build.patch && ~/devel/docker-utils/kolla-build-images.sh oraclelinux binary ^base $br;done09:52
hrwbbl09:53
mnasiadkayoctozepto: well, they released 8.009:56
mnasiadkayoctozepto: so maybe it has some pulse09:56
mgoddardhrw: https://bugs.launchpad.net/kolla/+bug/1814233 what do you mean we are hitting this again? Is rabbitmq failing to build and we don't know it?09:59
openstackLaunchpad bug 1814233 in kolla "rabbitmq image fails to build" [Critical,Fix released]09:59
mgoddardhrw: is it actually the same bug or a similar one?09:59
mgoddardhrw: I'll add a patch to reinstate OL testing to test your fix10:00
mgoddard(DNM)10:00
mgoddardactually, it will be easier to just backport it10:03
openstackgerritMark Goddard proposed openstack/kolla stable/stein: base: fix OracleLinux build  https://review.opendev.org/68285210:04
*** openstackgerrit has quit IRC10:06
*** openstackgerrit has joined #openstack-kolla10:09
openstackgerritMark Goddard proposed openstack/kolla stable/stein: base: fix OracleLinux build  https://review.opendev.org/68285210:09
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284210:10
*** zbr has quit IRC10:11
yoctozeptomgoddard: ^10:11
yoctozeptoleaving to grab some food10:11
*** dciabrin has joined #openstack-kolla10:12
*** zbr has joined #openstack-kolla10:13
*** dciabrin has quit IRC10:17
openstackgerritMerged openstack/kolla master: Add effective and permitted capability to blackbox exporter  https://review.opendev.org/67715610:33
openstackgerritMerged openstack/kolla-ansible master: Update stackalytics link for README  https://review.opendev.org/68280610:33
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284210:37
*** dciabrin has joined #openstack-kolla10:37
*** jdwidari has joined #openstack-kolla11:01
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284211:01
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Reinstate use of Docker registry mirror  https://review.opendev.org/68284611:05
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284211:07
openstackgerritDincer Celik proposed openstack/kolla master: build: enable Octavia for Ubuntu/binary  https://review.opendev.org/68267111:14
*** pcaruana has quit IRC11:19
*** kplant has joined #openstack-kolla11:19
*** dave-mccowan has joined #openstack-kolla11:23
openstackgerritDincer Celik proposed openstack/kolla master: build: enable Octavia for Ubuntu/binary  https://review.opendev.org/68267111:24
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Remove support for OracleLinux  https://review.opendev.org/68285811:26
*** pcaruana has joined #openstack-kolla11:28
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637611:32
*** Luzi has quit IRC11:35
openstackgerritMark Goddard proposed openstack/kolla master: Remove support for OracleLinux  https://review.opendev.org/68286511:41
*** zhangfei has quit IRC11:42
openstackgerritMerged openstack/kolla-ansible master: Fix prometheus-alertmanager cluster bug  https://review.opendev.org/68213211:43
openstackgerritMerged openstack/kolla-ansible master: Update python_path for "source" install type  https://review.opendev.org/67582211:43
openstackgerritMerged openstack/kolla-ansible master: Ignore create_cells and discover_computes when nova-api is disabled  https://review.opendev.org/68137611:43
openstackgerritMerged openstack/kolla master: Update skydive to v0.25.0  https://review.opendev.org/68176811:43
*** Luzi has joined #openstack-kolla11:49
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637611:58
*** AJaeger has joined #openstack-kolla12:02
hrwmgoddard: commented12:02
*** AJaeger has left #openstack-kolla12:02
openstackgerritDincer Celik proposed openstack/kolla master: build: enable Octavia for Ubuntu/binary  https://review.opendev.org/68267112:03
*** janki has quit IRC12:07
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: base: enable rabbitmq and elasticsearch repos for ppc64le  https://review.opendev.org/68261812:09
hrwrebased12:09
*** goldyfruit_ has quit IRC12:15
*** k_mouza has quit IRC12:16
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637612:17
hrwyoctozepto: how goes mirroring that opensuse repo?12:18
mnasiadkaI think it's merged12:26
openstackgerritDincer Celik proposed openstack/kolla-ansible master: Fixes destroys.  https://review.opendev.org/66828812:26
*** redrobot has joined #openstack-kolla12:26
hrwmnasiadka: but is it mirrored...12:26
mnasiadkayup - http://mirror.iad.rax.openstack.org/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/12:27
*** luksky has quit IRC12:27
hrwthey are12:28
openstackgerritMark Goddard proposed openstack/kolla master: Remove support for OracleLinux  https://review.opendev.org/68286512:43
*** scottsol has joined #openstack-kolla12:48
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix prometheus-alertmanager cluster bug  https://review.opendev.org/68288812:50
*** janki has joined #openstack-kolla12:52
*** janki has quit IRC12:53
*** vmixor has joined #openstack-kolla13:02
yoctozeptohrw: yeah, today merged our usage of it13:09
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284213:11
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Configure the upgrade jobs from the current branch  https://review.opendev.org/68284213:11
*** luksky has joined #openstack-kolla13:19
hrwyoctozepto: series of recheck sent13:21
yoctozeptohrw: good13:22
*** k_mouza has joined #openstack-kolla13:22
yoctozeptoit's worth doing what mnasiadka suggested - confirming we mirror everything we need and use only mirrored13:22
*** k_mouza has quit IRC13:26
*** scottsol has quit IRC13:29
*** k_mouza has joined #openstack-kolla13:29
*** scottsol has joined #openstack-kolla13:30
*** goldyfruit_ has joined #openstack-kolla13:30
openstackgerritRadosław Piliszek proposed openstack/kolla master: Add volume-related packages to zun-compute image  https://review.opendev.org/68243713:34
*** hjensas|afk has quit IRC13:36
openstackgerritMerged openstack/kolla-ansible stable/rocky: Add support to enable l3 port-forwarding plugin  https://review.opendev.org/68163613:42
*** BjoernT has joined #openstack-kolla13:48
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Zun: add volume support  https://review.opendev.org/68290013:48
*** Luzi has quit IRC13:49
*** k_mouza has quit IRC13:51
*** JamesBenson has joined #openstack-kolla14:02
andreinmgoddard: got the VLAN stuff sorted out! YAY!14:08
mgoddardandrein: nice work14:09
andreinnow, the next two things in my todo list: setting up LACP and Software RAID (https://storyboard.openstack.org/#!/story/2004581 - I know, I'm going to have to upgrade to master to get it)14:09
andreinany pointers for the LACP?14:09
mgoddardandrein: we normally configure bonds to fall back to a single interface on the switch, to allow PXE to work14:11
kplant^14:11
mgoddardit depends on the switch model if/how to do it14:11
kplantjunos is force-up14:11
kplantextremely useful14:11
mgoddard+114:11
kplantmake sure you only do it on one lag member though14:11
kplanti also recommend doing active lacp and 1s (fast) instead of 30s (slow) intervals14:12
andreinmgoddard: ahead of you on the switch config, it's all there :)14:12
kplantas well as layer2 and layer3 lb14:13
andreinkplant:14:13
andreinkplant: why do it on a single LAG member though?14:13
kplantso i'll use mgoddard's example of PXE14:13
mgoddardandrein: so now you hit the problem that bifrost can't configure bond interfaces14:13
kplantif both interfaces are up/up, and the switch is configured to force-up the lag from both members14:14
kplantit will try to send traffic over both lag members14:14
mgoddardwe do it after deploy using kayobe, although it doesn't currently work with a VLAN subinterface and a bond14:14
kplantso in the case of pxe, where both ports are L1 up, it will try to send traffic to both interfaces14:14
kplantwhere really you only want the interface that's doing pxe14:14
kplantmake sense?14:14
kplantforce-up basically says to send traffic over the lag member if it's not speaking lacp14:14
andreinkplant: yeah, makes sense, i'll double check with my nework engineer, thanks for the heads up!14:15
kplantyeah i got burned by that early on14:15
kplantpxe over a vlan without a native vlan is difficult @mgoddard14:16
kplantsometimes the NIC will just put the tag on for you, sometimes it doesn't14:16
kplantblah14:16
yoctozeptomy fujitsu systems do not support pxe with tagged-only interfaces ;/14:21
*** openstackgerrit has quit IRC14:21
andreinmgoddard: no vlans over bonds? :(14:23
yoctozeptomgoddard: some ironic failure in CI: https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/zuul_opendev_logs_3af/682618/2/check/kolla-ansible-centos-source-ironic/3af4e4a/primary/logs/ansible/test-ironic - it is easier for you to digest probably14:26
mgoddardyoctozepto: nom nom nom14:27
*** openstackgerrit has joined #openstack-kolla14:27
openstackgerritMerged openstack/kolla-ansible master: Remove support for OracleLinux  https://review.opendev.org/68285814:27
openstackgerritMerged openstack/kolla stable/stein: base: fix OracleLinux build  https://review.opendev.org/68285214:27
yoctozeptomgoddard: yeah, iron is healthy14:28
*** wuchunyang has joined #openstack-kolla14:39
mgoddardyoctozepto: which patch? it's hard to see what actually went wrong14:40
mgoddardpossibly that it's SHUTOFF?14:41
mgoddardcould be related to new nova-ironic callbacks14:41
mgoddardconfig added for it here: https://review.opendev.org/68073714:42
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637614:46
*** priteau has quit IRC14:47
yoctozeptomgoddard: unrelated hrw's patch14:47
hrw?14:47
yoctozeptohttps://review.opendev.org/#/c/682618/14:47
*** k_mouza has joined #openstack-kolla14:48
*** luksky has quit IRC14:48
yoctozeptohrw: I was asking mgoddard about the failure reason14:48
hrwah14:48
mgoddardmeeting in 10 minutes14:50
mgoddardmgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne14:50
hrwok14:51
*** priteau has joined #openstack-kolla14:51
mnasiadkaso, is there a way to get ARA visualisation for steps inside kolla-ansible deploy now? Or do we only get what is in Zuul console tab, which is Zuul level?14:53
*** wuchunyang has quit IRC14:53
*** gkadam has joined #openstack-kolla14:54
mnasiadkayoctozepto: nice error, timeout connecting to Docker API on localhost? :)14:55
mnasiadkahttps://zuul.opendev.org/t/openstack/build/6fba26aab357413e8e6dc416891db478/log/primary/logs/ansible/deploy#8112514:55
*** gkadam_ has joined #openstack-kolla14:55
*** gkadam has quit IRC14:56
*** priteau has quit IRC14:56
yoctozeptomnasiadka: we need to add argument to ara so that it generates a static version of it14:57
yoctozeptoso far been lazy14:58
mnasiadkayoctozepto: yeah, I thought so14:58
*** wuchunyang has joined #openstack-kolla14:58
*** priteau has joined #openstack-kolla14:58
yoctozeptois that one of my changes?14:58
yoctozeptowith that sad error14:58
*** chason has joined #openstack-kolla14:58
mnasiadkayoctozepto: the one you referenced a couple of lines earlier :)14:59
yoctozeptoah, the nfv case15:00
hrwding! 170015:00
yoctozeptothere was some loss of network it seems15:01
yoctozeptomgoddard15:01
yoctozeptodo your job15:01
yoctozepto:D15:01
mgoddardzzz15:01
mgoddard#startmeeting kolla15:01
openstackMeeting started Wed Sep 18 15:01:24 2019 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
*** openstack changes topic to " (Meeting topic: kolla)"15:01
openstackThe meeting name has been set to 'kolla'15:01
mgoddard#topic rollcall15:01
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:01
mgoddard\o15:01
hrw /o\15:01
yoctozeptoo/15:01
scottsolo/15:01
chasono/15:01
mnasiadkao/15:01
generalfuzzo/15:01
priteauo15:02
priteauo/15:02
yankcrime\o15:02
mgoddardwelcome back yankcrime15:02
yankcrimehi mgoddard15:02
*** wuchunyang has quit IRC15:03
mgoddard#topic agenda15:03
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:03
mgoddard* Roll-call15:03
mgoddard* Announcements15:03
mgoddard** Vote for Kayobe Train planning meeting time: https://doodle.com/poll/y7fakbhbkfx5hqyk15:03
mgoddard* Review action items from last meeting15:03
mgoddard* CI status15:03
mgoddard* Train release planning15:03
mgoddard* Review priorities15:03
mgoddard* Kayobe Train planning15:03
mgoddard#topic announcements15:03
*** openstack changes topic to "announcements (Meeting topic: kolla)"15:03
mgoddard#info Vote for Kayobe Train planning meeting time15:03
mgoddard#link https://doodle.com/poll/y7fakbhbkfx5hqyk15:03
mgoddardWill discuss this a little more later15:04
mgoddard#info CentOS 7.7 released yesterday15:04
mgoddardonly minimal CI breakage this time15:04
*** gkadam_ has quit IRC15:04
mgoddard#info CentOS 8 released next week, 24th15:05
*** jovial[m] has joined #openstack-kolla15:05
hrwIn october I may take a look at centos:8 builds15:06
mgoddard#info OpenStack projects releasing RC1 next week, final release 14-18 Oct15:06
mgoddardthat will be good hrw15:06
mgoddardwe should try to get ahead as soon as possible in case there are problems we need to report back15:07
mgoddardAny others?15:07
yoctozeptoipv6 is not a goal of deployment projects in train15:07
yoctozeptoprobably in ussuri15:07
mgoddardRight15:07
yoctozeptoso we are step ahead 8-)15:07
mgoddardstill nice to have15:07
mgoddardalways15:07
*** chason has quit IRC15:07
yoctozeptoindeed nice15:07
mgoddard(when complete)15:07
mgoddard#topic Review action items from last meeting15:08
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)"15:08
mgoddardThere were none15:08
*** chason has joined #openstack-kolla15:08
mgoddard#topic CI status15:08
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:08
openstackgerritRadosław Piliszek proposed openstack/kolla stable/rocky: base: fix OracleLinux build  https://review.opendev.org/68292715:08
openstackgerritRadosław Piliszek proposed openstack/kolla stable/queens: base: fix OracleLinux build  https://review.opendev.org/68292815:09
mgoddardWe have issues with oraclelinux on stable branches. Fixed in stein, looks like yoctozepto is backporting as we speak15:09
yoctozeptogetting unbroken as we speak 8-)15:09
hrw+2 on both15:09
yoctozeptool is not an issue of train15:09
yoctozeptoand we are dropping it nicely15:09
mgoddardwe had some flakiness in kayobe CI bare metal deploys, 'fixed' with iptables15:10
yoctozeptowhy 'fixed'?15:10
mgoddardhttps://review.opendev.org/#/c/681700/15:10
yoctozeptoweird indeed15:11
yoctozeptoat least it should always fail15:11
mgoddardcause was docker CE adding a default DROP policy to the FORWARD chain, in combination with bridge-nf-call-iptables15:11
mgoddardI never really understood why it only sometimes failed, and exactly what the problem was15:11
mgoddardbut a blanket ACCEPT on the FORWARD chain did the trick15:12
yoctozeptoworth noting if it does not break real deploys somewhere15:12
mgoddard+1, docker's policy has caused a few different issues for us15:12
priteauI've had issues on a Queens deployment using DVR15:12
priteauI ended up running docker with --iptables=false15:13
yoctozeptomaybe it does something funny on the lower layers as well (raw, mangle, nat stuff)15:13
yoctozeptoI seem to forget about raw and it can be tricky left unchecked :D15:14
yoctozepto(vide that ipv6 problem with firewalld on)15:14
mnasiadkamaybe it's time to add iptables: false to /etc/docker/daemon.json ;-)15:14
mgoddardpriteau: that is tempting. It wouldn't hurt kolla-ansible, I was just worried in case people run other containers without net=host15:14
priteauThere are security implications15:15
yoctozeptomnasiadka, mgoddard, priteau: zun impact15:15
mnasiadkayoctozepto: I don't care about Zun, who uses it anyway :)15:15
mgoddardouch15:16
yoctozeptobehave, mnasiadka, behave15:16
mnasiadkaFrankly speaking I never understood why we use the same container runtime for control plane and Zun15:16
yoctozeptomnasiadka: because we cannot otherwise15:17
mnasiadkayoctozepto: I would challenge that, but this is not the meeting to do it :)15:17
yoctozeptoyeah, I would like to hear that15:17
mgoddardI think we have digressed15:17
yoctozeptoindeed15:17
mgoddard#topic Train release planning15:17
*** openstack changes topic to "Train release planning (Meeting topic: kolla)"15:17
*** ivve has quit IRC15:18
mgoddardAccording to our release status section, we have 2 weeks until our feature freeze15:18
mgoddardKolla feature freeze: Sep 30 - Oct 0415:18
mgoddardI think we'll need to exclude centos8 & centos py3 from that15:19
mgoddardLet's assess what other priorities look like15:19
hrwyep15:19
yoctozeptomgoddard: or do the standard 2.5 month delay15:20
mgoddardsupport matrix won't be affected by freeze15:20
*** goldyfruit_ has quit IRC15:20
mgoddardyoctozepto: we still froze a few weeks after everyone else IIRC15:20
yoctozeptomgoddard: seems it got hit by hot winds then15:20
mgoddardwe'd just need to backport anything after RC115:21
openstackgerritMerged openstack/kayobe stable/stein: Fix kayobe-overcloud-centos CI job flakiness  https://review.opendev.org/68282915:21
mgoddardI think that's still a good approach, agree?15:21
mgoddard^ oh nice I can mark kayobe CI GREEN again15:21
mgoddardNova cells could be at risk - dougsz is away next week15:22
mgoddardI'd like to get the nova part of that work merged this cycle15:23
mgoddard(at least)15:23
mgoddardhttps://review.opendev.org/67565915:23
mgoddardWe finished the ansible upgrade \o/15:24
mgoddardInternal TLS is coming along nicely, first patch merged15:24
mgoddardI don't know if we'll get the backend patch in this cycle15:25
mgoddardbut we'll see15:25
mgoddardthis one is now a dependency of TLS: https://review.opendev.org/#/c/58759015:26
mgoddardand then it's this one: https://review.opendev.org/#/c/66451715:26
mnasiadkaI love those multi file small fixes :)15:26
hrwmaybe we should start use BP+1/-1 to mark what we need in Train and what goes for Ussuri?15:27
mgoddardoh yeah, get all those CI jobs going15:27
mgoddardI've been adding RP+1 for train priorities15:27
mgoddardRP-1 seems a bit mean :)15:28
mgoddardbut maybe nearer the freeze it will help15:28
mgoddardwhat's the status of health checks mnasiadka?15:28
mnasiadkaI doubt I'll get them in this cycle, looks like some debugging needed on docker-py level to get this working - and then there's a lot of work ahead15:29
yoctozepto:-(15:29
mgoddardk, good to know so we can focus elsewhere15:29
mgoddardyoctozepto: reckon IPv6 can land in time?15:30
yoctozeptomgoddard: I confirm, it works, CI is just grumpy :-)15:30
yoctozepto(well, ubuntu worked well)15:30
mgoddardok15:31
mgoddardI added patches to generate PDF docs15:32
mgoddardhttps://review.opendev.org/68261015:32
mgoddardhttps://review.opendev.org/68261115:32
mgoddardhttps://review.opendev.org/68261615:32
mgoddardshamelessly adapted from ironic's hard work15:32
mgoddardit's a community goal, so I'll mark RP+115:32
yoctozeptoseems we need more irrelevant files around15:33
mgoddardmaybe15:33
yoctozeptono need to test docs with ansible15:33
mgoddardhad to fix requirements.txt for those: https://review.opendev.org/#/c/68264615:33
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637615:34
hrw+2 on pdf15:34
mgoddardI think that's all the big stuff. I marked some things as being pushed out to Ussuri15:34
mgoddardDid I miss anything?15:35
mgoddard#topic Review priorities15:36
*** openstack changes topic to "Review priorities (Meeting topic: kolla)"15:36
mgoddardAnything we should focus on this week?15:36
*** zbr has quit IRC15:36
mgoddardalready marked PDFs15:36
mgoddardCells15:37
yoctozeptovxlan15:37
*** zbr has joined #openstack-kolla15:37
yoctozeptoipv615:37
yoctozeptoI do, you review15:37
hrw-OL15:37
mgoddardI think -OL went in already?15:37
yoctozeptosame here15:37
mgoddardah, not kolla15:37
mgoddardadded15:38
hrwOL stuff: https://review.opendev.org/#/c/682927/ https://review.opendev.org/#/c/682928/ https://review.opendev.org/#/c/682865/15:38
mgoddardyoctozepto: could you add bp/ topic to your IPv6 patches?15:39
mgoddardhttps://review.opendev.org/#/q/topic:bp/ipv6-control-plane15:39
mgoddardyoctozepto: feel free to add RP+1 to your IPv6 patches15:40
mnasiadkamgoddard: what's the plan with ceph-ansible spec? do we want to merge it now, in Ussuri or never? ;-)15:40
mgoddardmnasiadka: hmm15:40
mnasiadkaUssuri might make sense, we'll know more after the experimental CI job with ceph-ansible will be merged ;-)15:40
mgoddardmnasiadka: +215:41
yoctozeptomgoddard: ok, right when done15:41
mgoddardmnasiadka: I think the direction is sound, even if we don't know when we'll do it15:42
mgoddardKayobe priorities this week?15:42
mgoddardpriteau, jovial[m]?15:42
*** k_mouza has quit IRC15:42
priteauI am going over the storyboard and identifying things we can fix in Train15:43
mgoddardgreat. we can discuss the train planning in the next section, just looking for review priorities for this week atm15:44
mgoddardwe have a few bug fixes waiting for review, I'll pick some and mark RP+115:45
priteauOh, sorry15:45
mgoddardpriteau: np15:45
mgoddardAnyone know about kolla-cli15:45
priteauReview priority: stable/queens backport https://review.opendev.org/#/c/679269/15:45
mgoddardpriteau: +1. Feel free to add RP+115:45
mgoddardIs kolla-cli going to be ready to release?15:46
yoctozeptomgoddard: it looks it was a temporary activity burst15:46
jovial[m]I'll try and review 679269 now15:46
mgoddardyoctozepto: I have seen a few patches recently15:47
mnasiadkamgoddard: do you have any contact with somebody that is merging that work? ;-)15:48
yoctozeptothey don't seem to visit irc channel15:48
mnasiadkaprobably great firewall of China is to blame15:48
yoctozeptoprobably15:48
mgoddardyeah15:48
yoctozeptolet's mail them then15:49
mgoddardmnasiadka: not really. Jeffrey4l is the only one from 99cloud who is on IRC AFAIK15:49
mgoddardok, last 10 minutes15:49
mnasiadkamgoddard: I guess this is not very easy for him, I worked with some Chinese people in IBM, and they had to do a lot of tunnelling :-)15:49
mgoddardI expect so15:50
yoctozeptomgoddard: maybe he has higher clearance ;-)15:50
mgoddard#action mgoddard to email Jeffrey4l & co about kolla-cli15:50
mgoddard#topic Kayobe Train planning15:50
*** openstack changes topic to "Kayobe Train planning (Meeting topic: kolla)"15:50
mgoddardOk kayobians15:50
mgoddardpriteau & jovial[m]15:50
* yoctozepto just eavesdropping15:51
mgoddardCurrently no slot next week suits everyone :(15:51
mgoddardIf I allow for even hours does that help?15:51
mgoddardcouldn't see any way to overlap in doodle15:51
mgoddardI think I'll just need to pick one and apologise15:53
mgoddardWe should have a think about what to talk about in advance. priteau started an etherpad15:53
jovial[m]+115:54
mgoddard#link https://etherpad.openstack.org/p/kayobe-train-release15:54
*** cah_link has quit IRC15:54
*** k_mouza has joined #openstack-kolla15:55
mgoddardI think we could follow a similar pattern to kolla PTG15:55
mgoddarddump a list of things to discuss, then vote on priorities15:55
mgoddardbearing in mind we don't have much time15:55
mgoddardbut priorities can always roll over15:55
mgoddardso, bring your ideas for discussion, and comb storyboard for anything that we should be picking up15:56
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637615:56
mgoddardanything else we should be doing in those 2 hours?15:56
priteauDiscuss path forward for Kolla and Kayobe?15:57
jovial[m]is remaining patches for stuff that already has a change up for review?15:57
priteauAre there things we need to change in how the two integrate together15:57
mgoddardpriteau: good question15:57
mgoddardit's a good time for open questions like that15:57
mgoddardchuck it on the pad15:58
mgoddardthe format of the pad probably needs cleaning up15:58
mgoddardit's more of a status tracking pad15:58
mgoddardshould we have a fresh one for ths discussion?15:59
priteau+115:59
mgoddardok15:59
mgoddardhttps://etherpad.openstack.org/p/kayobe-train-planning15:59
priteauSince we just released Stein, this is a kind of PTG meeting15:59
mgoddard#link Welcome to Etherpad Lite!15:59
mgoddardThis pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!15:59
priteauwithout the snow15:59
mgoddardEtherpad Lite on Github: http://j.mp/ep-lite15:59
mgoddardn15:59
mgoddardoops15:59
mgoddard#link https://etherpad.openstack.org/p/kayobe-train-planning15:59
mgoddardcould be snowing somewhere16:00
mgoddardit's 1600UTC, regrettably we must end16:01
mgoddardthanks everyone16:01
mgoddardthe train is fast approaching16:01
mgoddard#endmeeting16:01
*** openstack changes topic to "Topic for #openstack-kolla is: support: ask.openstack.org | New to Kolla: docs @ https://docs.openstack.org/kolla/latest/ | | Kolla IRC meetngs on Wednesdays @ 15:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"16:01
openstackMeeting ended Wed Sep 18 16:01:51 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-18-15.01.html16:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-18-15.01.txt16:01
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-18-15.01.log.html16:01
*** henriqueof has quit IRC16:02
*** rpittau is now known as rpittau|afk16:03
yankcrimefyi i called kolla 'fucking magic' during a lightning talk i did at the london openstack meetup16:04
scottsol^^ he did16:04
yankcrimejust thought you should know i'm still doing my advocacy bit for the project ❤️16:04
openstackgerritMerged openstack/kolla master: base: enable rabbitmq and elasticsearch repos for ppc64le  https://review.opendev.org/68261816:05
*** zbr is now known as zbr|ruck16:07
*** hjensas|afk has joined #openstack-kolla16:08
mnasiadkamgoddard: keystone role refactor change is merging - do we still have any usage for kolla_keystone* modules?16:08
mgoddardmnasiadka: boom16:09
yoctozeptomnasiadka: now tell me about your idea for running a separate zun backend16:09
mnasiadkamgoddard: I think it should be at least boom^216:09
mgoddardI don't believe so. But we should wait before removing to provide a transition16:09
mgoddardmaybe deprecate for removal in U?16:09
mnasiadkayoctozepto: I don't know Zun, but what's the problem to run multiple container runtimes on one host?16:10
mnasiadkamgoddard: yeah, I was thinking about that16:10
mgoddardnice16:10
mgoddardyankcrime: you legend16:10
yoctozeptomnasiadka: what multiple you mean16:10
yoctozeptoport kolla to podman and run zun on podman-deployed docker? :D16:11
mgoddardyankcrime: how was the meetup?16:11
yankcrimedown on numbers, but it's been nearly 2 years since the last one so it's lost all momentum16:12
yankcrimehopefully it'll pick up again though, we've got the next one scheduled for november 28th at the microsoft reactor of all places16:12
*** gfidente has quit IRC16:12
noxoidyoctozepto, you good with me just copying what you did for cinder for designate's coordination backend? https://review.opendev.org/#/c/682095/16:12
yoctozeptonoxoid: sure, we can later globalize it16:13
noxoidperfect. ill try to get that done in the next day or two16:13
yoctozeptorefer the bug as related16:13
noxoidthanks16:13
noxoidyep16:14
mnasiadkayoctozepto: no, run another docker/crio/podman in addition :)16:14
mnasiadkayoctozepto: never done that? :)16:15
*** radeks has quit IRC16:16
mnasiadkayoctozepto: there's dind I guess, but that's not really production grade :)16:17
yoctozeptomnasiadka: what benefits does it bring though? If I enable iptables in either it's going to feel like the king of firewall anyway :-)16:17
mnasiadkayoctozepto: I just mean that we do crazy stuff with docker daemon config for control plane, which might influence it's behaviour16:18
mnasiadkacrazy == configure Etcd :)16:18
yoctozeptomnasiadka: I see16:18
yoctozeptowell, it's food for thought re: stability of solution16:18
mnasiadkayoctozepto: maybe I'm just over-cautious, or something16:19
yoctozeptonah, that's ok16:19
yoctozeptomgoddard, mnasiadka: https://review.opendev.org/682842 <- works16:20
yoctozeptonow can rebase vxlan on that16:20
yoctozeptoand make mnasiadka happy again16:20
mnasiadkalol16:20
yoctozeptomake CI green again 8-)16:20
*** chason has quit IRC16:21
*** gmann is now known as gmann_afk16:23
*** goldyfruit_ has joined #openstack-kolla16:28
*** ivve has joined #openstack-kolla16:32
*** k_mouza has quit IRC16:43
*** scottsol has quit IRC16:50
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069016:52
*** dougsz has quit IRC16:56
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069016:58
andreinmgoddard: are you still around?17:04
*** luksky has joined #openstack-kolla17:06
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069017:07
openstackgerritMerged openstack/kolla-ansible master: Build pdf doc  https://review.opendev.org/68261117:11
shubjeroI am trying to follow the multinode instructions for kolla-ansible and its saying a docker registry is mandatory but I've never done that and the instructions dont tell me how to get the 'kolla' directory to run a 'start-registry'. Do i just need to clone the specific github repo? https://docs.openstack.org/project-deploy-guide/kolla-ansible/rocky/multinode.html17:14
kplantit's not actually mandatory17:15
kplantif you don't specify one, it will use docker.io17:15
shubjerooh17:16
shubjerofor testing thats fine thenn, thanks kplant17:16
kplantnp17:16
kplantif you're wanting to setup a registry though it's not hard, you can just plop docker on a machine/vm start a registry container and start pushing images to it17:17
kplanti think it needs to be tls with a legitimate certificate though17:17
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069017:17
shubjeroyeah i did a bit of reading and looked in to the start-registry file on github so now i know what it does17:17
shubjeroi'll get there eventually17:18
kplantit's definitely the way to go, way faster and more reliable17:18
kplantyou don't want docker.io/cloudflare being down stopping your deployment17:18
shubjeroexactly17:18
openstackgerritMerged openstack/kayobe stable/queens: Allocate IPs separately from configuring network  https://review.opendev.org/67926917:24
shubjerofatal: [kolla-c1]: FAILED! => {"changed": false, "msg": "Unable to find any of pip3 to use.  pip needs to be installed."}17:27
shubjerohmm?17:28
shubjerothis is when im doing a multinode bootstrap17:28
kplantdid you get kolla via git or pip17:29
shubjero'pip install kolla-ansible'17:29
kplanthmm17:30
shubjerothere was one option i was not confident in selecting.. the binary/source choice in globals.yml17:31
shubjeroi chose source17:31
kplantyeah source is the better choice17:31
kplantwhat about version?17:31
shubjerorocky17:31
kplanti'm unsure why it's trying to use py317:31
kplanti've deployed stein on centos and i don't have python3 at all :-\17:32
shubjerothe underlying OS is 18.04.. I know it said for new users to use 16.04 but our prod env is all 18.04 now so i didnt want to back pedal17:32
shubjero(ubuntu)17:32
shubjero1s17:33
kplanthmm, i've only ever done centos17:33
kplantno me gusta ubuntu17:33
openstackgerritMerged openstack/kolla-ansible master: Refactor service, endpoint and user registration  https://review.opendev.org/58759017:34
shubjeroi told ansible to use python3 as the ansible intpreter in the inventory file.. let me inset that17:34
shubjero*unset17:34
kplantoh yeah17:34
kplantdon't do that17:34
shubjeroyup, fixed17:34
kplantare you using virtualenvs on the target hosts?17:34
shubjerodont think so. I'm just running 'kolla-ansible -i ./multinode bootstrap-servers' at this point17:35
kplantthen you're fine not specifying17:35
shubjeroi spun up 3 controller nodes and 1 compute node and just testing kolla for the first time17:35
*** zbr|ruck is now known as zbr17:36
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069017:40
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069017:51
*** priteau has quit IRC18:01
openstackgerritMerged openstack/kolla master: Remove support for OracleLinux  https://review.opendev.org/68286518:11
openstackgerritMerged openstack/kolla master: build: enable Octavia for Ubuntu/binary  https://review.opendev.org/68267118:11
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/stein: Fix for haproxy precheck failing on CentOS running non-root  https://review.opendev.org/68295718:14
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: CI: Use VXLAN overlay network  https://review.opendev.org/67069018:15
*** k_mouza has joined #openstack-kolla18:19
*** k_mouza has quit IRC18:23
*** openstackgerrit has quit IRC18:37
*** goldyfruit___ has joined #openstack-kolla18:38
*** goldyfruit_ has quit IRC18:41
*** openstackgerrit has joined #openstack-kolla19:02
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for self-signed SSL certificates  https://review.opendev.org/66451719:02
*** jbadiapa has quit IRC19:44
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637619:44
*** kplant has quit IRC19:46
*** jbadiapa has joined #openstack-kolla19:57
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637620:02
*** pcaruana has quit IRC20:10
openstackgerritJoseph M proposed openstack/kolla-ansible master: [designate] Add coordination backend for designate workers  https://review.opendev.org/68209520:20
openstackgerritJoseph M proposed openstack/kolla-ansible master: [designate] Add coordination backend for designate workers  https://review.opendev.org/68209520:22
*** goldyfruit___ has quit IRC20:44
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for self-signed SSL certificates  https://review.opendev.org/66451720:50
*** goldyfruit___ has joined #openstack-kolla20:56
*** signed8bit has quit IRC21:01
*** signed8bit has joined #openstack-kolla21:02
*** vmixor has quit IRC21:14
*** goldyfruit_ has joined #openstack-kolla21:22
*** goldyfruit___ has quit IRC21:25
*** JamesBenson has quit IRC21:25
*** JamesBenson has joined #openstack-kolla21:30
*** JamesBenson has quit IRC21:35
*** jonaspaulo has quit IRC21:56
*** goldyfruit_ has quit IRC22:15
*** goldyfruit_ has joined #openstack-kolla22:40
*** vmixor has joined #openstack-kolla23:05
*** luksky has quit IRC23:13
*** goldyfruit_ has quit IRC23:24
*** goldyfruit_ has joined #openstack-kolla23:29
*** BjoernT has quit IRC23:30
*** JamesBenson has joined #openstack-kolla23:46
*** ivve has quit IRC23:48
*** JamesBenson has quit IRC23:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!