Wednesday, 2023-06-28

opendevreviewMerged openstack/kolla master: Revert "neutron-mlnx-agent: pin pyzmq at 21.*"  https://review.opendev.org/c/openstack/kolla/+/88205103:14
opendevreviewDr. Jens Harbott proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88702005:45
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88702006:18
mnasiadkafrickler: thanks for fixing linter, but I missed editing the keepalived.conf template as well ;-)06:21
fricklermnasiadka: ah, ok, I was thinking it was fine if no service check is actually made, but yes, we can drop it then altogether06:48
fricklerI'm still thinking a reno and a bug reference would be good for backporting06:48
mnasiadkafrickler: yup, let me create a bug06:49
mnasiadkafrickler: just thinking if we shouldn't set keepalived_track_script: "{{ groups['loadbalancer'] | length > 1 }}" - this way it would also help operators (although probably it doesn't happen that often to have a single node and run deploy --tags keystone just after upgrade)06:54
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88702007:01
mnasiadkafrickler: done07:01
opendevreviewAdam Stackhouse proposed openstack/kolla-ansible master: Adding mariadb_port to wsrep sync status so alterative ports can be used  https://review.opendev.org/c/openstack/kolla-ansible/+/88658107:31
fricklermnasiadka: checking for group size may create issues when upgrading an initial deployment with more nodes or if people have a two-LB setup with one node temporarily down, so I'd prefer the explicit solution08:24
frickleranother possibility would be to not do this at all and instead add a check into the CI waiting for keepalived to be up again before proceeding with the upgrade. that might even be helpful outside of CI08:27
mnasiadkawell, we run deploy --tags keystone in CI after the upgrade - that kind of check would need to be in prechecks or site.yml08:31
mnasiadkaor we would need to have that check in https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/tasks/bootstrap.yml08:33
mnasiadkaso ideally create a service-db-register role and import it in all other roles08:33
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88702008:58
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88702008:58
SvenKieskefrickler: would it be possible to even skip the complete keepalived deployment in single node installs? :thinking:09:06
mnasiadkait's possible to skip haproxy on single node, we recommend that on dev installs09:07
mnasiadkabut moving from no keepalived to multiple controllers and keepalived by default? I have the same concerns as frickler 09:07
mnasiadkaanyway, let's go with what we have now, no reason to waste more time on a CI-only issue :)09:11
SvenKieskewell, given the state of CI i wished we would waste more time on CI only issues ;P09:15
mnasiadkait's getting better and better as you can see ;)09:16
SvenKieskeyes! :)09:16
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9  https://review.opendev.org/c/openstack/kolla-ansible/+/88448809:49
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9  https://review.opendev.org/c/openstack/kolla-ansible/+/88448809:49
mnasiadkaand that one (once it's green) should get us to CI greenland09:52
hrwo, looks like bookworm requirements are merged. now time for bookworm patches10:55
mnasiadkafrickler: https://review.opendev.org/c/openstack/kolla-ansible/+/887020 - it's passing, debian-upgrade passed multiple times during rework - so we should be fine10:59
mnasiadkabbezak: ^^10:59
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134011:27
mnasiadkakevko: I see you're brave :)11:35
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9  https://review.opendev.org/c/openstack/kolla-ansible/+/88448811:46
kevkomnasiadka: haha i am :D 12:21
kevkoand found some issues :D 12:21
mnasiadkamgoddard mnasiadka hrw  bbezak frickler kevko SvenKieske mmalchuk gkoper - meeting in 612:54
SvenKieskewhere's my 10 minute warning? :(12:55
mnasiadkathere's 6 minutes, I'm unfortunately not a bot :)12:55
SvenKieskedamn :D12:57
mnasiadka#startmeeting kolla13:00
opendevmeetMeeting started Wed Jun 28 13:00:13 2023 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
opendevmeetThe meeting name has been set to 'kolla'13:00
mnasiadka#topic rollcall13:00
mnasiadkao/13:00
gkopero/13:00
mgoddard\o13:00
fricklero/13:01
hrw\~o~\13:01
mattcreeso/13:01
SvenKieskeo/13:02
darmach\o13:02
mnasiadka#topic agenda13:03
mnasiadka* Announcements13:03
mnasiadka* Review action items from the last meeting13:03
mnasiadka* CI status13:03
mnasiadka* Release tasks13:03
mnasiadka* Regular stable releases (first meeting in a month)13:03
mnasiadka* Current cycle planning13:03
mnasiadka* Additional agenda (from whiteboard)13:03
mnasiadka* Open discussion13:03
mnasiadka#topic Announcements13:03
mnasiadkaAs discussed previously - I'm off for the next three weeks13:03
mnasiadkaI cancelled weekly meetings for that time13:03
mnasiadka#topic Review action items from the last meeting13:04
mnasiadkamnasiadka to raise EOL for stable/wallaby and send email to the ML - done13:04
mnasiadka#topic CI status13:04
mnasiadkaI guess green on average13:04
mnasiadkafixes ongoing here and there13:04
mnasiadka#topic Release tasks13:04
mnasiadkaI think we restored using master on the master branch13:05
mnasiadkaSo no other tasks planned in the calendar13:05
mnasiadka#topic Regular stable releases 13:05
mnasiadkaDo we want to post stable releases next week?13:06
frickleror maybe now, lest we forget?13:06
hrwwas there wallaby release before eol? or do we not do such?13:06
SvenKieskethat would be good imho, there where some users here asking for bugs that where closed in git, but not on pypi releases13:06
mnasiadkawe can't do releases when branch is EM13:06
fricklerwallaby was long em, no releases13:06
hrwok13:06
mnasiadkafrickler: willing to raise a patch or do we need another volunteer?13:07
hrwif erlang/rmq got sorted out on stable/* then maybe release?13:07
fricklerI can do that, just need to know what to wait for. erlang rmq seems sensible13:07
mnasiadkayeah, it got sorted-ish, we need to sort out Ubuntu/aarch64 it seems13:07
SvenKieskeI _guess_ rmq should be stable, but I just moved a bug about crashing rmq to kolla-ansible:13:08
SvenKieskehttps://bugs.launchpad.net/bugs/202366813:08
mnasiadkaisn't 3.9 EOL?13:08
SvenKieskeat least it was claimed the installation was done according to the kolla docs, so I would be interested in finding out why this crashes on yoga13:09
mnasiadkahttps://www.rabbitmq.com/versions.html13:09
fricklerdidn't we have this last week already? they're using binary build it seems13:09
SvenKieskeI think I found 3.9 referenced in some docs/files on an older branch of us?13:09
mnasiadkabinary, uh13:09
mnasiadkaanyway, bug is there, we can have a look13:10
mnasiadka#topic Current cycle planning13:10
mnasiadkaI see kevko has revived Let's Encrypt - so there's a chance we'll merge it finally13:10
hrwDebian/bookworm?13:10
mnasiadkahrw: stage is yours for Bookworm ;)13:10
kevkojust started to debug why it is not working out of the box on my env :P 13:11
hrwmnasiadka: it is https://review.opendev.org/q/topic:for-debian-bookworm-upgrade as it was before13:11
mnasiadkathat's kolla side13:11
mnasiadkawhat about nodepool instances and testing on those?13:11
hrwmnasiadka: building we can do on anything13:12
fricklerstill need to look into building nodepool image13:12
mnasiadkahrw: we can, if you're fine we drop Debian in kolla-ansible - let's just build images :)13:13
opendevreviewMerged openstack/kolla-ansible master: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88702013:13
mnasiadkaGiven we have vacation season ahead - I'm pretty sure we're going be last minute merging kolla-ansible support13:13
mnasiadkaAnyway, let's not forget about building nodepool image, I think it's fine if we don't have a mirror and download over the internet (just like in Rocky case)13:14
hrwmnasiadka: patches can be hold with 2 CR+2, V+1 and without W+113:14
SvenKieskethe support matrix at https://docs.openstack.org/kolla/xena/support_matrix doesn't indicate btw that we don't support binary builds of rabbitmq containers, should we change that?13:14
fricklermirror is done13:15
mnasiadkaah, mirror is done, fantastic13:15
hrwmnasiadka: but without reviews what is a point of doing anything further?13:15
SvenKieskesorry, maybe defer my point to "open discussion"13:15
fricklerI'm reviewing those patches13:15
mnasiadkahrw: I'll have a look later today13:15
frickleralso maybe we want RP+1 for those? or +2?13:15
SvenKieskeI did reviews there as well, there are open discussion points, without answer, so I didn't look again yet13:16
mnasiadkahrw: it would also help if those would be passing Zuul ;)13:17
hrwmnasiadka: sure13:17
mnasiadkaso run a recheck, let's see fresh results, and try to get somewhere :)13:18
mnasiadkaOk, podman guys are not here today - but I've seen some updates13:18
hrwINFO:kolla.common.utils.openstack-base:    The user requested cachetools13:18
hrwINFO:kolla.common.utils.openstack-base:    The user requested (constraint) cachetools===5.3.013:18
hrwthat kind of bugs are "not my fault" ;D13:18
SvenKieskeI notice rabbitmq 3.10 in bookworm is only supported upstream until end of july, this year, is the backporting in debian good? :)13:18
mnasiadkahrw: if it's ord rax nodepool provider - they have some weird networking issues towards pypi13:18
kevkobtw, yoga is unbuildable on localhost 13:19
mnasiadkakevko: I'm still amazed how it works in CI13:19
kevkobut passing in CI ...do you know why ? 13:19
hrwSvenKieske: it can/will move to 3rdparty repo13:19
mnasiadkakevko: even the experimental job that doesn't use infra wheel is failing?13:19
kevkohorizon is really unbuildable on localhost ..just try it ...i am fixing it in my downstream repo by limiting setuptools 13:19
kevkoi really don't know :D ... just try it on localhost 13:20
mnasiadkaIs anybody able to help kevko - or we just merge the backport?13:20
SvenKieskekevko: mhm I know some ppl who will be very sad about it13:20
SvenKieskekevko: do you have any patch I can look at?13:20
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible stable/2023.1: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88706913:20
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible stable/zed: loadbalancer: Add option to not define track script  https://review.opendev.org/c/openstack/kolla-ansible/+/88717013:20
mnasiadkaSvenKieske: https://review.opendev.org/c/openstack/kolla/+/87391313:21
kevkoSvenKieske: it's abandoned for now i think13:21
kevkomnasiadka thanks, yeah that one13:22
SvenKieskety13:22
mnasiadkanot abandoned, I tried building locally long time ago - and it also failed for me13:22
mnasiadkaso the mystery is - how does it work in zuul13:22
opendevreviewMichal Nasiadka proposed openstack/kolla stable/yoga: Pin setuptools=67.2.*  https://review.opendev.org/c/openstack/kolla/+/87391313:22
mnasiadkaok then, let's move on13:23
mnasiadka#topic Additional agenda (from whiteboard)13:23
mnasiadkagkoper: Infloblox designate driver (mdns running as root)13:23
mnasiadkagkoper: stage is yours13:23
gkopero/13:24
darmach(stage fright phase)13:24
gkoperInfoblox does not support changing the port used to communicate with MDNS to request zone transfers(AXFRs)13:24
gkoperTherefore MDNS containers need to be bound to port 53. Kolla built designate containers are starting with user designate, so are unable to bound the service on privileged port 5313:24
gkoperDirty workaround is to locally build containers to start with root user 13:24
gkoper(This poses security risk)13:24
gkoperAnother approach is to use CAP_NET_BIND_SERVICE to provide the capability for user designate to bind a service to a privileged port (0-1024) [Testing now ]13:24
mnasiadkaI think we're doing something similar to something related to prometheus - to be able to run ping13:24
gkoperWe also found some issues while templating pools.yml13:24
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm'  https://review.opendev.org/c/openstack/kolla/+/88608813:25
mnasiadkagkoper: one thing at a time13:25
fricklerbuilding your own containers is the documented solution for this13:25
mnasiadkafrickler: I know you -2d a patch to run as root - but that's logical - do you see any problems with using CAP_NET_BIND_SERVICE?13:26
mnasiadkafrickler: we build downstream one set of container images, wouldn't want to have a separate one for infoblox, and separate mdns for bind ;-)13:26
SvenKieskewell it provides privileges that most deployments don't need13:26
fricklercould we make that optional? I'd have to check what we do for prometheus13:26
mnasiadkain prometheus we setcap for blackbox exporter binary13:26
* SvenKieske is also thinking about how to make this conditional13:27
gkoperfrickler: https://github.com/openstack/kolla/blob/70f74eb64101431e23d56c6a7df96d7aab37ce2f/docker/prometheus/prometheus-blackbox-exporter/Dockerfile.j2#L3213:27
mnasiadkait still runs as unprivileged user13:27
mnasiadkawhich should be fine13:27
darmachAs for the pools.yml templating we ran into wrong templating of ns_records (list was used as sigle record resulting in string[fqdn]. - easyfix) and nameservers templated using dqdn resulting in designate-manage failing with: "(proper fqdn) is not IP address or host name" 13:27
mnasiadkadarmach: one thing at a time13:27
darmach^ nothing that can't be fixed when we are done with port 53 bind 13:27
mnasiadkalet's get some agreement on the root stuff13:28
SvenKieskeso I think it would be okay to setcap this as well, should be documented of course.13:28
* SvenKieske wondering if this really works with podman13:28
mnasiadkait would only add privileges to bind a low port, which most probably is not a security issue13:29
fricklerit is13:29
mnasiadkaok then, can we make it optional for greater good?13:29
fricklerany privilege that is not needed is a security issue13:29
mnasiadkaThen I'm pretty sure majority of our containers is insecure in those terms ;)13:29
SvenKieskewell, even "needed" ones are :)13:30
fricklerI'm not sure we want to do much special casing for a weird non-free backend13:30
gkoperand running as root is a less one ?13:30
SvenKieskeif I'm not mistaken the default is to run all containers in the host network namespace, correct?13:30
mnasiadkafrickler: we support it out of the box with some kolla-ansible variables, so it would make sense to make it easier and better13:30
fricklermnasiadka: or drop support for it completely?13:31
darmachI can agree with weird non-free backend - there are customers out there in the wild with infoblox deployed though...13:31
mnasiadkafrickler: as you can see - SHPC needs that for it's customers13:31
mnasiadkaso as long as there's somebody that wants to maintain it - I don't see a reason to drop it13:31
fricklerso ... why is building a custom container locally not feasible?13:32
SvenKieskecan we talk about the actual issue, please?13:32
fricklercan we then add a special container build upstream13:32
fricklerand deploy a different container in the infoblox case?13:33
mnasiadkaSvenKieske: the issue is there's an infoblox backend in designate, which is not tested in designate CI by the way, and requires designate-mdns to run on port 53.13:33
SvenKieskei know...13:33
mnasiadkathe other backend, which is bind (and probably powerdns) does not require that13:33
SvenKieskewhat indeed might be a problem, if my assumption above is correct regarding the host network namespace, is that other services might already be bound to port 5313:34
mnasiadkathat's a deployment specific, if somebody wants infoblox, then he needs to deal with that in his env ;)13:34
mnasiadkacurrently we direct users to build it on their own, which is most probably fine - but still requires root user, so we basically direct them to do insecure installation of designate13:35
SvenKieskemnasiadka: well not really, if per default, e.g. systemd-resolved is listening on localhost:53 and you deploy designate with k-a and infoblox and it breaks the default local resolver I would indeed refuse to merge such patches13:35
fricklerso if we amend the doc to use CAP_NET_BIND_SERVICE that is enough for you?13:35
mnasiadkanot really, since we use one source of downstream images for N clients13:36
SvenKieskeit doesn't make much sense to patch something, if the patch doesn't work in major deployment scenarios, so that should at least be tested and be guaranteed to work13:36
fricklerso ... extra container? designate-mdns-insecure?13:37
fricklerjust joking on the name of course13:37
mnasiadkaextra container - why not, and some logic in kolla-ansible to use it when infoblox is enabled13:38
fricklerack13:38
mnasiadkaor the same container and some extend_start logic to do setcap13:38
SvenKieskeunder the provision that this works at all (I have doubts): couldn't we introduce a conditional, that, _if_ infoblox is enabled the container gets (re)started with cap_net_bind_service?13:38
mnasiadkajust let's agree on one of those13:38
fricklerbut that would modify the container at runtime?13:38
SvenKieskeat restart time ;)13:38
mnasiadkait would, but is that something new? we remove default http certs at start time ;)13:39
mnasiadkawe mangle opensearch-dashboards plugins at start time13:39
* SvenKieske thinking about which is more pain, a second container or to modify the existing one13:39
darmach"or the same container and some extend_start logic to do setcap" < I quite like that, we can give it a try and test how it works13:39
SvenKieskebut it really should only setcap on the infoblox conditional13:40
SvenKieskesounds mostly fine to me, and imho less maintenanceburden than a whopping complete container, no?13:40
mnasiadkaSvenKieske: second container with one extra layer that runs one command sounds funny, but that's also nothing new in Kolla land13:40
fricklerso if that works, I think that should be acceptable, then13:40
mnasiadkaok then13:40
darmachGreat13:40
mnasiadkano new images, optional setcap in the existing one based on an ENV variable and we're fine13:41
mnasiadkagreat13:41
mnasiadkadarmach: the other issue seems like pools.yaml template misconfiguration, just propose a new patch and let's discuss it there?13:41
darmachYes, going to do that. It's nothing complicated.13:42
fricklerthere is https://review.opendev.org/c/openstack/kolla-ansible/+/878270 already, which I need to get back to13:42
fricklerplease check if that covers your issue, too13:42
mnasiadkadarmach: can you also have a look at ^^?13:42
SvenKieskeyeah, that would be good :)13:42
* SvenKieske praying for more designate maintainers13:43
mnasiadkafrom my perspective that patch needs to be backwards compatible, and now it's not.13:43
mnasiadkabut let's discuss in the patch itself13:43
darmachWill do, looping in that template was exactly what I was thinking about.13:43
fricklermaybe it needs to be split into some parts, too13:43
darmachMaybe we could split-out the pools.yml part, and I could take a stab at it.13:44
mnasiadkafrickler: might be, but we need to support designate_ns_record, or at least have prechecks saying you need to rework your config13:44
mnasiadkaok then, let's move on13:45
gkoperwe have an edge case , that needs to support more than 1 ns_group to be updated on Infoblox (WAN and LAN)13:45
mnasiadka#topic Open discussion13:45
mnasiadkagkoper: you can override pools.yaml just as other files in kolla-ansible13:46
mnasiadkalet's just cover the usual case, not the oddities13:46
gkopersure, i think Jakub already had an idea howto template that.13:46
gkoperlets move on.13:46
darmachWe could use the loop in junja as @frickler did - to create separate pools13:46
SvenKieskeregarding the crashing rmq 3.9 release I just asked in the bug report to provide reproducer steps as I'm 90% sure we don't mention this release anywhere.13:47
SvenKieskeI also advised to maybe not use binary builds13:47
mnasiadkaSvenKieske: we deprecated it in Yoga and removed in Zed, if they like using deprecated content - fine by me ;)13:48
SvenKieskemaybe we should make our support matrix more clear? or did we already? I think we did..half13:49
SvenKieskewe just dropped the distinction binary/source from our image support matrix, maybe a big disclaimer shouting "we do not support binary images anymore" would be good?13:50
SvenKieskemaybe not worth the effort, I guess there where maybe not even ten users with binary problems, I think.13:50
SvenKieskethat bothered to ask, at least :D13:50
mnasiadkayeah well, binary builds always had its own issues13:53
mnasiadkaok, anything else?13:54
mnasiadkathanks for coming - see you again 26th July13:56
mnasiadkain case of an urgent review request - please bug bbezak and mgoddard ;-)13:56
mnasiadka#endmeeting13:56
opendevmeetMeeting ended Wed Jun 28 13:56:19 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:56
opendevmeetMinutes:        https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-06-28-13.00.html13:56
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-06-28-13.00.txt13:56
opendevmeetLog:            https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-06-28-13.00.log.html13:56
fricklerthx mnasiadka enjoy your time off13:56
SvenKieskethx mnasiadka, happy holidays!13:57
darmachHave fun! Thanks!13:57
fricklerdo we want mariadb bump before bookworm or after? currently they conflict https://review.opendev.org/c/openstack/kolla/+/88292415:58
SvenKieskefirst come, first serve? :) imho the lts patch is much older, no?16:01
SvenKieskeit seems we have a lot of temp CI issues with pypi mirror connectivity, would it maybe possible to host our own partly mirror on #opendev e.g. via https://pypi.org/project/python-pypi-mirror/ ? what do you think?16:02
SvenKieskea pull through cache like gitlab offers would be nice.. I don't think gitea has something like that builtin?16:03
SvenKieskethere's a feature request at least: https://github.com/go-gitea/gitea/issues/2122316:05
SvenKieskee.g. here: https://review.opendev.org/c/openstack/kolla/+/882924/6#message-aad622078f973cee5ff7efc7245b71e08b2e84e116:08
SvenKieskewrong channel -.-16:08
hrwfrickler: before16:11
hrwfrickler: bookworm needs stuff around: nodes, k-a support etc. while mariadb update can be done on what we have now16:12
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm'  https://review.opendev.org/c/openstack/kolla/+/88608816:17
hrwfrickler: thanks for comments.16:18
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: base: install adduser on Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/88194816:21
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: base: Debian 12 uses new APT sources file  https://review.opendev.org/c/openstack/kolla/+/88195016:21
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm'  https://review.opendev.org/c/openstack/kolla/+/88608816:21
hrwhad to rebase whole set16:21
fricklerhrw: actually I'm not sure we can downgrade rmq to 3.10 if we are running 3.11 now on bullseye? that wouldn't work during an upgrade, would it?16:25
hrwfrickler: I will look at having 3.1116:26
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm'  https://review.opendev.org/c/openstack/kolla/+/88608816:27
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm'  https://review.opendev.org/c/openstack/kolla/+/88608816:34
hrwfrickler: rmq 3.1116:34
SvenKieskefyi: regarding pypi CI issues I reraised https://github.com/pypi/warehouse/issues/8568 again, maybe we're having more luck this time (this got closed in 2020)..16:35
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: build: distutils.StrictVersion() is deprecated  https://review.opendev.org/c/openstack/kolla/+/88674116:40
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134016:41
mnasiadkawe probably should move to rmq 3.12 in Bobcat, given how fast they EOL RMQ16:51
hrw3.11 is just to the end of 202316:53
mnasiadkamaybe we also need to think how to do upgrades16:53
mnasiadkawe have 3.9 in Yoga which is EOL16:53
mnasiadkaand we already moved from 3.8 to 3.9 in Yoga16:54
mnasiadkaupgrades inside a stable release I mean16:54
opendevreviewWill Szumski proposed openstack/kayobe master: Adds VGPU support  https://review.opendev.org/c/openstack/kayobe/+/88720016:54
mnasiadkaor maybe it's a way to tell users nicely to move forward :)16:54
hrwmnasiadka: I suggest to stay with one erlang version per release and update rabbitmq until erlang is too old16:55
hrwand then mark branch as 'existing systems only'16:56
mnasiadkaeither way, we need to move to 3.1216:56
mnasiadkaand bump ansible to 2.1516:56
mnasiadkasooner than last time16:56
hrwmnasiadka: propose patches16:56
mnasiadkain August most probably, going for three weeks vacation ;-)16:56
mnasiadkaalthough RMQ might be easy16:57
mnasiadkabut first we should probably fix ubuntu/aarch64 rmq 3.11 so we can backport that easily16:57
hrwmnasiadka: vacations... something I hope for16:57
SvenKieskemy vacation will be moving my household to a different city roughly at the beginning of august, wish me luck :)16:58
hrwSvenKieske: have fun!17:02
hrwSvenKieske: lot of carton boxes and packing tape. Then friends and moving company to pack all that.17:03
hrwI moved ~2y ago. But just 2km only17:03
* hrw out17:05
mnasiadkaI moved 150 meters 8 years ago :)17:11
mnasiadkawell, counting 4th floor down and up to 3rd floor it was a couple more meters ;-)17:12
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: haproxy: support single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/82339517:53
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123917:53
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123917:56
mnasiadkakevko: interested in merging that as well? Finally not only me :)17:57
guesswhat[m]Where is the path to store crt, metadata and mapping for keystone federation host https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L1268 ? 17:57
guesswhat[m]I guess its not possible to use kolla config directory...17:57
kevkomnasiadka: well, letsencrypt working but need some love 18:01
kevkomnasiadka: and for example in my deployment i am using api.master.ultimum.cloud and horizon.master.ultimum.cloud ...and it created only api.master.ultimum.cloud cert as it is defined in globals as fqdn ..18:02
kevkoi remember that you had a patch to use fqdn per service ..it is little bit related ...so i wanted to see how it is working toherher18:03
kevkoworse is on letsencrypt that few times i tried someting and i am blocked from letsencrypt api :D 18:03
kevkomnasiadka: i think we should ask for *.domain if kolla fqdn is set for example api.domain18:04
kevkomnasiadka what do you think ? 18:04
mnasiadkakevko: wildcards are only supported in dns plugins, I don't think http one supports that18:04
kevkomnasiadka: really, i think i used to use wildcard also with http18:06
kevkobut yeah, it is in faq on letsencrypt website18:07
kevko okay, so we should have some list of fqdns then ..18:07
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: haproxy: support single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/82339518:14
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123918:14
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123918:23
guesswhat[m]Anyone using Keystone + OIDC, its quite confusing.. 18:33
kevkowas tried :P 18:34
guesswhat[m]didnt work ?18:37
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134019:12
opendevreviewMichal Nasiadka proposed openstack/kolla master: rabbitmq: Fix repo for ubuntu aarch64  https://review.opendev.org/c/openstack/kolla/+/88722319:45
opendevreviewMichal Nasiadka proposed openstack/kolla master: rabbitmq: bump version to 3.12  https://review.opendev.org/c/openstack/kolla/+/88722519:53
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: ansible: bump min version to 2.14 and max to 2.15  https://review.opendev.org/c/openstack/kolla-ansible/+/88722719:58
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134020:13
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: Fix loop label syntax error with ansible-core 2.15  https://review.opendev.org/c/openstack/kolla-ansible/+/88641220:23
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: ansible: bump min version to 2.14 and max to 2.15  https://review.opendev.org/c/openstack/kolla-ansible/+/88722720:23
mnasiadkaok then, enough for today ;)20:25
opendevreviewMichal Nasiadka proposed openstack/kolla master: cadvisor: bump version to 0.47.2  https://review.opendev.org/c/openstack/kolla/+/88723220:41
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134020:53
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: haproxy: support single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/82339523:44
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123923:44

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!