| opendevreview | Merged openstack/kolla master: Revert "neutron-mlnx-agent: pin pyzmq at 21.*" https://review.opendev.org/c/openstack/kolla/+/882051 | 03:14 |
|---|---|---|
| opendevreview | Dr. Jens Harbott proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887020 | 05:45 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887020 | 06:18 |
| mnasiadka | frickler: thanks for fixing linter, but I missed editing the keepalived.conf template as well ;-) | 06:21 |
| frickler | mnasiadka: ah, ok, I was thinking it was fine if no service check is actually made, but yes, we can drop it then altogether | 06:48 |
| frickler | I'm still thinking a reno and a bug reference would be good for backporting | 06:48 |
| mnasiadka | frickler: yup, let me create a bug | 06:49 |
| mnasiadka | frickler: just thinking if we shouldn't set keepalived_track_script: "{{ groups['loadbalancer'] | length > 1 }}" - this way it would also help operators (although probably it doesn't happen that often to have a single node and run deploy --tags keystone just after upgrade) | 06:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887020 | 07:01 |
| mnasiadka | frickler: done | 07:01 |
| opendevreview | Adam Stackhouse proposed openstack/kolla-ansible master: Adding mariadb_port to wsrep sync status so alterative ports can be used https://review.opendev.org/c/openstack/kolla-ansible/+/886581 | 07:31 |
| frickler | mnasiadka: checking for group size may create issues when upgrading an initial deployment with more nodes or if people have a two-LB setup with one node temporarily down, so I'd prefer the explicit solution | 08:24 |
| frickler | another possibility would be to not do this at all and instead add a check into the CI waiting for keepalived to be up again before proceeding with the upgrade. that might even be helpful outside of CI | 08:27 |
| mnasiadka | well, we run deploy --tags keystone in CI after the upgrade - that kind of check would need to be in prechecks or site.yml | 08:31 |
| mnasiadka | or we would need to have that check in https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/tasks/bootstrap.yml | 08:33 |
| mnasiadka | so ideally create a service-db-register role and import it in all other roles | 08:33 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887020 | 08:58 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887020 | 08:58 |
| SvenKieske | frickler: would it be possible to even skip the complete keepalived deployment in single node installs? :thinking: | 09:06 |
| mnasiadka | it's possible to skip haproxy on single node, we recommend that on dev installs | 09:07 |
| mnasiadka | but moving from no keepalived to multiple controllers and keepalived by default? I have the same concerns as frickler | 09:07 |
| mnasiadka | anyway, let's go with what we have now, no reason to waste more time on a CI-only issue :) | 09:11 |
| SvenKieske | well, given the state of CI i wished we would waste more time on CI only issues ;P | 09:15 |
| mnasiadka | it's getting better and better as you can see ;) | 09:16 |
| SvenKieske | yes! :) | 09:16 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9 https://review.opendev.org/c/openstack/kolla-ansible/+/884488 | 09:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9 https://review.opendev.org/c/openstack/kolla-ansible/+/884488 | 09:49 |
| mnasiadka | and that one (once it's green) should get us to CI greenland | 09:52 |
| hrw | o, looks like bookworm requirements are merged. now time for bookworm patches | 10:55 |
| mnasiadka | frickler: https://review.opendev.org/c/openstack/kolla-ansible/+/887020 - it's passing, debian-upgrade passed multiple times during rework - so we should be fine | 10:59 |
| mnasiadka | bbezak: ^^ | 10:59 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 11:27 |
| mnasiadka | kevko: I see you're brave :) | 11:35 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9 https://review.opendev.org/c/openstack/kolla-ansible/+/884488 | 11:46 |
| kevko | mnasiadka: haha i am :D | 12:21 |
| kevko | and found some issues :D | 12:21 |
| mnasiadka | mgoddard mnasiadka hrw bbezak frickler kevko SvenKieske mmalchuk gkoper - meeting in 6 | 12:54 |
| SvenKieske | where's my 10 minute warning? :( | 12:55 |
| mnasiadka | there's 6 minutes, I'm unfortunately not a bot :) | 12:55 |
| SvenKieske | damn :D | 12:57 |
| mnasiadka | #startmeeting kolla | 13:00 |
| opendevmeet | Meeting started Wed Jun 28 13:00:13 2023 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 13:00 |
| mnasiadka | #topic rollcall | 13:00 |
| mnasiadka | o/ | 13:00 |
| gkoper | o/ | 13:00 |
| mgoddard | \o | 13:00 |
| frickler | o/ | 13:01 |
| hrw | \~o~\ | 13:01 |
| mattcrees | o/ | 13:01 |
| SvenKieske | o/ | 13:02 |
| darmach | \o | 13:02 |
| mnasiadka | #topic agenda | 13:03 |
| mnasiadka | * Announcements | 13:03 |
| mnasiadka | * Review action items from the last meeting | 13:03 |
| mnasiadka | * CI status | 13:03 |
| mnasiadka | * Release tasks | 13:03 |
| mnasiadka | * Regular stable releases (first meeting in a month) | 13:03 |
| mnasiadka | * Current cycle planning | 13:03 |
| mnasiadka | * Additional agenda (from whiteboard) | 13:03 |
| mnasiadka | * Open discussion | 13:03 |
| mnasiadka | #topic Announcements | 13:03 |
| mnasiadka | As discussed previously - I'm off for the next three weeks | 13:03 |
| mnasiadka | I cancelled weekly meetings for that time | 13:03 |
| mnasiadka | #topic Review action items from the last meeting | 13:04 |
| mnasiadka | mnasiadka to raise EOL for stable/wallaby and send email to the ML - done | 13:04 |
| mnasiadka | #topic CI status | 13:04 |
| mnasiadka | I guess green on average | 13:04 |
| mnasiadka | fixes ongoing here and there | 13:04 |
| mnasiadka | #topic Release tasks | 13:04 |
| mnasiadka | I think we restored using master on the master branch | 13:05 |
| mnasiadka | So no other tasks planned in the calendar | 13:05 |
| mnasiadka | #topic Regular stable releases | 13:05 |
| mnasiadka | Do we want to post stable releases next week? | 13:06 |
| frickler | or maybe now, lest we forget? | 13:06 |
| hrw | was there wallaby release before eol? or do we not do such? | 13:06 |
| SvenKieske | that would be good imho, there where some users here asking for bugs that where closed in git, but not on pypi releases | 13:06 |
| mnasiadka | we can't do releases when branch is EM | 13:06 |
| frickler | wallaby was long em, no releases | 13:06 |
| hrw | ok | 13:06 |
| mnasiadka | frickler: willing to raise a patch or do we need another volunteer? | 13:07 |
| hrw | if erlang/rmq got sorted out on stable/* then maybe release? | 13:07 |
| frickler | I can do that, just need to know what to wait for. erlang rmq seems sensible | 13:07 |
| mnasiadka | yeah, it got sorted-ish, we need to sort out Ubuntu/aarch64 it seems | 13:07 |
| SvenKieske | I _guess_ rmq should be stable, but I just moved a bug about crashing rmq to kolla-ansible: | 13:08 |
| SvenKieske | https://bugs.launchpad.net/bugs/2023668 | 13:08 |
| mnasiadka | isn't 3.9 EOL? | 13:08 |
| SvenKieske | at least it was claimed the installation was done according to the kolla docs, so I would be interested in finding out why this crashes on yoga | 13:09 |
| mnasiadka | https://www.rabbitmq.com/versions.html | 13:09 |
| frickler | didn't we have this last week already? they're using binary build it seems | 13:09 |
| SvenKieske | I think I found 3.9 referenced in some docs/files on an older branch of us? | 13:09 |
| mnasiadka | binary, uh | 13:09 |
| mnasiadka | anyway, bug is there, we can have a look | 13:10 |
| mnasiadka | #topic Current cycle planning | 13:10 |
| mnasiadka | I see kevko has revived Let's Encrypt - so there's a chance we'll merge it finally | 13:10 |
| hrw | Debian/bookworm? | 13:10 |
| mnasiadka | hrw: stage is yours for Bookworm ;) | 13:10 |
| kevko | just started to debug why it is not working out of the box on my env :P | 13:11 |
| hrw | mnasiadka: it is https://review.opendev.org/q/topic:for-debian-bookworm-upgrade as it was before | 13:11 |
| mnasiadka | that's kolla side | 13:11 |
| mnasiadka | what about nodepool instances and testing on those? | 13:11 |
| hrw | mnasiadka: building we can do on anything | 13:12 |
| frickler | still need to look into building nodepool image | 13:12 |
| mnasiadka | hrw: we can, if you're fine we drop Debian in kolla-ansible - let's just build images :) | 13:13 |
| opendevreview | Merged openstack/kolla-ansible master: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887020 | 13:13 |
| mnasiadka | Given we have vacation season ahead - I'm pretty sure we're going be last minute merging kolla-ansible support | 13:13 |
| mnasiadka | Anyway, let's not forget about building nodepool image, I think it's fine if we don't have a mirror and download over the internet (just like in Rocky case) | 13:14 |
| hrw | mnasiadka: patches can be hold with 2 CR+2, V+1 and without W+1 | 13:14 |
| SvenKieske | the support matrix at https://docs.openstack.org/kolla/xena/support_matrix doesn't indicate btw that we don't support binary builds of rabbitmq containers, should we change that? | 13:14 |
| frickler | mirror is done | 13:15 |
| mnasiadka | ah, mirror is done, fantastic | 13:15 |
| hrw | mnasiadka: but without reviews what is a point of doing anything further? | 13:15 |
| SvenKieske | sorry, maybe defer my point to "open discussion" | 13:15 |
| frickler | I'm reviewing those patches | 13:15 |
| mnasiadka | hrw: I'll have a look later today | 13:15 |
| frickler | also maybe we want RP+1 for those? or +2? | 13:15 |
| SvenKieske | I did reviews there as well, there are open discussion points, without answer, so I didn't look again yet | 13:16 |
| mnasiadka | hrw: it would also help if those would be passing Zuul ;) | 13:17 |
| hrw | mnasiadka: sure | 13:17 |
| mnasiadka | so run a recheck, let's see fresh results, and try to get somewhere :) | 13:18 |
| mnasiadka | Ok, podman guys are not here today - but I've seen some updates | 13:18 |
| hrw | INFO:kolla.common.utils.openstack-base: The user requested cachetools | 13:18 |
| hrw | INFO:kolla.common.utils.openstack-base: The user requested (constraint) cachetools===5.3.0 | 13:18 |
| hrw | that kind of bugs are "not my fault" ;D | 13:18 |
| SvenKieske | I notice rabbitmq 3.10 in bookworm is only supported upstream until end of july, this year, is the backporting in debian good? :) | 13:18 |
| mnasiadka | hrw: if it's ord rax nodepool provider - they have some weird networking issues towards pypi | 13:18 |
| kevko | btw, yoga is unbuildable on localhost | 13:19 |
| mnasiadka | kevko: I'm still amazed how it works in CI | 13:19 |
| kevko | but passing in CI ...do you know why ? | 13:19 |
| hrw | SvenKieske: it can/will move to 3rdparty repo | 13:19 |
| mnasiadka | kevko: even the experimental job that doesn't use infra wheel is failing? | 13:19 |
| kevko | horizon is really unbuildable on localhost ..just try it ...i am fixing it in my downstream repo by limiting setuptools | 13:19 |
| kevko | i really don't know :D ... just try it on localhost | 13:20 |
| mnasiadka | Is anybody able to help kevko - or we just merge the backport? | 13:20 |
| SvenKieske | kevko: mhm I know some ppl who will be very sad about it | 13:20 |
| SvenKieske | kevko: do you have any patch I can look at? | 13:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2023.1: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887069 | 13:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/zed: loadbalancer: Add option to not define track script https://review.opendev.org/c/openstack/kolla-ansible/+/887170 | 13:20 |
| mnasiadka | SvenKieske: https://review.opendev.org/c/openstack/kolla/+/873913 | 13:21 |
| kevko | SvenKieske: it's abandoned for now i think | 13:21 |
| kevko | mnasiadka thanks, yeah that one | 13:22 |
| SvenKieske | ty | 13:22 |
| mnasiadka | not abandoned, I tried building locally long time ago - and it also failed for me | 13:22 |
| mnasiadka | so the mystery is - how does it work in zuul | 13:22 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/yoga: Pin setuptools=67.2.* https://review.opendev.org/c/openstack/kolla/+/873913 | 13:22 |
| mnasiadka | ok then, let's move on | 13:23 |
| mnasiadka | #topic Additional agenda (from whiteboard) | 13:23 |
| mnasiadka | gkoper: Infloblox designate driver (mdns running as root) | 13:23 |
| mnasiadka | gkoper: stage is yours | 13:23 |
| gkoper | o/ | 13:24 |
| darmach | (stage fright phase) | 13:24 |
| gkoper | Infoblox does not support changing the port used to communicate with MDNS to request zone transfers(AXFRs) | 13:24 |
| gkoper | Therefore MDNS containers need to be bound to port 53. Kolla built designate containers are starting with user designate, so are unable to bound the service on privileged port 53 | 13:24 |
| gkoper | Dirty workaround is to locally build containers to start with root user | 13:24 |
| gkoper | (This poses security risk) | 13:24 |
| gkoper | Another approach is to use CAP_NET_BIND_SERVICE to provide the capability for user designate to bind a service to a privileged port (0-1024) [Testing now ] | 13:24 |
| mnasiadka | I think we're doing something similar to something related to prometheus - to be able to run ping | 13:24 |
| gkoper | We also found some issues while templating pools.yml | 13:24 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm' https://review.opendev.org/c/openstack/kolla/+/886088 | 13:25 |
| mnasiadka | gkoper: one thing at a time | 13:25 |
| frickler | building your own containers is the documented solution for this | 13:25 |
| mnasiadka | frickler: I know you -2d a patch to run as root - but that's logical - do you see any problems with using CAP_NET_BIND_SERVICE? | 13:26 |
| mnasiadka | frickler: we build downstream one set of container images, wouldn't want to have a separate one for infoblox, and separate mdns for bind ;-) | 13:26 |
| SvenKieske | well it provides privileges that most deployments don't need | 13:26 |
| frickler | could we make that optional? I'd have to check what we do for prometheus | 13:26 |
| mnasiadka | in prometheus we setcap for blackbox exporter binary | 13:26 |
| * SvenKieske is also thinking about how to make this conditional | 13:27 | |
| gkoper | frickler: https://github.com/openstack/kolla/blob/70f74eb64101431e23d56c6a7df96d7aab37ce2f/docker/prometheus/prometheus-blackbox-exporter/Dockerfile.j2#L32 | 13:27 |
| mnasiadka | it still runs as unprivileged user | 13:27 |
| mnasiadka | which should be fine | 13:27 |
| darmach | As for the pools.yml templating we ran into wrong templating of ns_records (list was used as sigle record resulting in string[fqdn]. - easyfix) and nameservers templated using dqdn resulting in designate-manage failing with: "(proper fqdn) is not IP address or host name" | 13:27 |
| mnasiadka | darmach: one thing at a time | 13:27 |
| darmach | ^ nothing that can't be fixed when we are done with port 53 bind | 13:27 |
| mnasiadka | let's get some agreement on the root stuff | 13:28 |
| SvenKieske | so I think it would be okay to setcap this as well, should be documented of course. | 13:28 |
| * SvenKieske wondering if this really works with podman | 13:28 | |
| mnasiadka | it would only add privileges to bind a low port, which most probably is not a security issue | 13:29 |
| frickler | it is | 13:29 |
| mnasiadka | ok then, can we make it optional for greater good? | 13:29 |
| frickler | any privilege that is not needed is a security issue | 13:29 |
| mnasiadka | Then I'm pretty sure majority of our containers is insecure in those terms ;) | 13:29 |
| SvenKieske | well, even "needed" ones are :) | 13:30 |
| frickler | I'm not sure we want to do much special casing for a weird non-free backend | 13:30 |
| gkoper | and running as root is a less one ? | 13:30 |
| SvenKieske | if I'm not mistaken the default is to run all containers in the host network namespace, correct? | 13:30 |
| mnasiadka | frickler: we support it out of the box with some kolla-ansible variables, so it would make sense to make it easier and better | 13:30 |
| frickler | mnasiadka: or drop support for it completely? | 13:31 |
| darmach | I can agree with weird non-free backend - there are customers out there in the wild with infoblox deployed though... | 13:31 |
| mnasiadka | frickler: as you can see - SHPC needs that for it's customers | 13:31 |
| mnasiadka | so as long as there's somebody that wants to maintain it - I don't see a reason to drop it | 13:31 |
| frickler | so ... why is building a custom container locally not feasible? | 13:32 |
| SvenKieske | can we talk about the actual issue, please? | 13:32 |
| frickler | can we then add a special container build upstream | 13:32 |
| frickler | and deploy a different container in the infoblox case? | 13:33 |
| mnasiadka | SvenKieske: the issue is there's an infoblox backend in designate, which is not tested in designate CI by the way, and requires designate-mdns to run on port 53. | 13:33 |
| SvenKieske | i know... | 13:33 |
| mnasiadka | the other backend, which is bind (and probably powerdns) does not require that | 13:33 |
| SvenKieske | what indeed might be a problem, if my assumption above is correct regarding the host network namespace, is that other services might already be bound to port 53 | 13:34 |
| mnasiadka | that's a deployment specific, if somebody wants infoblox, then he needs to deal with that in his env ;) | 13:34 |
| mnasiadka | currently we direct users to build it on their own, which is most probably fine - but still requires root user, so we basically direct them to do insecure installation of designate | 13:35 |
| SvenKieske | mnasiadka: well not really, if per default, e.g. systemd-resolved is listening on localhost:53 and you deploy designate with k-a and infoblox and it breaks the default local resolver I would indeed refuse to merge such patches | 13:35 |
| frickler | so if we amend the doc to use CAP_NET_BIND_SERVICE that is enough for you? | 13:35 |
| mnasiadka | not really, since we use one source of downstream images for N clients | 13:36 |
| SvenKieske | it doesn't make much sense to patch something, if the patch doesn't work in major deployment scenarios, so that should at least be tested and be guaranteed to work | 13:36 |
| frickler | so ... extra container? designate-mdns-insecure? | 13:37 |
| frickler | just joking on the name of course | 13:37 |
| mnasiadka | extra container - why not, and some logic in kolla-ansible to use it when infoblox is enabled | 13:38 |
| frickler | ack | 13:38 |
| mnasiadka | or the same container and some extend_start logic to do setcap | 13:38 |
| SvenKieske | under the provision that this works at all (I have doubts): couldn't we introduce a conditional, that, _if_ infoblox is enabled the container gets (re)started with cap_net_bind_service? | 13:38 |
| mnasiadka | just let's agree on one of those | 13:38 |
| frickler | but that would modify the container at runtime? | 13:38 |
| SvenKieske | at restart time ;) | 13:38 |
| mnasiadka | it would, but is that something new? we remove default http certs at start time ;) | 13:39 |
| mnasiadka | we mangle opensearch-dashboards plugins at start time | 13:39 |
| * SvenKieske thinking about which is more pain, a second container or to modify the existing one | 13:39 | |
| darmach | "or the same container and some extend_start logic to do setcap" < I quite like that, we can give it a try and test how it works | 13:39 |
| SvenKieske | but it really should only setcap on the infoblox conditional | 13:40 |
| SvenKieske | sounds mostly fine to me, and imho less maintenanceburden than a whopping complete container, no? | 13:40 |
| mnasiadka | SvenKieske: second container with one extra layer that runs one command sounds funny, but that's also nothing new in Kolla land | 13:40 |
| frickler | so if that works, I think that should be acceptable, then | 13:40 |
| mnasiadka | ok then | 13:40 |
| darmach | Great | 13:40 |
| mnasiadka | no new images, optional setcap in the existing one based on an ENV variable and we're fine | 13:41 |
| mnasiadka | great | 13:41 |
| mnasiadka | darmach: the other issue seems like pools.yaml template misconfiguration, just propose a new patch and let's discuss it there? | 13:41 |
| darmach | Yes, going to do that. It's nothing complicated. | 13:42 |
| frickler | there is https://review.opendev.org/c/openstack/kolla-ansible/+/878270 already, which I need to get back to | 13:42 |
| frickler | please check if that covers your issue, too | 13:42 |
| mnasiadka | darmach: can you also have a look at ^^? | 13:42 |
| SvenKieske | yeah, that would be good :) | 13:42 |
| * SvenKieske praying for more designate maintainers | 13:43 | |
| mnasiadka | from my perspective that patch needs to be backwards compatible, and now it's not. | 13:43 |
| mnasiadka | but let's discuss in the patch itself | 13:43 |
| darmach | Will do, looping in that template was exactly what I was thinking about. | 13:43 |
| frickler | maybe it needs to be split into some parts, too | 13:43 |
| darmach | Maybe we could split-out the pools.yml part, and I could take a stab at it. | 13:44 |
| mnasiadka | frickler: might be, but we need to support designate_ns_record, or at least have prechecks saying you need to rework your config | 13:44 |
| mnasiadka | ok then, let's move on | 13:45 |
| gkoper | we have an edge case , that needs to support more than 1 ns_group to be updated on Infoblox (WAN and LAN) | 13:45 |
| mnasiadka | #topic Open discussion | 13:45 |
| mnasiadka | gkoper: you can override pools.yaml just as other files in kolla-ansible | 13:46 |
| mnasiadka | let's just cover the usual case, not the oddities | 13:46 |
| gkoper | sure, i think Jakub already had an idea howto template that. | 13:46 |
| gkoper | lets move on. | 13:46 |
| darmach | We could use the loop in junja as @frickler did - to create separate pools | 13:46 |
| SvenKieske | regarding the crashing rmq 3.9 release I just asked in the bug report to provide reproducer steps as I'm 90% sure we don't mention this release anywhere. | 13:47 |
| SvenKieske | I also advised to maybe not use binary builds | 13:47 |
| mnasiadka | SvenKieske: we deprecated it in Yoga and removed in Zed, if they like using deprecated content - fine by me ;) | 13:48 |
| SvenKieske | maybe we should make our support matrix more clear? or did we already? I think we did..half | 13:49 |
| SvenKieske | we just dropped the distinction binary/source from our image support matrix, maybe a big disclaimer shouting "we do not support binary images anymore" would be good? | 13:50 |
| SvenKieske | maybe not worth the effort, I guess there where maybe not even ten users with binary problems, I think. | 13:50 |
| SvenKieske | that bothered to ask, at least :D | 13:50 |
| mnasiadka | yeah well, binary builds always had its own issues | 13:53 |
| mnasiadka | ok, anything else? | 13:54 |
| mnasiadka | thanks for coming - see you again 26th July | 13:56 |
| mnasiadka | in case of an urgent review request - please bug bbezak and mgoddard ;-) | 13:56 |
| mnasiadka | #endmeeting | 13:56 |
| opendevmeet | Meeting ended Wed Jun 28 13:56:19 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:56 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-06-28-13.00.html | 13:56 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-06-28-13.00.txt | 13:56 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-06-28-13.00.log.html | 13:56 |
| frickler | thx mnasiadka enjoy your time off | 13:56 |
| SvenKieske | thx mnasiadka, happy holidays! | 13:57 |
| darmach | Have fun! Thanks! | 13:57 |
| frickler | do we want mariadb bump before bookworm or after? currently they conflict https://review.opendev.org/c/openstack/kolla/+/882924 | 15:58 |
| SvenKieske | first come, first serve? :) imho the lts patch is much older, no? | 16:01 |
| SvenKieske | it seems we have a lot of temp CI issues with pypi mirror connectivity, would it maybe possible to host our own partly mirror on #opendev e.g. via https://pypi.org/project/python-pypi-mirror/ ? what do you think? | 16:02 |
| SvenKieske | a pull through cache like gitlab offers would be nice.. I don't think gitea has something like that builtin? | 16:03 |
| SvenKieske | there's a feature request at least: https://github.com/go-gitea/gitea/issues/21223 | 16:05 |
| SvenKieske | e.g. here: https://review.opendev.org/c/openstack/kolla/+/882924/6#message-aad622078f973cee5ff7efc7245b71e08b2e84e1 | 16:08 |
| SvenKieske | wrong channel -.- | 16:08 |
| hrw | frickler: before | 16:11 |
| hrw | frickler: bookworm needs stuff around: nodes, k-a support etc. while mariadb update can be done on what we have now | 16:12 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm' https://review.opendev.org/c/openstack/kolla/+/886088 | 16:17 |
| hrw | frickler: thanks for comments. | 16:18 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: base: install adduser on Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/881948 | 16:21 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: base: Debian 12 uses new APT sources file https://review.opendev.org/c/openstack/kolla/+/881950 | 16:21 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm' https://review.opendev.org/c/openstack/kolla/+/886088 | 16:21 |
| hrw | had to rebase whole set | 16:21 |
| frickler | hrw: actually I'm not sure we can downgrade rmq to 3.10 if we are running 3.11 now on bullseye? that wouldn't work during an upgrade, would it? | 16:25 |
| hrw | frickler: I will look at having 3.11 | 16:26 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm' https://review.opendev.org/c/openstack/kolla/+/886088 | 16:27 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm' https://review.opendev.org/c/openstack/kolla/+/886088 | 16:34 |
| hrw | frickler: rmq 3.11 | 16:34 |
| SvenKieske | fyi: regarding pypi CI issues I reraised https://github.com/pypi/warehouse/issues/8568 again, maybe we're having more luck this time (this got closed in 2020).. | 16:35 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: build: distutils.StrictVersion() is deprecated https://review.opendev.org/c/openstack/kolla/+/886741 | 16:40 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 16:41 |
| mnasiadka | we probably should move to rmq 3.12 in Bobcat, given how fast they EOL RMQ | 16:51 |
| hrw | 3.11 is just to the end of 2023 | 16:53 |
| mnasiadka | maybe we also need to think how to do upgrades | 16:53 |
| mnasiadka | we have 3.9 in Yoga which is EOL | 16:53 |
| mnasiadka | and we already moved from 3.8 to 3.9 in Yoga | 16:54 |
| mnasiadka | upgrades inside a stable release I mean | 16:54 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Adds VGPU support https://review.opendev.org/c/openstack/kayobe/+/887200 | 16:54 |
| mnasiadka | or maybe it's a way to tell users nicely to move forward :) | 16:54 |
| hrw | mnasiadka: I suggest to stay with one erlang version per release and update rabbitmq until erlang is too old | 16:55 |
| hrw | and then mark branch as 'existing systems only' | 16:56 |
| mnasiadka | either way, we need to move to 3.12 | 16:56 |
| mnasiadka | and bump ansible to 2.15 | 16:56 |
| mnasiadka | sooner than last time | 16:56 |
| hrw | mnasiadka: propose patches | 16:56 |
| mnasiadka | in August most probably, going for three weeks vacation ;-) | 16:56 |
| mnasiadka | although RMQ might be easy | 16:57 |
| mnasiadka | but first we should probably fix ubuntu/aarch64 rmq 3.11 so we can backport that easily | 16:57 |
| hrw | mnasiadka: vacations... something I hope for | 16:57 |
| SvenKieske | my vacation will be moving my household to a different city roughly at the beginning of august, wish me luck :) | 16:58 |
| hrw | SvenKieske: have fun! | 17:02 |
| hrw | SvenKieske: lot of carton boxes and packing tape. Then friends and moving company to pack all that. | 17:03 |
| hrw | I moved ~2y ago. But just 2km only | 17:03 |
| * hrw out | 17:05 | |
| mnasiadka | I moved 150 meters 8 years ago :) | 17:11 |
| mnasiadka | well, counting 4th floor down and up to 3rd floor it was a couple more meters ;-) | 17:12 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: haproxy: support single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/823395 | 17:53 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 17:53 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 17:56 |
| mnasiadka | kevko: interested in merging that as well? Finally not only me :) | 17:57 |
| guesswhat[m] | Where is the path to store crt, metadata and mapping for keystone federation host https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L1268 ? | 17:57 |
| guesswhat[m] | I guess its not possible to use kolla config directory... | 17:57 |
| kevko | mnasiadka: well, letsencrypt working but need some love | 18:01 |
| kevko | mnasiadka: and for example in my deployment i am using api.master.ultimum.cloud and horizon.master.ultimum.cloud ...and it created only api.master.ultimum.cloud cert as it is defined in globals as fqdn .. | 18:02 |
| kevko | i remember that you had a patch to use fqdn per service ..it is little bit related ...so i wanted to see how it is working toherher | 18:03 |
| kevko | worse is on letsencrypt that few times i tried someting and i am blocked from letsencrypt api :D | 18:03 |
| kevko | mnasiadka: i think we should ask for *.domain if kolla fqdn is set for example api.domain | 18:04 |
| kevko | mnasiadka what do you think ? | 18:04 |
| mnasiadka | kevko: wildcards are only supported in dns plugins, I don't think http one supports that | 18:04 |
| kevko | mnasiadka: really, i think i used to use wildcard also with http | 18:06 |
| kevko | but yeah, it is in faq on letsencrypt website | 18:07 |
| kevko | okay, so we should have some list of fqdns then .. | 18:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: haproxy: support single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/823395 | 18:14 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 18:14 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 18:23 |
| guesswhat[m] | Anyone using Keystone + OIDC, its quite confusing.. | 18:33 |
| kevko | was tried :P | 18:34 |
| guesswhat[m] | didnt work ? | 18:37 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 19:12 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: rabbitmq: Fix repo for ubuntu aarch64 https://review.opendev.org/c/openstack/kolla/+/887223 | 19:45 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: rabbitmq: bump version to 3.12 https://review.opendev.org/c/openstack/kolla/+/887225 | 19:53 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: ansible: bump min version to 2.14 and max to 2.15 https://review.opendev.org/c/openstack/kolla-ansible/+/887227 | 19:58 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 20:13 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Fix loop label syntax error with ansible-core 2.15 https://review.opendev.org/c/openstack/kolla-ansible/+/886412 | 20:23 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: ansible: bump min version to 2.14 and max to 2.15 https://review.opendev.org/c/openstack/kolla-ansible/+/887227 | 20:23 |
| mnasiadka | ok then, enough for today ;) | 20:25 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: cadvisor: bump version to 0.47.2 https://review.opendev.org/c/openstack/kolla/+/887232 | 20:41 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 20:53 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: haproxy: support single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/823395 | 23:44 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 23:44 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!