| opendevreview | Petr Slavchenkov proposed openstack/kolla-ansible master: Fix proxysql get mariadb_backup_database_password https://review.opendev.org/c/openstack/kolla-ansible/+/900979 | 07:58 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Run Tempest in test-core-openstack https://review.opendev.org/c/openstack/kolla-ansible/+/878826 | 08:42 |
| opendevreview | Petr Slavchenkov proposed openstack/kolla-ansible master: Fix proxysql get mariadb_backup_database_password https://review.opendev.org/c/openstack/kolla-ansible/+/900979 | 08:55 |
| opendevreview | Petr Slavchenkov proposed openstack/kolla-ansible master: Fix proxysql get mariadb_backup_database_password https://review.opendev.org/c/openstack/kolla-ansible/+/900979 | 08:59 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: CI: add podman jobs https://review.opendev.org/c/openstack/ansible-collection-kolla/+/899468 | 12:00 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: CI: add podman jobs https://review.opendev.org/c/openstack/ansible-collection-kolla/+/899468 | 12:01 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Run Tempest in test-core-openstack https://review.opendev.org/c/openstack/kolla-ansible/+/878826 | 12:10 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Run Tempest in test-core-openstack https://review.opendev.org/c/openstack/kolla-ansible/+/878826 | 12:10 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Run Tempest in test-core-openstack https://review.opendev.org/c/openstack/kolla-ansible/+/878826 | 12:31 |
| mnasiadka | frickler: is TC thinking what to do with Masakari, or are we just waiting for the next cycle and lack of PTL? | 12:41 |
| frickler | mnasiadka: I'm not sure, gmann is currently handling all the pending PTL things. if it is currently working for kolla as far as CI coverage goes, I'm fine with deferring to next cycle | 12:50 |
| mnasiadka | Well, bobcat is failing, but since there are users - I'm not going to raise the issue now :) | 12:50 |
| mnasiadka | so let's wait if the situation will resolve | 12:51 |
| mnasiadka | https://review.opendev.org/q/project:openstack/masakari+is:open+branch:stable/2023.2 | 12:51 |
| mnasiadka | this looks just fantastic | 12:51 |
| frickler | well since it is failing in kolla since over a month, we could add a deprecation warning in bobcat, that would make it easier to drop next cycle and also alarm possible consumers | 12:53 |
| frickler | "because CI jobs are failing and no contributors are around to fix it" something like that? | 12:53 |
| frickler | https://zuul.opendev.org/t/openstack/builds?job_name=kolla-ansible-ubuntu-masakari&project=openstack/kolla-ansible | 12:54 |
| SvenKieske | seems reasonable | 12:54 |
| SvenKieske | btw should we backport https://review.opendev.org/c/openstack/kolla-ansible/+/881707 ? | 12:54 |
| frickler | then if someone cares about the support in kolla, they have a chance to show up | 12:55 |
| opendevreview | Martin Hiner proposed openstack/kolla-ansible master: Rename kolla_docker to kolla_container https://review.opendev.org/c/openstack/kolla-ansible/+/900412 | 12:55 |
| mnasiadka | SvenKieske: seems like it would make sense backporting back to the first release when we had systemd | 12:56 |
| frickler | ha, jinx, I was just about to ask mhiner if they are around to update ^^ :-) | 12:56 |
| mnasiadka | which is Antelope I guess? | 12:56 |
| mnasiadka | yes, antelope | 12:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2023.1: Introduce oneshot docker_restart_policy https://review.opendev.org/c/openstack/kolla-ansible/+/900985 | 12:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2023.1: Introduce oneshot docker_restart_policy https://review.opendev.org/c/openstack/kolla-ansible/+/900985 | 12:57 |
| frickler | hmm, where do the conflicts come from, do we need more backports? | 12:57 |
| mnasiadka | no podman in antelope | 12:58 |
| mnasiadka | so basically two files do not needed ;-) | 12:58 |
| mnasiadka | *not needed | 12:58 |
| frickler | ah, ack. you also need to unwip the patch then | 12:59 |
| mnasiadka | already done | 12:59 |
| frickler | hmm, seems gerrit is slow with updating things, then. thx | 13:00 |
| opendevreview | Merged openstack/kolla-ansible master: Adds condition to handle QoS extension in sriov_agent.ini template https://review.opendev.org/c/openstack/kolla-ansible/+/899629 | 13:14 |
| opendevreview | Merged openstack/ansible-collection-kolla master: podman: Support registry mirror https://review.opendev.org/c/openstack/ansible-collection-kolla/+/899657 | 13:14 |
| mnasiadka | https://review.opendev.org/q/project:openstack/ansible-collection-kolla+is:open+label:Review-Priority - these two would like reviews, so we could standardise podman/docker settings in kolla-ansible CI | 13:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2023.1: Adds condition to handle QoS extension in sriov_agent.ini template https://review.opendev.org/c/openstack/kolla-ansible/+/900986 | 13:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/zed: Adds condition to handle QoS extension in sriov_agent.ini template https://review.opendev.org/c/openstack/kolla-ansible/+/900987 | 13:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/yoga: Adds condition to handle QoS extension in sriov_agent.ini template https://review.opendev.org/c/openstack/kolla-ansible/+/900988 | 13:17 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: CI: add podman jobs https://review.opendev.org/c/openstack/ansible-collection-kolla/+/899468 | 13:40 |
| opendevreview | Merged openstack/kayobe master: ipa-images: use changed openstack.cloud output https://review.opendev.org/c/openstack/kayobe/+/900753 | 13:49 |
| mnasiadka | mgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial - meeting in 8 minutes | 13:52 |
| mnasiadka | #startmeeting kolla | 14:01 |
| opendevmeet | Meeting started Wed Nov 15 14:01:24 2023 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:01 |
| mnasiadka | #topic rollcall | 14:01 |
| mnasiadka | o/ | 14:01 |
| jovial[m] | 0/ | 14:01 |
| mmalchuk | o/ | 14:01 |
| jangutter | o/ | 14:01 |
| mhiner | o/ | 14:01 |
| frickler | \o | 14:02 |
| kevko | \o/ | 14:03 |
| mnasiadka | #topic Agenda | 14:03 |
| mnasiadka | * CI status | 14:03 |
| mnasiadka | * Release tasks | 14:03 |
| mnasiadka | * Regular stable releases (first meeting in a month) | 14:03 |
| mnasiadka | * Current cycle planning | 14:03 |
| mnasiadka | * Additional agenda (from whiteboard) | 14:03 |
| mnasiadka | * Open discussion | 14:03 |
| mnasiadka | #topic CI Status | 14:03 |
| mnasiadka | Updated all CI statuses, marked Masakari as broken | 14:03 |
| jovial[m] | Did the Ironic job get fixed in the end? | 14:04 |
| mnasiadka | frickler: just as we discussed - do we want to mark Masakari as deprecated and fire off the mail to openstack-ml? | 14:04 |
| SvenKieske | o/ | 14:04 |
| mnasiadka | jovial[m]: yes, by https://review.opendev.org/c/openstack/kolla-ansible/+/900985 (well, the master patch, not the backport) | 14:05 |
| frickler | mnasiadka: I'd say yes, I can create a patch if you take care of the mail in return | 14:05 |
| mnasiadka | frickler: we have a deal | 14:05 |
| mnasiadka | ok, let's continue | 14:06 |
| mnasiadka | #topic Release tasks | 14:06 |
| mnasiadka | I've marked most of the patches we should try to squeeze in Bobcat as RP+1 | 14:07 |
| mnasiadka | Please try to review them, ask those that are missing - but no more major features please | 14:07 |
| mnasiadka | add those that are missing | 14:08 |
| mnasiadka | not ask | 14:08 |
| SvenKieske | would quorum queues be considered a major feature? :) | 14:08 |
| frickler | let's get the cat out of the house | 14:08 |
| frickler | good question. "maybe"? | 14:09 |
| mnasiadka | I think we agreed we should do it this cycle | 14:09 |
| mnasiadka | so - SvenKieske - what do you need to get it merged? :) | 14:09 |
| jovial[m] | I also added RP+1 to a few kayobe patches: https://review.opendev.org/q/project:openstack/kayobe+label:Review-Priority%253D1+status:open to try and get them into the release | 14:10 |
| SvenKieske | hopefully just a new CI run and then inspect the logs again, there was a recent fix in oslo.messaging that was missing, I hope that's all at least | 14:11 |
| mnasiadka | SvenKieske: would be good | 14:11 |
| mnasiadka | SvenKieske: can you post a link so we can RP+1? | 14:11 |
| SvenKieske | might still need to disect more of the upstream docs - which are cluttered - and revisit what to do for upgrades, but I guess as a first try it's good to support new deployments/go with downtime? | 14:12 |
| SvenKieske | #link https://review.opendev.org/c/openstack/kolla-ansible/+/898543 | 14:12 |
| SvenKieske | the oslo change got merged: https://review.opendev.org/c/openstack/oslo.messaging/+/889313 | 14:12 |
| jovial[m] | SvenKieske: Are you going to add some docs explaining how to migrate from the old style? | 14:13 |
| frickler | so we still need that in bobcat and a new release https://review.opendev.org/c/openstack/oslo.messaging/+/900891 | 14:13 |
| jovial[m] | sorry, missed your earlier message | 14:13 |
| SvenKieske | frickler: right, but as it's a bugfix I think that should be hopefully backported quickly? maybe too optimistic :) | 14:14 |
| mnasiadka | SvenKieske: we should be fine backporting the upgrade/migration docs later on | 14:15 |
| mnasiadka | if that requires a lot more work and we can't get it out through the door now | 14:15 |
| SvenKieske | ok | 14:16 |
| mnasiadka | ok then let's go forward | 14:16 |
| mnasiadka | #topic Current cycle planning | 14:16 |
| mnasiadka | Current means sort of Bobcat, I've tidied up the Bobcat priorities on the Whiteboard | 14:17 |
| mnasiadka | Will organise the Caracal priorities and send out a PTG summary | 14:17 |
| mnasiadka | #topic Additional agenda | 14:18 |
| mnasiadka | none | 14:18 |
| mnasiadka | #topic Open discussion | 14:18 |
| mnasiadka | so, we've got 40 minutes for open discussion ;) | 14:19 |
| mhiner | I want to ask if you're interested in the migration patch https://review.opendev.org/c/openstack/kolla-ansible/+/836941 | 14:19 |
| SvenKieske | certainly, as there are a lot of docker deployments out there :) | 14:19 |
| mhiner | It's goal is to migrate from Docker to Podman and vice versa within the same version | 14:19 |
| SvenKieske | same version meaning openstack release, or..? | 14:20 |
| mhiner | yes, that | 14:20 |
| SvenKieske | that would be good information to add to the commit message imho :) | 14:20 |
| jovial[m] | mhiner: very nice - 'migrate' as a verb seems a bit generic though | 14:21 |
| mnasiadka | Well, we probably are interested, but I think main case is that there is a CI job that tests that | 14:21 |
| mnasiadka | Second of all, ansible/migrate.yml is too generic, we have ansible/opensearch-migration.yml - so we should have ansible/container-engine-migration.yml and ideally support both directions? ;-) | 14:22 |
| mnasiadka | on other front, shouldn't it be that simple as changing kolla_container_runtime to podman? | 14:22 |
| mnasiadka | ah, volumes migration | 14:23 |
| mnasiadka | ok then, we need a command for the migration surely | 14:23 |
| mhiner | mnasiadka: both directions are supported and yeah ansible/migrate.yml can be renamed | 14:23 |
| jovial[m] | how close is it to working? | 14:24 |
| SvenKieske | ack, nice :) | 14:24 |
| mnasiadka | and then we need CI jobs testing that | 14:24 |
| mnasiadka | don't assume reviewers will go the extra mile and test that on their env | 14:24 |
| mhiner | jovial[m]: I worked on it relatively long time ago but iirc it should be functional | 14:25 |
| mhiner | mnasiadka: am I able to create that CI job? | 14:25 |
| SvenKieske | yeah, might be tricky to test | 14:25 |
| mnasiadka | mhiner: sure, just add it in .zuul.d/ - copy cat the opensearch migration jobs and you'll be fine | 14:26 |
| mhiner | okay, will look into it | 14:26 |
| mnasiadka | sorry, zuul.d (not a hidden one) | 14:26 |
| mnasiadka | if you need help, just reach out :) | 14:26 |
| mhiner | alright, thanks | 14:27 |
| jangutter | mhiner, you don't need special permissions to run and test a zuul job in the CI, and while developing it you can temporarily disable the other jobs till you've got it ironed out. | 14:27 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Add keystone_federation_oidc_additional_options variable for additional options https://review.opendev.org/c/openstack/kolla-ansible/+/892831 | 14:27 |
| SvenKieske | that commit message looks..interesting :D | 14:28 |
| SvenKieske | I heard you like additional options, so I put addtional options in your additional options. | 14:28 |
| mnasiadka | yeah, the author got bored, I'll refresh it | 14:29 |
| mnasiadka | ok, anybody has anything else or should we wrap up? | 14:32 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: keystone: Add support for OIDC options https://review.opendev.org/c/openstack/kolla-ansible/+/892831 | 14:35 |
| mnasiadka | SvenKieske: better? :) | 14:35 |
| jovial[m] | just needs a reno ;-) | 14:36 |
| mnasiadka | makes sense | 14:36 |
| SvenKieske | very nice :) | 14:37 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: keystone: Add support for OIDC options https://review.opendev.org/c/openstack/kolla-ansible/+/892831 | 14:40 |
| mnasiadka | and there's the reno | 14:40 |
| mnasiadka | ok, I guess nothing more | 14:41 |
| mnasiadka | thanks for coming! | 14:41 |
| mnasiadka | #endmeeting | 14:41 |
| opendevmeet | Meeting ended Wed Nov 15 14:41:08 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:41 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-11-15-14.01.html | 14:41 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-11-15-14.01.txt | 14:41 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-11-15-14.01.log.html | 14:41 |
| opendevreview | Merged openstack/kolla-ansible master: CI: Log details about failed containers https://review.opendev.org/c/openstack/kolla-ansible/+/892488 | 14:41 |
| mmalchuk | mnasiadka thanks | 14:43 |
| SvenKieske | mhm thanks | 14:43 |
| SvenKieske | seems gerrit is really slow I don't see the reno | 14:43 |
| SvenKieske | even force reloading the page doesn't help. did notice slow gerrit beginning yesterday in the evening (took ages to load) | 14:44 |
| greatgatsby | Mind if I just ask for quick clarification on masakari being deprecated? So it's going to be removed entirely from future kolla releases? I'd just like to be able to pass this info on to my team. | 14:45 |
| SvenKieske | that depends on masakari: it's currently not well developed afaik, so if that doesn't change it will get dropped I guess? | 14:47 |
| SvenKieske | so if you depend on it that is a good point to get involved upstream :) | 14:47 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: keystone: Add support for OIDC options https://review.opendev.org/c/openstack/kolla-ansible/+/892831 | 14:49 |
| mnasiadka | SvenKieske: now it's there, sorry ;) | 14:49 |
| mnasiadka | greatgatsby: it's going to be deprecated in Bobcat, if it won't shape up in Caracal - then we'll drop it | 14:50 |
| greatgatsby | interesting, thanks. I'm not sure we "depend" on it, but we do currently enable it. Good to know there's a chance it might be removed. | 14:51 |
| mnasiadka | jangutter: so now Ubuntu podman has been building over 3 hours - sure it doesn't need your patch? :) | 14:53 |
| jangutter | :o | 14:53 |
| mnasiadka | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7a8/899231/4/check/kolla-build-ubuntu-podman/7a80523/job-output.txt | 14:53 |
| jangutter | We can check, the podman info should tell us. | 14:53 |
| jangutter | Native Overlay Diff: "true" <--- It doesn't look like the same issue (from: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7a8/899231/4/check/kolla-build-ubuntu-podman/7a80523/kolla/podman-info.txt ) | 14:54 |
| mnasiadka | ok then | 14:54 |
| jangutter | from reading around, _weird_ things can cause slowdowns that snowball. One thing a friend hit was old versions of rpm needed a limit to open files. | 14:57 |
| jangutter | But this isn't rpm based containers, so https://www.linuxquestions.org/questions/linux-software-2/docker-fedora-31-installing-rpms-for-redhat-variants-slow-at-scriptlet-start-4175667314/ shouldn't apply. | 14:58 |
| jangutter | mnasiadka: do you have a link to the zuul page? | 15:01 |
| jangutter | something stalled on that, the build was progressing fine then, no output and it timed out. | 15:03 |
| jangutter | This one just hung.... https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7a8/899231/4/check/kolla-build-ubuntu-podman/7a80523/kolla/build/ironic-prometheus-exporter.log | 15:06 |
| kevko | does anyone using pdf generation from kolla-ansible ? :) | 15:06 |
| kevko | i mean tox ... | 15:06 |
| opendevreview | Merged openstack/kolla-ansible master: Fix OpenStack exporter scrape with internal TLS & FQDN https://review.opendev.org/c/openstack/kolla-ansible/+/877483 | 15:10 |
| SvenKieske | kevko: I never used that, didn't even know it is a thing, why do you ask? | 15:24 |
| opendevreview | Pedro Alvarez proposed openstack/kolla-ansible stable/zed: Fix OpenStack exporter scrape with internal TLS & FQDN https://review.opendev.org/c/openstack/kolla-ansible/+/900990 | 15:33 |
| opendevreview | Pedro Alvarez proposed openstack/kolla-ansible stable/2023.1: Fix OpenStack exporter scrape with internal TLS & FQDN https://review.opendev.org/c/openstack/kolla-ansible/+/900992 | 15:35 |
| opendevreview | Merged openstack/kolla-ansible master: ovn: Fix wording in limit check https://review.opendev.org/c/openstack/kolla-ansible/+/898910 | 15:52 |
| SvenKieske | mhm, just glancing at https://etherpad.opendev.org/p/kolla-stable-yoga-eom should we maybe merge this for yoga? https://review.opendev.org/c/openstack/kolla/+/880799 (rmq 3.10) | 15:55 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2023.1: ovn: Fix wording in limit check https://review.opendev.org/c/openstack/kolla-ansible/+/900994 | 15:57 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/zed: ovn: Fix wording in limit check https://review.opendev.org/c/openstack/kolla-ansible/+/900995 | 15:58 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/yoga: ovn: Fix wording in limit check https://review.opendev.org/c/openstack/kolla-ansible/+/900996 | 15:58 |
| opendevreview | Verification of a change to openstack/kayobe stable/yoga failed: Generate local Kolla Ansible config in check mode https://review.opendev.org/c/openstack/kayobe/+/900589 | 16:19 |
| opendevreview | Mark Goddard proposed openstack/kolla-ansible master: fact gathering: Group hosts before gathering facts https://review.opendev.org/c/openstack/kolla-ansible/+/899592 | 17:09 |
| opendevreview | Mark Goddard proposed openstack/kolla-ansible master: fact gathering: Remove gathering of package facts https://review.opendev.org/c/openstack/kolla-ansible/+/901056 | 17:09 |
| mnasiadka | jangutter: zuul.openstack.org ? | 17:15 |
| jangutter | mnasiadka: i mean the specific zuul result page for the log you sent. | 17:15 |
| mnasiadka | jangutter: https://review.opendev.org/c/openstack/kolla/+/899231?tab=change-view-tab-header-zuul-results-summary | 17:16 |
| jangutter | thanks! | 17:17 |
| jangutter | oh yeah, this is a Zuul timeout, raw logs are needed. But regardless, I think the build process hung on one job rather than being generally slow. | 17:18 |
| jangutter | You can see https://zuul.opendev.org/t/openstack/build/7a80523666914aa9a625d7b5f178d114/log/job-output.txt#1001 <--- it hung here for hours and didn't progress | 17:19 |
| mnasiadka | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7a8/899231/4/check/kolla-build-ubuntu-podman/7a80523/kolla/build/ironic-prometheus-exporter.log | 17:20 |
| mnasiadka | it didn't even properly start building anything | 17:20 |
| opendevreview | Verification of a change to openstack/kayobe stable/yoga failed: Fix setting kolla_admin_openrc_cacert https://review.opendev.org/c/openstack/kayobe/+/900586 | 17:44 |
| opendevreview | Juan Pablo Suazo proposed openstack/kolla-ansible master: Enable the Fluentd Plugin Systemd https://review.opendev.org/c/openstack/kolla-ansible/+/875983 | 17:50 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: Remove Monasca and co remnants https://review.opendev.org/c/openstack/kayobe/+/901059 | 17:53 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: Drop not used grafana-conf from reqs https://review.opendev.org/c/openstack/kayobe/+/901060 | 17:56 |
| opendevreview | Merged openstack/kayobe stable/zed: Generate local Kolla Ansible config in check mode https://review.opendev.org/c/openstack/kayobe/+/900588 | 17:58 |
| opendevreview | Merged openstack/kayobe stable/yoga: Fix an issue when user forgot combine custom passwords https://review.opendev.org/c/openstack/kayobe/+/900455 | 17:58 |
| opendevreview | Verification of a change to openstack/kayobe stable/2023.1 failed: Fix an issue when user forgot combine custom passwords https://review.opendev.org/c/openstack/kayobe/+/900461 | 18:11 |
| opendevreview | Merged openstack/kayobe stable/zed: Fix an issue when user forgot combine custom passwords https://review.opendev.org/c/openstack/kayobe/+/900462 | 18:12 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.1: Introduce oneshot docker_restart_policy https://review.opendev.org/c/openstack/kolla-ansible/+/900985 | 18:36 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!