| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Update horizon local settings for Django 4 https://review.opendev.org/c/openstack/kolla-ansible/+/906254 | 06:49 |
|---|---|---|
| frickler | hmm, kevko does a rebase of my patch and when I rebase it again, gerrit says I'm doing this on behalf of kevko? weird. but let's please get this ^^ fix in now and check the local_settings.d stuff later | 07:01 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Just test temporarily https://review.opendev.org/c/openstack/kolla-ansible/+/906423 | 07:32 |
| opendevreview | Rafal Lewandowski proposed openstack/kolla-ansible master: [WIP] Enable ML2/OVN and distributed FIP by default https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 07:45 |
| kevko | frickler: i have a question ... what version should be installed before kolla-ansible upgrade in upgrade.sh ? i thought there should be 2023.2, shouldn't be ? | 08:28 |
| kevko | frickler: why there is master and then tag change_xxxx ? | 08:29 |
| kevko | frickler: let's merge it ..as it is CI blocker ...It will fix the gate ...but I still don't understand why my patch is not working | 08:33 |
| kevko | frickler: deploy jobs are working ..because it is master images and master kolla-ansible where it is everything ok, why upgrade job is not working ? i thought that upgrade job is zuul-mirror 2023.2 tag and kolla-ansible 2023.2 ....then tag zuul-mirror:master (in case not depends-on) and then gerrit review upgrade ... or change_XXX (if depends-on | 08:36 |
| kevko | kolla review) tag ...with the current gerrit review kolla-ansible patch ...or no ? | 08:36 |
| kevko | we should draw some pipeline workflow diagram :D | 08:37 |
| frickler | kevko: it is well possible that there is a bug in our testing, I don't claim to understand much of it either. probably mnasiadka is the one who remains with the most insight | 09:16 |
| kevko | frickler: bbezak promised me that he will check it today :) | 09:17 |
| kevko | frickler: okay, let's merge your django local_settings as it's gate blocker | 09:18 |
| opendevreview | Rafal Lewandowski proposed openstack/kolla-ansible master: [WIP] Enable ML2/OVN and distributed FIP by default https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 09:32 |
| * SvenKieske has to read up about all the CI breakage | 09:59 | |
| opendevreview | Dr. Jens Harbott proposed openstack/kolla master: Revert "Fix gnocchi and skyline after requirements change" https://review.opendev.org/c/openstack/kolla/+/906479 | 10:09 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test jobs workflow https://review.opendev.org/c/openstack/kolla-ansible/+/906503 | 10:32 |
| opendevreview | Merged openstack/kolla-ansible master: Update horizon local settings for Django 4 https://review.opendev.org/c/openstack/kolla-ansible/+/906254 | 10:50 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Fix podman cross-dependency build https://review.opendev.org/c/openstack/kolla-ansible/+/906229 | 11:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rework horizon role to support local_settings.d https://review.opendev.org/c/openstack/kolla-ansible/+/906347 | 11:10 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible stable/2023.2: use 2023.2 images in stable/2023.2 branch https://review.opendev.org/c/openstack/kolla-ansible/+/906509 | 11:48 |
| kevko | frickler ^^ bbezak is my savior ..I almost lost my temper | 11:49 |
| bbezak | :) | 11:51 |
| bbezak | I've seen couple of other gems, will post some patches | 11:51 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rework horizon role to support local_settings.d https://review.opendev.org/c/openstack/kolla-ansible/+/906347 | 11:52 |
| kevko | bbezak: can u share them in short ? :) | 11:52 |
| bbezak | we didn't update openstack_previous_release_name in master - that is used for ironic rolling update | 11:54 |
| bbezak | and we didn't update previous release in kayobe | 11:54 |
| kevko | bbezak: yes i saw this at night | 11:54 |
| bbezak | I'm updating this shortly | 11:55 |
| kevko | ^ ironic | 11:55 |
| kevko | when i was grepping almost everytihing in kolla-ansible repo :D | 11:55 |
| bbezak | :) | 11:55 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: update openstack_previous_release_name var for bobcat https://review.opendev.org/c/openstack/kolla-ansible/+/906511 | 11:57 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe master: update previous release to 2023.2 https://review.opendev.org/c/openstack/kayobe/+/906513 | 12:00 |
| kevko | bbezak: shouldn't we have some script to do this automatically when stable release from "previous" master is released ? | 12:01 |
| kevko | bbezak: btw, do exist some way how i can access logs dir even if tests are still running ? :D | 12:02 |
| bbezak | we've got a procedure to follow - https://docs.openstack.org/kolla/latest/contributor/release-management.html looks like it was not followed properly this time :) | 12:05 |
| kevko | that's what i am saying all the time ...noone reads docs :D | 12:06 |
| bbezak | I think for that you would need to get access to the host directly, but I think frickler could help you with that if that's really needed | 12:06 |
| kevko | bbezak: i am asking for the future ... | 12:15 |
| kevko | bbezak: here you go -> podman depends on fail ... (i rebased and dropped that patch which was solving podman cross-dependency issue ... and you see..) https://zuul.opendev.org/t/openstack/build/c95b2d80c13145dfbff71973966c7dad | 12:16 |
| bbezak | I'll look into podman thingy in a bit | 12:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Fix podman cross-dependency build https://review.opendev.org/c/openstack/kolla-ansible/+/906229 | 12:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rework horizon role to support local_settings.d https://review.opendev.org/c/openstack/kolla-ansible/+/906347 | 12:17 |
| kevko | bbezak: it's simple, if job is running with podman container engine ...it's built also using podman --> https://github.com/openstack/kolla-ansible/blob/6ec1dc97e2246442fc601b022a4baf783ce052ec/tests/setup_gate.sh#L82C1-L82C29 | 12:19 |
| kevko | bbezak: as I saw fail, i've already rebased and included fix for cross-dependency again | 12:20 |
| bbezak | cool | 12:21 |
| opendevreview | Piotr Parczewski proposed openstack/kolla master: Use full binary path when invoking ip https://review.opendev.org/c/openstack/kolla/+/906516 | 12:24 |
| SvenKieske | kevko: bbezak: +1 on automating release management. I fail to see why one would need host access for that? I must overlook something. | 13:23 |
| kevko | SvenKieske: during debugging I wrote some own output files which ran early on pipeline beggining ..and because I am impatient most of the time .. I was just curious if it is possible to check my debug even if job is still running | 13:25 |
| kevko | SvenKieske: sometimes it's really helpful to try some scenarios manually ... you probably agree that if you develop some feature ..you execute code locally 20x more than in one review ... | 13:26 |
| SvenKieske | sure, but I was just talking about automation of release management, not about debugging :) maybe I misunderstood. | 13:27 |
| kevko | SvenKieske: for me - for example it would be helpful to create bucket in swift on the beginning and upload intermediate logfiles to swift even if job is already running ... then you will be able to check the URL for logs from running console job and just open in your browser ...unfortuntatelly ... current implementation is that logs are uploaded | 13:29 |
| kevko | at once on the end | 13:29 |
| kevko | SvenKieske: aaaa .. well there were several variables misconfigured ..and because of this ..we tested (for example) master->master ..because variable was master in 2023.2 branch (:() | 13:30 |
| kevko | bbezak sent me docs ..what is the process of release .. | 13:30 |
| kevko | I was asking ..why there is not some script to do it automatically and ask user on the end ...is this everything OK ? and he just click ..yes | 13:31 |
| SvenKieske | kevko: yes, but it at least seems we should be able to take the docs and replace them by some automation script thingy, no? :) | 13:31 |
| SvenKieske | agreed :) | 13:31 |
| kevko | yeah, of course .. | 13:32 |
| kevko | if the release process is always same ..i mean, create new branch from the master branch, change previous version in new master to previous released version ..change version in stable release to stbale release not master ...etc ...etc | 13:32 |
| kevko | from my point of view it's really easy script | 13:33 |
| kevko | some greps, seds, ...or python + yaml | 13:33 |
| kevko | that's it | 13:33 |
| frickler | so just write it and submit it as a tool, I don't think anyone would object | 13:35 |
| bbezak | Kolla meeting in 10 minutes - mgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees | 13:50 |
| bbezak | #startmeeting kolla | 14:00 |
| opendevmeet | Meeting started Wed Jan 24 14:00:33 2024 UTC and is due to finish in 60 minutes. The chair is bbezak. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:00 |
| bbezak | #topic rollcall | 14:00 |
| SvenKieske | o/ | 14:00 |
| frickler | \o | 14:00 |
| kevko | \o/ | 14:00 |
| mattcrees | o/ | 14:01 |
| bbezak | #topic agenda | 14:02 |
| bbezak | * Roll-call | 14:02 |
| bbezak | * Agenda | 14:02 |
| bbezak | * Announcements | 14:02 |
| bbezak | * Review action items from the last meeting | 14:02 |
| bbezak | * CI status | 14:02 |
| bbezak | * Release tasks | 14:02 |
| bbezak | * Regular stable releases (first meeting in a month) | 14:02 |
| bbezak | * Current cycle planning | 14:02 |
| bbezak | * Additional agenda (from whiteboard) | 14:02 |
| bbezak | * Open discussion | 14:02 |
| bbezak | #topic CI status | 14:02 |
| bbezak | I think looks good after yesterday fixes by kevko and frickler | 14:03 |
| bbezak | (I've been on sick leave for couple of days) | 14:03 |
| kevko | podman cross-dependency missing | 14:03 |
| SvenKieske | (me too ftr) | 14:03 |
| bbezak | thx SvenKieske - sorry I was not around | 14:04 |
| mmalchuk | o/ | 14:04 |
| SvenKieske | ah no problem, health comes first :) | 14:04 |
| bbezak | kevko: yeah I wanted first to merge the 2023.2 k-a fix and then retest upgrades in your patch | 14:04 |
| bbezak | if everything is sorted as it should | 14:05 |
| kevko | bbezak: we have the same thoughts | 14:05 |
| kevko | +1 | 14:05 |
| bbezak | nice | 14:05 |
| bbezak | there was red status in kolla whiteboard, I've changed it to green, as it is If I'm not mistaken | 14:05 |
| kevko | I hypnotize zuul status with a look | 14:06 |
| bbezak | :) | 14:06 |
| bbezak | #topic Release tasks | 14:06 |
| bbezak | we were lacking several patches for release tasks unfortunately | 14:06 |
| bbezak | I've raised them today | 14:07 |
| bbezak | kevko has some automation/tooling ideas | 14:07 |
| bbezak | patches are welcome | 14:07 |
| kevko | i will propose some maybe ...if I will have some time :) | 14:08 |
| bbezak | ok, thx | 14:08 |
| bbezak | I've raised monthly stable release last week | 14:08 |
| kevko | as you surely know, I have never done a release, I was completely surprised that there is no automated procedure for this | 14:08 |
| bbezak | https://review.opendev.org/c/openstack/releases/+/906046 | 14:09 |
| bbezak | the process is pretty well documented, however it surely need an update - https://docs.openstack.org/kolla/latest/contributor/release-management.html | 14:09 |
| bbezak | will add a note to myself to update this doc | 14:11 |
| kevko | +1 | 14:11 |
| bbezak | #topic Current cycle planning | 14:12 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L209 | 14:12 |
| bbezak | any updates/questions on those priorities? | 14:12 |
| bbezak | I don't have any update myself this week | 14:13 |
| SvenKieske | not from my part at least, partly due to being sick the last two days :) | 14:13 |
| bbezak | as far as my tasks | 14:13 |
| SvenKieske | *technically three days | 14:15 |
| bbezak | we've been mostly fire fighting CI since last week I can see. Let's hope next week will be more fruitful | 14:16 |
| bbezak | #topic Additional agenda (from whiteboard) | 14:17 |
| bbezak | I can see Container engine migration from mhiner | 14:17 |
| bbezak | however I recall it was covered last week | 14:17 |
| SvenKieske | I don't remember if this question was answered? > current CI tests don't gather container logs after migrating, any hints how to fix? | 14:18 |
| bbezak | oh I remember that I didn't want to remove it from whiteboard as it has some valuable info | 14:18 |
| kevko | SvenKieske: i think it is .. | 14:19 |
| SvenKieske | in general I think it would make sense to update the whiteboard if questions are brought up and answered, no? not sure about the current approach here. | 14:19 |
| frickler | I think I still need to look at those tests | 14:19 |
| SvenKieske | at least - if I'm not misinterpreting - I don't see any answers to any of the questions there, like which cases we want to test. these answers are also missing from the patchset afaik. | 14:20 |
| bbezak | yeah, it got valuable info/question - which is not yet fully answered, we can leave it here to track every week. if that makes sense | 14:21 |
| SvenKieske | imho it only makes sense if it either contains the actual status of info/question, or a link to the status, or at the very list a link to the person working on answering the questions/providing the needed information | 14:22 |
| bbezak | it probably should be discussed in the change itself | 14:22 |
| SvenKieske | I'm fine with any solution, but I think only having the question there with no update serves little benefit. afaik the CI question is already in the review, but no answers :) | 14:23 |
| bbezak | ok, let's continue discussion on that in the change. I'll double check if this info is in the change and then will remove from additional agenda part for now | 14:25 |
| SvenKieske | ok | 14:25 |
| bbezak | #topic Open discussion | 14:25 |
| bbezak | Anything to discuss today? some changes to review? | 14:27 |
| kevko | well, I wanted to review horizon patches :P | 14:27 |
| kevko | and DNS | 14:27 |
| mmalchuk | https://review.opendev.org/c/openstack/kolla-ansible/+/904400 | 14:28 |
| kevko | I am sure that after CI patches will be merged ..it will pass ..so if anyone have a time | 14:28 |
| mmalchuk | please merge last fix to Yoga | 14:28 |
| mmalchuk | and this: https://review.opendev.org/c/openstack/kolla-ansible/+/902382 | 14:28 |
| mmalchuk | should be abandoned since not needed | 14:28 |
| mmalchuk | https://review.opendev.org/c/openstack/tenks/+/904333 | 14:29 |
| mmalchuk | not K/K-A but please review too | 14:29 |
| mattcrees | I'm looking for second core reviews on two bugfixes, if anyone has the time: https://review.opendev.org/c/openstack/kolla-ansible/+/905500 https://review.opendev.org/c/openstack/kolla-ansible/+/905851 | 14:29 |
| frickler | added to my list | 14:31 |
| bbezak | speaking of tenks, I've just notice it is failing in kayobe and k-a ironic jobs in master | 14:32 |
| bbezak | "msg": "exceeded retry limit of 15 whilst waiting for resources to become available" | 14:32 |
| bbezak | we need to look into that | 14:32 |
| bbezak | I'll add info to whiteboard | 14:33 |
| mmalchuk | have a link to failed job? | 14:33 |
| bbezak | https://zuul.opendev.org/t/openstack/build/ea7e2d6d8f0d471e849e66cee46d3422 | 14:34 |
| bbezak | and in k-a https://zuul.opendev.org/t/openstack/build/935effc2148b436287d1ab1a057c7d7f | 14:34 |
| bbezak | looks pretty fresh case | 14:35 |
| mmalchuk | interesting | 14:36 |
| mmalchuk | some regression in ironicclient? | 14:36 |
| mmalchuk | oh... placementclinet | 14:36 |
| bbezak | not sure, I literally just seen it in my patch for kayobe | 14:37 |
| mmalchuk | strange... 4.2.0 released in Aug 2023 | 14:37 |
| SvenKieske | don't know if related but I proposed a fix for networking-baremetal, which I think is used in ironic, with regards to quorum queues: https://review.opendev.org/c/openstack/networking-baremetal/+/903995 | 14:37 |
| SvenKieske | didn't look into your issue though, so maybe totally unrelated. | 14:38 |
| mmalchuk | imho not related since not used even if exists in container | 14:38 |
| bbezak | interesting, let's see. I'll try to look into that tenks case probably tomorrow | 14:38 |
| mmalchuk | it used when segments activated | 14:38 |
| kevko | btw, i am too lazy to study code ..but from where is tenks installed in ci ? | 14:39 |
| mmalchuk | have a look too | 14:39 |
| kevko | tenks.yml | 14:39 |
| kevko | where i can find it ? | 14:40 |
| mmalchuk | deploy-tenks.sh | 14:40 |
| kevko | hh, i know | 14:41 |
| mmalchuk | tests/deploy-tenks.sh | 14:41 |
| kevko | and there is tenks.yml | 14:41 |
| mmalchuk | function deploy_tenks_logged | 14:41 |
| mmalchuk | line #10 | 14:41 |
| mmalchuk | installed from source | 14:41 |
| kevko | thanks | 14:42 |
| bbezak | nice, so more firefighting, as usual | 14:42 |
| bbezak | anything more to discuss? | 14:43 |
| bbezak | ok, I guess no. thank you everybody! | 14:44 |
| bbezak | #endmeeting | 14:44 |
| opendevmeet | Meeting ended Wed Jan 24 14:44:21 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:44 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-01-24-14.00.html | 14:44 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-01-24-14.00.txt | 14:44 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-01-24-14.00.log.html | 14:44 |
| mmalchuk | thanks bbezak | 14:44 |
| SvenKieske | thank you for running the meeting :) | 14:44 |
| mattcrees | Cheers all | 14:45 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix ironic quorum queues don't support auto_delete property https://review.opendev.org/c/openstack/kolla-ansible/+/906527 | 15:04 |
| kevko | let's see | 15:05 |
| opendevreview | Merged openstack/kolla-ansible master: update openstack_previous_release_name var for bobcat https://review.opendev.org/c/openstack/kolla-ansible/+/906511 | 15:36 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.2: use 2023.2 images in stable/2023.2 branch https://review.opendev.org/c/openstack/kolla-ansible/+/906509 | 15:56 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/2023.1: Support older Octavia var names in Antelope https://review.opendev.org/c/openstack/kolla-ansible/+/905500 | 16:10 |
| opendevreview | Merged openstack/kolla-ansible master: Fix OpenSearch upgrade tasks idempotency https://review.opendev.org/c/openstack/kolla-ansible/+/905851 | 16:45 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Fix podman cross-dependency build https://review.opendev.org/c/openstack/kolla-ansible/+/906229 | 17:03 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rework horizon role to support local_settings.d https://review.opendev.org/c/openstack/kolla-ansible/+/906347 | 17:03 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Enable instance usage audit only when ceilometer is enabled https://review.opendev.org/c/openstack/kolla-ansible/+/905502 | 18:51 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 18:51 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 18:51 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger octavia tests - neutron internal DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/906099 | 18:51 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 20:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 20:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger octavia tests - neutron internal DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/906099 | 20:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 20:57 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 20:57 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger octavia tests - neutron internal DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/906099 | 20:57 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.1: Support older Octavia var names in Antelope https://review.opendev.org/c/openstack/kolla-ansible/+/905500 | 21:05 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 21:09 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger octavia tests - neutron internal DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/906099 | 21:09 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 21:33 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 21:33 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger octavia tests - neutron internal DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/906099 | 21:33 |
| opendevreview | Merged openstack/kolla-ansible master: ironic: only bootstrap ironic-tftp when service is enabled https://review.opendev.org/c/openstack/kolla-ansible/+/896820 | 23:41 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rework horizon role to support local_settings.d https://review.opendev.org/c/openstack/kolla-ansible/+/906347 | 23:58 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!