| opendevreview | Merged openstack/openstack-ansible-haproxy_server master: Enable stats to use dedicated server certificate and allow for client cert auth https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/890640 | 04:39 |
|---|---|---|
| opendevreview | Christian Rohmann proposed openstack/openstack-ansible-haproxy_server stable/2023.1: Enable stats to use dedicated server certificate and allow for client cert auth https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/898529 | 06:03 |
| opendevreview | Christian Rohmann proposed openstack/openstack-ansible-haproxy_server stable/zed: Enable stats to use dedicated server certificate and allow for client cert auth https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/898530 | 06:17 |
| noonedeadpunk | mornings | 06:24 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Remove common nova playbook https://review.opendev.org/c/openstack/openstack-ansible/+/898493 | 06:25 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Remove common nova playbook https://review.opendev.org/c/openstack/openstack-ansible/+/898493 | 06:26 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Run nova db post setup from nova playbook https://review.opendev.org/c/openstack/openstack-ansible/+/898494 | 06:28 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Run nova db post setup from nova playbook https://review.opendev.org/c/openstack/openstack-ansible/+/898494 | 06:30 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Fix linters for example playbook https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/898531 | 06:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Override pyScss version with a bugfixed one https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/898463 | 06:33 |
| noonedeadpunk | Ok, seems that bookworm jobs are happy now. The only thing left is wtf with galera upgrade | 06:34 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible/+/894561 | 06:35 |
| jrosser | good morning | 07:36 |
| jrosser | do we think that the galera upgrade fail is only seen in CI? | 07:53 |
| noonedeadpunk | I tried to reproduce couple of times and was not able to | 08:04 |
| noonedeadpunk | Actually I added some debug yesterday | 08:04 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/817bd115ce6649d5a176f51422c66a61/log/job-output.txt#25363-25377 | 08:05 |
| noonedeadpunk | That is really o_O | 08:05 |
| noonedeadpunk | Like repo is not there at all | 08:06 |
| noonedeadpunk | hm. but we somewhat don't update cache when re-adding repo | 08:08 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Bump galera version to 10.11.5 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894740 | 08:09 |
| noonedeadpunk | so kinda makes sense why debug won't show it. But then during package install it should be updated... | 08:10 |
| jrosser | looks like it | 08:12 |
| jrosser | if it still breaks could put that debug task in again after the package installation | 08:12 |
| jrosser | to show before / after | 08:13 |
| noonedeadpunk | Yeah | 08:24 |
| noonedeadpunk | I was thinking on adding block actually there for rescue | 08:24 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Bump galera version to 10.11.5 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894740 | 08:26 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova stable/2023.1: Run nova_db_post_setup from playbook directly https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/898495 | 08:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova stable/2023.1: Run nova_db_post_setup from playbook directly https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/898495 | 08:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Bump galera version to 10.11.5 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894740 | 11:08 |
| noonedeadpunk | ok, seems the issue was in update_cache indeed.... | 11:08 |
| noonedeadpunk | it's weird it doesn't work in apt itself somehow... like as if apt module try to resolve dependencies before cache is updated | 11:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server stable/2023.1: Apply haproxy-service-config tag on include https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/898500 | 11:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server stable/2023.1: Apply haproxy-service-config tag on include https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/898500 | 11:16 |
| noonedeadpunk | damiandabrowski: that's what we hit just now during upgrade ^ | 11:17 |
| noonedeadpunk | or well, backport of that :D | 11:17 |
| noonedeadpunk | also I wonder how I wasn;t able to reproduce that locally.... | 11:35 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: Add tag for creating magnum resources https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/893362 | 12:36 |
| noonedeadpunk | actually, 1 thing that made our CI slower - is that we build wheels by default. This is smth we had disabled in CI before, that is known to take more time | 12:37 |
| noonedeadpunk | (when only 1 host that needs wheels) | 12:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Adopt for usage openstack_resources role https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/889879 | 12:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Add openstack_resources role skeleton https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/878794 | 12:42 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Convert haproxy_service_config_external.yml to a symlink https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/897841 | 12:49 |
| spatel | admin1 your solution doesn't work.. same error :( | 13:08 |
| spatel | I don't think that is how it will work.. there must be some more puzzle pieces | 13:08 |
| noonedeadpunk | ugh.... "Failed to download packages: Status code: 503 for https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=extras-9 (IP: 199.232.194.132)" | 13:13 |
| noonedeadpunk | At least https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894740 doesn't fail anymore with ubuntu upgrade job | 13:15 |
| NeilHanlon | ugh. looking again at this. | 13:25 |
| NeilHanlon | it's weird because everything else is seemingly fine. | 13:26 |
| opendevreview | Merged openstack/ansible-role-systemd_service master: Use TOX_CONSTRAINTS_FILE https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/851327 | 13:28 |
| noonedeadpunk | Nice thing is that our CI is good at catching failures :D | 13:35 |
| noonedeadpunk | NeilHanlon: where discussions about having infra mirrors in opendev stopped? | 13:36 |
| NeilHanlon | yeah, i think just stalled from before last PTG, as the AFS was pretty full iirc | 13:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Run nova db post setup from nova playbook https://review.opendev.org/c/openstack/openstack-ansible/+/898494 | 14:04 |
| NeilHanlon | noonedeadpunk: okay. I found a small bug in my remediation that was preventing some requests from being properly shielded. I believe those spurious 503s should _really_ be fixed now. | 14:15 |
| noonedeadpunk | jrosser: I have a question about this thing: https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/master/tasks/nova_db_post_setup.yml#L25 | 14:34 |
| jrosser | yes | 14:35 |
| noonedeadpunk | how that kinda supposed to work? As it's run against conductor, right? So... With Ironic we have only 1 possible virt_type per setup? | 14:35 |
| noonedeadpunk | or we assumed that fact from the compute we're delegating from is ironic? and only then it works? | 14:37 |
| noonedeadpunk | Even if look back to Yoga, let's say and forget about recent changes | 14:39 |
| noonedeadpunk | (very recent ones I mean) | 14:40 |
| jrosser | even then it is run_once: true | 14:41 |
| jrosser | here https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/stable/yoga/tasks/main.yml#L299 | 14:43 |
| noonedeadpunk | well, you kinda can run with limits... | 14:48 |
| noonedeadpunk | so it's THE node | 14:48 |
| noonedeadpunk | the lucky one | 14:48 |
| noonedeadpunk | I guess it should be kinda getting all `nova_virt_type` for all hosts and then check if ironic is in there? | 14:49 |
| noonedeadpunk | but then why not to add `--by-service` always... | 14:49 |
| noonedeadpunk | Like efficiency? | 14:50 |
| noonedeadpunk | https://docs.openstack.org/nova/latest/cli/nova-manage.html#cell-v2-discover-hosts | 14:50 |
| jrosser | i wonder what the actual difference is with --by-service | 14:51 |
| jrosser | and tbh the logic we have looks just wrong when there is any kind of mix of types | 14:51 |
| jrosser | *have always had | 14:51 |
| noonedeadpunk | `This is less efficient and is only necessary when using compute drivers that may manage zero or more actual compute nodes at any given time (currently only ironic).` | 14:52 |
| noonedeadpunk | But then we could just check for length of ironic compute or smth.... | 14:53 |
| jrosser | yes, that sounds much better | 14:53 |
| noonedeadpunk | ok. I just have found bugs in https://review.opendev.org/q/topic:bugfix-2034583 | 14:54 |
| noonedeadpunk | and no idea how that passed CI on master | 14:54 |
| noonedeadpunk | an one of these is that `nova_virt_type` is undefined | 14:54 |
| noonedeadpunk | (when included with tasks_from) | 14:55 |
| noonedeadpunk | but probably not in CI somehow (as we have kvm-compute_hosts or smth like that) | 14:55 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:01 |
| opendevmeet | Meeting started Tue Oct 17 15:01:01 2023 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:01 |
| NeilHanlon | o/ | 15:01 |
| noonedeadpunk | #topic office hours | 15:01 |
| noonedeadpunk | o/ | 15:01 |
| jrosser | o/ hello | 15:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix vars-file include for os-nova-install https://review.opendev.org/c/openstack/openstack-ansible/+/898557 | 15:06 |
| noonedeadpunk | Ah, I clean forgot to make a rollcall (though I thought I did lol) | 15:07 |
| noonedeadpunk | I just was about to say that we have pretty much clean CI now, once I've found another bug right before the meeting :( | 15:07 |
| noonedeadpunk | Anyway it looks like things are way better now | 15:08 |
| NeilHanlon | :\ oh no | 15:08 |
| damiandabrowski | hi! | 15:08 |
| noonedeadpunk | we got quite some stuff merged lately | 15:08 |
| NeilHanlon | i tried to push what I could see, through over the night time | 15:08 |
| noonedeadpunk | I've also triggered bunch of rechecks today - hopefully they won't fail on rocky mirrors :) | 15:08 |
| NeilHanlon | 🤞 | 15:08 |
| NeilHanlon | I configured the CDN to lie to users again -- so you should always at least get dl.rockylinux.org as a response 😉 | 15:09 |
| noonedeadpunk | I've come up with the etherpad for the PTG and even added some content there | 15:09 |
| noonedeadpunk | #link https://etherpad.opendev.org/p/osa-caracal-ptg | 15:09 |
| NeilHanlon | "origin gave a 503? we'll make our own response, thanks" | 15:09 |
| noonedeadpunk | Right now I realized that I've missed capi driver discussion | 15:09 |
| jamesdenton | o/ | 15:09 |
| NeilHanlon | i just realized I missed the voting for 'Caracal' heh | 15:10 |
| noonedeadpunk | hehe | 15:11 |
| noonedeadpunk | Not the most catchy name to be frank, still trying to remember it | 15:11 |
| noonedeadpunk | Also looks like stein branch is being EOLed right now | 15:12 |
| noonedeadpunk | FWIW, I did not sent ML yet with the PTG info - will do it right after the meeting | 15:12 |
| noonedeadpunk | Bumps for Zed and Yoga were merged and I am going to propose releases by end of the day | 15:13 |
| noonedeadpunk | Out of good news - Debian 12 seem to pass most of the things - at least MariaDB upgrade is not an issue anymore | 15:13 |
| noonedeadpunk | I've also reproduced upgrade failure caused by swiching to 2023.2 - I'm still investigating though the reasons | 15:14 |
| noonedeadpunk | #link https://review.opendev.org/c/openstack/openstack-ansible/+/897434 | 15:15 |
| noonedeadpunk | But I think we're generally in a good shape | 15:15 |
| NeilHanlon | that's great | 15:15 |
| noonedeadpunk | Functional jobs were also fixed - though not in a very appropriate way, but they do pass at least | 15:15 |
| NeilHanlon | heh | 15:16 |
| noonedeadpunk | And I was not able to work on pushing more patches for quorum queues. | 15:17 |
| noonedeadpunk | Though this is smth I think we should finilize for 2023.2 | 15:17 |
| noonedeadpunk | So, everyone interested, please, do populate the PTG doc. PTG is the next week, and doc is pretty much empty, so don't be shy :) | 15:20 |
| noonedeadpunk | #link https://etherpad.opendev.org/p/osa-caracal-ptg | 15:21 |
| NeilHanlon | 👍 | 15:30 |
| noonedeadpunk | #endmeeting | 15:55 |
| opendevmeet | Meeting ended Tue Oct 17 15:55:55 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:55 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-10-17-15.01.html | 15:55 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-10-17-15.01.txt | 15:55 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-10-17-15.01.log.html | 15:55 |
| noonedeadpunk | despite of huge spam regarding bugs in launchpad, at least we're down to 284 open bugs.... | 17:02 |
| opendevreview | Merged openstack/ansible-role-systemd_service master: Reduce output by leveraging loop labels https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/876302 | 17:41 |
| opendevreview | Merged openstack/ansible-role-systemd_service master: Remove conditions on systemd version https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/897832 | 17:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Adopt for usage openstack_resources role https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/889879 | 17:47 |
| noonedeadpunk | Ugh, no, functional jobs seems to be broken for plugins repo. But kinda failures are valid I guess.... | 17:49 |
| noonedeadpunk | There's a check on delegation to container with jsut IP, and since we don't have SSH on containers anymore.... | 17:50 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Bump galera version to 10.11.5 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894740 | 18:13 |
| opendevreview | Merged openstack/openstack-ansible-os_horizon master: Fix linters for example playbook https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/898531 | 18:48 |
| opendevreview | Merged openstack/openstack-ansible-lxc_container_create master: Properly apply tags for include_tasks https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/894294 | 18:48 |
| noonedeadpunk | Ok, so what we see in https://review.opendev.org/c/openstack/openstack-ansible/+/897434 seems to be a nova bug.... | 18:50 |
| noonedeadpunk | I;ve filled in one just in case https://bugs.launchpad.net/nova/+bug/2039597 | 18:50 |
| noonedeadpunk | And, Debian 12 is green now! https://review.opendev.org/c/openstack/openstack-ansible/+/894561 | 18:51 |
| opendevreview | Merged openstack/openstack-ansible-os_horizon master: Override pyScss version with a bugfixed one https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/898463 | 19:13 |
| jrosser | i have been trying to think if there is a good way to drop needing the wheel build selectively | 19:15 |
| jrosser | we had bugs merged with no repo server which was sad, but maybe we don't need it for upgrade jobs or something? | 19:16 |
| jrosser | though some of the heavier roles like magnum are pretty wedged up at the moment with timeouts | 19:17 |
| noonedeadpunk | yeah, true, not talking about telemetry even... | 19:29 |
| noonedeadpunk | but that kinda boils all down to functional jobs or some kind of their re-incarnation with molecule that we're talking about for a while, but don't have time to execute basically | 19:31 |
| noonedeadpunk | where we can test heavy parts with really minimalistic things - ie build wheels | 19:32 |
| noonedeadpunk | or generate certs | 19:33 |
| jrosser | i would hope that a future thing with incus would reduce the runtime a bunch too | 19:37 |
| jrosser | as we could convert pretty much all the container config stuff into config data on the incus side, and just boot them | 19:37 |
| noonedeadpunk | oh, yes, incus, the thing I forgot to add to etherpad ) | 19:43 |
| mgariepy | jrosser, have you started playing with it ? | 19:45 |
| jrosser | no, i havent, but it's basically LXD isnt it | 19:45 |
| jrosser | which we use here anyway | 19:45 |
| mgariepy | pretty much yeah. | 19:45 |
| mgariepy | no snap no maas no fan no cacnonical rbac etc. | 19:46 |
| mgariepy | i did tried on arch but there is something weird with the build. | 19:46 |
| jrosser | i am hoping that using all the builtin stuff for networking / cloud-init etc could save really many many ansible tasks that we currently have | 19:48 |
| mgariepy | yeah it probably could. | 19:49 |
| opendevreview | Merged openstack/openstack-ansible stable/2023.1: Remove common nova playbook https://review.opendev.org/c/openstack/openstack-ansible/+/898493 | 21:45 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!