Monday, 2025-01-20

opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Add multi-AZ deployment configuration sample  https://review.opendev.org/c/openstack/openstack-ansible/+/93960909:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Add multi-AZ deployment configuration sample  https://review.opendev.org/c/openstack/openstack-ansible/+/93960909:41
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Add multi-AZ deployment configuration sample  https://review.opendev.org/c/openstack/openstack-ansible/+/93960910:13
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Molecule to respect depends-on for test-requirements update  https://review.opendev.org/c/openstack/openstack-ansible/+/93929010:23
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Add noble to molecule testing  https://review.opendev.org/c/openstack/openstack-ansible/+/93930610:23
noonedeadpunkfwiw, ansible-compat 25.0.0 was dropped from pypi: https://pypi.org/project/ansible-compat/25.0.0/10:24
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Add httpd role to a-r-r  https://review.opendev.org/c/openstack/openstack-ansible/+/93827110:24
jrosserhmm interesting10:24
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Ensure repo_server proper username  https://review.opendev.org/c/openstack/openstack-ansible/+/93827510:24
jrosseri wonder where the discussion about that is happening10:24
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Return upgrade jobs back to voting  https://review.opendev.org/c/openstack/openstack-ansible/+/93930710:24
jrosseranyway thats good - we have been wedged up for 1 week now on master branch, so we can start rechecking stuff again10:26
noonedeadpunknot in IRC for sure10:26
noonedeadpunkI also spent weekend on documenting AZ setup: https://zuul.opendev.org/t/openstack/build/4755b757ba944c9f80a40404e811f408/artifacts10:27
noonedeadpunkit includes also some smaller examples, which I guess could be avoided... But also I realized we don't have any example showing how to do them all together.10:28
noonedeadpunkto it's quite big one....10:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-repo_server master: Use standalone httpd role  https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/93827010:39
jrosseri wonder if we should follow up this https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/939599 to remove the need for a privileged port10:44
jrosserto allow global use of priv ports on potentially a metal infra host when we could just move the backends to 8443/8080 is surprising10:47
jrosserthis is likely from the past when direct access to the horizon backend by the user might be expected10:48
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-os_ironic master: Fix quorum queue support for ironic-inspector  https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/93954410:53
opendevreviewMerged openstack/openstack-ansible master: Update prod example to reflect OVN setup  https://review.opendev.org/c/openstack/openstack-ansible/+/93898111:03
f0oI'm just ever so slightly confused by the plethora of "action required" emails haha11:31
opendevreviewJonathan Rosser proposed openstack/ansible-role-systemd_networkd master: DNM - test vars inclusion  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/93962812:26
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Make sysctl configuration path configurable Defaults to /etc/sysctl.conf to retain current behavior  https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/93960112:27
opendevreviewMerged openstack/openstack-ansible-os_swift master: Make sysctl configuration path configurable Defaults to /etc/sysctl.conf to retain current behavior  https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/93960312:57
noonedeadpunkf0o: yeah, it looks like weve introduced couple of issues lately13:00
noonedeadpunkbut gerrit is quite noisy indeed13:01
opendevreviewMerged openstack/openstack-ansible-repo_server stable/2023.2: Fix tags usage for the role  https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/93905113:01
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ops master: Upgrade magnum-cluster-api to 0.24.2  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/93622913:02
f0onoonedeadpunk: no worries; I'm running my changes happily locally13:07
f0oif there _IS_ something I need to address, feel free to shout at me ;)13:07
noonedeadpunksure thing, but it looks like issues are on our side for now13:14
opendevreviewMerged openstack/openstack-ansible-os_horizon master: Make sysctl configuration path configurable Defaults to /etc/sysctl.conf to retain current behavior  https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/93959913:21
jrosserhuh13:41
jrosserso why only warnings here for rabbitmq and galera https://zuul.opendev.org/t/openstack/build/86a170c3e142470cb343e20ecac59f2a/log/job-output.txt#20458-2050413:41
jrosserand how is this possible https://zuul.opendev.org/t/openstack/build/86a170c3e142470cb343e20ecac59f2a/log/job-output.txt#20973-2111913:42
jrosser_systemd_networkd_distro_packages is a list with values, but systemd_networkd_distro_packages somehow ends up as []13:43
jrosseraaaaaaahhhhhhhh https://opendev.org/openstack/openstack-ansible-lxc_container_create/src/branch/master/tasks/lxc_container_network.yml#L2213:45
jrossernoonedeadpunk: so we either need to remove that ^ or make the service restart somehow conditional13:46
noonedeadpunkI'm really not sure why we have that13:56
noonedeadpunkoh13:56
noonedeadpunkI know....13:57
noonedeadpunkthese should be part of the image13:57
noonedeadpunknot to spend time on installation during the runtime13:57
noonedeadpunkso pretty much we need to extend this list then: https://opendev.org/openstack/openstack-ansible-lxc_hosts/src/branch/master/vars/debian.yml#L4813:57
noonedeadpunkjrosser: ^13:58
jrosserok - maybe we do both then? extend the list and remove that line?13:58
jrosserbecasue the behaviour is very counterintuitive with that list set externally to []13:59
noonedeadpunkthe thing is that we skip whole this block for all containers: https://opendev.org/openstack/ansible-role-systemd_networkd/src/branch/master/tasks/main.yml#L32-L10014:00
noonedeadpunkeven though it will be mostly skipped individually (except package), it still should be faster to skip the block14:01
noonedeadpunkand given the multiplier....14:01
jrosserso but the issue is that the restart handler still tries to restart all the services defined in vars/<os>.yml14:02
jrosserand adjusting the base image package list is somewhat complicated because of systemd_resolved being optional14:02
opendevreviewMerged openstack/ansible-hardening master: Make sysctl configuration path configurable Defaults to /etc/sysctl.conf to retain current behavior  https://review.opendev.org/c/openstack/ansible-hardening/+/93960014:03
jrosserand even if we did that successfully it would not fix these upgrade jobs14:03
jrosseras the contaienr is not rebuilt from a new image during the upgrade14:03
noonedeadpunkwe have some mapping already? https://opendev.org/openstack/openstack-ansible-lxc_hosts/src/branch/master/vars/debian.yml#L6314:03
jrosserah ok, sire14:04
jrosser*sure14:04
noonedeadpunkI wonder if we should rebuild the container image in run_upgrade14:04
jrosserwe would need to do a rolling destroy/create too14:04
noonedeadpunkdoh14:05
noonedeadpunktrue14:05
jrosserso yes absolutely we can update the base image packages, that makes total sense14:05
jrosserbut thats not the same as for an in-place upgrade14:05
noonedeadpunkand actually jobs are not failing for non-upgrade, do they?14:05
jrosserthose work, its just an upgrade that fails14:06
noonedeadpunkso I'm looking now at 2024.1 and I hardly see a difference?14:06
jrossertheres an example here with my added debug https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/939601?tab=change-view-tab-header-zuul-results-summary14:06
noonedeadpunkah, udev is missing14:07
jrosseryeah, becasue its not in the base image (in the 'upgrade from' branch) and thr package list for the networkd role is []14:07
jrossermaybe we need to split the handler into two parts14:08
jrosserone to restart networkd14:08
noonedeadpunkbut well. how does then networkd works there at all14:08
jrosserand the other part to restart other things, but only if systemd_networkd_distro_packages is not []14:08
noonedeadpunkas systemd-udevd was kinda crucial to set an IP to interfaces14:08
jrosserin docker ^14:09
noonedeadpunkyes14:09
jrosseri think that we have made a fix for the molecule case, which would be working anyway in lxc14:09
noonedeadpunkthough I thought it would be working for LXC , as LXC does have a service running anyway14:10
noonedeadpunkwhich is apparently not true14:10
jrosserwierd isnt it14:10
noonedeadpunkI'm just trying to think if it is this, which should be conditional: https://opendev.org/openstack/ansible-role-systemd_networkd/src/branch/master/handlers/main.yml#L16-L2514:12
jrosseri wonder if we have some difference between udev and systemd-udevd14:13
jrosserbecasue we install udev, but the thing that fails is restarting systemd-udevd14:13
noonedeadpunkum, yes, but also depends on debian/ubuntu version14:13
noonedeadpunkas jammy and debian 11 had systemd provided with a single package14:13
noonedeadpunkand 24.04/12 have it split into pieces14:14
jrosseri was wondering if the serivce name needs to depend on the os version14:14
noonedeadpunkas starting udev would fail/block systemd-udevd eecution at least in docker until reboot14:14
noonedeadpunkthe problem is - they're valid in non-upgrade (because it's already in the image) and in docker...14:15
noonedeadpunkso could be systemd was not relying on udev that much before...14:16
noonedeadpunkdunno14:16
jrosseri'm just looking at a jammy deploy14:16
noonedeadpunkjust pretty much we need to handle an upgrade scenario14:16
jrossertheres no udev installed at all14:16
jrosserin my keystone container, as example14:16
noonedeadpunkyeah, so it could be not needed14:16
noonedeadpunkand then it's the handler who should be conditional14:17
noonedeadpunkor at least move list of services to vars14:17
opendevreviewJonathan Rosser proposed openstack/ansible-role-systemd_networkd master: Only restart non-networkd services when the role is configured to install them  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/93964014:27
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Make sysctl configuration path configurable Defaults to /etc/sysctl.conf to retain current behavior  https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/93960114:28
jrosserthat feels like the simplest change and also matching the behaviour of testing the distro package length elsewhere14:29
noonedeadpunkactually it's true....14:34
noonedeadpunkdespite being counter-intuitive as well. as if services installed/running we don't need to start them14:35
noonedeadpunkand likely we can drop that after 2025.1 at all14:35
opendevreviewJonathan Rosser proposed openstack/ansible-role-systemd_networkd master: Add test coverage for upgrades  https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/93964114:43
opendevreviewMerged openstack/openstack-ansible stable/2024.2: Use the same version of ansible for linters as is used for deployments.  https://review.opendev.org/c/openstack/openstack-ansible/+/93935615:26
opendevreviewMerged openstack/openstack-ansible master: Add httpd role to a-r-r  https://review.opendev.org/c/openstack/openstack-ansible/+/93827116:00
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_horizon master: Remove use of async when compressing static files  https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/92795316:40
opendevreviewMerged openstack/openstack-ansible master: Ensure repo_server proper username  https://review.opendev.org/c/openstack/openstack-ansible/+/93827518:59
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-repo_server master: Use standalone httpd role  https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/93827019:02

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!