Thursday, 2023-07-13

*** ptoal is now known as Sage90502:18
noonedeadpunkfolks, can we kindly land leftovers for 2023.1? https://review.opendev.org/q/parentproject:openstack/openstack-ansible+branch:%255Estable/2023.1+status:open+08:20
noonedeadpunkespecially for the integrated repo08:20
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/zed: Include proper vars_file for rally  https://review.opendev.org/c/openstack/openstack-ansible/+/88843608:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_rally stable/zed: Include proper commit in rally_upper_constraints_url  https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/88768008:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/zed: Re-enable CI jobs after rally is fixed  https://review.opendev.org/c/openstack/openstack-ansible/+/88843708:29
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/zed: Pin version of setuptools  https://review.opendev.org/c/openstack/openstack-ansible/+/88715708:30
jrosserdone08:31
noonedeadpunkthanks!08:33
noonedeadpunkwill try to fix other stable branches today as well08:33
jrosserdo we really mean br-octavia here? https://github.com/openstack/openstack-ansible/commit/d35e3e0daef67546eded457a92186eb043575d6909:51
noonedeadpunklikely this works for OVN...09:52
noonedeadpunknot sure about ovs though09:52
noonedeadpunk(or anything else)09:53
jrosserit's different to this https://opendev.org/openstack/openstack-ansible/src/branch/master/etc/openstack_deploy/openstack_user_config.yml.aio#L7209:53
jrosserand not what AIO bootstrap creates09:53
noonedeadpunkyeah, jsut found that as well09:53
noonedeadpunkwe need either to revert one or update another as well...09:54
jrosseri just found octavia in an LXC AIO is broken from this09:54
noonedeadpunkwell.... we both were reviewing that change...09:54
noonedeadpunknobody else to blame :D09:55
jrosserindeed09:55
jrosseroh well10:00
jrossermaybe there is br-lbaas on the controllers for containers10:00
jrosserand OVN managed br-octavia on computes10:01
jrosserwhich are actually the same flat network?10:01
noonedeadpunkwell... then there should be 2 records in provider_networks10:02
noonedeadpunkas currently we're trying to build br-octavia to octavia-worker10:02
noonedeadpunkand not only to neutron_ovn_gateway10:02
noonedeadpunkand well, not sure if neutron_linuxbridge_agent/ovs_agent removed10:05
jrosserit needs also to work for AIO scenario10:07
jrosserwhen controller == compute10:07
opendevreviewMerged openstack/openstack-ansible-plugins stable/2023.1: Skip updating service password by default  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/88815210:27
opendevreviewMerged openstack/openstack-ansible-os_rally stable/2023.1: Include proper commit in rally_upper_constraints_url  https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/88767910:27
opendevreviewMerged openstack/openstack-ansible-os_keystone stable/2023.1: Fix SSL logic in keystone-httpd.conf.j2  https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/88815810:28
opendevreviewMerged openstack/openstack-ansible-os_nova stable/2023.1: Apply always tag to nova_virt_detect.yml  https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/88787810:31
opendevreviewMerged openstack/openstack-ansible-galera_server stable/2023.1: Add optional compression to mariabackup  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/88713910:43
opendevreviewMerged openstack/openstack-ansible stable/2023.1: Adjust default value for *_backend_ssl  https://review.opendev.org/c/openstack/openstack-ansible/+/88768311:01
opendevreviewMerged openstack/openstack-ansible stable/2023.1: Pin version of setuptools  https://review.opendev.org/c/openstack/openstack-ansible/+/88833211:01
noonedeadpunkok, zed also seems "fixed" with https://review.opendev.org/c/openstack/openstack-ansible/+/888436 and https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/88768011:06
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/88821611:44
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Fix linters and metadata  https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/88822711:48
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Fix linters issue and metadata  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/88845511:58
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Fix linters issue and metadata  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/88813212:03
noonedeadpunkhuh, looking at what takes time in our CI jobs, and that looks like taking quite a lot for such simple task? https://paste.openstack.org/show/b5uS80Y5DBfPjLhOMtqN/12:12
noonedeadpunklike 6sec for pasting a file12:12
jrosserthats surprising12:13
jrosserjust copy module i think12:13
noonedeadpunkbut then same job for rally is fast12:14
noonedeadpunkhttps://paste.openstack.org/show/bQ9njmsVGIzjzDEZfaGE/12:14
jrosseri guess we have no idea what is happening with the storage at that moment12:15
jrossercould be anything12:15
noonedeadpunkyeah, true12:18
noonedeadpunkI'm not sure dstat shows iostat (I bet it's not)12:18
noonedeadpunkah, it does, but we indeed don't know what is on layer below12:19
noonedeadpunkhttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_906/886527/12/check/openstack-ansible-deploy-aio_metal-rockylinux-9/90602d2/logs/openstack/instance-info/dstat.html12:19
jrosseransible should be using real tmpfs for all intermediate file operations though12:23
jrosserperhaps we have some case where that doesnt happen and we use real disk12:23
noonedeadpunkI assume we're disabling swap in our jobs12:28
noonedeadpunkanyway.12:29
noonedeadpunkdamiandabrowski: fwiw, this new approach with haproxy_ackends configured per playbook acting really crazy for metal deployments - it would configure everything when running anything12:31
noonedeadpunkas metal host is part of all API groups basically, I guess12:32
damiandabrowskiahh :/ will look into that12:34
noonedeadpunkbut it could be my weird sandbox with ansible 2.15 as well :D12:34
noonedeadpunkactually, looking at current logs - it seems to work as expected12:43
noonedeadpunkSo indeed it might got broken jsut with ansible-core 2.15.112:44
noonedeadpunk(logs in CI)12:44
mgariepyhnn12:49
mgariepyi think i miss a part for my haproxy patch.12:50
mgariepythe haproxy_check_port probably need to match the other one12:50
mgariepyhttps://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/templates/service.j2#L9-L13 this won't do it since it won't monitor the backend correclty 12:54
mgariepyhttps://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/88831412:54
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Reduce output by leveraging loop labels  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/87630212:55
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_glance master: Apply tags to systemd_service include  https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/88845812:58
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Apply tags to included tasks  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/88846113:02
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Apply tags to included tasks  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/88846113:02
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Fix linters and metadata  https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/88822713:04
jamesdentonas a git newb, how does one resolve these types of failures when running bootstrap-ansible.sh? https://paste.opendev.org/show/bWo13GvOVpbeUJIJ3Iyt/   -- "Failed to download remote objects and refs:  fatal: error in object: unshallow xxxx"13:18
opendevreviewMerged openstack/openstack-ansible-openstack_openrc master: Update role metadata  https://review.opendev.org/c/openstack/openstack-ansible-openstack_openrc/+/88823113:22
Sage905jamesdenton: What OS / version are you running it on?13:23
jamesdenton22.04.213:23
jrosseryou can perhaps try to make this larger https://github.com/openstack/openstack-ansible/blob/ea0bbd2c582b313fc7174cc714a52e17e27e6e14/scripts/get-ansible-role-requirements.yml#L16713:23
mgariepyjamesdenton, https://github.com/openstack/openstack-ansible/blob/master/ansible-role-requirements.yml it's a configuration from there: 13:23
mgariepyyou try to upgrade an existing env ?13:24
jamesdentonno, this was a 2023.1 install from the beginning about a week ago13:24
jrosserit might be that when the number of commits in a repo beyond the shallow-since date is more than `role_clone_default_depth`, then the SHA is question is not available to check out13:25
jamesdentoni will bump to 40 and see if that makes a difference, thank you13:26
noonedeadpunkI think that depth should be ignored by parallel clone when shallow_since is provided13:30
noonedeadpunkAnd legacy (non-parallel) ignores shallow_since13:30
noonedeadpunkjamesdenton: I think it's due to pacemaker and etcd actually13:31
jamesdentonparallel is failing, too.13:31
jamesdentonyes, pacemaker, etcd, neutron andf tempest13:31
noonedeadpunkas they're master. We have a bug report for that, and it will be fixed in 27.0.1 - already merged to stable/2023.113:31
jamesdentonhmm, well lemme check i just did a pull on stable/2023.113:32
noonedeadpunkhttps://bugs.launchpad.net/openstack-ansible/+bug/202551313:32
jamesdentoni ended up recloning it to get the latest bits - changes seem to be there now, thanks13:37
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_adjutant master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/88846913:55
opendevreviewMerged openstack/openstack-ansible-rabbitmq_server master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/88823313:56
opendevreviewMerged openstack/openstack-ansible-memcached_server master: Fix linters issue and metadata  https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/88814613:57
opendevreviewMerged openstack/openstack-ansible-lxc_container_create master: Fix linters issue and metadata  https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/88817913:59
opendevreviewMerged openstack/openstack-ansible master: Allow to skip roles/collections bootstrap separately  https://review.opendev.org/c/openstack/openstack-ansible/+/88828314:00
opendevreviewMerged openstack/openstack-ansible master: Remove haproxy_accept_both_protocols from repo_all  https://review.opendev.org/c/openstack/openstack-ansible/+/88658614:04
hamburglerdamiandabrowski: ty :) re:ssl14:05
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_aodh master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/88847114:11
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_barbican master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-os_barbican/+/88847214:25
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Fix linters issue and metadata  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/88813214:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Fix linters issue and metadata  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/88845514:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_blazar master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-os_blazar/+/88847314:41
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_ceilometer master: Fix linters and metadata  https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/88847615:01
*** dviroel_ is now known as dviroel15:22
noonedeadpunkTbh, I start thinking how wise is that to set "0640" for local ansible facts...15:40
noonedeadpunkLike I do....15:41
noonedeadpunkMaybe worth reverting that back to 0644 until it's not tooooooo late15:41
opendevreviewMerged openstack/openstack-ansible-os_zun master: Switch jobs to Jammy  https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/88436216:18
damiandabrowskinoonedeadpunk: I spawned metal AIO on master branch and ubuntu jammy, executed gate-check-commit.sh and everything looks fine16:39
damiandabrowskieach playbook configures only its own haproxy services16:39
damiandabrowskiex. https://paste.openstack.org/show/b6Pzag6prypbiqiZhtZt/16:39
opendevreviewMarc Gariépy proposed openstack/openstack-ansible-haproxy_server master: Add ability to have different backend port.  https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/88831417:18
mgariepyhmm that's not quite flexible enough :/ https://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/templates/service.j2#L55 18:20
mgariepyor i'm not doing something supported.. :/ haha18:21
mgariepyhttps://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/templates/letsencrypt_renew_certbot_distro.j2#L5-L718:21
mgariepyI added both domains to this list: haproxy_ssl_letsencrypt_domains 18:22
mgariepyadded another ip via : keepalived_instances_overrides18:23
mgariepythen i add haproxy_extra_services and bind to the other domain.18:24
opendevreviewMarc Gariépy proposed openstack/openstack-ansible-haproxy_server master: Add possibility to override haproxy_ssl_path  https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/88849819:12
mgariepythis fixes my issue..19:14
mgariepycould be better tho. we could generate the path the same way in the letsencrypt renewal hook and the haproxy template :D19:14

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!