Thursday, 2022-05-26

« Wednesday, 2022-05-25 Index Friday, 2022-05-27 »
opendevreviewMerged openstack/openstack-ansible-rabbitmq_server master: Add support for centos-9  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/82398500:00
opendevreviewMerged openstack/openstack-ansible-ceph_client stable/wallaby: Use global package_state  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/84240700:48
*** ysandeep|out is now known as ysandeep01:27
*** ysandeep is now known as ysandeep|afk02:50
noonedeadpunkI wonder if I did +W before fix got really merged06:33
opendevreviewMerged openstack/openstack-ansible-ceph_client stable/victoria: Disable setuptools auto discovery  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/84294906:42
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts stable/wallaby: Do not mess up with distro-provided machines mount  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/84339206:52
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_rally stable/victoria: Ensure that the rally venv build is self contained  https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/84339306:56
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/victoria: Switch Victoria to stable branch  https://review.opendev.org/c/openstack/openstack-ansible/+/84267406:57
*** ysandeep|rover is now known as ysandeep|rover|lunch07:35
opendevreviewMerged openstack/openstack-ansible-openstack_hosts master: Use pam_env for su commands on Centos-9  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/84298808:19
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Check the actual upper-constraints file for heathcheck-infrastructure.  https://review.opendev.org/c/openstack/openstack-ansible/+/84341508:28
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-repo_server master: Avoid a race condition between mounting and using repo filesystem  https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/84341808:34
opendevreviewMerged openstack/openstack-ansible master: [doc] Mark Bionic and Buster as unsupported for Y  https://review.opendev.org/c/openstack/openstack-ansible/+/84334008:37
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Add Centos-9 Stream support  https://review.opendev.org/c/openstack/openstack-ansible/+/82341708:54
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Bump ansible version to 2.12.6  https://review.opendev.org/c/openstack/openstack-ansible/+/84342008:56
opendevreviewMerged openstack/openstack-ansible-ceph_client stable/victoria: Use global package_state  https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/84240808:59
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-tests master: Bump ansible version to 2.12.6  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/84342108:59
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Use integrated repo 'hosts' jobs to test different backing stores  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/84302909:01
jrosser_noonedeadpunk: i didnt test out that race condition fix locally yet - was going to see what happened in zuul first09:03
jrosser_it's a bit speculative about whats actually happening tbh09:03
noonedeadpunkIt should not hurt anyway09:08
noonedeadpunkexcept cases when repo mounts are not defined09:08
jrosser_I think it currently makes the mount even for just one server09:09
*** ysandeep|rover|lunch is now known as ysandeep|rover09:23
opendevreviewMerged openstack/openstack-ansible-openstack_hosts master: Add passwd for rhel8.  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/84035310:16
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: WIP: Fix CI for elk_metrics_7x on Focal  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84342310:17
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Add Centos-9 Stream support  https://review.opendev.org/c/openstack/openstack-ansible/+/82341710:27
jrosser_something is broken with bullseye for SSL10:30
jrosser_i wonder what changed there10:30
jrosser_there are lots of these right now https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_59c/843420/1/check/openstack-ansible-deploy-aio_lxc-debian-bullseye/59c67fb/logs/ara-report/results/3325.html10:31
*** carloss_ is now known as carloss10:31
jrosser_oh right, i think i messed that up10:32
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: WIP: Fix CI for elk_metrics_7x on Focal  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84342310:32
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Fix pam_env config for debian  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/84342911:10
*** dviroel_ is now known as dviroel11:17
jrosser_oh nice https://review.opendev.org/c/openstack/skyline-apiserver/+/84290711:32
noonedeadpunkuh, it depended on poetry11:42
noonedeadpunkthat was kind of nasty....11:42
*** ysandeep|rover is now known as ysandeep|rover|break11:44
noonedeadpunkjrosser_: did you happen to make molecule work with github actions?11:51
noonedeadpunkthis is some nightmare... I've spent half of day trying to make molecule use proper python interpreter (where it's installed actually) without any success....11:53
jrosser_i havent by i think some others here might - do you have an example?12:00
noonedeadpunkhttps://github.com/noonedeadpunk/ansible-unbound/runs/660898762612:00
noonedeadpunkSo python and molecule and everyting is installed into /opt/hostedtoolcache/Python/3.8.12/x6412:01
noonedeadpunkoh wayt....12:02
noonedeadpunkah, no nothing12:02
noonedeadpunkI though it's inside container laready... 12:02
noonedeadpunkas whatever I do, it uses  /opt/pipx/venvs/ansible-core/bin/python as interpreter12:03
noonedeadpunkI think I made it work12:16
jrosser_it's sort of like this https://github.com/ansible-community/molecule/issues/248912:16
noonedeadpunknah, it was expecting to have ansible installed. And if it's not - molecule creates new venv, which don't have requirements...12:16
jrosser_ahh12:17
noonedeadpunkAnd I was trying to beat it somehow, but have up and just installing ansible as pre-step...12:17
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: elk7: minimal fixes to CI jobs  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84342312:18
noonedeadpunkBut I can't find where in the world I can pass requirements.txt for molecule to isntall to venv...12:18
noonedeadpunkanyway12:19
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: Update beat templates from 7.11 reference  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/77884912:19
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: beats: fix potential for indexes to be set up incorrectly  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84344512:20
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: Add journald input support to filebeat  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84344712:22
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: kibana: fix bug in templating variable selection  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84345112:24
opendevreviewAndrew Bonney proposed openstack/openstack-ansible-ops master: kibana: add configuration option for base url  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84345212:26
*** ysandeep|rover|break is now known as ysandeep|rover12:36
mgariepyupgrade job focal .. https://zuul.opendev.org/t/openstack/build/c5c6449d9fe04bbda75567271a57892a/log/logs/openstack/aio1_galera_container-35d09063/mariadbcheck.socket.journal-08-01-56.log.txt#112:38
mgariepygalera is down in haproxy for this reason i guess 12:39
jrosser_interesting12:40
mgariepydid we change the galera check between the 2 release ?12:40
*** mnaser_ is now known as mnaser12:41
mgariepyin the galera logs it seems all ok.12:41
mgariepyand,. ara is missing which is no fun :/12:41
jrosser_also we don't collect `ss` output inside containers which would say what was using port 920012:42
noonedeadpunkRly? I thought we did... Or that only for hosts?12:43
noonedeadpunkyeah, likely only for hosts...12:43
jrosser_i just had a quick look in that link ^^ and didnt see it, but maybe i'm looking in the wrong place12:43
jrosser_i think for these upgrade jobs we are managing the transition from xinetd to a system socket service12:43
jrosser_so its possible there is another kind of race condition there12:44
jrosser_here https://github.com/openstack/openstack-ansible-galera_server/blob/81615669c6f2fd69062918dcabb7735e1c0f7827/tasks/galera_server_post_install.yml#L16-L3612:45
mgariepyhttps://github.com/openstack/openstack-ansible-galera_server/blob/master/tasks/galera_server_post_install.yml#L32-L3612:46
mgariepydo we need xinetd ?12:46
mgariepyif the file is not there how is xinetd suppose to stop the service ? :/12:49
mgariepyhow does it knows about it.. 12:49
mgariepyhmm..12:53
jrosser_doesnt xinetd start things on demand as the sockets are connected to?12:59
mgariepyxinetd would reserve the socket i guess ?13:00
mgariepyit seems to work correctly here on a older release.13:01
jrosser_i did the "discussion" bit of this https://www.oreilly.com/library/view/linux-security-cookbook/0596003919/ch03s03.html13:01
mgariepyhmm. recheck and i'll ask infra to hold the vm.13:04
mgariepybecause it doesn't make much sense to me:)13:10
jrosser_we need to merge this to unbreak debian https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/84342913:53
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Change unbound role source  https://review.opendev.org/c/openstack/openstack-ansible/+/84346813:57
*** ysandeep|rover is now known as ysandeep|dinner14:27
jrosser_remember we did all this cleanup of setup.py on master branch https://review.opendev.org/q/topic:cleanup-setup14:41
jrosser_i have the same sort of failure here on victoria https://zuul.opendev.org/t/openstack/build/4db37e82c65344639a6d9d4cedb7c0f414:41
jrosser_how did we deal with the stable branches for this.....14:42
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_glance stable/victoria: DNM - test linters job on victoria  https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/84347614:46
mnaserjrosser_: https://github.com/mnaser/squip =P14:47
jrosser_mnaser: o/ hello14:48
mnaserhiya :)14:49
*** ysandeep|dinner is now known as ysandeep14:50
noonedeadpunkYeah, I have more dumb versin of that... https://gist.github.com/noonedeadpunk/6fb0fdd442e520a440f8cd52d77c53ce14:51
noonedeadpunkbut threaded :D14:52
* noonedeadpunk not going to show that to infra folks14:52
jrosser_i think maybe i was wondering if either 1) we never spot this on victoria 2) we just didnt ever fix it 3) theres something wierd in os_rally that triggers it14:54
jrosser_or 4) we did some alternative hack compared to master14:54
noonedeadpunkI think for stable branches we have setuptools fixed in global requirements14:54
jrosser_yeah, so linters job seems to be not using that14:55
noonedeadpunkhttps://opendev.org/openstack/openstack-ansible/src/branch/stable/victoria/global-requirement-pins.txt#L1014:55
noonedeadpunkbut yeah...14:55
noonedeadpunkWell it should, but seems it's not14:55
jrosser_i think this is "old linters" before we made it use the integrated repo14:56
noonedeadpunkyup, sure14:56
noonedeadpunkintegrated linters started landing in W I guess and wasn't covering everything14:57
* jrosser_ wishes there was a better visualisation of periodic jobs to understand this14:57
jrosser_hmm so theres probably not a low-pain way of fixing this which doesnt involve patching all the stable/victoria repos and dealing with allllll the broken things....15:00
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Align collection_file naming  https://review.opendev.org/c/openstack/openstack-ansible/+/84348315:02
mgariepyfun. not the same issue this time around.15:09
mgariepy:/15:09
mgariepyi'll try to find time this afternoon to run the test in a vm.15:09
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_rally stable/victoria: Disable setuptools auto discovery  https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/84348715:14
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_rally stable/victoria: Ensure that the rally venv build is self contained  https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/84339315:14
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Use PKI role for certificate generation  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/83906815:31
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia stable/xena: Fix distro support option for Octavia  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/84340815:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia stable/wallaby: Fix distro support option for Octavia  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/84340915:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia stable/wallaby: Fix distro support option for Octavia  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/84340915:32
opendevreviewMerged openstack/openstack-ansible-openstack_hosts master: Fix pam_env config for debian  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/84342915:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia stable/xena: Fix distro support option for Octavia  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/84340815:33
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Add tempest repos to required-repos  https://review.opendev.org/c/openstack/openstack-ansible/+/83428915:38
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Test interaction with keystone in healthcheck-infrastructure  https://review.opendev.org/c/openstack/openstack-ansible/+/84349515:49
*** frenzyfriday|ruck is now known as frenzy_friday15:54
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Use integrated repo 'hosts' jobs to test different backing stores  https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/84303015:58
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Add Centos-9 Stream support  https://review.opendev.org/c/openstack/openstack-ansible/+/82341716:03
*** ysandeep is now known as ysandeep|out16:06
jrosser_something is wierdly inconsistent with the linter https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/84341816:09
jrosser_i've seen this a few times now where it blows up on loads of things, but then not on other patches in the same repo16:10
opendevreviewMerged openstack/openstack-ansible-ops master: elk7: minimal fixes to CI jobs  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84342316:22
opendevreviewMerged openstack/openstack-ansible-ops master: beats: fix potential for indexes to be set up incorrectly  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84344516:27
opendevreviewMerged openstack/openstack-ansible-ops master: kibana: add configuration option for base url  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84345216:27
opendevreviewMerged openstack/openstack-ansible-ops master: kibana: fix bug in templating variable selection  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84345116:29
opendevreviewMerged openstack/openstack-ansible-ops master: Update beat templates from 7.11 reference  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/77884916:29
opendevreviewMerged openstack/openstack-ansible-ops master: Add journald input support to filebeat  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/84344716:29
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-repo_server master: Avoid a race condition between mounting and using repo filesystem  https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/84341817:58
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Add Centos-9 Stream support  https://review.opendev.org/c/openstack/openstack-ansible/+/82341718:05
noonedeadpunkwhat's weird with linters, as they should take config from integrated repo. but then for warn list report only `experimental` and `no-changed-when` while in fact should be way more https://opendev.org/openstack/openstack-ansible/src/branch/master/.ansible-lint19:22
noonedeadpunkah, no....19:22
noonedeadpunkI read that wrong19:22
noonedeadpunkSo it was blaiming `../../../../../../etc/ansible/roles/repo_server/tasks/repo_pre_install.yml:80 Task/Handler: Wait until {{ repo_service_home_folder }} is a mounted` there only19:23
noonedeadpunkWe just print all warnings always, which is super confusing19:23
noonedeadpunkso it was failing for reason here :)19:24
mgariepylooks like race condidition for galera stuff. 19:32
noonedeadpunkdoes anybody recall what ansible needs to collect network facts? iproute2?19:39
spatelnoonedeadpunk any update here - https://bugs.launchpad.net/openstack-ansible/+bug/197564319:50
spatelcurrently i am using hacks but would like to have good solution for that bug19:50
noonedeadpunkI bet we've backported smth....19:50
noonedeadpunkbut likely that was smth different...19:51
jrosser_noonedeadpunk: looks like it https://github.com/ansible/ansible/pull/70796/commits/6a7c2597734cd41842a79d9bb26fca4d32329ace19:51
noonedeadpunkYeah already found here https://github.com/ansible/ansible/blob/ccdb552d90363a6160ddbf6858ffc3828ed3363c/lib/ansible/module_utils/facts/network/linux.py#L4919:52
mgariepyho. not a race..19:53
noonedeadpunkI guess we discussed on a meeting and did nothing at the end?19:53
noonedeadpunkok, I guess I'm tired... I created a list of packages and for name hardcoded jsut package name isntead of "{{ item }}". Was fighting for last hour with that...19:54
mgariepyjrosser_, https://zuul.opendev.org/t/openstack/build/ce67c16a4cac4d29994aee390aeb05df/log/job-output.txt#2252019:55
mgariepythe socket doesn't start on restart of the container.19:55
jrosser_ooooohhhh19:55
jrosser_doh19:56
jrosser_how are those jobs different to the normal upgrade jobs then19:58
mgariepy2022-05-26 19:07:01.199964 | ubuntu-focal | ### NOW RUNNING: /home/zuul/src/opendev.org/openstack/openstack-ansible/scripts/upgrade-utilities/galera-cluster-rolling-restart.yml19:59
mgariepy2247019:59
jrosser_https://github.com/openstack/openstack-ansible-galera_server/blob/master/tasks/galera_server_post_install.yml#L5019:59
jrosser_hmm19:59
mgariepyhaha.20:00
mgariepyi'll try to find the good depend on mariadb stuff.20:00
jrosser_but https://github.com/openstack/openstack-ansible-galera_server/blob/master/tasks/galera_server_post_install.yml#L5520:01
jrosser_i think this is quite wierd behaviour with the service and the socket20:01
jrosser_i had a lot off difficulty to get systemd_service to place the units in a way that didnt just fail immediately20:02
noonedeadpunkyeah, role is a bit overcomplicated now.20:05
noonedeadpunk(a lot overcomplicated)20:05
mgariepyit's unclear to me how the @.service and the .socket are linked togetter .. 20:05
noonedeadpunkand I bet there're several race conditions in the way...20:05
jrosser_https://mgdm.net/weblog/systemd/20:06
jrosser_and https://mgdm.net/weblog/systemd-socket-activation/20:06
jrosser_probably most interesting is seeing what systemctl status of both of them is after restarting the container20:09
jrosser_compared to before20:10
mgariepyi'm still in the instance. 20:13
mgariepyhttps://paste.openstack.org/show/bbyyYR3TGL7PnOCQffqR/20:21
mgariepyhttps://paste.openstack.org/show/binUXCNaHTbFIYM66FPU/20:21
mgariepyafter the restart ^^20:21
mgariepykinda weird..20:22
mgariepylate enough for me i'll continue dig on that tomorrow.20:22
opendevreviewMerged openstack/openstack-ansible-os_sahara master: Use common service setup tasks from a collection rather than in-role  https://review.opendev.org/c/openstack/openstack-ansible-os_sahara/+/84067220:30
opendevreviewMarc Gariépy proposed openstack/openstack-ansible-galera_server master: Fix race on boot for mariadb-check socket  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/84354720:35
mgariepylet's try that ^^20:35
mgariepyjrosser_,  noonedeadpunk that's the one that made the issue appears. https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843284 20:49
*** dviroel is now known as dviroel|out21:00
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Use integrated repo 'hosts' jobs to test different backing stores  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/84302921:29
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Use integrated repo 'hosts' jobs to test different backing stores  https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/84303021:30
mgariepyplease merge whne you have a minute to unblock the repo patches :D https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/84354722:43
« Wednesday, 2022-05-25 Index Friday, 2022-05-27 »

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!