| *** krumelmonster_ is now known as krumelmonster | 07:43 | |
| jrosser | morning | 08:12 |
|---|---|---|
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Replace use of deprecated ANSIBLE_CALLBACK_WHITELIST https://review.opendev.org/c/openstack/openstack-ansible/+/829002 | 08:44 |
| jrosser | slightly surprised that this is passing https://review.opendev.org/c/openstack/openstack-ansible/+/828267 | 09:31 |
| jrosser | i would expect it to fail in an lxc job on os_cinder trying to install kmod-devel | 09:53 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Remove unwanted repositories installed with the zuul configure-mirrors role https://review.opendev.org/c/openstack/openstack-ansible/+/828267 | 10:28 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Disable unwanted repositories from the zuul configure-mirrors role https://review.opendev.org/c/openstack/openstack-ansible/+/828267 | 10:33 |
| *** dviroel_ is now known as dviroel | 11:18 | |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts master: Simplify installtion of libpython https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/827458 | 11:19 |
| admin1 | morning .. so over the weekend, i decided to upgrade to 24.1.0 .. and in the 1st try .. it did upgrade a few components .. while getting stuck at gnocchi .. i control C .. .. now when i want to run the playbooks again, in this task TASK [python_venv_build : Create the virtualenv (if it does not exist)] .. it gives the error i am getting: | 11:32 |
| admin1 | virtualenv --no-download --python=python3 --always-copy /openstack/venvs/keystone-24.0.1", "msg": "[Errno 2] No such file or directory: b'virtualenv'", "rc": 2, "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} .. this is the same error when i try to run any playbooks. | 11:32 |
| admin1 | so os glance or anything also complains now about virtualenv not found | 11:32 |
| admin1 | i am unable to push changes and updates due to this | 11:32 |
| jrosser | admin1: you mean 24.0.1? | 11:41 |
| jrosser | which OS? | 11:41 |
| admin1 | yes .. sorry .. typo in my side | 11:41 |
| admin1 | ubuntu-focal | 11:41 |
| jrosser | root@infra1-keystone-container-8ea445de:~# which virtualenv | 11:43 |
| jrosser | /usr/bin/virtualenv | 11:43 |
| jrosser | do you have that? | 11:43 |
| jrosser | admin1: ^ | 11:58 |
| jrosser | noonedeadpunk: could do with a SHA bump on master if you have a moment | 12:06 |
| jrosser | this unwedges nova on rocky linux https://review.opendev.org/c/openstack/nova/+/822749 | 12:06 |
| jrosser | NeilHanlon: i got a tempest pass on rocky/lxc \o/ | 12:30 |
| admin1 | which virtualenv returns empty | 12:30 |
| admin1 | i did destory /create on keystone after it was on 24.0.1 | 12:31 |
| jrosser | admin1: the openstack_hosts role should install python3-virtualenv https://github.com/openstack/openstack-ansible-openstack_hosts/blob/stable/xena/vars/ubuntu-20.04.yml#L53 | 12:36 |
| jrosser | really there is no need to destroy/create the containers | 12:37 |
| jrosser | however, even if you do, the container create playbook should re-run the openstack_hosts role https://github.com/openstack/openstack-ansible/blob/master/playbooks/containers-lxc-create.yml#L66-L78 | 12:38 |
| jrosser | so i would guess that you had a failure during the containers create playbook which did not get resolved before carrying on to deploy keystone | 12:40 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Fix condtional for powertools/crb repo installation https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/829019 | 12:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use native ansible module to control dnf module_hotfixes https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/829020 | 12:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Enable powertools/crb repository for the repo server https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/829021 | 12:41 |
| admin1 | jrosser, i also updated this https://review.opendev.org/c/openstack/openstack-ansible-os_gnocchi/+/822905 ... .. it is missing redis | 12:58 |
| damiandabrowski[m] | that's weird, during distro upgrades on shared_infra hosts, every time rabbitmq_user tries to ensure that specified user exists, it fails because it already exists...? | 13:00 |
| damiandabrowski[m] | have You seen it before guys? | 13:00 |
| damiandabrowski[m] | `rabbitmqctl exited with non-zero code: Error:\nUser \"keystone\" already exists\n` | 13:00 |
| damiandabrowski[m] | i'm running 22.3.2, tried to update rabbitmq ansible collection from 1.0.1 to 1.1.0 but did not help | 13:01 |
| jrosser | damiandabrowski[m]: i do remember something about the rabbitmq ansible module getting out of step with what rabbitmqctl (?) returned | 13:13 |
| damiandabrowski[m] | thanks for Your input! i didn't have any other ideas so I just restarted rabbitmq cluster and....it helped lol | 13:18 |
| jrosser | hah | 13:18 |
| NeilHanlon | jrosser: nice! | 15:13 |
| jrosser | it was not too bad in the end | 15:14 |
| jrosser | i had to make vars/rocky.yml in lxc_hosts | 15:15 |
| NeilHanlon | yeah i had been messing with such things as well as patching os_nova | 15:15 |
| jrosser | ah well | 15:16 |
| jrosser | nova is currently broken by this https://review.opendev.org/c/openstack/nova/+/822749 | 15:16 |
| jrosser | well, fixed, i mean | 15:16 |
| NeilHanlon | oh, nice. i can remove this then https://paste.opendev.org/show/b5yuCOsWUdmkP1gwF6oR/ | 15:17 |
| jrosser | so i've had to update the SHA for openstack/nova and openstack/requirements in playbooks/defaults/repo_packages/openstack_services.yml | 15:17 |
| jrosser | correct | 15:17 |
| jrosser | the real fix is to pull in a newer version of nova with that problem addressed | 15:17 |
| NeilHanlon | ohh i bet the openstack/requirements is what I was running into.. i had only been setting `nova_git_install_branch` to master | 15:18 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Enable powertools/crb repository for the repo server https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/829021 | 15:20 |
| jrosser | NeilHanlon: then theres a stack of 3 patches there ^ for openstack_hosts | 15:21 |
| jrosser | after you update the git SHA for requirements_git_install_branch there is a hidden banana skin | 15:21 |
| NeilHanlon | 😍 | 15:21 |
| NeilHanlon | it really IS valentines day! :P | 15:22 |
| jrosser | you need to re-run the repo-server role :( because it caches a copy of upper-constraints.txt there on the local web server | 15:22 |
| NeilHanlon | ah | 15:22 |
| jrosser | thats a bit ugly, but that was all part of fixing a rumble of CI failures when downloading it from opendev.org occasionally broke | 15:23 |
| jrosser | i think that this is all very very close to working when the rocky nodes are available, both for metal and lxc deploys | 15:24 |
| jrosser | there is still something funky with the handling of the powertools repo, and i made this https://review.opendev.org/c/zuul/zuul-jobs/+/829028 | 15:25 |
| NeilHanlon | metal appears to be good from what I can tell. i set no_containers: true on my distributed test bed last night (three nodes, infra, a nova server, and a cinder server) and it worked just swimmingly | 15:25 |
| jrosser | that should hopefully close the gap between what happens in CI and what happens in the cloud-image vm | 15:25 |
| jrosser | thats excellent | 15:25 |
| NeilHanlon | great re: zuul jobs and mirrors. looks like a good change | 15:26 |
| *** dviroel is now known as dviroel|lunch | 15:26 | |
| jrosser | i am a bit surprised we didnt run into more things like (foo == 'centos') which would have needed generalising | 15:32 |
| NeilHanlon | I'm glad about that | 15:32 |
| NeilHanlon | heh | 15:32 |
| NeilHanlon | what did you end up needing in your lxc_hosts vars/rocky.yml ? | 15:34 |
| NeilHanlon | https://paste.opendev.org/show/b3Hm8rFXHdMJ22cDKTsa/ this is what i had been using | 15:34 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Allow redhat.yml to support any distribution and major release https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/829062 | 15:39 |
| jrosser | ^ like that being hardwired for centos | 15:39 |
| jrosser | well, i was on a bit of a hack to see if it worked at all so i copied the redhat.yml and made the same change to the dnf command as you've done | 15:42 |
| jrosser | what would be excellent is if we could fix up vars/redhat.yml so that it works for any rhel-ish distro | 15:43 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Allow redhat.yml to support any distribution and major release https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/829062 | 16:05 |
| NeilHanlon | yeah i have a branch that i'm working on making the lxc_container_create more general | 16:16 |
| NeilHanlon | did lxc yell at you about a missing file in /var/lib/lxc/ ? | 16:16 |
| jrosser | no, it didnt moan | 16:17 |
| *** dviroel|lunch is now known as dviroel | 16:24 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Generalise redhat variables to handle any distro and major release https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/829106 | 16:39 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Allow redhat.yml to support any distribution and major release https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/829062 | 16:54 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Do not install extra repos with the zuul configure_mirrors role https://review.opendev.org/c/openstack/openstack-ansible/+/829111 | 16:59 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Do not install extra repos with the zuul configure-mirrors role https://review.opendev.org/c/openstack/openstack-ansible/+/829111 | 17:00 |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts stable/victoria: Ensure that the legacy network-scripts package is present https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/828237 | 18:19 |
| opendevreview | Merged openstack/openstack-ansible-repo_server stable/xena: Ensure insist=true is always set for lsyncd https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/828835 | 18:42 |
| opendevreview | Merged openstack/openstack-ansible-os_keystone stable/xena: Drop ProxyPass out of VHost https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/828838 | 19:09 |
| admin1 | another issue observed on os-nova .. 24.0.1 -> resolution) from re", "alembic.util.exc.CommandError: Can't locate revision identified by 'b30f573d3377'" | 19:47 |
| admin1 | TASK [os_nova : Synchronize the nova API DB schema | 19:48 |
| spatel | admin1 are you upgrading from W-> X ? | 19:51 |
| admin1 | spatel, yes | 21:07 |
| admin1 | sorry for the late response | 21:07 |
| spatel | i also did upgrade but had no issue | 21:07 |
| *** dviroel is now known as dviroel|out | 21:17 | |
| *** fdegir16 is now known as fdegir | 23:25 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!