| *** akahat|out is now known as akahat|ruck | 05:15 | |
| jrosser_ | morning | 07:05 |
|---|---|---|
| noonedeadpunk | mornings! | 07:17 |
| noonedeadpunk | btw I see these OSError quite frequently nowadays | 07:19 |
| jrosser_ | for neutron in particular? | 07:22 |
| jrosser_ | anyway - we merged a lot of stuff | 07:23 |
| jrosser_ | but still a few to go | 07:23 |
| noonedeadpunk | mmm, can't recall. Once it was for neutron for sure, but can't say if it was _only_ for neutron | 07:24 |
| jrosser_ | wallaby is kind of broken for bionic https://zuul.opendev.org/t/openstack/build/b3c78680f6354aadbec7f050a8cfdf72/log/job-output.txt#8144 | 07:24 |
| noonedeadpunk | Yeah, on rabbit | 07:25 |
| jrosser_ | did we have a patch to switch the repos to cloudsmith | 07:25 |
| noonedeadpunk | I wonder where does erlang 25 come from | 07:26 |
| noonedeadpunk | yes, but it fails as well | 07:26 |
| noonedeadpunk | I think I will spawn AIO to play with | 07:26 |
| jrosser_ | ah yes https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/846122 | 07:26 |
| jrosser_ | yeah there is something pretty wrong there | 07:26 |
| jrosser_ | POST_FAILURE has kind of gone away | 07:27 |
| jrosser_ | though this is just as possible to be a swift backend being quick rather than any of the patches i made | 07:27 |
| jrosser_ | log uploads are taking between 2 and 20 minutes so there is something hugely variable still going on | 07:27 |
| jrosser_ | it would be worth double checking the backport situation | 07:28 |
| jrosser_ | i did a few, but may have missed some | 07:28 |
| jrosser_ | and then there is how far back do we want to go with these things | 07:29 |
| noonedeadpunk | My guess is that some provider got minorly faster that now allows jobs to pass... As this haven't been backported to Xena or Wallaby, but they don't have post failures either | 07:35 |
| noonedeadpunk | And I don't think it's worth going lower then W anyway with these backports. Most of them are even for Xena | 07:35 |
| jrosser_ | i've also spent a bunch of time on our grafana dashboards | 07:37 |
| jrosser_ | and looking at adding extra debug for these POST_FAILURES into the zuul jobs/roles | 07:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/wallaby: Use cloudsmith repo for rabbit and erlang https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/846122 | 09:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/wallaby: Use cloudsmith repo for rabbit and erlang https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/846122 | 09:43 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun stable/yoga: Move etcd installation to playbook https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/848062 | 09:53 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila stable/xena: Delegate share type creation to setup host https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/848063 | 09:55 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun stable/yoga: Move etcd installation to playbook https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/848062 | 09:57 |
| *** dviroel|out is now known as dviroel | 11:20 | |
| opendevreview | shahab taee proposed openstack/openstack-ansible-os_neutron stable/xena: Allow to provide custom configuration for VPNaaS https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/848066 | 11:34 |
| opendevreview | shahab taee proposed openstack/openstack-ansible-os_neutron stable/wallaby: Allow to provide custom configuration for VPNaaS https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/848067 | 11:35 |
| noonedeadpunk | jrosser_: so rabbitmq now seems passing with cloudsmith, but it required to bump erlang version https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/846122 | 11:45 |
| noonedeadpunk | and with old repo locally I failed to verify repo signature with the key they've provided... | 11:46 |
| noonedeadpunk | which is super weird and I'd avoided using it given that tbh,,, | 11:46 |
| jrosser_ | noonedeadpunk: there are a bunch of other jobs (forgot which one) where there was repo signing error stuff | 11:50 |
| jrosser_ | i think that was also bionic, perhaps UCA | 11:50 |
| noonedeadpunk | well, in AIO I catched only erlang | 11:53 |
| opendevreview | Merged openstack/openstack-ansible stable/xena: Exclude /var/log/journal from log collection https://review.opendev.org/c/openstack/openstack-ansible/+/847828 | 12:22 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Run wheels build for each unique distro/arch https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/848106 | 13:09 |
| noonedeadpunk | damiandabrowski[m]: would be great if you could test and review that as bug owner ^ | 13:09 |
| jrosser_ | i've not looked at it but i am surprised that is needed | 13:13 |
| noonedeadpunk | Well, that makes sense kind of. As venv_build_targets is build against current play host https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/vars/main.yml#L44 | 13:16 |
| noonedeadpunk | And basically delegated also to specific host for each play host https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/defaults/main.yml#L127 | 13:16 |
| jrosser_ | why do we not start with the lsyncd master as the first host to upgrade? | 13:16 |
| * jrosser_ checks upgrade notes | 13:16 | |
| noonedeadpunk | Well, I think this fix is more about cross-distro/arch support then only about OS upgrades | 13:17 |
| mgariepy | so the file are not deleted when you run on the master repo ? | 13:17 |
| noonedeadpunk | as you need to build wheels for several distros as example | 13:17 |
| noonedeadpunk | I feel I need to re-phrase commit message but not sure where exactly :) | 13:18 |
| jrosser_ | the bug seems entirely about upgrades | 13:19 |
| jrosser_ | and if it's really cross-distro/arch then thats pretty broken really until master anyway | 13:19 |
| noonedeadpunk | looking on code I'd say it's broken on master still, yes | 13:20 |
| noonedeadpunk | and distro upgrade is just one of user cases where cross-distro might be needed | 13:20 |
| noonedeadpunk | *use cases | 13:20 |
| jrosser_ | i think i'm completely failing to understand | 13:20 |
| jrosser_ | the build host is already selected on OS / architecture https://github.com/openstack/ansible-role-python_venv_build/blob/master/defaults/main.yml#L121-L127 | 13:21 |
| noonedeadpunk | yes, and it's used as delegate_to destination https://github.com/openstack/ansible-role-python_venv_build/blob/master/tasks/python_venv_wheel_build.yml#L17 | 13:22 |
| noonedeadpunk | BUT, venv_build_host will be different based on the host against which play will run | 13:22 |
| noonedeadpunk | and we have run_once there, which means that just random wheels will be build | 13:22 |
| noonedeadpunk | As venv_build_targets will contain all set of arch/distros of repo_all group | 13:23 |
| noonedeadpunk | Well, currently you likely can workaround this by limiting play only to specific "group" of arch/distro | 13:24 |
| noonedeadpunk | I either bad in explaining things or just wrong in my conclusions | 13:27 |
| jrosser_ | well theres a lot of things to get confused with here | 13:30 |
| jrosser_ | our OS upgrades notes say to do one node at a time and use --limit | 13:30 |
| noonedeadpunk | let's forget about OS upgrades for a while and just consider having multiple architectures, or centos/ubuntu mixed deployment | 13:31 |
| noonedeadpunk | we have fixed repo_containers with regards to that with shared filesystem. Now wheels that are build on different targets will appear everywhere | 13:32 |
| jrosser_ | https://github.com/openstack/ansible-role-python_venv_build/blob/4a1490377ccde785d4f066f94608d18978894e0c/vars/main.yml#L38-L57 | 13:33 |
| noonedeadpunk | Yes, sure. But it's used only to select venv_build_host, right? | 13:35 |
| noonedeadpunk | And venv_build_host would be different and will depend on which inventory_hostname we're playing against? | 13:37 |
| noonedeadpunk | because we use ansible_facts there to select proper repo_container for this specific inventory_hostname | 13:37 |
| noonedeadpunk | And then, we ruin everything with run_once. | 13:37 |
| noonedeadpunk | here https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/tasks/main.yml#L67 | 13:38 |
| noonedeadpunk | so basically, if you play against set of hosts with different arch, only one type of wheels will be built | 13:38 |
| noonedeadpunk | so basically you need to build wheels only for the first host of each type | 13:39 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Switch galera_root_user default value https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/847547 | 13:40 |
| jrosser_ | right | 13:40 |
| jrosser_ | maybe some reworking of the commit message needed | 13:41 |
| noonedeadpunk | yeah, I already relized that but still not sure how to re-phrase in a betterway | 13:42 |
| jrosser_ | i think my brain is still melting trying to understand | 13:43 |
| jrosser_ | there may be multiple architecures and OS in ansible_play_hosts | 13:43 |
| jrosser_ | so we must build wheels for all of the architectures and OS in ansible_play_hosts | 13:44 |
| noonedeadpunk | yes | 13:44 |
| jrosser_ | the patch changes the code from arbitrarily picking one of them, to properly selecting each required combination | 13:45 |
| jrosser_ | i can try and rewrite the commit message? | 13:45 |
| noonedeadpunk | sure! | 13:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Run wheels build for each unique distro/arch https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/848106 | 14:08 |
| noonedeadpunk | I've updated a bit, but feel free to re-write! Update was mostly about adding release note. | 14:08 |
| noonedeadpunk | jrosser_: btw, there's another WIP thing from ebbex_ with kind of same approach https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/714483/1/vars/main.yml | 14:28 |
| noonedeadpunk | it also drops run_once in favor of that condition | 14:28 |
| noonedeadpunk | regarding backports - I think this one is close the only left https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/848062 and I can bump roles afterwards | 14:39 |
| opendevreview | Merged openstack/openstack-ansible master: Add timestamp to written journal file https://review.opendev.org/c/openstack/openstack-ansible/+/846439 | 15:09 |
| opendevreview | Merged openstack/openstack-ansible-os_zun master: Move etcd installation to playbook https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/847363 | 15:24 |
| *** dviroel is now known as dviroel|lunch | 15:24 | |
| opendevreview | Merged openstack/openstack-ansible stable/yoga: Do not try to run etcd in serial mode https://review.opendev.org/c/openstack/openstack-ansible/+/848002 | 15:29 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Run wheels build for each unique distro/arch https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/848106 | 15:37 |
| *** dviroel|lunch is now known as dviroel | 16:37 | |
| opendevreview | Merged openstack/openstack-ansible-os_zun stable/yoga: Move etcd installation to playbook https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/848062 | 18:10 |
| mgariepy | NeilHanlon, any update on the cloud image issue ? https://bugs.rockylinux.org/view.php?id=102 | 19:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Bump OpenStack-Ansible roles for RC2 https://review.opendev.org/c/openstack/openstack-ansible/+/848188 | 20:16 |
| noonedeadpunk | mgariepy: damiandabrowski[m] jrosser_ ^ would be great to land that asap... | 20:16 |
| *** dviroel is now known as dviroel|biab | 20:30 | |
| damiandabrowski[m] | done | 20:46 |
| *** dviroel|biab is now known as dviroel | 22:09 | |
| *** dviroel is now known as dviroel|out | 22:28 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!