| opendevreview | Merged openstack/openstack-ansible-os_placement stable/yoga: Install git into placement containers https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859162 | 00:07 |
|---|---|---|
| opendevreview | Merged openstack/openstack-ansible-os_placement stable/xena: Install git into placement containers https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859163 | 00:07 |
| opendevreview | Merged openstack/openstack-ansible-os_keystone stable/yoga: Bootstrap when running against last backend https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/859232 | 00:09 |
| opendevreview | Merged openstack/openstack-ansible-os_placement stable/wallaby: Install git into placement containers https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859164 | 01:35 |
| *** ysandeep|out is now known as ysandeep | 01:45 | |
| *** ysandeep is now known as ysandeep|afk | 03:42 | |
| *** ysandeep|afk is now known as ysandeep | 05:14 | |
| opendevreview | Merged openstack/openstack-ansible master: Cleanup py27 support https://review.opendev.org/c/openstack/openstack-ansible/+/853110 | 08:24 |
| *** ysandeep is now known as ysandeep|sick | 08:27 | |
| jrosser_ | damiandabrowski: where are these variables used? https://review.opendev.org/c/openstack/openstack-ansible-os_barbican/+/857753 | 09:56 |
| noonedeadpunk | they're not | 10:07 |
| noonedeadpunk | and already dropped on Y | 10:08 |
| damiandabrowski | ah, that's a fair point :D i just used codesearch to find where 'TLSv1.0' is defined | 10:19 |
| noonedeadpunk | NeilHanlon: fwiw I was able to reproduce issue from Rocky9 image | 10:29 |
| noonedeadpunk | specifically this one: https://dl.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud-9.0-20220830.0.x86_64.qcow2 | 10:30 |
| noonedeadpunk | https://paste.openstack.org/show/bKW6DhMDF1Q9Oc7H8zrq/ | 10:30 |
| noonedeadpunk | jrosser_: btw I'm not sure that this https://docs.ansible.com/ansible/latest/collections/ansible/posix/selinux_module.html module also has switched to using c bindings... | 10:34 |
| noonedeadpunk | as I guess we're mixing up on how ansible deals with selinux vs when we want to manage selinux with ansible | 10:35 |
| noonedeadpunk | https://github.com/ansible-collections/ansible.posix/blob/main/plugins/modules/selinux.py#L99-L104 | 10:36 |
| noonedeadpunk | So I guess it's jsut matter that centos ci image don't have selinux enabled that we don't occur this issue there | 10:37 |
| noonedeadpunk | NeilHanlon: also regarding that ceph bug I talked about yestarday - Rocky 9 image won't have that problem ,as discovered_interpreter_python is among facts there | 10:41 |
| noonedeadpunk | And it's Rocky 8 that's affected I believe | 10:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Add Rocky Linux 9 to zuul and docs https://review.opendev.org/c/openstack/openstack-ansible/+/857191 | 11:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Replace usage of which with command https://review.opendev.org/c/openstack/openstack-ansible/+/859550 | 11:22 |
| *** dviroel|afk is now known as dviroel | 11:22 | |
| noonedeadpunk | I hope that will fix Rocky 9 jobs... | 11:24 |
| noonedeadpunk | aio runs nicely through now. as well as in CI | 11:55 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-plugins stable/yoga: Introduce variables for rocky linux 9 support in gluster https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/859424 | 11:57 |
| noonedeadpunk | hm, btw, I can bet I saw discovered_python_interpreter when bootstrapping host, but I couldn't see it afterwards | 12:02 |
| noonedeadpunk | Oh...... | 12:03 |
| noonedeadpunk | I know - it's result of OSA_ANSIBLE_PYTHON_INTERPRETER being defined | 12:03 |
| noonedeadpunk | As it simply disables interpreter discovery | 12:07 |
| NeilHanlon | noonedeadpunk: catching up now, but seems like we'll have some zuul results soon | 12:50 |
| NeilHanlon | ty for looking into that. i've been swamped | 12:51 |
| noonedeadpunk | np, I just need to get time to see how to workaround this discovered_python_interpreter issue. But at least now I know what triggers it | 12:51 |
| noonedeadpunk | NeilHanlon: fwiw https://zuul.opendev.org/t/openstack/build/9d83849c67d74adc8bc84760b08ffda8 | 12:52 |
| NeilHanlon | nice i was just going looking for that :) | 12:53 |
| NeilHanlon | hopefully this will be easy to backport if we want to | 12:53 |
| noonedeadpunk | yeah, seems like it is waaay easier then rocky 8 | 12:54 |
| NeilHanlon | :D | 13:07 |
| NeilHanlon | are there any docs/guides on backporting? I think i may have asked this already | 13:08 |
| noonedeadpunk | not sure. But idea is - use cherry-pick from gerrit menu once master is merged | 13:16 |
| noonedeadpunk | We usually don't backport new fetures, but exceptions can be made | 13:17 |
| noonedeadpunk | and it makes sense here as effort/value will prevail | 13:17 |
| opendevreview | Marcus Klein proposed openstack/ansible-config_template master: document the removal of keys in ini files https://review.opendev.org/c/openstack/ansible-config_template/+/859584 | 13:19 |
| noonedeadpunk | the only thing - we can't backport release notes. So when backporting it needs to be dropped from cherry-pick and re-added with follow-up patch | 13:20 |
| noonedeadpunk | it's due to how reno does identify to which release note should be added (as it check on SHA and where this SHA included) | 13:21 |
| NeilHanlon | ah, makes sense | 13:24 |
| kleini | https://docs.openstack.org/openstack-ansible/wallaby/admin/upgrades/distribution-upgrades.html I am testing distro upgrade in staging for a non-primary infra host. keystone deployment wants to read from repo container os-release/23.4.1/ubuntu-20.04-x86_64/wheels. This does only exist on the non-primary repo container but haproxy sends requests to primary repo container. Should primary repo container be put into | 13:34 |
| kleini | maintenance when deploying non-primary infra hosts or should content of non-primary repo container be synced to primary? | 13:34 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Add Rocky Linux 9 to zuul and docs https://review.opendev.org/c/openstack/openstack-ansible/+/857191 | 13:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Replace usage of which with command https://review.opendev.org/c/openstack/openstack-ansible/+/859550 | 13:40 |
| kleini | answering my own question: rsync just receives files to be synced for repo and lsync sends them. lsync only runs on primary repo container and therefore ubuntu-20.04 is not synced from non-primary to primary. So the guide misses the point to put primary repo into maintenance mode in haproxy when running setup-openstack.yml on non-primary infra hosts | 13:54 |
| noonedeadpunk | kleini: well, I think we have mentioned that one day... | 13:56 |
| jrosser_ | i think it's even more subtle than that | 13:56 |
| noonedeadpunk | and it's already not a case on yoga as there's no lsync anymore :D | 13:56 |
| jrosser_ | you have to ensure that requests go to the right OS release infra host that is also the build host | 13:56 |
| jrosser_ | so its not just !primary | 13:56 |
| noonedeadpunk | or disable wheels build for older OS hosts, and ensure you're having serial 1 as otherwise you will ddos infra mirrors | 13:57 |
| kleini | is there some parameter to redirect the used repo container? | 13:59 |
| noonedeadpunk | I think you can put rest of backends in haproxy to maint state | 13:59 |
| *** spotz__ is now known as spotz | 14:04 | |
| kleini | or copy ubuntu-20.04 directory to primary repo | 14:06 |
| noonedeadpunk | or that, yes :) | 14:08 |
| noonedeadpunk | but well, no | 14:08 |
| noonedeadpunk | as it won't be populated - build will still happen on container running 20.04 | 14:09 |
| noonedeadpunk | and primary one will delete wheels from there as lsyncd runs with --delete iirc | 14:09 |
| kleini | okay, will put all other repo into maintenance. seems to be easier | 14:14 |
| *** dviroel is now known as dviroel|lunch | 15:29 | |
| opendevreview | Merged openstack/ansible-config_template master: document the removal of keys in ini files https://review.opendev.org/c/openstack/ansible-config_template/+/859584 | 15:45 |
| jrosser_ | oh looks like skyline does have a stable/zed branch | 16:29 |
| jrosser_ | thats good | 16:29 |
| jrosser_ | are we still adding more distro jobs - they are there for centos-9 and rocky-9 even though i thought we agreed at last PTG to not replace them | 16:32 |
| *** dviroel|lunch is now known as dviroel | 16:35 | |
| ThiagoCMC | Folks, OSA is creating a '/etc/default/tftpd-hpa' file in ironic-api-container without an address... Wrong line: "TFTP_ADDRESS=address_undefined:69", I looked the default value and it's just "ansible_host", which clearly doesn't work | 16:39 |
| noonedeadpunk | I think ansible_host is always defined... | 16:41 |
| noonedeadpunk | Quite wierd if it's not | 16:41 |
| noonedeadpunk | jrosser_: well... not sure if it was intentional or not for centos9 | 16:41 |
| noonedeadpunk | maybe I jsut copy-pasted | 16:41 |
| noonedeadpunk | Though I believe they should be passing once we set release to zed | 16:42 |
| noonedeadpunk | (and once packages will be released) | 16:42 |
| jrosser_ | ThiagoCMC: where do you see it should be ansible_host? | 16:44 |
| jrosser_ | is it this? https://github.com/openstack/openstack-ansible-os_ironic/search?q=ironic_tftp_server_address | 16:44 |
| ThiagoCMC | It's the 'templates/tftpd-hpa.j2' | 16:56 |
| ThiagoCMC | It's also undefined in ironic.conf | 16:57 |
| ThiagoCMC | Maybe I forgot to set `ironic_tftp_server_address`... | 17:00 |
| ThiagoCMC | Sorry, this: ironic_bmaas_address | 17:00 |
| jrosser_ | ? | 17:00 |
| jrosser_ | https://github.com/openstack/openstack-ansible-os_ironic/blob/c78e5c19f99b880ca87a408e5998ed83394eb174/defaults/main.yml#L127 | 17:01 |
| ThiagoCMC | I see it should be set automatically, but it isn't, so I'm wondering what I missed | 17:01 |
| ThiagoCMC | checking again | 17:01 |
| jrosser_ | you've got an eth15 in the container? | 17:01 |
| jrosser_ | i.e br-bmaas is correctly wired to an interface in the lxc? | 17:02 |
| ThiagoCMC | On the Ironic PAI, no, it doesn't have eth15. | 17:02 |
| ThiagoCMC | *API | 17:02 |
| ThiagoCMC | I'm trying without Ironic inspector containers now, but can't make it work with just apt and compute containers... | 17:03 |
| jrosser_ | i'm not sure that is going to work | 17:03 |
| ThiagoCMC | Ok | 17:04 |
| ThiagoCMC | I'll put it back... | 17:04 |
| ThiagoCMC | But, why TFTP is in Ironic API container? | 17:04 |
| jrosser_ | becasue the people who worked on os_ironic only did a metal deploy, i think | 17:05 |
| jrosser_ | and the whole thing is a mess | 17:05 |
| ThiagoCMC | Ewww... I see | 17:05 |
| jrosser_ | on a metal deploy ironic and inspector containers all collapse onto the same host and then things "work" | 17:05 |
| ThiagoCMC | Makes sense... I managed to make one machine to request IP, Inspector's dnsmasq answered, but then, it failed to download the kernel/ramdisk, didn't booted. | 17:06 |
| ThiagoCMC | So I guess that it's because TFTP is "too far", not reachable | 17:07 |
| ThiagoCMC | Getting there! lol | 17:07 |
| jrosser_ | becasue probably the http server for downloading those things is in the ironic api container (?) | 17:07 |
| jrosser_ | and if you don't have br-bmaas wired there then it's not going to work | 17:07 |
| jrwr | DHCP and "next-server" is what says where the tftp server needs to be | 17:07 |
| ThiagoCMC | Well, there's that too... | 17:07 |
| ThiagoCMC | :-D | 17:07 |
| jrosser_ | here is my networks in ironic-api https://paste.opendev.org/show/bvikYWM40n48ufnP7nFS/ | 17:08 |
| ThiagoCMC | But the documented "group_binds" for "br-bmaas" doesn't include ironic_api, only ironic_inspector | 17:08 |
| jrosser_ | right, but it's a mess | 17:08 |
| ThiagoCMC | Hmmmmm lol | 17:08 |
| jrosser_ | we are about to tear down ironic from our lab having made it work | 17:08 |
| jrosser_ | and generate a new load of patches | 17:09 |
| ThiagoCMC | Sounds awesome! I'd love to help testing! | 17:09 |
| jrosser_ | here is the inspector container networks https://paste.opendev.org/show/bdtr7JlVglcmfoqFQvBL/ | 17:09 |
| ThiagoCMC | Cool, from what I'm seeing, only thing missing here now is the eth15 on Ironic API container. | 17:10 |
| ThiagoCMC | I'll try that! | 17:10 |
| ThiagoCMC | Perhaps we could polish OSA Ironic for next 'stable/zed' release! ^_^ | 17:10 |
| ThiagoCMC | What would be the "group_binds" to add br-bmaas to ironic-api ? | 17:11 |
| ThiagoCMC | It's 'ironic_api', got it | 17:13 |
| jrosser_ | here is user_variables_ironic.yml https://paste.opendev.org/show/bDho5Orn9rdBNqjH5ito/ | 17:13 |
| jrosser_ | but note that this is not an "example" as it includes a bunch of hacks needed for things that i've not properly patched yet in os_ironic | 17:13 |
| ThiagoCMC | Thank you! | 17:13 |
| ThiagoCMC | Found this on master: https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio.j2#L137 - Gonna review everything! Learning a lot, thanks agai! | 17:14 |
| jrosser_ | then Ironic_Network is br-bmaas, and it's also available as a vlan neutron network on the network nodes | 17:15 |
| jrosser_ | as in our setup there is a mixture of ironic container dnsmasq dhcp (for things that are discovered) and neutron dhcp (for nodes that are already enrolled) | 17:15 |
| ThiagoCMC | That sounds complicated... But interesting idea! | 17:17 |
| jrosser_ | this is the neutron network https://paste.opendev.org/show/bfhHiPOfqtNXrPD1j1ZV/ | 17:17 |
| jrosser_ | well it is complicated, but thats how ironic works | 17:17 |
| jrosser_ | or how it can work - trouble is this is all configurable / pluggable and some architecture decisions have to be made somewhere | 17:18 |
| jrosser_ | you can't dhcp a thing with neutron that neutron does not already know about | 17:18 |
| ThiagoCMC | Yep, that's fair... I won't give up! =P | 17:18 |
| jrosser_ | so for discovering nodes with inspector you need something else as well | 17:18 |
| jrosser_ | and conversely, for things you know about you want to assign IP known to neutron, which you can't do with dnsmasq | 17:19 |
| jrosser_ | so it all makes sense | 17:19 |
| ThiagoCMC | Cool, I'll rewrite my lab based in your examples, thanks for sharing | 17:20 |
| jrosser_ | no problem - i would hope that we can make the majority of that user_variables_ironic go away by patching things some more | 17:20 |
| opendevreview | Merged openstack/openstack-ansible-plugins stable/yoga: Introduce variables for rocky linux 9 support in gluster https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/859424 | 17:20 |
| jrosser_ | weve also got the IPMI serial consoles working but that's needing really lots of extra work | 17:21 |
| ThiagoCMC | Sounds like a plan, I'll test it extensively. | 17:21 |
| jrosser_ | without that you are really in trouble - having our test node connected to a VGA raritan KVM was vital | 17:21 |
| jrosser_ | once you get it starting to boot but something goes wrong you need to be in front of the console of the thing | 17:22 |
| ThiagoCMC | Ok, well, right now, the BMC is reachable via separated (stable) network | 17:22 |
| ThiagoCMC | Baby steps ^_^ | 17:23 |
| jrosser_ | cool - ok i need to head off | 17:23 |
| jrosser_ | let me know if you make any progress | 17:23 |
| ThiagoCMC | Sure, thanks again! | 17:23 |
| *** dviroel is now known as dviroel|walk | 19:54 | |
| opendevreview | Merged openstack/ansible-role-python_venv_build stable/yoga: Change default value for venv_wheel_build_enable https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/859231 | 20:10 |
| *** dviroel|walk is now known as dviroel | 20:44 | |
| opendevreview | Merged openstack/openstack-ansible master: Add weight decrease to keepalived checks https://review.opendev.org/c/openstack/openstack-ansible/+/856721 | 21:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Bump OpenStack-Ansible Yoga https://review.opendev.org/c/openstack/openstack-ansible/+/859728 | 21:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/xena: Bump OpenStack-Ansible Xena https://review.opendev.org/c/openstack/openstack-ansible/+/859730 | 21:27 |
| *** dviroel is now known as dviroel|out | 21:37 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/wallaby: Bump OpenStack-Ansible Wallaby https://review.opendev.org/c/openstack/openstack-ansible/+/859731 | 21:37 |
| prometheanfire | I think the elastic_ilm uri calls need to have retries, at least on this 15 year old hardware | 23:01 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!