| *** ysandeep|out is now known as ysandeep|rover | 06:04 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Add centos-9 support https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/842236 | 06:15 |
|---|---|---|
| *** ysandeep|rover is now known as ysandeep|rover|lunch | 07:24 | |
| noonedeadpunk | oh, just realized NeilHanlon already published lxc for centos! | 07:49 |
| noonedeadpunk | that is awesome, thanks a lot for help! | 07:49 |
| jrosser | i think there is a little more to do, i'm just trying a centos-9 lxc aio with that | 08:08 |
| jrosser | lxc-templates seems to be missing the actual templates | 08:08 |
| jrosser | which stacks up with this https://bugzilla.redhat.com/show_bug.cgi?id=1860566 | 08:08 |
| jrosser | i've dropped the centos template file in by hand copying it from an ubuntu box and it seems to be working just fine | 08:09 |
| jrosser | setup-hosts seems to work | 08:11 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Add centos-9 support https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/842236 | 08:13 |
| jrosser | noonedeadpunk: question here - do we really want to uninstall rsync? | 08:15 |
| jrosser | https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837859 | 08:15 |
| noonedeadpunk | I'd say it's good to ensure that rsync is present in post task either way | 08:22 |
| noonedeadpunk | as for rhel it's rsyncd which is quite different from what rsync provides as I got | 08:23 |
| jrosser | oh right good point | 08:23 |
| jrosser | that will be why it fails only on ubuntu then | 08:24 |
| jrosser | NeilHanlon: i have containers created with you packages which is great, i think we are affected by this https://bugzilla.redhat.com/show_bug.cgi?id=1860566 | 08:32 |
| jrosser | when i drop the missing files into /usr/share/lxc/... it all works. See https://packages.ubuntu.com/jammy/amd64/lxc-templates/filelist | 08:32 |
| *** ysandeep|rover|lunch is now known as ysandeep|rover | 08:38 | |
| damiandabrowski[m] | morning everyone | 09:24 |
| damiandabrowski[m] | jrosser: Yesterday I've tested gluster on focal | 09:25 |
| damiandabrowski[m] | recovery from single/multi node failure and complete outage works fine, recreating repo container and lsync->gluster transition works fine as well | 09:26 |
| jrosser | damiandabrowski[m]: thats good news | 09:26 |
| damiandabrowski[m] | but I've found one issue, /var/www/repo gluster mount doesn't always come up when restarting container | 09:26 |
| damiandabrowski[m] | https://paste.openstack.org/raw/bZ5GlMKUD0lqzoIthExM/ | 09:26 |
| damiandabrowski[m] | tried to play with 'after'/'requires' parameters in systemd mount definition but haven't succeeded :/ | 09:27 |
| jrosser | i was thinking that there must be a race condition | 09:28 |
| jrosser | each repo server uses it's own gluster server daemon as the thing to connect to for the mount https://review.opendev.org/c/openstack/openstack-ansible/+/837589/15/inventory/group_vars/repo_all.yml | 09:29 |
| damiandabrowski[m] | yeah i've noticed that, but even when I set 'requires=glusterd.service' it wasn't working, probably because gluster needs few secs to peer? | 09:31 |
| jrosser | damiandabrowski[m]: omg https://github.com/systemd/systemd/issues/4468 | 09:48 |
| damiandabrowski[m] | :/ | 09:53 |
| noonedeadpunk | ugh | 10:09 |
| opendevreview | Merged openstack/openstack-ansible-repo_server master: Add upgrade path from lsyncd to shared filesystem. https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/839411 | 11:16 |
| *** dviroel_ is now known as dviroel | 11:19 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Drop default nova client_uuid https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/842335 | 11:20 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Remove all code for lsync, rsync and ssh https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837588 | 11:39 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Clean up legacy lsycnd, rsync and ssh key config https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837859 | 11:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Clean up legacy lsycnd, rsync and ssh key config https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837859 | 11:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Use the same vars file for all versions of centos https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/841618 | 11:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Use distro packages for nginx on centos. https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/841619 | 11:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Allow service to be absent https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/842339 | 11:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Drop duplicated group creation tasks https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/842340 | 11:59 |
| noonedeadpunk | what a mess we have in ceph_client role .... | 11:59 |
| noonedeadpunk | Just tried to use it outside of osa to install ceph clients on hosts.... | 11:59 |
| jrosser | the whole business with getting the keys with several methods is messy | 12:18 |
| SiavashSardari | hey everyone, could you please review these two patches? | 12:25 |
| SiavashSardari | https://review.opendev.org/c/openstack/openstack-ansible/+/837742 | 12:25 |
| SiavashSardari | https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/839373 | 12:25 |
| jrosser | SiavashSardari: there are comments on the os_octavia one already | 12:28 |
| jrosser | and was there a companion part to the first one in lxc_container_create or similar? | 12:28 |
| SiavashSardari | @jrosser, oops I've missed the ocatvia comments. thanks for mentioning that | 12:33 |
| SiavashSardari | and yeah this one is the complementary patch. https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/837734 | 12:34 |
| *** arxcruz_ is now known as arxcruz | 12:34 | |
| jrosser | there are comments on that lxc_container_create one as well, and an error with default(omit) | 12:39 |
| SiavashSardari | @jrosser, thanks, i will take a look at that. | 12:55 |
| SiavashSardari | is everything alright with gerrit? I'm getting siavash.sardari@review.opendev.org: Permission denied (publickey) | 12:57 |
| SiavashSardari | I'm sure nothing changed on my side. except updating to ubuntu 22 | 12:58 |
| mouaa | Hi guys. I'm trying to upgrade a cluster from train to wallaby. Upgrade infrastrucure nodes (underlay) from ubuntu 18.04 to 20.04 is planned on ussuri version. The distribution upgrade notice a "repo_container". What is that ? Just in the case of "Repo Server" role usage (not the case here)? Thanks for help/reply | 13:09 |
| jrosser | mouaa: do you have something you can show us, either a paste somewhere or a link to a document? I'm not sure exactly what you need | 13:17 |
| mouaa | for example in this document: https://docs.openstack.org/openstack-ansible/victoria/admin/upgrades/distribution-upgrades.html, repo_container is mentioned. I can't find a reference in the code or any other document for this. I don't understand what this refers to | 13:25 |
| noonedeadpunk | mouaa: Just in case - aren't you running tripleo or smth?:) | 13:28 |
| noonedeadpunk | and underlay/overlay is mainly applicable there... | 13:28 |
| noonedeadpunk | s/and/as | 13:28 |
| noonedeadpunk | or maybe you're running metal deploy? | 13:29 |
| noonedeadpunk | Basically repo_container is the one, where we are building and storing wheels for packages | 13:30 |
| mouaa | newbee on openstack-ansible, worked before with tripleO, my bad. | 13:30 |
| noonedeadpunk | if you're running source install with lxc - it's just lxc container with "<control>_repo_*" in it | 13:31 |
| jrosser | lxc-ls will show them, i have one here called aio1_repo_container-638ff2ad for example | 13:32 |
| mouaa | I couldn't find this container on any controller node. I assume this is due to not using the "Repo Server" role? Am I correct? | 13:35 |
| noonedeadpunk | what install_method are you using? | 13:37 |
| noonedeadpunk | as in case of distro it might not be needed indeed. | 13:37 |
| noonedeadpunk | but for source I'd expect deployment to fail without it.... | 13:37 |
| noonedeadpunk | but if you can find container, likely it's even that it's not defined in openstack_user_config | 13:38 |
| mouaa | Ok. We use distro install_method | 13:38 |
| noonedeadpunk | then you shouldn't care about repo container :) | 13:39 |
| mouaa | Ok. Thanks you ! 2h for searching what's for | 13:39 |
| jrosser | are you testing these upgrades out on a lab environment first? | 13:39 |
| mouaa | yes. Openstack DEV in openstack PROD whit mtu nightmare (ml2) | 13:40 |
| mouaa | with sorry. Vxlan in a vxlan | 13:41 |
| jrosser | we have much much less test coverage of distro install_method, and i don't think we test upgrades at all for that | 13:41 |
| NeilHanlon | jrosser: interesting.. i will take a peek this morning. i bet i just need to enable some flag | 13:42 |
| mouaa | thanks for this warning ! | 13:42 |
| noonedeadpunk | well yes, distro thing is kind of weird, as we don't have much control over packages | 13:44 |
| jrosser | default and preferred really is source | 13:45 |
| jrosser | and we talk every PTG about removing distro install, and i think it will go away as OS support is dropped | 13:45 |
| noonedeadpunk | and it becomes more nasty when publisher decides not to have "OS upgrade release" | 13:46 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_nova stable/xena: Pass valid cert regen variable to pki role https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/842323 | 14:50 |
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-pki stable/xena: Fix variable names for certificate CSR fields https://review.opendev.org/c/openstack/ansible-role-pki/+/842324 | 14:51 |
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-pki stable/wallaby: Fix variable names for certificate CSR fields https://review.opendev.org/c/openstack/ansible-role-pki/+/842325 | 14:51 |
| opendevreview | Merged openstack/openstack-ansible master: Define tempest network features for ovs/ovn https://review.opendev.org/c/openstack/openstack-ansible/+/842153 | 14:59 |
| opendevreview | Merged openstack/openstack-ansible master: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842132 | 14:59 |
| opendevreview | Merged openstack/openstack-ansible master: haproxy: fix csp for glance image uploads via horizon https://review.opendev.org/c/openstack/openstack-ansible/+/842111 | 15:00 |
| opendevreview | Merged openstack/openstack-ansible stable/wallaby: Limit mistral worker https://review.opendev.org/c/openstack/openstack-ansible/+/841942 | 15:00 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/xena: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842327 | 15:08 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/wallaby: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842328 | 15:09 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/victoria: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842329 | 15:09 |
| *** ysandeep|rover is now known as ysandeep|dinner | 15:17 | |
| *** dviroel is now known as dviroel|lunch | 15:39 | |
| *** ysandeep|dinner is now known as ysandeep | 15:52 | |
| *** ysandeep is now known as ysandeep|out | 15:57 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Bump ansible-core to 2.13.0 https://review.opendev.org/c/openstack/openstack-ansible/+/842387 | 16:20 |
| *** dviroel|lunch is now known as dviroel | 16:31 | |
| opendevreview | Merged openstack/openstack-ansible stable/xena: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842327 | 16:32 |
| spatel | I have keepalived question, any problem if i keep state: BACKUP on both node? | 16:32 |
| jrosser | that means neither has the VIP - do i misunderstand? | 16:36 |
| spatel | I have set both node to be BACKUP and things working fine.. | 16:36 |
| spatel | BACKUP state is just to tell when keepalived come online it doesn't need election and act like BACKUP (pre define) | 16:37 |
| jrosser | oh, ok | 16:37 |
| spatel | But if both are BACKUP then they do election and highest priority win | 16:37 |
| jrosser | doesnt this all have reasonable defaults anyway | 16:38 |
| opendevreview | James Denton proposed openstack/openstack-ansible-os_neutron master: Enable FDB extension for OVS Agent when using SR-IOV https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/842389 | 16:38 |
| spatel | I believe so. | 16:38 |
| opendevreview | Merged openstack/openstack-ansible stable/wallaby: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842328 | 16:38 |
| opendevreview | Merged openstack/openstack-ansible stable/victoria: [doc] Be even more explicit about lb_vip_address https://review.opendev.org/c/openstack/openstack-ansible/+/842329 | 16:38 |
| spatel | Just trying to create my keepalived server for firewall and thinking to keep both BACKUP and let election decide who will be master | 16:39 |
| jrosser | oh, i thought you meant for openstack-ansible use of keepalived | 16:40 |
| spatel | I was also looking at openstack-ansible config to see what options we are setting up and found we predefined MASTER and BACKUP | 16:41 |
| spatel | Trying to understand any downside of setting up both BACKUP :) | 16:41 |
| jrosser | this patch is cursed, more failures https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/840353 | 16:50 |
| mgariepy | hmm. sad. | 16:57 |
| jrosser | good news is ansible 2.13.0 appears to print more output for templating errors https://paste.opendev.org/show/bGJTSS3N0ZYx0xvsMN1K/ | 17:01 |
| jrosser | bad news is i dont understand it :) | 17:02 |
| jrosser | i expect this is "Increase minimum jinja2 dependency to 3.0+" | 17:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_mount master: Do not report changed for shell https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/842414 | 17:09 |
| mgariepy | anyone performed the bionic > focal upgrade? | 17:21 |
| damiandabrowski[m] | i did | 17:22 |
| mgariepy | how does the repo stuff went ? | 17:22 |
| mgariepy | on Ussuri ? | 17:22 |
| damiandabrowski[m] | Victoria | 17:24 |
| damiandabrowski[m] | I filled one bug: https://bugs.launchpad.net/openstack-ansible/+bug/1964535 | 17:24 |
| damiandabrowski[m] | and i remember fixing one more thing but i'm not sure if that fix was backported, let me find it | 17:24 |
| mgariepy | i was thinking of doing a quick/dirty hack to only do-release upgrade a repo container to accomodate a new host.. | 17:25 |
| damiandabrowski[m] | https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/829270 | 17:26 |
| damiandabrowski[m] | it was backported up to victoria | 17:26 |
| damiandabrowski[m] | hmm haven't tried this do-release-upgrade hack :D | 17:27 |
| mgariepy | i did it once or twice before.. lol | 17:27 |
| opendevreview | Merged openstack/openstack-ansible-os_octavia master: Updated from OpenStack Ansible Tests https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/835717 | 17:28 |
| mgariepy | last time was to install 7 or 8 nodes.. and didn't want to have to re-upgrade them a couple weeks after the initial installation.. | 17:28 |
| mgariepy | this time i only have one to do. | 17:28 |
| damiandabrowski[m] | I am glad it's already behind me and all our regions are running focal now... :D | 17:31 |
| damiandabrowski[m] | spent a lot time reprovisioning servers, performing firmware upgrade, evacuating vms etc... | 17:31 |
| mgariepy | do you have local storagE? | 17:32 |
| damiandabrowski[m] | only on few computes, it's ceph in 95% | 17:32 |
| mgariepy | ok | 17:39 |
| mgariepy | i don't have local storage on a new cluster and migrating vms is much more fun. | 17:40 |
| mgariepy | also jammy is comming soon ! :P haha | 17:42 |
| damiandabrowski[m] | not sure if I should be happy about it or just start crying... :D | 17:49 |
| mgariepy | haha. | 17:50 |
| mgariepy | the worst one was xenial to bionic. | 17:50 |
| mgariepy | imo. | 17:50 |
| opendevreview | Merged openstack/ansible-role-pki stable/wallaby: Fix variable names for certificate CSR fields https://review.opendev.org/c/openstack/ansible-role-pki/+/842325 | 17:51 |
| mgariepy | some package are named by the os version and pkg-1.x-xenial is higher than pkg-1.x-bionic | 17:51 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Use global package_state https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/842426 | 18:45 |
| noonedeadpunk | mgariepy: you just don't know how jammy upgrade would look like :p | 18:46 |
| noonedeadpunk | But with shared fs instead of lsyncd.... It looks very promising | 18:46 |
| mgariepy | it will be awesome | 18:47 |
| mgariepy | it's always a pain. | 18:47 |
| mgariepy | the memcached stuff is is not fun at all either. | 18:47 |
| opendevreview | Merged openstack/openstack-ansible master: Use glusterfs to synchronise repo server contents https://review.opendev.org/c/openstack/openstack-ansible/+/837589 | 18:50 |
| opendevreview | Merged openstack/openstack-ansible-os_nova stable/xena: Pass valid cert regen variable to pki role https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/842323 | 18:53 |
| jrosser | noonedeadpunk: I wonder if we should try to drop using ansible_sshd role | 18:57 |
| jrosser | it breaks quite badly on ansible 2.13.0 and I was very surprised how gross the template is | 18:58 |
| jrosser | https://github.com/willshersystems/ansible-sshd/blob/master/templates/sshd_config.j2 | 18:59 |
| SiavashSardari | apparently openssh 8.8 dropped the support of rsa signatures using sha1, which still is used by gerrit. in case anyone wants to upgrade your ssh client this link may save you from some hair pulling over why git review stops working :D | 19:01 |
| SiavashSardari | https://confluence.atlassian.com/bitbucketserverkb/ssh-rsa-key-rejected-with-message-no-mutual-signature-algorithm-1026057701.html | 19:01 |
| *** dviroel is now known as dviroel|out | 20:28 | |
| NeilHanlon | jrosser: looked into the templates thing. it appears the upstream package changed at some point to stop providing a lot of templates. I'll just add some back in :) | 22:16 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!