| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Implement installation method selection for MariaDB role https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/914530 | 09:20 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Add distro infra jobs https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/914691 | 09:20 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Implement installation method selection for MariaDB role https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/914530 | 09:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Add distro infra jobs https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/914691 | 09:21 |
| KSR-DK | Hi? Anyone online tha can answer question about major upgrade with openstack-ansible? | 10:47 |
| noonedeadpunk | hey | 10:50 |
| noonedeadpunk | sure, fire it | 10:50 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [Feature] Add skyline deployment capability https://review.opendev.org/c/openstack/openstack-ansible/+/859446 | 10:56 |
| KSR-DK | Thanks | 11:01 |
| noonedeadpunk | I mean - feel free to ask :) | 11:03 |
| KSR-DK | I'm running the upgrade script, and reached the setup-openstack.yml part. I've reached the - import_playbook: os-gnocchi-install.yml part, and it failed. When restarting the upgrade.. Can I comment out the playbooks in setup-openstack.yml above the gnocchi part, and go again (so I don't have to wait hours again), or will subsequent playbooks be dependent on variables gathered in the previous playbooks? | 11:04 |
| noonedeadpunk | KSR-DK: yeah, you don't have to re-run everything from beginning | 11:09 |
| noonedeadpunk | eventually upgrade script should have output what's remaining | 11:09 |
| noonedeadpunk | so you can even manually execute left playbooks, as gnocchi is somewhere closer to the end IIRC | 11:09 |
| noonedeadpunk | at least after most core services | 11:09 |
| noonedeadpunk | also, I did couple of patches towards gnocchi recently | 11:10 |
| noonedeadpunk | so if you provide more details I could be able to help you out narrowing down the issue | 11:10 |
| KSR-DK | Thanks for the help! The issue with Gnocchi, was a missing ceph keyring - my own fault. | 11:20 |
| noonedeadpunk | ah, ok, as Gnocchi frequently fails to be built due to not following/using upper-constraints from OpenStack | 11:21 |
| KSR-DK | Now that I'm here.. This fix for https://bugs.launchpad.net/nova/+bug/2004555 was backported to nova xena-eom, However this is not working with https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/unmaintained/xena/templates/nova.conf.j2 | 11:21 |
| KSR-DK | It's simply missing the [service_user ] part of the config | 11:22 |
| noonedeadpunk | Ok, yeah, that can be the case indeed | 11:23 |
| noonedeadpunk | But, I can't recall IF this was backported in cinder | 11:23 |
| KSR-DK | and the variable nova_service_token_roles_required. I've grabbed the part from https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/unmaintained/yoga/templates/nova.conf.j2 and tested on mys setup - seems to work | 11:24 |
| noonedeadpunk | As this requires patch to be present in Cinder as well, and there were some troubles with that. But I can't recall if it was Xena or smth else... | 11:24 |
| KSR-DK | https://docs.openstack.org/nova/xena/admin/configuration/service-user-token.html - The bug is referenced here | 11:25 |
| noonedeadpunk | So it could be that this bug was never properly addressed on Xena at all... But let me double-check | 11:25 |
| noonedeadpunk | KSR-DK: yes, but as I said - it requires Cinder to support it as well | 11:25 |
| noonedeadpunk | It's not only Nova thing - it was quite combined | 11:25 |
| noonedeadpunk | KSR-DK: also , I think you should be able to place an override to cover that fwiw | 11:26 |
| KSR-DK | Ok.. I can tell you that upgrade of nova from wallaby to xena fails, unless the service_user is added to nova.conf | 11:27 |
| noonedeadpunk | nah, I guess it was backported by cinder to Xena, not to Wallaby I guess | 11:27 |
| noonedeadpunk | https://review.opendev.org/c/openstack/cinder/+/882839 | 11:28 |
| noonedeadpunk | So yeah, it was never fixed on Wallaby | 11:28 |
| noonedeadpunk | What I was trying to explain, is that this bug was backported by Nova down to Victoria | 11:29 |
| noonedeadpunk | So latest Wallaby was also "borked" in a way, because nova is requesting service_user, but other services, like cinder, still won't provide it | 11:29 |
| noonedeadpunk | And that was patched around EM cycle | 11:30 |
| noonedeadpunk | KSR-DK: so you should be able to add that to your user_variables: https://paste.openstack.org/show/bRsXAMXaM17TaYYeunna/ | 11:31 |
| noonedeadpunk | and pretty much same for cinder and for glance.... | 11:31 |
| KSR-DK | together with the nova_service_token_roles_required variable. Thanks man! | 11:31 |
| noonedeadpunk | but I think our eventual idea was to move Xena to EM before this bug was dicovered | 11:32 |
| noonedeadpunk | So pre-EM releases should be fine, I guess, while not covering the bug | 11:33 |
| noonedeadpunk | you might need to add `nova_service_token_roles_required` explicitly though :D | 11:35 |
| noonedeadpunk | but frankly, I guess I'd jumped just to smth supported asap... | 11:35 |
| KSR-DK | I'm planning to do 2 more major upgrades in the next weeks eventually hitting Zed so, I will be moving on ;-) | 11:36 |
| noonedeadpunk | X->Y upgrade should be quite trivial, and then you can have Y-> 2023.1 directly | 11:36 |
| noonedeadpunk | as we tested this path in CI as first "unofficial" SLURP release | 11:36 |
| KSR-DK | Is Y a SLURP ? | 11:36 |
| noonedeadpunk | unofficialy | 11:36 |
| noonedeadpunk | basically when TC made a resolution and we practised on how to do that before doing 2023.1/2024.1 | 11:37 |
| noonedeadpunk | But CI were happy and we upgraded without any issues internally as well | 11:37 |
| noonedeadpunk | (actually, internally we even jumped X->2023.1 somewhere but there were some quirks) | 11:38 |
| noonedeadpunk | let me find the resolution for you | 11:38 |
| KSR-DK | Ooh sweet! I'll have to look into that.. However I'm a bit conerned about the linux-bridge becomming experimental from ZED - Wondering if a migration to OVN is the way to go before moving on from Y or Z | 11:38 |
| KSR-DK | Thank | 11:38 |
| noonedeadpunk | well, LXB still works on 2023.1 | 11:40 |
| noonedeadpunk | I think some folks doing 2023.2 upgrades with lxb... | 11:41 |
| noonedeadpunk | we handling logic for that: https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/templates/neutron.conf.j2#L123-L126 | 11:41 |
| noonedeadpunk | fwiw, there;s some blogpost for such migration: https://www.jimmdenton.com/migrating-lxb-to-ovn/ | 11:42 |
| KSR-DK | Oh yeah, but I mean some features in neutron will be unsupported down the line, and I guess if any bugs occur noone in the community as is will look into it if its related to LXB - At least that's how I've understood their statement | 11:43 |
| noonedeadpunk | yeah. prettty much you're right | 11:43 |
| noonedeadpunk | or well, they won't reject patches or maintaienance, but won't do it on their own more or less | 11:44 |
| noonedeadpunk | As still quite some deployments are using LXB | 11:44 |
| KSR-DK | LOL I like the GIF in the jimmdenton blog | 11:44 |
| noonedeadpunk | (and OVN bar of getting into understanding isn't low) | 11:44 |
| noonedeadpunk | but I'd get to the first official slurp release first I guess before doing lxb->ovn migration | 11:45 |
| noonedeadpunk | as there you potentially also might want/need to do OS upgrade | 11:45 |
| noonedeadpunk | (depending on what you're running) | 11:45 |
| noonedeadpunk | as you'd better have the most modern OVN available... | 11:46 |
| KSR-DK | That's good advice. And yes.. I have an upcomming OS upgrade from Ubuntu 20.04 -> 22.04 also | 11:46 |
| noonedeadpunk | not saying about tons of patches/backports made to 2023.1 for ovn | 11:46 |
| KSR-DK | And that is from Z -> it's possibleI guess | 11:46 |
| noonedeadpunk | (I'd save time and jsut jumped Y->2023.1) | 11:46 |
| noonedeadpunk | and yeah, 20.04 not supported in 2023.2 | 11:47 |
| noonedeadpunk | so it's Z and 2023.1 where you can upgrade | 11:47 |
| noonedeadpunk | iirc | 11:47 |
| noonedeadpunk | anyway | 11:48 |
| KSR-DK | Yeah.. I'll look into Y -> 2023.1 - In regards to the quirks.. Can you share something on that? | 11:48 |
| noonedeadpunk | Well, we had when we upgraded from X -> 2023.1 which was not expected to work at all | 11:49 |
| KSR-DK | Ok, so It will be deployment specific I suppose | 11:50 |
| noonedeadpunk | IIRC, you should be pretty much carefull with Octavia, as we've migrated to the pki role there, so handling of Octavia certs has changed and jsut make sure you backup them as well (as they have weird location historically) | 11:50 |
| KSR-DK | Oh, we don't have that (we run the bare minimum of services) | 11:51 |
| noonedeadpunk | And for X -> 2023.1 upgrade Nova doesn't support RPC microversions for such upgrade, so conductor/computes go into chicken-egg loop, where conductor expects to see modern computes, but these fail to start as see too modern conductor | 11:51 |
| noonedeadpunk | So we jsut updated version in DB for services | 11:52 |
| noonedeadpunk | But it's not an issue for Y->2023.1 | 11:52 |
| noonedeadpunk | And we've also polished https://docs.openstack.org/openstack-ansible/latest/admin/upgrades/distribution-upgrades.html - I'm not sure of all docs adjustments were actually backported to 2023.1 | 11:53 |
| KSR-DK | #noonedeadpunk Nice! | 11:58 |
| KSR-DK | Again - Thanks alot for your help - guess I will be joining the chat again later in my upgrade process! Gotta go, have a nice weeken y'all | 11:59 |
| spatel | did you see error "Running setup.py install for netifaces did not run successfully." when running pip install python-openstackclient ? | 20:05 |
| spatel | damn !!! this fixed it apt-get install build-essential | 20:10 |
| gokhani | Hello folks, when running masakari installation second time, I am getting "msg": "AnsibleUndefinedVariable: 'dict object' has no attribute 'br_mgmt'" error. How I solve this issue ? | 21:42 |
| gokhani | it gives error when creating corosync config | 21:43 |
| gokhani | error logs in detail https://paste.openstack.org/show/b3Qh0cegGxZhjoBNtrO6/ | 22:00 |
| gokhani | I get the problem. If it can not reach any masakari monitor host, it can not get facts. so it throws error. | 22:25 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!