| opendevreview | OpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-ansible/+/948290 | 04:22 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Unfreeze roles after milestone release https://review.opendev.org/c/openstack/openstack-ansible/+/946281 | 07:43 |
| noonedeadpunk | wow, 35m aio build right here: https://zuul.opendev.org/t/openstack/build/16ff2eb6a5f044afb3e254f0c317d940 | 07:50 |
| noonedeadpunk | good morning:) | 07:50 |
| jrosser | we have a grafana dashboard for this dont we | 08:03 |
| jrosser | can probably see when the new modern nodes in the raxflex regions became available, and i would hope we also see an improvement from ansible forks too | 08:03 |
| jrosser | oh, we really do need to update that again, it's looking at non-existant jobs | 08:05 |
| opendevreview | Merged openstack/openstack-ansible master: docs: minor fixes with RabbitMQ and HAProxy https://review.opendev.org/c/openstack/openstack-ansible/+/947569 | 08:08 |
| noonedeadpunk | yeah, dashboard needs some love for sure | 08:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix SHA test scenario https://review.opendev.org/c/openstack/openstack-ansible/+/948319 | 09:26 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Unfreeze roles after milestone release https://review.opendev.org/c/openstack/openstack-ansible/+/946281 | 09:26 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Stop creating the `member` role in Horizon https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/948321 | 10:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Loop around module list in horizon_translations_update https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/656045 | 10:01 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Loop around module list in horizon_translations_update https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/656045 | 10:02 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947673 | 10:13 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947673 | 10:13 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Ensure post-install is idempotent https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947680 | 10:14 |
| noonedeadpunk | so pretty much for the release we have keystone -> httpd and figure out smth with Zun | 10:28 |
| noonedeadpunk | and we also have manila broken I believe... | 10:29 |
| noonedeadpunk | as ganesha is replaced with in-tree code / plugin IIRC | 10:29 |
| * noonedeadpunk planning to work on keystone right away | 10:30 | |
| * noonedeadpunk instantly regred that decision | 10:41 | |
| noonedeadpunk | *regret | 10:41 |
| noonedeadpunk | I'm very o_O an handlers even.... | 10:49 |
| noonedeadpunk | it feels even now we'd restart uwsgi twice under certain conditions | 10:50 |
| noonedeadpunk | I guess I will just replace the apache setup for now and leave all clean-ups as a follow-up anyway... | 11:04 |
| noonedeadpunk | And we'd need to add some kind of oidc test I guess... | 11:04 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: wip https://review.opendev.org/c/openstack/openstack-ansible/+/948111 | 12:19 |
| noonedeadpunk | jrosser: I'm looking at these proxy passes and wonder - do they have to be at the very end? basically, does the order matters in apache? really can't recall that... I believe that for locations/directories it does, but can't recall about options... | 13:38 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-os_keystone/src/branch/master/templates/keystone-httpd.conf.j2#L141-L145 | 13:38 |
| noonedeadpunk | as i nthe httpd role we don't have anything at the end to add https://opendev.org/openstack/ansible-role-httpd/src/branch/master/templates/httpd_vhost.conf.j2 | 13:39 |
| jrosser | i dont know tbh | 13:42 |
| jrosser | the order they are configured in does matter, ie first matching proxypass wins | 13:43 |
| jrosser | but as for order in the file with everything else, not sure | 13:43 |
| NeilHanlon | Yeah the order matters, as well as if you have any rules that end evaluation | 13:43 |
| jrosser | you mean they have to go at the end of the config file? or just the order of them | 13:44 |
| noonedeadpunk | I know the order of same value does for sure | 13:44 |
| noonedeadpunk | But really can't recall about order of different ones... | 13:44 |
| NeilHanlon | no not at the end necessarily | 13:44 |
| noonedeadpunk | So if smth will break if I add ProxyPass as `options` | 13:45 |
| jpw_ | im setting up openstack for the first time. I'm really struggling to get a working network configuration though. are the docs up to date on this? Even using the example configs i seem to find myself in the situation where neutron is in a crash loop | 13:45 |
| noonedeadpunk | jpw_: Highly depend :) | 13:45 |
| NeilHanlon | what if instead of your jinja conditional, you actually conditionalized the ProxyPass? | 13:45 |
| noonedeadpunk | jpw_: if you give the link to example and give a bit more of context - we probably can figure smth out | 13:46 |
| jpw_ | my original plan was to use linux bridge for vxlan. As i understand using the "neutron_linuxbridge_agent" but neutron was complaining vxlan wan't a valid plugin. | 13:46 |
| jpw_ | i'll make a pastebin of my user config/interfaces | 13:46 |
| noonedeadpunk | NeilHanlon: well, I'm rewriting the thing to httpd role. So was thinking about moving ProxyPass here: https://opendev.org/openstack/ansible-role-httpd/src/branch/master/templates/httpd_vhost.conf.j2#L40-L42 | 13:47 |
| noonedeadpunk | which will be above <Location> and <Directory> | 13:47 |
| noonedeadpunk | jpw_: LinuxBridge is dropped from Neutron in 2025.1 | 13:47 |
| noonedeadpunk | So it can not be used as an option anymore | 13:47 |
| noonedeadpunk | jpw_: but also by default we're deploying OVN since 2023.1 IIRC. So unless you define `neutron_plugin_type: ml2.lxb` - you are using OVN | 13:49 |
| NeilHanlon | hmm i may be wrong here, anyways.. it looks from a quick test that ProxyPass does terminate further execution | 13:49 |
| noonedeadpunk | and then there're also several more overrides needed for LXB | 13:49 |
| noonedeadpunk | But it's pointelss to attempt a fresh deployment on LXB as of today | 13:49 |
| noonedeadpunk | NeilHanlon: gotcha | 13:49 |
| noonedeadpunk | bad news then :( | 13:49 |
| NeilHanlon | Yeah... the only thing I can think of that might work would be to use mod_macro | 13:50 |
| jpw_ | https://paste.debian.net/1372056/ | 13:51 |
| NeilHanlon | but then you're templating out a template which sounds.. not ideal | 13:51 |
| jpw_ | i got that far re lxb deprecation even though it wasn't explicitly stated. I've since tried again with ovs but not having any more luck | 13:52 |
| noonedeadpunk | jpw_: I see that `neutron_plugin_type: ml2.ovs` is commented out right now | 13:52 |
| noonedeadpunk | but I think you need couple of more vars for OVS to work | 13:53 |
| jpw_ | i'll uncomment it. i wasn't sure if it was default. i've not checked the defaults yet. | 13:54 |
| noonedeadpunk | I think you need this as well: neutron_plugin_base: ['router', 'metering'] | 13:54 |
| jpw_ | ok, done. any other comments or shall i try sending it? | 13:55 |
| jpw_ | one other question i had about ovs. does it require any other configuration on the host or does osa handle it all? | 13:57 |
| noonedeadpunk | it probably should be fine otherwise.... | 13:57 |
| noonedeadpunk | I did not check your network setup thoroughly though | 13:57 |
| noonedeadpunk | I think you might need less configuration actually. Ie, drop configuration of `br-vlan`, as I think this should be OVS bridge osa creates | 13:58 |
| noonedeadpunk | but there should be a proper mapping for it in place... | 13:59 |
| noonedeadpunk | so `enp86s0` will be enslaved by OVS | 14:00 |
| jpw_ | ok. that won't cause an issue with the other bridges will it? | 14:00 |
| noonedeadpunk | I just somehow can't recall what is needed in `provider_networks` for this to happen :D | 14:00 |
| noonedeadpunk | no, it will not | 14:00 |
| noonedeadpunk | it would if you'd relied on `enp86s0` as untagged for something | 14:01 |
| noonedeadpunk | like using it for SSH access or smth :D | 14:01 |
| jpw_ | nah native vlan is for pxe stuff | 14:01 |
| jpw_ | ssh is coming in over vlan 40 | 14:02 |
| noonedeadpunk | yes, right, that is good:) | 14:02 |
| jpw_ | i'll try commenting it out after this run has completed. i'll let you know how i get on. | 14:03 |
| jpw_ | thank you | 14:03 |
| noonedeadpunk | one thing though, is that as it's an HCI setup, you might want to consider not mixing up OVS and LXB bridges on same hosts... It's generally should not matter, just wanted to say that it's possible to connect LXC containers to OVS as well | 14:03 |
| jpw_ | as in build the bridges within ovs? | 14:04 |
| noonedeadpunk | yeah | 14:04 |
| noonedeadpunk | but it's possibility, not requirement I guess | 14:04 |
| noonedeadpunk | jsut wanted to mention that | 14:04 |
| jpw_ | that would be pushing the boundaries of my knowledge. | 14:05 |
| jpw_ | would that be following the process here? https://docs.openstack.org/openstack-ansible-os_neutron/2024.2/app-openvswitch.html | 14:05 |
| noonedeadpunk | I'm trying to find the doc | 14:06 |
| noonedeadpunk | it looks like it's not really documented :( | 14:07 |
| noonedeadpunk | except https://opendev.org/openstack/openstack-ansible/src/branch/master/etc/openstack_deploy/openstack_user_config.yml.example#L141-L144 | 14:08 |
| jamesdenton__ | jpw_ what sort of errors are you seeing with neutron logs? And if you can share your openstack_user_config.yml and user_variables.yml we can prob help you out | 14:08 |
| noonedeadpunk | jamesdenton__: it;'s https://paste.debian.net/1372056/ | 14:08 |
| jamesdenton__ | thank you | 14:08 |
| jpw_ | i'll share some logs once this run completes. shouldn't be too much longer | 14:09 |
| jamesdenton__ | ok - so i see in your interfaces file you;ve defined "br-vlan", but we also have that defined in openstack_user_config as an OVS bridge (it's not clear, i know) | 14:10 |
| noonedeadpunk | jpw_: so I had smth like this for using OVS as the bridge: https://paste.openstack.org/show/bcOLpjKoPCIl5mbHGiE1/ | 14:10 |
| jamesdenton__ | so, i would remove the one in the interfaces file | 14:10 |
| jamesdenton__ | you also have a br-vlan defined twice - once at 42 and another at 51. I would remove 51-59 | 14:11 |
| noonedeadpunk | jamesdenton__: do you recall if all options are there for proper mapping? As I'm using a neutron_provider_networks for a while now and somehow forgot what are required options would be in `provider_network` for os_neutron to create the bridge and wire interface to it | 14:11 |
| jamesdenton__ | noonedeadpunk 42-50 should be enough to build an ovs bridge named 'br-vlan' and plug enp86s0 into it | 14:12 |
| jamesdenton__ | actually i take that bacjk | 14:12 |
| jamesdenton__ | change line 46 from host_bind_override to network_interface, then it will plug it in | 14:12 |
| jamesdenton__ | but now there is likely a linux bridge named br-vlan causing conflicts | 14:13 |
| jamesdenton__ | but i guess what also isn't clear here is whether or not LXC is to be used? | 14:14 |
| jamesdenton__ | ie. is the intention to use it or not | 14:14 |
| jpw_ | I am deploying in to LXC yes | 14:16 |
| jpw_ | next steps. provide some logs, remove vlan from interfaces, remove :51-59, change :46 to `network_interface` | 14:18 |
| jamesdenton__ | ok - hang tight | 14:18 |
| jamesdenton__ | yes, change that to network_interfaces | 14:20 |
| jamesdenton__ | remove br-vlan from interfaces file | 14:20 |
| jamesdenton__ | and ip link delete br-vlan | 14:20 |
| jpw_ | legends, neutron is happy now. i can load horizon :D | 14:21 |
| jamesdenton__ | also, uncomment 194 | 14:21 |
| jamesdenton__ | if you hadn't already | 14:21 |
| jamesdenton__ | might also need to fix the spacing on that one | 14:21 |
| jpw_ | give me a few minutes. i need to do the interfaces change manually. | 14:23 |
| noonedeadpunk | jamesdenton__: ah, right, I was looking at 51:59 and thinking that smth is missing there | 14:25 |
| jamesdenton__ | been a minute | 14:25 |
| jrosser | feels like we need to extract an example config from all this | 14:25 |
| jamesdenton__ | ... since i've looked at it | 14:25 |
| jamesdenton__ | yes jrosser i think we can probably simplify it a bit | 14:26 |
| * jamesdenton__ adds it to the backlog | 14:26 | |
| jamesdenton__ | see you in 2027 | 14:26 |
| noonedeadpunk | lol | 14:26 |
| jpw_ | yes some clarity would be welcome. that deprecation note for lxb is something i found in the bug tracker | 14:29 |
| noonedeadpunk | we have not released the 2025.1, but this is present in release notes: https://docs.openstack.org/releasenotes/openstack-ansible/unreleased.html#upgrade-notes | 14:30 |
| noonedeadpunk | with reference to jamesdenton__ plog post :) | 14:31 |
| noonedeadpunk | * blog | 14:31 |
| jamesdenton__ | hey, lookie there :D | 14:31 |
| jpw_ | removing that interface has made something very unhappy :s | 14:36 |
| jamesdenton__ | oh? | 14:36 |
| jpw_ | hanging on boot | 14:36 |
| jamesdenton__ | whats the last message on console? | 14:37 |
| jamesdenton__ | it should be unrelated | 14:37 |
| jpw_ | "fauled to start console-font" | 14:37 |
| jamesdenton__ | hmm let's wait and see | 14:38 |
| jpw_ | i'll give it a few more minutes. im tempted to do a full redeploy so i can start fresh tomorrow | 14:38 |
| jpw_ | oh it just came back | 14:39 |
| jpw_ | "Dependency failed for openvswithch something" then the screen cleared | 14:40 |
| jpw_ | i'll make those other changes | 14:40 |
| noonedeadpunk | NeilHanlon: I've read the doc https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass and it seems that only relative position of ProxyPass to each other matters. And seems that all <Location>s are processed before any Vhost ones, so location of ProxyPass relative to Location does not matter.,.. | 14:41 |
| noonedeadpunk | I'm not 100% sure still, but I guess I will try my luck... | 14:42 |
| jamesdenton__ | jpw_ if you decide to do a full redeploy, ping me here and i can take a look at those files and your networking to hopefully ensure a good experience. then we can take your config and use it as an example | 14:47 |
| jamesdenton__ | no doubt the docs have gotten stale in a few areas | 14:47 |
| jpw_ | yeah sure, will do | 14:49 |
| noonedeadpunk | jpw_: as another thing to consider - you can try adding `neutron_provider_networks` - these will take precedence over stuff you configured in `provider_networks` in openstack_user_config | 14:49 |
| noonedeadpunk | and they are waaaaay less confusing, IMO | 14:50 |
| noonedeadpunk | from the doc, I meanL: https://docs.openstack.org/openstack-ansible-os_neutron/2024.2/app-openvswitch.html#openstack-ansible-user-variables | 14:50 |
| jpw_ | here's me thinking i had already picked the easy option | 14:53 |
| jpw_ | if i understand that correctly you hand off a bridge to neutron_openvswitch_agent and it configures everything else? | 14:54 |
| noonedeadpunk | for me personally - networking is hardest part of the OpenStack | 14:54 |
| noonedeadpunk | you don't need to have a bridge | 14:54 |
| noonedeadpunk | you hand off an interface and namings/mappings | 14:54 |
| noonedeadpunk | then the role will create the bridge, add interface, configure neutron | 14:55 |
| jpw_ | oh right it's directly against the interface | 14:55 |
| jpw_ | that's nifty | 14:55 |
| noonedeadpunk | you probably can add the bridge to another bridge as well | 14:55 |
| noonedeadpunk | but naming should be different. and that is weird... | 14:55 |
| noonedeadpunk | but you should be able to do that as well :D | 14:55 |
| jamesdenton__ | here's an example i have: https://gisty.link/7fa4a117ff8af23bec5784554f77c5c0a0985408 | 14:56 |
| noonedeadpunk | I do have very much alike to that, yes ^ | 14:56 |
| jpw_ | specifically the hardest part about it for me was the terminology | 14:56 |
| jpw_ | as in there's plenty of openstack specific terms that are new to me | 14:58 |
| jamesdenton__ | yes, that tends to be the case | 14:58 |
| *** gmaan_pto is now known as gmaan | 14:59 | |
| jamesdenton__ | no question is too basic to be asked here - so if there's any confusion don't hesitate | 15:00 |
| jamesdenton__ | jrosser doesn't bite | 15:00 |
| NeilHanlon | noonedeadpunk: hmm okay, i did a really quick test on my machine here but I could've been misinterpretting what I saw... good you read the docs :D | 15:03 |
| noonedeadpunk | you might be right though, as I did not come to the test part yet :( | 15:03 |
| noonedeadpunk | I just really want you to be wrong here and avoid introducing messy variable to the template :D | 15:06 |
| noonedeadpunk | what makes things worse - we don't use any federation over here, so I kinda struggle to do tests either | 15:07 |
| jpw_ | jamesdenton__: it looks like that change of host_bind_override to network_interface may have causeed an issue. | 15:14 |
| jpw_ | /usr/bin/ovs-vsctl -t 5 add-port br-vlan enp86s0", "interface_mapping": "br-vlan:enp86s0", "msg": "ovs-vsctl: cannot create a port named enp86s0 because a bridge named enp86s0 already exists" | 15:14 |
| jamesdenton__ | host_bind_override was ineffective, and network_interface would plug enp86s0 into br-vlan, but if there;s some bridge named enp86s0 then it would be problematic. Can you post output of "ip link show"? | 15:15 |
| noonedeadpunk | it could be some leftover from prior runs | 15:16 |
| jpw_ | im just rebooting to see the state of things after that br-vlan removal | 15:16 |
| jpw_ | i think there's some lingering config here. | 15:17 |
| jpw_ | i'm going to burn it down and try again | 15:17 |
| noonedeadpunk | `ovs-vsctl list-br` could be iunteresting, but whatever I guess | 15:23 |
| jpw_ | `ip l` after a rebuild https://paste.debian.net/1372075/ | 15:43 |
| noonedeadpunk | that looks fine | 15:48 |
| jamesdenton__ | that looks good, i would expect no complaints | 15:52 |
| jpw_ | great. i'm deploying the rest of osa now. it'll take a while | 15:53 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/948338 | 16:10 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947673 | 16:12 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947673 | 16:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947673 | 16:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Ensure post-install is idempotent https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947680 | 16:17 |
| jrosser | jamesdenton__: sorry in meeting overload here :) | 16:20 |
| jamesdenton__ | :D | 16:20 |
| jrosser | was there something specific | 16:20 |
| jamesdenton__ | you and me both | 16:20 |
| jamesdenton__ | no no, just teasing | 16:21 |
| jrosser | oh right i see -ECONTEXT | 16:21 |
| jpw_ | that looks like it's all deploying successfully now. thank you for your help all. I'll move on and do some more testing. | 17:05 |
| jpw_ | jamesdenton__: i'll give you a ping later this week once i have refined things some more and give you my configs. | 17:05 |
| noonedeadpunk | awesome news | 17:08 |
| jamesdenton__ | sounds good! | 17:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Add EPEL repo for distro-based installs https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/947142 | 17:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Add EPEL repo for distro-based installs https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/947142 | 17:23 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947673 | 19:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon master: Ensure post-install is idempotent https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/947680 | 19:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Use standalone httpd role https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/948338 | 19:41 |
| noonedeadpunk | fwiw, EL9 distro setup is fixed with https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/947142 | 19:48 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-os_trove master: Remove tags from README https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/948372 | 22:26 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-os_trove master: Remove tags from README https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/948372 | 22:26 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!