| noonedeadpunk | yep, I haven't patched that yet. | 06:59 |
|---|---|---|
| jrosser | also take a look at jamesdenton patch for octavia | 07:00 |
| noonedeadpunk | but it's affecting ovn more then ovs, despite it's an ovs bug.... | 07:00 |
| noonedeadpunk | regarding ovn driver? | 07:00 |
| jrosser | CI/AIO setup is broken totally under OVN | 07:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Fix example playbook linters https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/897350 | 07:01 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Add security rule for octavia healthmanager https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/897316 | 07:01 |
| noonedeadpunk | btw, I have no idea how octavia was working without that at all since train ^ | 07:01 |
| noonedeadpunk | or well, how operational_status was updated | 07:02 |
| jrosser | interesting, my magnum/k8s AIO seems ok | 07:02 |
| noonedeadpunk | so the thing is that octavia is working, but it always reporting it's operational status as down and don't have any stats. This somehow worked for us on Xena, but it doesn;t with octaviav2 driver on 2023.1 | 07:10 |
| jrosser | did we switch to amphorav2 driver for antelope? | 07:11 |
| noonedeadpunk | yeah, v1 is deprecated and removed early in 2023.2 | 07:12 |
| noonedeadpunk | old v1 amphoras are handled by v2 driver | 07:12 |
| noonedeadpunk | (in 2023.2 | 07:12 |
| jrosser | this could be why all of ours did a failover on upgrade perhaps | 07:13 |
| noonedeadpunk | Ours did as well actually... | 07:13 |
| noonedeadpunk | Though I though I've messed up with certificates or smth... | 07:13 |
| jrosser | we have some other antelope upgrade issues that Andrew is looking into as well | 07:13 |
| jrosser | check your mariadb is still actually a cluster :/ | 07:14 |
| noonedeadpunk | um.... | 07:14 |
| jrosser | which quite bizarrely became split for us in two regions during cinder upgrade | 07:15 |
| noonedeadpunk | we've spotted quite different behaviour of mariabackup | 07:15 |
| noonedeadpunk | or well, incremental backups | 07:15 |
| jrosser | we had to move ours to the non primary node | 07:16 |
| noonedeadpunk | Like looks like each incremental backup is created not from previous incremental one, but from full. And this causes prolonged marking of backend as "DOWN" in haproxy | 07:16 |
| noonedeadpunk | nah, clusters are fine, phew | 07:17 |
| jrosser | no idea how the cinder upgrade could do that | 07:18 |
| noonedeadpunk | We just hit bugs with running nova handlers in wrong place and vpnaas-related things on top of that. | 07:19 |
| noonedeadpunk | Ah, and re-adding nodes to cluster (like after re-setup or upgrade) now needs either wipe from conductor or to place node uuid localy to /var/lib/nova | 07:19 |
| noonedeadpunk | /var/lib/nova/compute_id to be precise, and there hypervisor uuid should be placed | 07:21 |
| jrosser | do we need a task to do that for upgraded nodes? | 07:25 |
| noonedeadpunk | Nope, during upgrade it will be created | 07:26 |
| noonedeadpunk | It's jsut during re-install with the same hostname | 07:26 |
| noonedeadpunk | we came up with smth like this to execute https://paste.openstack.org/show/b8sP3Yao9xVinLGLfChw/ | 07:27 |
| noonedeadpunk | as basically error is raised inside the condition when RPC version of compute matches with the one stored for conductor | 07:27 |
| jrosser | andrewbonney: ^ something to note here | 07:28 |
| noonedeadpunk | if RPC version is lower nova will create that file as part of "upgrade" process | 07:28 |
| noonedeadpunk | I was thinking if that might be worth to push to ops repo, but we're on early days now with 2023.1 so monitoring if smth else will raise first | 07:29 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: WIP - test Vexxhost CAPI driver for magnum https://review.opendev.org/c/openstack/openstack-ansible/+/893240 | 08:51 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: WIP - test Vexxhost CAPI driver for magnum https://review.opendev.org/c/openstack/openstack-ansible/+/893240 | 08:52 |
| jrosser | ^ this is really close to working now | 08:53 |
| jrosser | very messy playbook but its POC really | 08:53 |
| noonedeadpunk | fwiw, there're couple of comments to the patches it depends on | 08:57 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-tests master: Bump ansible-core to 2.12.8 https://review.opendev.org/c/openstack/openstack-ansible-tests/+/897403 | 09:01 |
| jrosser | yeah i just don't know what to do about the lxc regex one | 09:03 |
| jrosser | i understand the comment but cannot see a reasonable solution | 09:03 |
| noonedeadpunk | jrosser: if, if, lxc.mount.auto param can't be repeated multiple times (like you can do with lxc.include or lxc.mount.entry), then maybe we can just have `regexp: "^lxc.mount.auto = "`? I mean, we don't have to define full string | 09:34 |
| noonedeadpunk | But if it can, then maybe it's worth to add `lxc_container_mount_auto` jsut as a new line in config | 09:35 |
| jrosser | shouldnt we bring this to the same version as the integrated repo? https://review.opendev.org/c/openstack/openstack-ansible-tests/+/897403 | 09:45 |
| noonedeadpunk | well.... if someone wanna fix linters for tests repo... | 09:50 |
| noonedeadpunk | I just tried to make the fastest/simpliest fix possible | 09:52 |
| noonedeadpunk | damn, how did this worked and why it got broken now.... https://zuul.opendev.org/t/openstack/build/51b8a01e984d415d86d1edccc3f6ea06 | 11:32 |
| noonedeadpunk | or we jsut rarely run it... | 11:32 |
| noonedeadpunk | looks like it ansible-lint version bump that got it borked | 11:33 |
| noonedeadpunk | and it's ansible-lint version and u-c requires different things... | 11:36 |
| noonedeadpunk | close to no idea how to solve that to be frank.... | 11:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Temporary don't use u-c for inventory tox test https://review.opendev.org/c/openstack/openstack-ansible/+/897421 | 11:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Switch classic queues to version 2 https://review.opendev.org/c/openstack/openstack-ansible/+/895806 | 11:42 |
| jrosser | we could put ansible-lint in it's own venv | 11:46 |
| jrosser | as we really don't care about u-c with that at all | 11:46 |
| noonedeadpunk | I think, that linters are actually not using u-c already... | 11:53 |
| noonedeadpunk | yeah, dunno.... | 11:54 |
| noonedeadpunk | maybe indeed place to other file instead, but then don't really want to make more places where versions should be adjusted | 11:54 |
| jrosser | oh awesome, my cluster-api job passed | 12:29 |
| mgariepy | woohoo jrosser :) | 12:33 |
| noonedeadpunk | sweet | 12:36 |
| noonedeadpunk | now we need to decide how to deal with it I guess :) | 12:36 |
| mgariepy | only 3 hours check | 12:36 |
| mgariepy | ahah | 12:36 |
| noonedeadpunk | ugh, yes | 12:36 |
| noonedeadpunk | I wonder why we become that slow | 12:37 |
| noonedeadpunk | or what else has changed.... | 12:37 |
| noonedeadpunk | I was thinking that it should be possible to add this job to magnum directly | 12:37 |
| noonedeadpunk | but not sure how to deal with the playbook.... | 12:38 |
| noonedeadpunk | Like ideally... IDeally... Would be to paste it to ops repo, if ops repo was a collection | 12:39 |
| noonedeadpunk | and then a shrt guide how to make use of it | 12:40 |
| noonedeadpunk | *short | 12:40 |
| noonedeadpunk | but we can add it to base ofc as well... Though I'm not sure how much opinionated that driver/concept is | 12:40 |
| jrosser | yeah the playbook is really just a description of "everything that needs to be done" to bring it all up | 12:44 |
| jrosser | ideally there would be a role of some kind | 12:44 |
| jrosser | i think we should get mnaser to come and talk to us at PTG about this, as I believe there are some decisions being made about how magnum will look in the future | 12:45 |
| mnaser | jrosser: i'm happy to do that, we've had some discussion on the goals moving forward with this | 12:46 |
| noonedeadpunk | I wonder if we might want to re-arrange previous agreement on the PTG slot from Tuesday to some other day | 12:55 |
| noonedeadpunk | As it looks like Tuesday is already really packed | 12:55 |
| noonedeadpunk | but well... except octavia - all rest projects have more slots other days | 12:57 |
| mnaser | im speaking at a conference (teleport connect) on the wednesday so thats the old day thats hard for me to be there :( | 12:57 |
| noonedeadpunk | mnaser: will smth like Tuesday 14 or 15UTC work for you | 13:00 |
| mnaser | 15 would be nicer, since i'll be PST.. so at least that's 8am | 13:00 |
| noonedeadpunk | Aha | 13:01 |
| noonedeadpunk | I was checking EST | 13:01 |
| noonedeadpunk | I can book 16UTC then as well, as 8 am is meh | 13:02 |
| mnaser | ah the event is in SF so i'll be there for the week :P | 13:02 |
| mnaser | later the merrier :P | 13:02 |
| noonedeadpunk | ok, agreed then | 13:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: [doc] Update example on how to define neutron_vpnaas_custom_config https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/897431 | 13:34 |
| jrosser | i think something has slowed down recently in the CI jobs | 13:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: [doc] Update example on how to define neutron_vpnaas_custom_config https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/897431 | 13:39 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server master: Fix example playbook linters https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/896613 | 14:09 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server master: [doc] Document usage of binding to interface https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/889691 | 14:09 |
| noonedeadpunk | I wonder if we could start running more tempest tests then before or smth like that | 14:16 |
| noonedeadpunk | https://c75a355f7aec62c918a1-10955244430898ac5659b860db970033.ssl.cf5.rackcdn.com/890640/3/check/openstack-ansible-deploy-aio_metal-ubuntu-jammy/c5ac59c/logs/openstack/aio1-utility/stestr_results.html looks pretty much the same | 14:16 |
| noonedeadpunk | tempest itself takes 7min which is pretty much fair | 14:18 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Track stable/2023.2 SHAs for upstream projects https://review.opendev.org/c/openstack/openstack-ansible/+/897434 | 14:33 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Deprecate OpenDayligh support https://review.opendev.org/c/openstack/openstack-ansible/+/897437 | 14:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Deprecate OpenDaylight support https://review.opendev.org/c/openstack/openstack-ansible/+/897437 | 14:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Deprecate OpenDaylight support https://review.opendev.org/c/openstack/openstack-ansible/+/897437 | 14:43 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Deprecate OpenDaylight support https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/897461 | 14:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Deprecate OpenDaylight support https://review.opendev.org/c/openstack/openstack-ansible/+/897437 | 14:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Deprecate OpenDaylight support https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/897461 | 14:48 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_rally stable/xena: Include proper commit in rally_upper_constraints_url https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/897447 | 17:03 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_rally stable/xena: Include proper commit in rally_upper_constraints_url https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/897447 | 17:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/xena: Use github to fetch ansible collections https://review.opendev.org/c/openstack/openstack-ansible/+/897356 | 17:05 |
| *** cloudnull7 is now known as cloudnull | 23:45 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!