| vanou | good morning ironic | 01:31 |
|---|---|---|
| Nisha_Agarwal | morning Ironic!!! | 06:41 |
| opendevreview | Verification of a change to openstack/ironic master failed: Make rbac enforced test non-voting for the time being https://review.opendev.org/c/openstack/ironic/+/882451 | 07:20 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-tempest-plugin master: Fix rbac tests, take 2 https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/882452 | 07:42 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-tempest-plugin master: Fix rbac tests, take 2 https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/882452 | 07:42 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-tempest-plugin master: Add RBAC specific tempest jobs to gate plugin https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/882312 | 07:42 |
| opendevreview | Merged openstack/ironic master: Make rbac enforced test non-voting for the time being https://review.opendev.org/c/openstack/ironic/+/882451 | 09:49 |
| iurygregory | good morning Ironic | 11:03 |
| *** iurygregory_ is now known as iurygregory | 11:18 | |
| TheJulia | good morning | 13:23 |
| dtantsur | morning iurygregory, TheJulia | 13:26 |
| * TheJulia tries to wake up | 13:37 | |
| arne_wiebalck | Good morning, Ironic! | 14:10 |
| TheJulia | good morning | 14:10 |
| arne_wiebalck | (and it is really morning for me: at a conference in the U.S. talking about Ironic :)) | 14:11 |
| dtantsur | \o/ | 14:14 |
| TheJulia | \o/ | 14:15 |
| TheJulia | Out of curiosity, if your willing to share, what is the conference? | 14:15 |
| arne_wiebalck | https://indico.jlab.org/event/459/contributions/11646/ | 14:16 |
| arne_wiebalck | Computing in High Energy Physics | 14:16 |
| TheJulia | nice! | 14:17 |
| * TheJulia is still trying to wake up | 14:20 | |
| JayF | I would've gone, arne_wiebalck, but I'm only an expert in Medium Energy Physics /s | 14:20 |
| JayF | lol | 14:20 |
| arne_wiebalck | JayF: LOL | 14:20 |
| JayF | Norfolk, VA is nice. You should try to head over to Virginia Beach while you're in town | 14:28 |
| JayF | that's a nice area of the states | 14:28 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Attempting to troubleshoot grenade https://review.opendev.org/c/openstack/ironic/+/882347 | 14:40 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DPU modeling - parent_node DB/Model/API https://review.opendev.org/c/openstack/ironic/+/880114 | 14:57 |
| JayF | Good morning. Meeting time! | 15:00 |
| JayF | #startmeeting ironic | 15:00 |
| opendevmeet | Meeting started Mon May 8 15:00:09 2023 UTC and is due to finish in 60 minutes. The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'ironic' | 15:00 |
| iurygregory | o/ | 15:00 |
| JayF | Who all is here this morning? | 15:00 |
| matfechner | o/ | 15:00 |
| dtantsur | o/ | 15:00 |
| JayF | We have a very simple agenda so unless folks have something for open discussion I'm going to try and make it quick. | 15:00 |
| TheJulia | o/ | 15:00 |
| JayF | #topic announcements/reminder | 15:00 |
| JayF | #note Standing reminder to review patches tagged ironic-week-prio and to hashtag any patches ready for review with ironic-week-prio: https://tinyurl.com/ironic-weekly-prio-dash | 15:00 |
| JayF | #note Please avoid running a `recheck` command bare, without any other comments. Obviously, best case is to troubleshoot and fix an issue in CI, but in case of some ephemeral failure, please just note that -- e.g. `recheck jobname failed`. This is tracked at an OpenStack level (https://etherpad.opendev.org/p/recheck-weekly-summary) and I've noticed more contributors doing | 15:01 |
| JayF | bare rechecks. | 15:01 |
| JayF | Although I recently confirmed those messages are not aggreggated, it can be useful for others coming to a patch later. | 15:01 |
| JayF | Those are the two announcements/reminders. | 15:01 |
| JayF | We had no action items last meeting; skipping agenda item. | 15:02 |
| JayF | #topic Review Ironic CI status | 15:02 |
| JayF | TheJulia: did you ever get that grenade-with-debug job to fail? | 15:02 |
| JayF | That's the only outstanding gate thing right now, yeah? Grenade? | 15:02 |
| TheJulia | yeah, trying to figure out if nova-api is just dying | 15:03 |
| dtantsur | the rbac job has been turned non-voting | 15:03 |
| TheJulia | I think we likely need to consider doing the same thing to grenade | 15:03 |
| dtantsur | le sigh, but yeah | 15:03 |
| JayF | Only for a short time :/ we have another grenade job (skip-level) to get running, too. | 15:03 |
| JayF | for SLURP | 15:04 |
| TheJulia | We likely need to reconsider nova being central to our upgrade testing | 15:04 |
| TheJulia | ... As well. | 15:04 |
| JayF | That sounds like a pretty brutal decision given we have a big nova-compute migration incoming | 15:04 |
| JayF | when sharding is complete | 15:04 |
| dtantsur | We have a bifrost upgrade job, but I wonder if the Nova-Ironic link is actually the weak point that we want to test | 15:05 |
| dtantsur | yep | 15:05 |
| TheJulia | well, the requirement has always been "however it is done", not mandating the tool precisely | 15:05 |
| TheJulia | unless slurp codified grenade | 15:05 |
| TheJulia | but without actual grenade maintenance, I can't see that as being reasonable | 15:05 |
| TheJulia | I'll put it this way | 15:06 |
| JayF | What is the maintenance work we need from grenade? | 15:06 |
| TheJulia | us running grenade is a convience upgrade test | 15:06 |
| dtantsur | you definitely cannot expect me to miss grenade :D | 15:06 |
| dtantsur | we trapped ourselves with devstack | 15:07 |
| JayF | The story I see developing, as told by our CI reliability, is that we have a good handle on testing standalone ironic, and a less-good one on testing ironic+nova, generally, with grenade being a pain point | 15:07 |
| JayF | It's hard for me to think it's grenade/devstack as a tooling problem versus actually implying functional issues without more data | 15:08 |
| opendevreview | Merged openstack/ironic master: Handle MissingAttributeError when using OOB inspections to fetch MACs https://review.opendev.org/c/openstack/ironic/+/880575 | 15:08 |
| TheJulia | afaik grenade is basically as needed only, and locks us in to a nova-centric model. Maybe that is fine. dunno. | 15:08 |
| JayF | I don't see us as locked-in to a nova-centric model so much as, two basic columns of integration testing: Ironic "standalone" and openstack integrated | 15:09 |
| JayF | but either way, we need to make patches mergable and keep upgrades testable | 15:09 |
| JayF | we should keep down the path of identifying if the nova-api is dying | 15:09 |
| JayF | because if it is, hopefully we can get some help from those associated teams | 15:09 |
| TheJulia | that is what I'm doing | 15:09 |
| JayF | ++ that's what I thought just getting it into the meeting log | 15:09 |
| JayF | Going to move on | 15:10 |
| dtantsur | Well, among many things that we did right with bifrost, is this one: the production tool is easy to use for testing | 15:10 |
| TheJulia | the underlying issue is basically people are ignoring the issue or going "it is not my problem" | 15:10 |
| TheJulia | so... at some point, we're going to be completely blocked unless we can pin it down | 15:10 |
| JayF | TheJulia: I'd request if you're not getting good results in IRC, lets make an explicit request on the list so if there is indifference, it's more visible at a project level | 15:10 |
| TheJulia | ack | 15:10 |
| JayF | #topic Review ongoing 2023.2 workstreams | 15:11 |
| JayF | #link https://etherpad.opendev.org/p/IronicWorkstreams2023.2 | 15:11 |
| TheJulia | if nova-api is dying will be a major data point, so time will tell | 15:11 |
| JayF | Please view and update workstream status. Thank you for putting the DPU information in there, I'll review those patches today | 15:11 |
| JayF | #topic Open Discussion | 15:12 |
| JayF | Anything for open discussion? | 15:12 |
| JayF | I'll note, Ironic cores please check your email and respond to the one I sent last week at some point :) | 15:12 |
| JayF | Sounds like nothing for open discussion; closing the meeting. Have a good week all o/ | 15:14 |
| JayF | #endmeeting | 15:14 |
| opendevmeet | Meeting ended Mon May 8 15:14:09 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:14 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/ironic/2023/ironic.2023-05-08-15.00.html | 15:14 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/ironic/2023/ironic.2023-05-08-15.00.txt | 15:14 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/ironic/2023/ironic.2023-05-08-15.00.log.html | 15:14 |
| JayF | TheJulia: lmk if there's something specific I can help out w/the grenade troubleshooting | 15:15 |
| TheJulia | just waiting at this point, I wasn't able to focus on it on friday due to a multi-hour meeting | 15:16 |
| sschmitt_ | I'm seeing this error constantly in my nova-compute-ironic service on a fresh Zed install: https://bugs.launchpad.net/nova/+bug/1910504 Is it safe to ignore? | 15:16 |
| TheJulia | ... unrelated, looks like they added a thing and defaulted it to an exception for other cases. | 15:18 |
| sschmitt_ | yeah thats what I assumed. Its just filling my logs and I would have assumed it might have for others as well | 15:19 |
| TheJulia | that method has been there for a very long time too | 15:20 |
| TheJulia | I don't think I've ever seen that recorded before. :\ | 15:21 |
| sschmitt_ | I wonder why its hitting it now | 15:22 |
| JayF | it looks like that method is only called explicitly | 15:23 |
| JayF | which makes me wonder what is calling it | 15:23 |
| JayF | like, AFAICT, nothing is calling get_host_uptime from inside nova except in response to a req for host uptime | 15:23 |
| TheJulia | Additionally, for bare metal usage it doesn't exactly map over data wise | 15:24 |
| TheJulia | looks like the present grenade run, with uwsgi turned off passed resource creation | 15:26 |
| TheJulia | ... I did so hoping we might see more process logging wise | 15:26 |
| JayF | heisenbug, away! | 15:28 |
| JayF | :| | 15:28 |
| sschmitt_ | hmmm maybe the openstack prometheus exporter is triggering that method | 15:32 |
| NobodyCam | Good morning and Happy Monday Ironic Folks | 15:36 |
| * TheJulia wonders good morning | 15:40 | |
| TheJulia | JayF: If your referring to the TV show... I've made it to the candy lady to get some of the "rock" (candy), but have never made it more than a few episodes into the show | 15:41 |
| JayF | I'm referring to https://en.wikipedia.org/wiki/Heisenbug | 15:42 |
| JayF | > In computer programming jargon, a heisenbug is a software bug that seems to disappear or alter its behavior when one attempts to study it.[1] The term is a pun on the name of Werner Heisenberg, the physicist who first asserted the observer effect of quantum mechanics, which states that the act of observing a system inevitably alters its state. In electronics, the | 15:42 |
| JayF | traditional term is probe effect, where attaching a test probe to a device changes its behavior. | 15:42 |
| TheJulia | heh | 15:42 |
| TheJulia | I was unaware of that reference | 15:42 |
| sschmitt_ | Yep, stopping the Prometheus OpenStack Exporter stops those error messages related to get_host_uptime. | 15:45 |
| JayF | sschmitt_: looks like you may need to configure/patch that service to not call that method for Ironic :/ | 15:46 |
| sschmitt_ | womp womp | 15:46 |
| JayF | I've been thinking about if we could implement that | 15:46 |
| JayF | but there's really nothing at all we could take to put there | 15:46 |
| JayF | that would even remotely make sense in context of what it'd mean for actual-hypervisors | 15:46 |
| sschmitt_ | Maybe just something so it doesn't return an exception? But yeah something like time since node creation doesn't really make sense either | 15:49 |
| TheJulia | yeah | 15:50 |
| TheJulia | That was where my mind immediately went | 15:51 |
| sschmitt_ | or time since last successful node deployment | 15:51 |
| TheJulia | That would actually make sense for nova-compute to keep in memory | 15:52 |
| JayF | how does that map to hypervisor uptime? | 15:52 |
| JayF | if anything; it seems like lifetime of nova-compute service might be closer | 15:52 |
| TheJulia | possibly, but even with baremetal that looses meaning really. | 15:54 |
| TheJulia | time since last deploy op *does* sort of make sense, is the proxy working or getting workload | 15:54 |
| TheJulia | I rechecked grenade, time to see again. | 15:54 |
| * JayF adds a /v1/uptime to Ironic which returns the first time an ironic service using that DB was ever spun up | 15:54 | |
| JayF | lol | 15:54 |
| TheJulia | hehehehe | 15:54 |
| TheJulia | Add an rbac policy around that! | 15:55 |
| dtantsur | I think we have registration time for conductors? | 15:57 |
| dtantsur | We can just expose it if not already :) | 15:57 |
| TheJulia | I don't think we expose it via the object->api | 15:59 |
| sschmitt_ | Anyone who does a vanilla kolla-ansible deployment with both ironic and prometheus enabled will hit this it seems | 15:59 |
| TheJulia | rpittau: Body: b'{"error_message": "{\\"faultcode\\": \\"Server\\", \\"faultstring\\": \\"\\\\\\"baremetal:node:delete:self_owned_node\\\\\\": \\\\\\"role:admin and project_id:%(node.owner)s\\\\\\" requires a scope of [\'project\'], request was made with system scope.\\", \\"debuginfo\\": null}"}' is a bug I think | 16:09 |
| TheJulia | running tests on fix now | 16:13 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Fix self_owned_node policy check https://review.opendev.org/c/openstack/ironic/+/882597 | 16:20 |
| TheJulia | heh, well... looks like grenade is going to pass as uwsgi, again | 16:58 |
| TheJulia | err, without uwsgi | 16:58 |
| TheJulia | o/ Nisha_Agarwal | 17:00 |
| Nisha_Agarwal | TheJulia, o/ | 17:02 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Attempting to troubleshoot grenade https://review.opendev.org/c/openstack/ironic/+/882347 | 17:04 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Attempting to troubleshoot grenade https://review.opendev.org/c/openstack/ironic/+/882347 | 17:04 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Change nova-api to not use uwsgi for grenade https://review.opendev.org/c/openstack/ironic/+/882607 | 17:06 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Change nova-api to not use uwsgi for grenade https://review.opendev.org/c/openstack/ironic/+/882607 | 17:07 |
| TheJulia | if it passes CI, might as well merge it | 17:07 |
| TheJulia | 2x passes in the troubleshooting run, reproposed that without it | 17:07 |
| TheJulia | if it passes on that change, it will be at a better pass rate than CI has been for us all week | 17:08 |
| JayF | dansmith: I wonder, do you all run nova in uwsgi mode for the !ironic grenade run? | 17:08 |
| dansmith | uwsgi everywhere, AFAIK | 17:08 |
| TheJulia | it is the overall default | 17:08 |
| JayF | this behavior implies some kind of bug in the uwsgi support, yeah? | 17:08 |
| TheJulia | too early to know for sure, we can see cases where neutron responds and it looks like the api grinds to a halt, but we need to see if the API is still actually running in the uwsgi case | 17:09 |
| TheJulia | (even then, I think we run 2 workers by default in that config) | 17:09 |
| dansmith | each service runs in its own uwsgi though, I'm sure you know | 17:09 |
| dansmith | ah I misread I thought you were saying neither nova nor neutron responds | 17:10 |
| TheJulia | we don't know if nova has stopped responding, but we can see where neutron responds | 17:11 |
| TheJulia | in the logs, the logs end for nova-api | 17:11 |
| TheJulia | I believe "weird" is an understatement | 17:12 |
| dansmith | I'd certainly be skeptical of running nova-api in a different-from-everywhere-else mode as a solution here of course | 17:13 |
| JayF | dansmith: I think the idea is more, flip the switch as a last-ditch before we mark it non-voting and can hopefully troubleshoot past that | 17:14 |
| TheJulia | of course | 17:14 |
| JayF | but I think we're rapidly approaching the end of the rope | 17:14 |
| JayF | given TheJulia's last debugging attempt appears to have fixed it | 17:14 |
| JayF | "fixed" lolsob | 17:14 |
| dansmith | is ironic's api running in uwsgi proxied through apache as well? | 17:14 |
| TheJulia | good question | 17:15 |
| dansmith | just wondering what else could be causing a difference in behavior (like if apache is getting stuck somewhere and having an additional component is adding pressure) | 17:15 |
| TheJulia | well, intrestingly enough, apache behaves as if there is just a raw timeout | 17:16 |
| dansmith | did this start when jobs got converted to jammy by chance? | 17:17 |
| TheJulia | so, Ironic's grenade job *does* run it's api in wsgi | 17:18 |
| TheJulia | so, I think we saw it before, but it has clearly gotten worse | 17:18 |
| dansmith | just wondering if zed->2023.1 grenade jobs show the problem (on focal) or not.. if not that'd be a data point | 17:19 |
| TheJulia | before it was more a timeout behavior with getting the test resource instance booted and occassionally something like this where we never see it get past validate_networks | 17:19 |
| TheJulia | indeed | 17:20 |
| dansmith | I just can't think of any reason why you all would be the only ones hitting this if it wasn't something related to your job configuration (i.e. having ironic in the mix, or your use of a different/larger image, IIRC) | 17:20 |
| TheJulia | it is cirros in the case of the resource creation | 17:20 |
| TheJulia | it is indeed bizzar | 17:21 |
| JayF | it is nested virt, we might be under more memory pressure | 17:21 |
| JayF | that should only matter if at the point of breakage we were majorly slow, e.g. significant swapping | 17:21 |
| JayF | which seems extremely unlikely | 17:21 |
| dansmith | nested virt? are you using the nested virt nodes with kvm enabled | 17:22 |
| dansmith | ? | 17:22 |
| JayF | when I say nested virt, I just mean we run Ironic fake-BM-VMs inside our grenade VM | 17:22 |
| TheJulia | we have no control over if the VMs are enabled with nested virt | 17:22 |
| JayF | I don't mean to imply a specific hardware config | 17:22 |
| dansmith | TheJulia: yes you do | 17:22 |
| TheJulia | well, we can constrain it | 17:22 |
| dansmith | okay because we have nested virt nodes, and that configuration would be be different than what everyone else is on, so just wondering | 17:23 |
| TheJulia | dansmith: I believe only via restricting providers for the jobs | 17:23 |
| dansmith | JayF: typically memory pressure manifests as an OOM, which we hit in lots of other places | 17:23 |
| JayF | yeah, that's my expectation. Just mainly thinking out loud | 17:23 |
| dansmith | TheJulia: no, there's a nodepool label that lets you select nodes that can do it, and flags to enable/allow it in devstack | 17:23 |
| TheJulia | we've got timeout adjustments embedded for where the devstack vm is fully emulated virt | 17:23 |
| TheJulia | it is a red herring here though | 17:24 |
| TheJulia | because we're not even to that point | 17:24 |
| TheJulia | nova-compute's spawn is never triggered | 17:24 |
| dansmith | the devstack vm is not emulated, but the instances created within are | 17:24 |
| TheJulia | indeed | 17:24 |
| dansmith | yeah | 17:24 |
| TheJulia | dansmith: even on rax? | 17:25 |
| dansmith | yeah devstack in emulated mode would be incredibly slow ;) | 17:25 |
| TheJulia | well, yes, it is | 17:25 |
| clarkb | rax are xen VMs (pvhvm iirc). Then on top of that you get qemu emulation for the workload cloud's resources | 17:26 |
| dansmith | right | 17:26 |
| dansmith | pvhvm being equivalent to a first-level kvm instance.. hardware virt support with a userspace device model | 17:27 |
| dansmith | different from the emulated guest where it's all fully emulated in userspace | 17:27 |
| TheJulia | ugh, looks like we're also seeing the old dhcp test which was once an absolute rare blue moon sort of thing becoming way more frequent under jammy | 17:30 |
| * TheJulia goes and works on that while waiting on cI | 17:30 | |
| TheJulia | CI | 17:30 |
| dansmith | fwiw, we (nova) have seen some differing test and service behavior under jammy, but from what I can see, it's likely because py3.10 is so much faster | 17:32 |
| dansmith | like we hit some race conditions more often than we used to | 17:32 |
| clarkb | it also made thread scheduling more determinstic so if you rely on proper threads a lot you could be exposed to more race conditions due to them existing under the order it ends up scheduling them | 17:32 |
| TheJulia | That would align with the unit test in particular that dislikes us :) | 17:32 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Modify dhcp client ID fail https://review.opendev.org/c/openstack/ironic/+/882609 | 18:08 |
| TheJulia | If anyone would like to offer thoughts on extending the step schema -> https://review.opendev.org/c/openstack/ironic/+/880545/4/ironic/api/controllers/v1/node.py it would be appreicated | 18:19 |
| opendevreview | Julia Kreger proposed openstack/ironic-python-agent master: Fix Bandit errors https://review.opendev.org/c/openstack/ironic-python-agent/+/879912 | 18:45 |
| opendevreview | Julia Kreger proposed openstack/ironic-python-agent master: Fix nvidia hardware manager url parser to permit https https://review.opendev.org/c/openstack/ironic-python-agent/+/881410 | 18:45 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove autocommit, again. https://review.opendev.org/c/openstack/ironic/+/862832 | 18:58 |
| opendevreview | Julia Kreger proposed openstack/metalsmith master: Update MD5 checksum references https://review.opendev.org/c/openstack/metalsmith/+/882170 | 19:14 |
| opendevreview | Merged openstack/ironic-lib master: Add jsonrpc client port capability https://review.opendev.org/c/openstack/ironic-lib/+/879211 | 19:19 |
| *** dmellado9 is now known as dmellado | 19:43 | |
| opendevreview | Merged openstack/python-ironicclient master: Allow several nodes for most node actions https://review.opendev.org/c/openstack/python-ironicclient/+/875756 | 20:11 |
| TheJulia | stevebaker[m]: https://review.opendev.org/c/openstack/ironic/+/882609 | 20:30 |
| JayF | stevebaker[m]: I ninja'd in my +2 before you landed it anyway ;) | 20:34 |
| stevebaker[m] | Dueling ninja +2s | 20:35 |
| opendevreview | Merged openstack/ironic master: CI: Modify dhcp client ID fail https://review.opendev.org/c/openstack/ironic/+/882609 | 20:48 |
| TheJulia | hmm, looks like another test needs to be fixed as well | 21:03 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Fix another network test https://review.opendev.org/c/openstack/ironic/+/882618 | 21:11 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove autocommit, again. https://review.opendev.org/c/openstack/ironic/+/862832 | 21:31 |
| sschmitt_ | Is there a reason a node wouldn't get its DHCP record switched away from ignore so that it would deploy? have a node getting to wait-callback and seeing dhcp-discover packets hitting the controller, but dnsmasq still has its ignore record so it's ignoring them | 21:40 |
| TheJulia | sschmitt_: neutron dhcp should be responding, are the packets getting in to the neutron dhcp namespace? | 21:42 |
| sschmitt_ | well this is my first time deploying with OVN so thats it | 21:44 |
| TheJulia | do you have neutron-dhcp-agent setup? | 21:44 |
| TheJulia | or at least present and running? | 21:44 |
| sschmitt_ | I see the dhcpdiscover coming into the interface I have set as the interface in the dnsmasq config for inspector | 21:44 |
| sschmitt_ | I thought with the newer OVN you didnt need the dhcp agent for ipv4 baremetal nodes, so I dont | 21:45 |
| opendevreview | Harald Jensås proposed openstack/ironic-tempest-plugin master: Fix rbac indicator tests https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/882619 | 21:45 |
| TheJulia | sschmitt_: so, we don't have a job to test that with OVN, because OVN doesn't support ipv6 DHCP configurations needed for Ironic to function, so we generally still point folks to using a neutron-dhcp-agent service | 21:47 |
| TheJulia | in theory, yes, ovn should work with dhcp for ipv4 | 21:48 |
| TheJulia | What that means configuration wise, I personally don't know. | 21:48 |
| sschmitt_ | Ok, ill try and dive in and see where the disconnect it with ovn and can deploy the dhcp agent as a last resort | 21:49 |
| TheJulia | okay, ironic-grenade has passed 10x in a row, this seems... odd | 21:55 |
| TheJulia | sschmitt_: any information you can share one you figure things out, would be helpful to aid our collective knowledge | 21:55 |
| opendevreview | Harald Jensås proposed openstack/ironic-tempest-plugin master: Fix rbac indicator tests https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/882619 | 22:04 |
| opendevreview | Harald Jensås proposed openstack/ironic-tempest-plugin master: Fix rbac indicator tests https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/882619 | 22:05 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove autocommit, again. https://review.opendev.org/c/openstack/ironic/+/862832 | 22:07 |
| opendevreview | Harald Jensås proposed openstack/ironic master: Fix api-ref v1-indicators https://review.opendev.org/c/openstack/ironic/+/882620 | 22:26 |
| JayF | TheJulia: in or out of wsgi? | 22:40 |
| TheJulia | across the board | 22:41 |
| JayF | Strange. I'll make a note tomorrow morning to "diff" a couple of jobs from today/last week and look at commit logs, see if I can identify a win | 22:43 |
| TheJulia | nothing seems inherently different, but it could just be the odds. hopefully things will get better with the other unit test fixes as well | 22:47 |
| opendevreview | Merged openstack/ironic master: CI: Fix another network test https://review.opendev.org/c/openstack/ironic/+/882618 | 23:12 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!