| TheJulia | Word has it mtus have been fixed. | 00:17 |
|---|---|---|
| opendevreview | Ghanshyam proposed openstack/ironic master: Run ironic-dbsync in grenade testing from venv https://review.opendev.org/c/openstack/ironic/+/932016 | 00:39 |
| gmann | JayF: ok, I rebased mine on top of your change and testing in https://review.opendev.org/c/openstack/grenade/+/930507/13 | 00:40 |
| opendevreview | Ghanshyam proposed openstack/ironic master: Continue disable the GLOBAL_VENV in grenade job https://review.opendev.org/c/openstack/ironic/+/932484 | 05:14 |
| rpittau | good morning ironic! o/ | 06:27 |
| opendevreview | Merged openstack/networking-generic-switch master: tox: Drop envdir https://review.opendev.org/c/openstack/networking-generic-switch/+/932260 | 07:08 |
| jssfr | If a node is in state active and in maintenance (because IPMI was unreachable) and we disable maintenance mode, which target_power_state will Ironic assume? | 07:44 |
| jssfr | (currently, target_power_state shows as None) | 07:44 |
| jssfr | in other words: do we risk Ironic turning off the nodes when we disable maintenance after restoring IPMI access? | 07:44 |
| jssfr | same question for state error (we haven't yet found the reason for that state) | 07:45 |
| dtantsur | jssfr: target_power_state is only changed by a user via API. However, your concern is valid: Ironic will enforce the current power_state. | 08:26 |
| jssfr | power_state is None because of lack of IPMI access. | 08:27 |
| jssfr | ([…] presumably because of […]) | 08:27 |
| dtantsur | jssfr: right. Then you're safe, I believe: on the first sync Ironic will merely record what the hardware reports. | 08:29 |
| jssfr | okay. I guess we'll try that first with some unimportant node (like a ceph node :>) | 08:29 |
| dtantsur | Who cares about this storage thing :D | 08:31 |
| jssfr | alternatively, could we stop conductor et al., update the database to change maintenance = false and power_state = power-on, or would that be an even more insane thing to do? | 08:31 |
| jssfr | dtantsur, well, ceph will cope with the loss of a node better than mariadb or nova-compute (we run openstack on top of ironic-deployed nodes), which are the only other choices :) | 08:31 |
| dtantsur | I'm reasonable sure in my answer above, so I'd rather not hack on the database. | 08:32 |
| jssfr | understood :) | 08:32 |
| dtantsur | This is the logic: https://opendev.org/openstack/ironic/src/branch/master/ironic/conductor/manager.py#L4116-L4129 | 08:33 |
| jssfr | ah lovely | 08:33 |
| jssfr | that looks pretty clear | 08:33 |
| jssfr | I'm gonna check this against the version we're running to be extra-plus-sure. | 08:34 |
| jssfr | thanks a lot! | 08:34 |
| opendevreview | Merged openstack/networking-generic-switch master: cumulus_nvue: Don't call enable() and set enter_config_mode = False https://review.opendev.org/c/openstack/networking-generic-switch/+/931572 | 08:57 |
| opendevreview | Merged openstack/networking-generic-switch master: Correct device_type in Cumulus NVUE comment https://review.opendev.org/c/openstack/networking-generic-switch/+/931655 | 08:57 |
| TheJulia | brraaaaaains | 09:40 |
| * TheJulia ... really didn't sleep | 09:48 | |
| TheJulia | rpittau: so I had some discussions with folks at OpenInfra Days NA, and there seems to be interest in us keeping BFV support "around", so we could potentially punt on the topic. | 09:53 |
| TheJulia | I have the Ironic update session later today, I expect the topic to come up | 09:55 |
| opendevreview | Mahnoor Asghar proposed openstack/sushy-tools master: Minor docs changes for better readability and consistency https://review.opendev.org/c/openstack/sushy-tools/+/932496 | 09:56 |
| opendevreview | Mahnoor Asghar proposed openstack/sushy-tools master: Minor docs changes for better readability and consistency https://review.opendev.org/c/openstack/sushy-tools/+/932496 | 10:23 |
| rpittau | TheJulia: sounds good to me, and let's see also what comes up form the ironic session :) | 10:46 |
| opendevreview | Stephen Finucane proposed openstack/ironic master: devstack: Remove IRONIC_USE_MOD_WSGI https://review.opendev.org/c/openstack/ironic/+/932501 | 10:48 |
| opendevreview | Stephen Finucane proposed openstack/ironic master: devstack: Remove IRONIC_USE_WSGI https://review.opendev.org/c/openstack/ironic/+/932502 | 10:48 |
| iurygregory | good morning ironic | 11:05 |
| rpittau | FYI PTG joint session with nova will be on Thursday October 24 at 1300 UTC in austin room, I've updated our schedule | 14:01 |
| iurygregory | tks rpittau | 14:14 |
| iurygregory | TheJulia, did you had a chance to give the pxe boot sticker to guilhermesp ? :D | 14:17 |
| TheJulia | iurygregory: no, still have it. | 14:38 |
| iurygregory | TheJulia, I think he is spending most of his time in the Vexxhost booth, if you have some time to pass by | 14:40 |
| cardoe | So what's up with the post failures? https://zuul.opendev.org/t/openstack/build/eff548ca29734d3f990d621353ba0f72 | 15:15 |
| cardoe | https://review.opendev.org/c/openstack/ironic-python-agent/+/930080 how do we get that merged? | 15:17 |
| cardoe | oh it's a chain ignore me | 15:17 |
| cardoe | dtantsur: So that above depends on https://review.opendev.org/c/openstack/ironic-python-agent/+/928779 the above has ironic-week-prio on it but 779 doesn't. Any chance you can clean the dependencies up? | 15:20 |
| dtantsur | cardoe: the rebase was done by JayF, I don't want to step on his shoes (even though I'm not fond of it - see my comment) | 16:04 |
| rpittau | good night! o/ | 16:10 |
| JayF | dtantsur: ...did I hook your patch up to the end of my giant patch chain? | 16:38 |
| JayF | dtantsur: if that was me, I don't know what I was thinking | 16:38 |
| dtantsur | You did :) | 16:41 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-python-agent master: A hardware manager call for a full sync before shutdown https://review.opendev.org/c/openstack/ironic-python-agent/+/931017 | 16:42 |
| JayF | dtantsur: yeah, sorry about that, that patch chain will be top of my list today | 16:43 |
| dtantsur | thx! | 16:43 |
| opendevreview | Michal Nasiadka proposed openstack/networking-generic-switch stable/2024.2: cumulus_nvue: Don't call enable() and set enter_config_mode = False https://review.opendev.org/c/openstack/networking-generic-switch/+/932541 | 17:21 |
| opendevreview | Michal Nasiadka proposed openstack/networking-generic-switch stable/2024.2: Correct device_type in Cumulus NVUE comment https://review.opendev.org/c/openstack/networking-generic-switch/+/932542 | 17:26 |
| * TheJulia hears of servers on fire in iceland | 17:42 | |
| opendevreview | Julia Kreger proposed openstack/ironic master: Handle Power On/Off for child node cases https://review.opendev.org/c/openstack/ironic/+/896570 | 18:13 |
| opendevreview | Jay Faulkner proposed openstack/ironic-tempest-plugin master: Fix test to not expect double-JSON-encoded errs https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/932544 | 18:16 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Advanced vmedia deployment test ops https://review.opendev.org/c/openstack/ironic/+/898010 | 18:16 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Fix double JSON encoding of error message https://review.opendev.org/c/openstack/ironic/+/931795 | 18:16 |
| cardoe | So to follow up on that BIOS settings application from the redfish driver... I did the following... @base.clean_step(priority=0, argsinfo=_APPLY_CONFIGURATION_ARGSINFO, requires_ramdisk=False) | 18:38 |
| cardoe | On my Dell's its successfully applying BIOS settings changes without booting the IPA. It's scheduling the iDRAC controller job to apply those settings. They are applied if I manually check the iDRAC BUT... | 18:39 |
| cardoe | It powered on the machine and it tried to do whatever the default boot was. Ironic's showing it's state is powered on. It's in "clean wait". And attempting to retrieve the bios settings shows the old values, likely because it's cached and it won't refresh its cache until the clean is done. | 18:41 |
| cardoe | But I don't think that's a huge lift and then we can give people a viable "reset your BIOS and set some options that are necessary to get into IPA back" | 18:42 |
| opendevreview | Nicholas Kuechler proposed openstack/ironic master: docs: Updates redfish metrics docs with required configuration file settings https://review.opendev.org/c/openstack/ironic/+/932551 | 18:50 |
| JayF | cardoe: I'd +1 such a change, and probably backport it | 18:54 |
| gmann | JayF: cardoe: As ironic fix for global venv can take time I am continue disabling the GLOBAL_VENV flag for ironic grenade job https://review.opendev.org/c/openstack/ironic/+/932484 | 19:02 |
| gmann | this will help to merge the greande setting for current master (stable/2024.2 -> master upgrade) otherwise grenade is not testing the right path | 19:03 |
| gmann | note: enabling the GLOBAL_VENV is fix to land the grenade current master setting | 19:03 |
| JayF | Yeah I think the problem with the test in grenade, the output I saw, is that it's changing from respecting global venv=false to not as it crosses over the barrier | 19:04 |
| JayF | but I'll take a closer look at that change, especially since it's failing current tempest CI, to make sure I didn't miss anything | 19:04 |
| gmann | Currently also ironic greande and all grenade job run with global venv=false so there is no change if we keep it disable in Ironic until we are ready in ironic | 19:06 |
| gmann | but at least we can proceed for grenade correct testing (stable/2024.2 -> master) and for venv for other projects | 19:06 |
| gmann | JayF: my suggestion is if we can merge this first and you can continue working your effort to move things o venv https://review.opendev.org/c/openstack/ironic/+/932484 | 19:07 |
| JayF | gmann: https://review.opendev.org/c/openstack/ironic/+/930776/17/devstack/lib/ironic is this change even right? I don't look at GLOBAL_VENV at all | 19:09 |
| JayF | this is making me wonder if my other patch is shaped right | 19:09 |
| JayF | it's sometimes hard to figure out how these patterns map over to devstack plugins | 19:09 |
| JayF | (or, I should say, I find it tough sometimes) | 19:09 |
| gmann | JayF: I think these are two set of effort GLOBAL_VENV and USE_VENV. former is devstack/grenade install/use everything from virtual env and later one services itself be in their own virtual env | 19:16 |
| JayF | Is there a doc or a spec somewhere that I can follow to implement both correctly? | 19:16 |
| gmann | JayF: your effort for USE_VENV is in right path and what I am trying to work in for GLOBAL_VENV in greande (devstack already enable it by default) | 19:16 |
| JayF | aha devstack being global_venv by default is the piece I was missing | 19:17 |
| JayF | so really I don't think ironic supports GLOBAL_VENV=false | 19:17 |
| JayF | when true it just works, because it installs into the /opt/stack/data/venv (I think that's the path? The venv that's in that data dir) | 19:17 |
| gmann | yeah that is done since 2023.2 and we did not do that in greande because of old node of devstack branch was not ready for global venv that time | 19:18 |
| JayF | does GLOBAL_VENV=false set USE_VENV=true? | 19:18 |
| JayF | meaning like, implementing USE_VENV is all I need? | 19:18 |
| gmann | I do not think we have any doc or spec as such but I find clarkb commit msg very detail about GLOBAL_VENV https://review.opendev.org/c/openstack/devstack/+/558930 | 19:19 |
| gmann | end goal is to enable them both and GLOBAL_VENV as soon as we can as devstack and greande is enabling them by default | 19:19 |
| gmann | JayF: your change is one step towards enabling USE_VENV also as default in devstack but we need to check how other plugins are at | 19:20 |
| JayF | oh, interesting | 19:21 |
| JayF | so Ironic needs no changes for GLOBAL_VENV, seemingly | 19:22 |
| JayF | except your path fix in ironic-dbsync | 19:22 |
| JayF | and my change is unrelated but still good to do | 19:22 |
| gmann | JayF: for GLOBAL_VENV, my plan is 1. to enable it in grenade by default 2. fix the grenade testing for current master so that we do not have any regression due to untested upgrade from stable/2024.2-> master 3. we can fix the projects if failing for 1st or disable the GLOBAL_VENV for time being until fix is ready | 19:22 |
| gmann | JayF: change in ironic-dbsync still fail with GLOBAL_VENV=true which I will debug more and at the end we will enable GLOBAL_VENV for ironic also | 19:23 |
| gmann | I do not want to delay tp fix the grenade testing for current master in case we add any regression due to this missing testing | 19:24 |
| gmann | JayF: I commented to correct the confusion on your change. I mixed the testing of both effort in single change but testing them separately is what I should do https://review.opendev.org/c/openstack/ironic/+/930776/comments/1c2b488c_16b0889f | 19:30 |
| gmann | iurygregory: rpittau: TheJulia: can any of you please check this one https://review.opendev.org/c/openstack/ironic/+/932484 | 19:43 |
| iurygregory | gmann, sure | 19:44 |
| gmann | iurygregory: thanks | 19:44 |
| iurygregory | gmann, newbie question, but since is related to grenade, do we need to backport this? Just want to be sure | 19:46 |
| gmann | iurygregory: no. I am changing the greande side from this cycle only. And this is needed for the upgrade.sh which is always run on new node (current master). for stable branch upgrade testing stay as it is | 19:47 |
| iurygregory | gmann, ack, tks! | 19:48 |
| opendevreview | Nicholas Kuechler proposed openstack/ironic master: docs: Updates redfish metrics docs with required configuration file settings https://review.opendev.org/c/openstack/ironic/+/932551 | 20:25 |
| opendevreview | Nicholas Kuechler proposed openstack/ironic master: docs: Updates redfish metrics docs with required configuration file settings https://review.opendev.org/c/openstack/ironic/+/932551 | 20:30 |
| opendevreview | Nicholas Kuechler proposed openstack/ironic-prometheus-exporter master: docs: Updates configuration documentation https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/932458 | 20:32 |
| cardoe | Folks okay with like 1400 UTC for neutron/ironic PTG? | 20:36 |
| JayF | I should be able to swing it, cid will have to answer for himself | 20:38 |
| cid | cardoe: works for me. | 20:55 |
| *** gmann is now known as gmann_afk | 20:58 | |
| opendevreview | Merged openstack/ironic master: Continue disable the GLOBAL_VENV in grenade job https://review.opendev.org/c/openstack/ironic/+/932484 | 21:12 |
| *** gmann_afk is now known as gmann_ | 21:41 | |
| *** gmann_ is now known as gmann | 21:41 | |
| cardoe | cid: JayF: had to go with 1500 | 21:58 |
| JayF | I basically have my calendar for that week cleared | 22:00 |
| JayF | I usually tell my bosses to treat it like I'm outta town at a conference for PTG week | 22:00 |
| cardoe | If anyone has a cycle to give Scott some feedback on https://review.opendev.org/c/openstack/sushy/+/932096 | 22:05 |
| cardoe | Once we discuss code style pre-commit and the future of sushy-oem-idrac landing into sushy I was going to have him activate more on bringing some other changes in. | 22:07 |
| cardoe | We've also scrapped a bunch of actual hardware to test against instead of just the DMTF examples which are currently used. | 22:07 |
| JayF | A thought spawned by chatter in #openstack-nova: Should Ironic, at some point, deprecate the idea of creating new nodes with node.owner=null | 22:35 |
| JayF | and starting moving towards the project-scope-required model of most other openstack services | 22:35 |
| JayF | (e.g. you can't create a system-scoped instance in nova, it just doesn't make sense) | 22:36 |
| JayF | sean_k_mooney[m] ^^ if you're here and just not visible | 22:36 |
| JayF | sean-k-mooney[m] ^^ if you're here and just not visible (hyphens, not underscores!) | 22:36 |
| cardoe | So maybe my setup is wrong but our nodes are always in a project. You gotta have a project scoped node. | 22:57 |
| cardoe | We've got a domain called infra and a project called baremetal where they live. Users have projects in the default domain that lease those nodes. | 22:58 |
| JayF | that is the "new" ironic model being using reasonably | 22:58 |
| JayF | but we don't force folks to | 22:58 |
| cardoe | It feels weird from an admin stand point | 23:00 |
| JayF | Honestly I agree, but it feels weird in all the services to me | 23:01 |
| JayF | so at some point being weird in the same way as other projects is beneficial :P | 23:01 |
| cardoe | Well I can't have two bucket projects of nodes. | 23:02 |
| cardoe | So like if I'm reserving some nodes extra special for James. I can't put them in their own project so everyone else doesn't see them by default. And still have it work with nova-ironic. Because nova-ironic only works with 1 project of nodes. | 23:02 |
| JayF | you should hook up your nova compute service with a service-scoped user | 23:03 |
| JayF | instead of a project-scoped user | 23:03 |
| JayF | if you want it to see cross-project nodes | 23:03 |
| cardoe | Hrm. It is a user with service role but not sure how to do that scope. Cause I know if system, domain and project scopes. | 23:06 |
| JayF | https://docs.openstack.org/ironic/latest/configuration/sample-config.html cardoe serach for 'rbac_service' | 23:08 |
| JayF | cardoe: what we have configurable via rbac_service_project_name is hardcoded to `service` in many other openstack projects | 23:09 |
| * JayF out o/ | 23:10 | |
| cardoe | Hmm. I’ll have to look. OSH is also a lot of hardcoded stuff. | 23:47 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!