opendevreview | Vanou Ishii proposed openstack/ironic master: Handle IPMI incompatibility in iRMC S6 2.x https://review.opendev.org/c/openstack/ironic/+/865075 | 00:59 |
---|---|---|
arne_wiebalck | Good morning, Ironic! | 07:26 |
kubajj | Good morning Ironic o/ | 08:35 |
rpittau | good morning ironic! | 10:02 |
*** jpodivin_ is now known as jpodivin | 10:14 | |
kubajj | dtantsur: I am trying to understand what you said yesterday. Is there any point in checking what was stored in Swift if I mock the return value, or did you mean mocking something else? | 10:22 |
opendevreview | Jakub Jelinek proposed openstack/ironic master: Enable alternative storage for inventory https://review.opendev.org/c/openstack/ironic/+/866056 | 10:35 |
iurygregory | good morning Ironic | 11:38 |
kubajj | good morning iurygregory | 11:50 |
iurygregory | o/ | 11:51 |
opendevreview | Jakub Jelinek proposed openstack/ironic master: Enable alternative storage for inventory https://review.opendev.org/c/openstack/ironic/+/866056 | 13:04 |
dtantsur | kubajj: you'll need to mock both storing and retrieving. You can check what was stored by checking the arguments the mock was called with (see assert_called_once_with and others) | 13:18 |
kubajj | dtantsur: for retrieving, do you mean some function for retrieving? I am already testing the storing with the assert in the change above | 13:21 |
dtantsur | lemme see the patch first | 13:21 |
dtantsur | kubajj: left some reviews, but I think it's really good overall | 13:35 |
dtantsur | no issues with the way tests are written | 13:35 |
kubajj | dtantsur: thanks | 13:36 |
dtantsur | kubajj: there is one aspect that we'll need to cover still. unlike data in the database, data in swift is not deleted on node deletion. | 13:36 |
dtantsur | (it may be another patch if this feels to heavy already) | 13:36 |
kubajj | dtantsur: could we do that in another one? I already started looking into the api after the current one. But I am happy to look into it. I have plenty of time now as I am working just on my project for next 2 weeks or so | 13:39 |
dtantsur | yes, absolutely! | 13:51 |
dtantsur | (and I'm very glad that we have you | 13:51 |
dtantsur | I'll be here until ~ x-mas, should be able to review things and help | 13:51 |
kubajj | dtantsur: thanks, I hope to implement as much as possible before then | 13:53 |
TheJulia | good morning | 14:17 |
*** jpodivin_ is now known as jpodivin | 14:17 | |
dtantsur | morning TheJulia | 14:22 |
kubajj | morning TheJulia | 14:27 |
dtantsur | ajya: hi, could you find a minute today for https://review.opendev.org/c/openstack/sushy/+/866612 please? | 15:01 |
ajya | dtantsur: ok, looking | 15:01 |
TheJulia | dtantsur: w/r/t inspector db stuffs, will I be more likely to get the patches moved forward if I just squash them? | 15:09 |
dtantsur | TheJulia: I'd prefer that, to be honest. Otherwise it feels like I need to review 2 things: one somewhat incorrect and one correct :) | 15:09 |
TheJulia | well, one correct for what was there and what is there with a working autocommit... the other... autocommit goes away | 15:12 |
TheJulia | Okay, I'll try to get that done in the next day or so | 15:12 |
TheJulia | I lost a note I made yesterday so need to find that for the patch I need to backport first | 15:12 |
opendevreview | Jakub Jelinek proposed openstack/ironic master: Enable alternative storage for inventory https://review.opendev.org/c/openstack/ironic/+/866056 | 15:20 |
kubajj | dtantsur: that should be your comments implemented ^ | 15:22 |
TheJulia | stevebaker[m]: you beat me to the backports | 15:22 |
dtantsur | kubajj: thanks, will check asap | 15:35 |
rpittau | TheJulia: are you already looking at the issues with python 3.11 ? | 15:53 |
opendevreview | Riccardo Pittau proposed openstack/ironic bugfix/21.0: Align iRMC driver with Ironic's default boot_mode https://review.opendev.org/c/openstack/ironic/+/866656 | 15:56 |
Kirill_ | I see some errors with freshly IDRAC 6.00.30.00 and delete_configuration step | 16:00 |
Kirill_ | Is it known behaviour or bug should be created? | 16:00 |
Kirill_ | 2022-12-06 13:38:25.706 7 INFO ironic.conductor.cleaning [req-a099f2cf-888d-4750-9ee7-29ce1c181c0e - - - - -] Executing {'step': 'delete_configuration', 'interface': 'raid', 'abortable': False, 'priority': 0} on node 734732c4-d0a7-4a03-882b-d6 | 16:00 |
Kirill_ | 82d5268f47 | 16:00 |
Kirill_ | 2022-12-06 13:38:31.778 7 INFO ironic.drivers.modules.drac.utils [req-a099f2cf-888d-4750-9ee7-29ce1c181c0e - - - - -] Completed: get real-time ready status with system 4c4c4544-004a-4610-8053-c7c04f584d33 manager 334d584f-c0c7-5380-4610-004a4c | 16:00 |
ajya | Kirill_: RAID? | 16:00 |
Kirill_ | 4c4544 for node 734732c4-d0a7-4a03-882b-d682d5268f47 | 16:00 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils [req-a099f2cf-888d-4750-9ee7-29ce1c181c0e - - - - -] Node 734732c4-d0a7-4a03-882b-d682d5268f47 failed step {'step': 'delete_configuration', 'interface': 'raid', 'abortable': False, 'priori | 16:00 |
Kirill_ | ty': 0}: HTTP DELETE https://dt-sl-pcb107-rmi.be.gc.onl/redfish/v1/Systems/System.Embedded.1/Storage/RAID.SL.3-1/Volumes/Disk.Virtual.0:RAID.SL.3-1 returned code 501. Base.1.12.GeneralError: Unsupported parameter name @Redfish.OperationApplyTi | 16:00 |
Kirill_ | me. Extended information: [{'Message': 'Unsupported parameter name @Redfish.OperationApplyTime.', 'MessageArgs': ['@Redfish.OperationApplyTime'], 'MessageArgs@odata.count': 1, 'MessageId': 'IDRAC.2.7.SYS029', 'RelatedProperties': ['#/@Redfish. | 16:00 |
Kirill_ | OperationApplyTime'], 'RelatedProperties@odata.count': 1, 'Resolution': 'Remove the unsupported parameter and retry the operation.', 'Severity': 'Warning'}]: sushy.exceptions.ServerSideError: HTTP DELETE https://dt-sl-pcb107-rmi.be.gc.onl/redf | 16:00 |
Kirill_ | ish/v1/Systems/System.Embedded.1/Storage/RAID.SL.3-1/Volumes/Disk.Virtual.0:RAID.SL.3-1 returned code 501. Base.1.12.GeneralError: Unsupported parameter name @Redfish.OperationApplyTime. Extended information: [{'Message': 'Unsupported paramete | 16:00 |
Kirill_ | r name @Redfish.OperationApplyTime.', 'MessageArgs': ['@Redfish.OperationApplyTime'], 'MessageArgs@odata.count': 1, 'MessageId': 'IDRAC.2.7.SYS029', 'RelatedProperties': ['#/@Redfish.OperationApplyTime'], 'RelatedProperties@odata.count': 1, 'R | 16:00 |
Kirill_ | esolution': 'Remove the unsupported parameter and retry the operation.', 'Severity': 'Warning'}] | 16:00 |
ajya | Kirill_: yes, it's a known issue :( and there is a patch - https://review.opendev.org/c/openstack/sushy/+/864845 | 16:00 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils Traceback (most recent call last): | 16:00 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/ironic/conductor/cleaning.py", line 170, in do_next_clean_step | 16:00 |
ajya | Cores, please review | 16:00 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils result = interface.execute_clean_step(task, step) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/ironic/drivers/base.py", line 337, in execute_clean_step | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils return self._execute_step(task, step) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/ironic/drivers/base.py", line 299, in _execute_step | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils return getattr(self, step['step'])(task) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/ironic/drivers/modules/drac/raid.py", line 1289, in delete_configuration | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils return super(DracRedfishRAID, self).delete_configuration(task) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/ironic/drivers/modules/redfish/raid.py", line 863, in delete_configuration | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils task) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/ironic/drivers/modules/redfish/raid.py", line 1177, in _submit_delete_configuration | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils response = volume.delete(apply_time=apply_time) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/sushy/resources/system/storage/volume.py", line 154, in delete | 16:01 |
JayF | Kirill_: please use a pastebin and do not spam chat | 16:01 |
JayF | holy cow | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils r = self._delete(payload, apply_time, timeout) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/sushy/resources/system/storage/volume.py", line 140, in _delete | 16:01 |
JayF | fungi: do you have access to mute Kirill_ real quick? | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils timeout=timeout) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/sushy/connector.py", line 412, in delete | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils **extra_session_req_kwargs) | 16:01 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/sushy/connector.py", line 148, in _op | 16:02 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils exceptions.raise_for_response(method, url, response) | 16:02 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils File "/var/lib/kolla/venv/lib/python3.6/site-packages/sushy/exceptions.py", line 170, in raise_for_response | 16:02 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils raise ServerSideError(method, url, response) | 16:02 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils sushy.exceptions.ServerSideError: HTTP DELETE https://dt-sl-pcb107-rmi.be.gc.onl/redfish/v1/Systems/System.Embedded.1/Storage/RAID.SL.3-1/Volumes/Disk.Virtual.0:RAID.SL.3-1 returned code 5 | 16:02 |
Kirill_ | 01. Base.1.12.GeneralError: Unsupported parameter name @Redfish.OperationApplyTime. Extended information: [{'Message': 'Unsupported parameter name @Redfish.OperationApplyTime.', 'MessageArgs': ['@Redfish.OperationApplyTime'], 'MessageArgs@odat | 16:02 |
Kirill_ | a.count': 1, 'MessageId': 'IDRAC.2.7.SYS029', 'RelatedProperties': ['#/@Redfish.OperationApplyTime'], 'RelatedProperties@odata.count': 1, 'Resolution': 'Remove the unsupported parameter and retry the operation.', 'Severity': 'Warning'}] | 16:02 |
Kirill_ | 2022-12-06 13:38:35.776 7 ERROR ironic.conductor.utils | 16:02 |
Kirill_ | 2022-12-06 13:38:35.841 7 ERROR ironic.conductor.task_manager [req-a099f2cf-888d-4750-9ee7-29ce1c181c0e - - - - -] Node 734732c4-d0a7-4a03-882b-d682d5268f47 moved to provision state "clean failed" from state "cleaning"; target provision state is "manageable": sushy.exceptions.ServerSideError: HTTP DELETE https://dt-sl-pcb107-rmi.be.gc.onl/redfish/v1/Systems/System.Embedded.1/Storage/RAID.SL.3-1/Volumes/Disk.Virtual.0:RAID.SL.3-1 returned code 501. | 16:02 |
Kirill_ | Base.1.12.GeneralError: Unsupported parameter name @Redfish.OperationApplyTime. Extended information: [{'Message': 'Unsupported parameter name @Redfish.OperationApplyTime.', 'MessageArgs': ['@Redfish.OperationApplyTime'], 'MessageArgs@odata.count': 1, 'MessageId': 'IDRAC.2.7.SYS029', 'RelatedProperties': ['#/@Redfish.OperationApplyTime'], 'RelatedProperties@odata.count': 1, 'Resolution': 'Remove the unsupported parameter and retry the operation.', | 16:02 |
Kirill_ | 'Severity': 'Warning'}] | 16:02 |
Kirill_ | got it,big thanks) | 16:02 |
Kirill_ | one more question - how can i find this patch? before i tried storyboard and in code. but didn't find any links to this patch | 16:03 |
ajya | Kirill_: storyboard hasn't been created, probably should. Can always check open patches | 16:03 |
ajya | JayF, was there a plan to move back to launchpad or I'm missing something? | 16:04 |
Kirill_ | ++, thanks | 16:04 |
JayF | we have plans to do it generally | 16:04 |
JayF | no work has been done to get there | 16:04 |
ajya | and to repeat in case it got lost in the floods - can cores review the patch for this issue - https://review.opendev.org/c/openstack/sushy/+/864845 It's already in prio list. Thanks. | 16:05 |
opendevreview | Merged openstack/sushy master: Handle a different error code for missing TransferProtocolType https://review.opendev.org/c/openstack/sushy/+/866612 | 16:05 |
ajya | ok, thanks, JayF | 16:05 |
opendevreview | Riccardo Pittau proposed openstack/ironic bugfix/20.2: Align iRMC driver with Ironic's default boot_mode https://review.opendev.org/c/openstack/ironic/+/866780 | 16:08 |
TheJulia | rpittau: I am not, I've got backports, issues, and end of year stuff to deal with for the foundation | 16:08 |
rpittau | TheJulia: ok, I'll have a look then, I should have the cycles this week | 16:08 |
TheJulia | thanks | 16:08 |
fungi | JayF: sorry, am presenting on a call, but get up with me later about setting you up with chanop access so you can do it | 16:14 |
dtantsur | was about to say that the PTL should have these rights | 16:16 |
TheJulia | change the topic? | 16:17 |
TheJulia | Yeah, the bot access has always been disjointed and weird. I think I last changed the topic via the bot back in... 2016/2017 | 16:18 |
opendevreview | Riccardo Pittau proposed openstack/sushy master: Handle proper code_status in unit test https://review.opendev.org/c/openstack/sushy/+/866782 | 16:23 |
opendevreview | Riccardo Pittau proposed openstack/sushy master: Handle proper code_status in unit test https://review.opendev.org/c/openstack/sushy/+/866782 | 16:24 |
JayF | TheJulia: got a sec to sanity check a couple of shards things?5 | 16:25 |
JayF | GET /v1/shards would return -> {'shards': [{'shard': 'shard1', 'count': 1}]} | 16:25 |
JayF | given we aren't doing pagination for this, or get-one (e.g. /v1/shards/shard1 won't return anything), I don't think we need a links section | 16:25 |
JayF | so I literally think the return is that simple | 16:25 |
TheJulia | I think it is literally that simple | 16:26 |
JayF | Like, my controllers.v1.shard.Shards.get_all is {'shards': dbapi.get_shards()} | 16:27 |
TheJulia | sounds about right | 16:28 |
TheJulia | heh | 16:28 |
JayF | sometimes when things look too simple it makes me think I screwd up | 16:28 |
opendevreview | Jay Faulkner proposed openstack/ironic master: API support for CRUD node.shard https://review.opendev.org/c/openstack/ironic/+/866235 | 16:38 |
opendevreview | Riccardo Pittau proposed openstack/sushy master: Handle proper code_status in unit test https://review.opendev.org/c/openstack/sushy/+/866782 | 16:54 |
opendevreview | Riccardo Pittau proposed openstack/sushy master: Handle proper code_status in unit test https://review.opendev.org/c/openstack/sushy/+/866782 | 16:55 |
opendevreview | Jay Faulkner proposed openstack/ironic master: API support for CRUD node.shard https://review.opendev.org/c/openstack/ironic/+/866235 | 16:57 |
JayF | https://review.opendev.org/c/openstack/ironic/+/866235/5/ironic/tests/unit/api/controllers/v1/test_shard.py#64 | 16:58 |
JayF | should that be returning 404? | 16:58 |
TheJulia | I *think* you need to use https://review.opendev.org/c/openstack/ironic/+/866235/5/ironic/api/controllers/v1/shard.py#67 with get_one instead for that to happen | 17:00 |
TheJulia | since get_all is a blanket (everything under this not otherwise routed) | 17:00 |
TheJulia | but I might bewrong | 17:00 |
kubajj | I have a question about the ironic API. There is the controller for most of the stuff (NodeVIF, NodeHistory, etc). Is this the thing that handles the commands? | 17:00 |
JayF | so basically, implement get_one and force it to 404 | 17:00 |
JayF | got it | 17:01 |
TheJulia | no | 17:01 |
TheJulia | I mean just only have a get_one method should just do it | 17:01 |
TheJulia | since get_one is a specific routing | 17:01 |
TheJulia | get_all is a everything | 17:01 |
JayF | I thought get_all was a list? | 17:01 |
TheJulia | so the call won't know where to go and 404 | 17:01 |
JayF | and get_one was for getting /v1/shard/lol | 17:01 |
JayF | looks like I need to go read some pecan docs, eh | 17:01 |
TheJulia | you could just also implement get_one to return 404 | 17:02 |
TheJulia | so kubajj, the whole thing with JayF is why controller are used | 17:02 |
TheJulia | you route buckets to controllers in the routing | 17:02 |
TheJulia | and each controller is a glass with one or more methods | 17:02 |
TheJulia | and those methods are what does the thing | 17:02 |
TheJulia | JayF: pecan docs ++ | 17:03 |
JayF | those docs indicate that get_all is for GET /resource/ | 17:04 |
JayF | get() is for GET /resource + GET /resource/specific_one | 17:04 |
JayF | so I think I do just need to implement get_one and 404 it | 17:04 |
JayF | https://pecan.readthedocs.io/en/latest/rest.html#url-mapping | 17:04 |
TheJulia | sounds good | 17:18 |
rpittau | good night! o/ | 17:25 |
JayF | arne_wiebalck: please take a look at that shards spec; see if your concerns are addressed | 17:26 |
JayF | arne_wiebalck: we're to the point where implementation is fully prototyped; almost done. We need to get the spec, and the supporting code, landed so the nova side can start | 17:27 |
opendevreview | Merged openstack/sushy master: Retry on iDRAC SYS518 errors for all requests https://review.opendev.org/c/openstack/sushy/+/864911 | 18:32 |
jrosser | where do i start debugging `ironic.conductor.utils [-] Asynchronous exception for node 31311f0c-5428-4c77-97b1-b07d94824850: Node failed to start the first cleaning step: not all arguments converted during string formatting` | 18:52 |
TheJulia | deja vu | 18:57 |
TheJulia | jrosser: manual cleaning? | 18:57 |
jrosser | well, this is when i first `openstack baremetal node provide <node>` | 18:58 |
jrosser | and it then starts cleaning at that point | 18:58 |
TheJulia | jrosser: any patches locally? | 18:59 |
jrosser | a little more log here https://paste.opendev.org/show/bm7bY1etR08VkHu86mOX/ | 18:59 |
jrosser | no, though i've had to build my own IPA image as this is aarch64 | 19:00 |
jrosser | image/kernel/initramfs | 19:00 |
jrosser | it's on yoga | 19:01 |
TheJulia | IPA is on yoga? | 19:02 |
jrosser | i think so, this is how i made it https://paste.opendev.org/show/bf0XgtjoAzd4D5pTePXD/ | 19:03 |
jrosser | i had a terrible wild goose chase to build that, finding an OS i could boot on an accessible arm system that didnt result in failing to build IPA | 19:05 |
jrosser | so thats actually rocky8 to build IPA | 19:05 |
TheJulia | hmm | 19:08 |
TheJulia | so, okay | 19:08 |
TheJulia | what version gets stored in driver_internal_info/agent_version ? | 19:08 |
jrosser | 'agent_version': '8.5.2.dev1' | 19:10 |
TheJulia | interesting | 19:11 |
TheJulia | jrosser: is there a mellenox card in that machine? | 19:13 |
jrosser | there are two, yes | 19:13 |
jrosser | no three | 19:13 |
TheJulia | heh | 19:13 |
TheJulia | I wonder if there is an issue with the hardware manager... | 19:14 |
JayF | /o\ | 19:14 |
jrosser | it's got an onboard cx4, and a PCIe and an OCP cx6 | 19:14 |
TheJulia | hmm, looking closer, maybe not | 19:14 |
TheJulia | this is weird | 19:15 |
TheJulia | oh, apparently forum cfp is also open | 19:27 |
opendevreview | Merged openstack/ironic-inspector bugfix/10.9: Fix compatibility with oslo.db 12.1.0 https://review.opendev.org/c/openstack/ironic-inspector/+/856506 | 19:38 |
opendevreview | Julia Kreger proposed openstack/ironic-inspector master: SQLAlchemy 2.0 Support https://review.opendev.org/c/openstack/ironic-inspector/+/860731 | 19:44 |
opendevreview | Julia Kreger proposed openstack/ironic-inspector master: SQLAlchemy 2.0 Support https://review.opendev.org/c/openstack/ironic-inspector/+/860731 | 19:45 |
jrosser | TheJulia: i took a hunch and set debug=False in ironic.conf, restarted conductor and to my surprise, cleaning has now succeeded | 19:49 |
jrosser | that hunch based on nothing more than this is the last thing in the log before the async exception https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/agent_base.py#L844 | 19:51 |
jrosser | it's kind of late here now but tomorrow i can turn debug back on and see if it fails again | 19:52 |
TheJulia | jrosser: oh, that is why it seems like deja vu for me | 19:53 |
TheJulia | jrosser: thanks! | 19:53 |
opendevreview | Jay Faulkner proposed openstack/ironic master: API support for CRUD node.shard https://review.opendev.org/c/openstack/ironic/+/866235 | 23:02 |
JayF | ^^ shards API implementation is complete enough for code review now; shards spec also needs review | 23:13 |
opendevreview | Jay Faulkner proposed openstack/ironic master: API support for CRUD node.shard https://review.opendev.org/c/openstack/ironic/+/866235 | 23:32 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!