| TheJulia | mnaser: hmm... crash on initial boot on one of the VMs | 00:11 |
|---|---|---|
| *** diurnalist has quit IRC | 00:18 | |
| TheJulia | subnode networking seems to be down in one of them | 00:19 |
| *** ociuhandu has joined #openstack-ironic | 00:33 | |
| *** ociuhandu has quit IRC | 00:37 | |
| *** tosky has quit IRC | 00:51 | |
| *** rh-jelabarre has quit IRC | 00:52 | |
| *** xinliang has joined #openstack-ironic | 00:57 | |
| *** trandles has quit IRC | 01:04 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Mark multinode non-voting due to high failure rate https://review.opendev.org/c/openstack/ironic/+/780133 | 01:06 |
| TheJulia | mnaser: fyi^ | 01:09 |
| *** rloo has quit IRC | 01:10 | |
| *** diurnalist has joined #openstack-ironic | 01:12 | |
| *** diurnalist has quit IRC | 01:17 | |
| *** bfournie has joined #openstack-ironic | 01:24 | |
| *** rh-jelabarre has joined #openstack-ironic | 01:29 | |
| TheJulia | lbragstad: Mar 12 00:58:33.721482 ubuntu-focal-ovh-bhs1-0023448869 nova-compute[113633]: ERROR nova.virt.ironic.driver [None req-06ae772a-081a-4315-930f-74e7a5adb043 None None] An unknown error has occurred when trying to get the list of nodes from the Ironic inventory. Error: Expecting to find domain in user. The server could not comply with the request since it is either malformed or otherwise incorrect. | 01:38 |
| TheJulia | The client is assumed to be in error. (HTTP 400) (Request-ID: req-d8dd6bc3-a918-4694-a9d0-34b71f744f4a): keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in user. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-d8dd6bc3-a918-4694-a9d0-34b71f744f4a) | 01:38 |
| TheJulia | lbragstad: that appears to be coming out of keystone, so maybe I've got the wrong config for nova? | 01:42 |
| *** k_mouza has joined #openstack-ironic | 02:07 | |
| *** k_mouza has quit IRC | 02:12 | |
| *** rh-jelabarre has quit IRC | 02:16 | |
| *** ociuhandu has joined #openstack-ironic | 02:21 | |
| lbragstad | TheJulia oh - let me check quick | 02:23 |
| TheJulia | lbragstad: so, nova uses openstacksdk to pull the nodes list. While it doesn't *appear* to be detonating deep inside of the keystoneauth1 library, it does seem to not like the config that is getting passed | 02:24 |
| TheJulia | *but* it looks like successful auths have passed a user-domain-id | 02:24 |
| TheJulia | of default | 02:24 |
| TheJulia | so I just edited the devstack patch | 02:25 |
| TheJulia | triggering recheck | 02:25 |
| lbragstad | TheJulia yeah - its the missing user domain ID | 02:26 |
| *** ociuhandu has quit IRC | 02:26 | |
| lbragstad | i just left a comment | 02:26 |
| lbragstad | but - domains are containers for projects and users | 02:26 |
| TheJulia | Thats what I thought | 02:27 |
| lbragstad | but keystone doesn't guarantee project and users to have unique names across domains | 02:27 |
| TheJulia | so its a keystone constraint ultimately? | 02:27 |
| lbragstad | yes | 02:27 |
| TheJulia | okay | 02:27 |
| lbragstad | if you supply a user name for authentication - you also need to supply the domain of the user, so keystone can namespace it | 02:27 |
| lbragstad | the same constraint applies to projects | 02:27 |
| TheJulia | except of course, if there is any hint of a project in the request, system-scope gets ignored | 02:27 |
| lbragstad | but - if you use IDs, for either projects or users, you don't need to supply the domain since they're globally unique | 02:28 |
| TheJulia | which makes sense | 02:28 |
| TheJulia | interesting | 02:28 |
| lbragstad | well - yeah, that's confusing | 02:28 |
| TheJulia | very much so | 02:28 |
| lbragstad | so username + user_domain_id == user_id | 02:28 |
| lbragstad | project_name + project_domain_id == project_id | 02:28 |
| TheJulia | well, we'll see since user domain id got preserved in base -> patch 3 | 02:30 |
| TheJulia | 1 hour to go | 02:30 |
| * TheJulia goes and foods the corgi overlord | 02:30 | |
| *** tzumainn has quit IRC | 02:38 | |
| *** diurnalist has joined #openstack-ironic | 02:46 | |
| *** diurnalist has quit IRC | 02:51 | |
| TheJulia | lbragstad: good news \o/ https://8211ecf8fed6b0b51c88-2d35d76e65f9afb322cf96e46e6932b6.ssl.cf2.rackcdn.com/778957/12/check/ironic-inspector-tempest-secure-rbac/6169b47/controller/logs/screen-n-cpu.txt | 03:04 |
| TheJulia | *looks* like we've got glance issues now, but that can be for tomorrow | 03:04 |
| *** ianychoi_ has joined #openstack-ironic | 03:27 | |
| iurygregory | +W in https://review.opendev.org/c/openstack/ironic/+/780133 to unblock (since we have the random failures =( ) | 03:30 |
| iurygregory | now time to sleep o/ | 03:30 |
| *** ianychoi__ has quit IRC | 03:30 | |
| *** jamesdenton has quit IRC | 03:33 | |
| *** jamesdenton has joined #openstack-ironic | 03:33 | |
| *** xinliang has quit IRC | 03:47 | |
| *** rcernin has quit IRC | 03:48 | |
| *** diurnalist has joined #openstack-ironic | 03:58 | |
| *** diurnalist has quit IRC | 04:03 | |
| *** rcernin has joined #openstack-ironic | 04:15 | |
| *** rcernin has quit IRC | 04:19 | |
| *** rcernin has joined #openstack-ironic | 04:19 | |
| *** lifeless has quit IRC | 04:42 | |
| *** k_mouza has joined #openstack-ironic | 04:45 | |
| *** k_mouza has quit IRC | 04:50 | |
| openstackgerrit | Merged openstack/ironic master: Mark multinode non-voting due to high failure rate https://review.opendev.org/c/openstack/ironic/+/780133 | 05:16 |
| *** stevebaker has quit IRC | 05:18 | |
| *** stevebaker has joined #openstack-ironic | 05:23 | |
| *** ociuhandu has joined #openstack-ironic | 05:58 | |
| *** ociuhandu has quit IRC | 06:02 | |
| *** AndreyK has quit IRC | 06:33 | |
| *** diurnalist has joined #openstack-ironic | 06:35 | |
| *** rcernin has quit IRC | 06:47 | |
| *** lifeless has joined #openstack-ironic | 06:49 | |
| *** k_mouza has joined #openstack-ironic | 06:54 | |
| *** rcernin has joined #openstack-ironic | 06:58 | |
| *** k_mouza has quit IRC | 06:59 | |
| *** ociuhandu has joined #openstack-ironic | 07:03 | |
| *** diurnalist has quit IRC | 07:03 | |
| *** ociuhandu has quit IRC | 07:07 | |
| openstackgerrit | Dhuldev Valekar proposed x/sushy-oem-idrac master: Add resources to put iDRAC in known good state https://review.opendev.org/c/x/sushy-oem-idrac/+/721596 | 07:22 |
| openstackgerrit | Dhuldev Valekar proposed x/sushy-oem-idrac master: Add resources to put iDRAC in known good state https://review.opendev.org/c/x/sushy-oem-idrac/+/721596 | 07:32 |
| *** rcernin has quit IRC | 07:33 | |
| *** rcernin has joined #openstack-ironic | 07:34 | |
| arne_wiebalck | Good morning, ironic! | 07:36 |
| *** rcernin has quit IRC | 07:40 | |
| *** rcernin has joined #openstack-ironic | 07:42 | |
| *** ociuhandu has joined #openstack-ironic | 07:45 | |
| *** ociuhandu has quit IRC | 07:45 | |
| *** rcernin has quit IRC | 07:47 | |
| *** diurnalist has joined #openstack-ironic | 07:50 | |
| *** diurnalist has quit IRC | 07:55 | |
| *** jamesdenton has quit IRC | 07:55 | |
| *** jamesdenton has joined #openstack-ironic | 07:56 | |
| *** ociuhandu has joined #openstack-ironic | 07:57 | |
| *** ociuhandu has quit IRC | 07:57 | |
| *** ociuhandu has joined #openstack-ironic | 07:58 | |
| *** gryf is now known as gryf_ | 07:58 | |
| *** _gryf is now known as gryf | 07:59 | |
| *** rcernin has joined #openstack-ironic | 08:06 | |
| *** zzzeek has quit IRC | 08:07 | |
| *** zzzeek has joined #openstack-ironic | 08:11 | |
| *** gyee has quit IRC | 08:11 | |
| *** rcernin has quit IRC | 08:11 | |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Update export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/759425 | 08:15 |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add import system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/770614 | 08:15 |
| *** rcernin has joined #openstack-ironic | 08:23 | |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add import system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/770614 | 08:26 |
| *** ociuhandu has quit IRC | 08:26 | |
| *** ociuhandu has joined #openstack-ironic | 08:27 | |
| *** rcernin has quit IRC | 08:28 | |
| arne_wiebalck | dtantsur|afk: Are there plans to tag/release ironic-lib any time soon, on stable/ussuri and upwards? Having the "clean EBRs" patch in a release would spare me rebuilding the RPM downstream ... and IIRC others were interested in this specific patch as well (e.g. kaifeng). | 08:31 |
| * arne_wiebalck is obviously unclear on the mechanics | 08:31 | |
| *** ianychoi_ has quit IRC | 08:32 | |
| *** rcernin has joined #openstack-ironic | 08:33 | |
| *** rcernin has quit IRC | 08:35 | |
| *** ociuhandu has quit IRC | 08:50 | |
| *** tosky has joined #openstack-ironic | 08:51 | |
| openstackgerrit | Devendra Kulkarni proposed x/sushy-oem-idrac master: Update virtual media boot related constants https://review.opendev.org/c/x/sushy-oem-idrac/+/773848 | 08:53 |
| *** diurnalist has joined #openstack-ironic | 09:01 | |
| *** lucasagomes has joined #openstack-ironic | 09:03 | |
| *** ociuhandu has joined #openstack-ironic | 09:06 | |
| *** derekh has joined #openstack-ironic | 09:07 | |
| *** diurnalist has quit IRC | 09:07 | |
| *** ociuhandu has quit IRC | 09:16 | |
| *** stevebaker has quit IRC | 09:17 | |
| *** ociuhandu has joined #openstack-ironic | 09:23 | |
| janders | good morning arne_wiebalck o/ | 09:43 |
| openstackgerrit | sonali bhausaheb borkar proposed x/sushy-oem-idrac master: Add export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/710327 | 09:46 |
| *** anuradha1904 has joined #openstack-ironic | 09:46 | |
| *** dtantsur|afk is now known as dtantsur | 09:51 | |
| dtantsur | morning ironic | 09:51 |
| janders | hey dtantsur :) | 09:51 |
| janders | Happy Friday | 09:51 |
| dtantsur | arne_wiebalck: master has been released, feel welcome to request stable releases | 09:51 |
| dtantsur | is anyone up for SPUC today? | 09:51 |
| *** diurnalist has joined #openstack-ironic | 09:54 | |
| *** k_mouza has joined #openstack-ironic | 09:56 | |
| janders | dtantsur sure! | 09:58 |
| *** diurnalist has quit IRC | 09:59 | |
| janders | arne_wiebalck? | 10:00 |
| janders | ajya? | 10:01 |
| arne_wiebalck | dtantsur: I hereby request! :-D | 10:02 |
| arne_wiebalck | dtantsur: srsly, how do I request? | 10:02 |
| dtantsur | arne_wiebalck: https://docs.openstack.org/ironic/latest/contributor/releasing.html#how-to-propose-a-release | 10:03 |
| arne_wiebalck | SPUC link? | 10:03 |
| dtantsur | arne_wiebalck: https://bluejeans.com/772893798/ | 10:03 |
| openstackgerrit | sonali bhausaheb borkar proposed x/sushy-oem-idrac master: Add export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/710327 | 10:15 |
| openstackgerrit | sonali bhausaheb borkar proposed x/sushy-oem-idrac master: Add export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/710327 | 10:29 |
| janders | back to... Android app development - that's my "other projects" idea | 10:43 |
| dtantsur | I always have https://github.com/dtantsur/rust-osauth/ to hack on :) | 10:45 |
| openstackgerrit | Moshiur Rahman proposed openstack/ironic-python-agent-builder master: Fix: IPA image building with OpenSuse. https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/778726 | 10:47 |
| openstackgerrit | Rotan proposed openstack/ironic-inspector master: There is a MismatchError when the test_without_root_disk function has been tested. https://review.opendev.org/c/openstack/ironic-inspector/+/780219 | 10:50 |
| openstackgerrit | Rotan proposed openstack/ironic-inspector master: Fix the MismatchError https://review.opendev.org/c/openstack/ironic-inspector/+/780219 | 10:54 |
| janders | dtantsur nice! | 10:55 |
| *** ociuhandu_ has joined #openstack-ironic | 11:18 | |
| *** ociuhandu has quit IRC | 11:21 | |
| *** ociuhandu_ has quit IRC | 11:22 | |
| openstackgerrit | sonali bhausaheb borkar proposed x/sushy-oem-idrac master: Add get PXE port macs for BIOS mode https://review.opendev.org/c/x/sushy-oem-idrac/+/770904 | 11:32 |
| openstackgerrit | Aija Jauntēva proposed openstack/ironic master: Add Redfish RAID interface to idrac HW type https://review.opendev.org/c/openstack/ironic/+/774433 | 11:32 |
| iurygregory | good morning arne_wiebalck janders dtantsur and Ironic o/ happy friday! | 11:40 |
| dtantsur | o/ | 11:42 |
| janders | good morning iurygregory o/ | 11:42 |
| janders | Happy Friday | 11:42 |
| *** jamesdenton has quit IRC | 11:50 | |
| *** jamesdenton has joined #openstack-ironic | 11:52 | |
| arne_wiebalck | hey iurygregory o/ | 11:53 |
| arne_wiebalck | dtantsur: do we have a policy to request stable releases without "holes", i.e. if I request one for ussuri (which then has a certain fix), should I also request one for victoria, so that the newer release has at least the same fixes? | 11:56 |
| iurygregory | arne_wiebalck, afaik we don't have this =) | 12:01 |
| iurygregory | but if you can request the release for victoria also it would be good =) | 12:01 |
| iurygregory | (I would say it's ok to have in the same patch since you will release the same project... but I would check with release team) | 12:02 |
| arne_wiebalck | iurygregory: I think it would make sense to not have fixes only in older releases as this could lead to some surprises | 12:03 |
| iurygregory | yeah | 12:04 |
| *** ociuhandu has joined #openstack-ironic | 12:09 | |
| *** ociuhandu has quit IRC | 12:15 | |
| openstackgerrit | Devendra Kulkarni proposed x/sushy-oem-idrac master: Updated RETRY_COUNT for virtual media boot https://review.opendev.org/c/x/sushy-oem-idrac/+/780240 | 12:15 |
| dtantsur | arne_wiebalck: not necessary, but desired | 12:16 |
| openstackgerrit | Devendra Kulkarni proposed x/sushy-oem-idrac master: Updated RETRY_COUNT for virtual media boot https://review.opendev.org/c/x/sushy-oem-idrac/+/780240 | 12:18 |
| janders | see you on Monday Ironic | 12:18 |
| janders | have a great weekend everyone o/ | 12:18 |
| dtantsur | enjoy your weekend | 12:18 |
| iurygregory | bye janders o/ | 12:19 |
| iurygregory | I'm surprised by https://review.opendev.org/c/x/sushy-oem-idrac/+/780240 O.o (I had the feeling we did a workaround in Ironic because of this.. - retry in insert media..) | 12:21 |
| openstackgerrit | Devendra Kulkarni proposed x/sushy-oem-idrac master: Updated RETRY_COUNT for virtual media boot https://review.opendev.org/c/x/sushy-oem-idrac/+/780242 | 12:25 |
| arne_wiebalck | dtantsur: thanks, I requested releases for both now | 12:47 |
| *** tkajinam has quit IRC | 12:54 | |
| *** rh-jelabarre has joined #openstack-ironic | 12:55 | |
| TheJulia | good morning | 13:13 |
| dtantsur | morning TheJulia | 13:16 |
| dtantsur | thank you arne_wiebalck, I'll check them a bit later | 13:16 |
| *** ociuhandu has joined #openstack-ironic | 13:28 | |
| openstackgerrit | Moshiur Rahman proposed openstack/ironic-python-agent-builder master: Fix: IPA image building with OpenSuse. https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/778726 | 13:35 |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add real-time status check to Lifecycle service https://review.opendev.org/c/x/sushy-oem-idrac/+/776224 | 13:36 |
| openstackgerrit | sonali bhausaheb borkar proposed x/sushy-oem-idrac master: Add export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/710327 | 13:48 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/ussuri: Stable-CI: Disable ansible testing on ussuri https://review.opendev.org/c/openstack/ironic/+/780062 | 14:06 |
| TheJulia | ^^ iurygregory, dtantsur, also marks the multinode non-voting due to limestone issues | 14:07 |
| dtantsur | le sigh | 14:07 |
| TheJulia | yup | 14:07 |
| dtantsur | what's wrong with limestone? | 14:07 |
| *** diurnalist has joined #openstack-ironic | 14:07 | |
| TheJulia | basically on every job I've looked at, the subnode can't get packets through on vxlan to the controller. We think it might be MTU, but we're going to need to actually get into a failed setup and dig around/try to reproduce it | 14:07 |
| TheJulia | fungi is happy to setup the auto-hold once someone has time to look at it. | 14:08 |
| *** rloo has joined #openstack-ironic | 14:09 | |
| fungi | yep | 14:09 |
| fungi | just let me know | 14:09 |
| TheJulia | <3 | 14:09 |
| * TheJulia sighs with gerrit's ui | 14:12 | |
| *** diurnalist has quit IRC | 14:12 | |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/710327 | 14:12 |
| TheJulia | lbragstad: I see our issue with tempest. I guess we can't see the images that got uploaded... when lib/tempest tries to run. Is that right? | 14:15 |
| *** jamesdenton has quit IRC | 14:24 | |
| *** jamesdenton has joined #openstack-ironic | 14:24 | |
| *** diurnalist has joined #openstack-ironic | 14:33 | |
| openstackgerrit | Merged x/sushy-oem-idrac master: Add export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/710327 | 14:35 |
| *** diurnalist has quit IRC | 14:38 | |
| lbragstad | TheJulia you can't see the images? is glance in enforcing mode? | 14:38 |
| lbragstad | or enforcing secure rbac/ | 14:38 |
| TheJulia | Only ironic | 14:39 |
| TheJulia | I think it is because we're uploading as system-admin | 14:39 |
| lbragstad | oh - you're uploading the image as a system-admin... | 14:40 |
| TheJulia | yup, I can tone that back on the command line, but I guess these are the quarks we need to find | 14:41 |
| TheJulia | That way it should pickup admin project instead | 14:41 |
| lbragstad | ok - i don't think glance scrubs the owner id from system-scoped tokens | 14:41 |
| lbragstad | yeah - you might be able to pass in the owner on the image and set it to a "project" as a workaround for now | 14:41 |
| TheJulia | interesting | 14:42 |
| *** tzumainn has joined #openstack-ironic | 14:42 | |
| TheJulia | That is going to be a stumbling point for many I think | 14:42 |
| lbragstad | cc dansmith - something to think about | 14:42 |
| dansmith | I'm not sure I get the problem | 14:43 |
| dansmith | you create a private image and it's only visible to you yeah? | 14:43 |
| TheJulia | I don't know off the top of my head, I thought they were public in the config, but I guess not | 14:44 |
| * TheJulia is juggling multiple things at the moment | 14:44 | |
| lbragstad | iiuc - the problem is that glance still allows system-admins to create images | 14:45 |
| lbragstad | but the owner attribute/project attribute isn't set properly? | 14:45 |
| dansmith | why wouldn't it be? | 14:45 |
| lbragstad | so - project-admin (who are still technically system-administrators in glance) can't view the image added by a system-scoped admin user | 14:45 |
| TheJulia | could be, could also just be wrong | 14:46 |
| dansmith | does a system-scoped admin not have a project_id or something? | 14:46 |
| TheJulia | it does not | 14:46 |
| TheJulia | any project_id presence makes it an project scoped request | 14:47 |
| dansmith | oh, I can see that breaking lots of stuff | 14:47 |
| dansmith | I mean, all over | 14:47 |
| dansmith | when anyone goes to create a resource and tries to set the field of an object that requires a project_id, what do they get? | 14:47 |
| dansmith | I mean, like nova instances have a project_id and it's fairly important :) | 14:47 |
| lbragstad | i think i just confirmed that - http://paste.openstack.org/raw/803510/ | 14:48 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Trivial: add a missing argument to an exception https://review.opendev.org/c/openstack/ironic/+/780270 | 14:48 |
| dansmith | lbragstad: so, I'm not really sure what to say.. that seems like it's going to be a problem everywhere right? | 14:49 |
| dansmith | other than just not letting you create anything as a system-admin, I'm not sure what to do about that | 14:49 |
| dansmith | if a resource requires a project_id, then a system admin shouldn't be able to create those things yeah? | 14:50 |
| dansmith | that's a little more complicated because lots of actions create resources with a project id, like instance actions if you go to shut down an instance, etc | 14:50 |
| lbragstad | unless they have an alternate way of passing that in | 14:50 |
| lbragstad | like this, | 14:50 |
| lbragstad | http://paste.openstack.org/raw/803511/ | 14:51 |
| lbragstad | so - as a system-admin, i can explicitly set the project_id for a private image | 14:52 |
| dansmith | but, you don't want every project to have to start taking a project_id in every API right? | 14:52 |
| TheJulia | I guess the lack of the owner is also why tempest fails on install because it doesn't see images that were uploaded at all | 14:52 |
| lbragstad | dansmith yeah - i don't think so | 14:52 |
| dansmith | lbragstad: so for nova, system-admin will not be able to create instances, or really do anything with them, | 14:52 |
| dansmith | and glance just needs to fail if you don't provide the actual owner on an image create | 14:53 |
| dansmith | s/fail/refuse/ | 14:53 |
| lbragstad | that seems reasonable | 14:53 |
| lbragstad | if the service requires a project ID, but doesn't have a reasonable thing to use, then giving a 400 seems like the right thing | 14:53 |
| dansmith | lbragstad: I guess the alternative is to have a config option for "default_project" which (shudder) is what we use if you show up with a system token and we need a project_id | 14:53 |
| *** rloo has quit IRC | 14:54 | |
| lbragstad | yeah - that's an option | 14:54 |
| dansmith | lbragstad: that's likely going to be a death by a thousand cuts kind of thing, I expect | 14:54 |
| *** rloo has joined #openstack-ironic | 14:54 | |
| dansmith | you might get pretty far down the rabbit hole before you break | 14:54 |
| lbragstad | agree - i think it'll just prolong clients from using the right behavior | 14:54 |
| dansmith | like you've shut down the instance and are going to record the action in the db, and.. oopsie, well, now I have to report failure, but I've already done stuff | 14:54 |
| TheJulia | re: just failing if not supplied or available, makes tons of sense. That is the path ironic is ending up in to an extent with physical node allocations, except our views allow for pure system level use/interaction/allocations too. *joy* | 14:56 |
| lbragstad | i know we said we weren't going to implement system-scope support for glance in Wallaby, but maybe a patch that does determines if glance is dealing with a system-scoped token is worth it to fail early | 14:56 |
| TheJulia | +1000000000 | 14:56 |
| TheJulia | fail if we can't right now because that seems like the best course of action to provide the feedback loop | 14:56 |
| TheJulia | as opposed to ending up with support cases down the road | 14:57 |
| dansmith | well, I'd phrase it differently | 14:57 |
| dansmith | sounds like we've kinda created a regression that needs a mitigation :) | 14:57 |
| TheJulia | It all comes down to perspective :) | 14:57 |
| dansmith | perspective matters in rc phase :) | 14:58 |
| TheJulia | But, it seems kind of reasonable to me that an system-admin shouldn't be able to upload an image unless there is a defined project_id they are doing it on behalf of. It is not a regression that they don't ahve a project id, it is intentional in order to create the necessary separation and be able to apply the delineating logic to enable rbac | 14:59 |
| dansmith | lbragstad: have you tried doing some nova stuff with a system token? might want to see if it needs the same treatment | 14:59 |
| dansmith | TheJulia: well, I'm calling it a regression because glance claims to support some of this experimentally in wallaby, but I'm mostly just trying to help lbragstad phrase this in a way that doesn't have to wait until X :) | 15:00 |
| TheJulia | Ahh | 15:00 |
| TheJulia | Yeah, still seems like a reasonable change to merge in before or even after release since it is providing a guard rail | 15:01 |
| lbragstad | http://paste.openstack.org/raw/803513/ | 15:02 |
| lbragstad | nova ^ isn't enforcing scope | 15:02 |
| lbragstad | and that's a system-scoped token | 15:03 |
| lbragstad | wait... hold on | 15:03 |
| dansmith | also, | 15:04 |
| dansmith | how would nova ask neutron for networks that it can attach, and then ports and things need ownership too | 15:04 |
| lbragstad | ERROR nova.api.openstack.wsgi ValueError: Field `project_id' cannot be None | 15:06 |
| dansmith | likely because our objects are very strict | 15:06 |
| lbragstad | i had an environment variable for OS_PROJECT_ANME set, even though i was trying to use --os-cloud devstack-system-admin | 15:06 |
| lbragstad | that error returns a 500 | 15:06 |
| dansmith | yeah, so the API should prevent us getting that far and return a 400 | 15:07 |
| lbragstad | yeah, agreed | 15:07 |
| dansmith | I guess that's already a bug because I could have shot a request at the api manually with no project and hit that eh? | 15:07 |
| dansmith | i.e. not a regression in wallaby | 15:07 |
| lbragstad | http://paste.openstack.org/raw/803514/ | 15:08 |
| *** anuradha1904 has quit IRC | 15:08 | |
| lbragstad | correct - and nova implemented this stuff in victoria i think | 15:08 |
| lbragstad | so - this particular 500 using system-scoped tokens has been around for at least a cycle | 15:08 |
| dansmith | but is this a result of that? probably not I'd guess | 15:08 |
| lbragstad | well - i need to find where nova gets the project id from | 15:09 |
| lbragstad | if it relies on the context object, then this would be been introduced when nova started adopting system-scoped tokens | 15:09 |
| lbragstad | s/adopting/adding support for/ | 15:10 |
| dansmith | okay | 15:10 |
| lbragstad | since ctx.project_id = None when a token is system-scoped | 15:10 |
| lbragstad | unless nova has another method for supplying a project ID that i'm not aware of? | 15:10 |
| dansmith | okay so maybe we had some inbuilt protection from that before oslo changes? | 15:11 |
| lbragstad | which oslo changes? | 15:12 |
| *** k_mouza has quit IRC | 15:13 | |
| dansmith | I assume there's some stuff in oslo_policy or context that parses the request and token and such and fills those fields based on you saying this would have come in during the system-scope stuff | 15:13 |
| *** k_mouza has joined #openstack-ironic | 15:14 | |
| dansmith | I thought you mean that project_id=None happens when we get a system-scoped token and create a context for it, which I assume is not nova code | 15:14 |
| dansmith | but obviously I don't know how this works :) | 15:14 |
| *** diurnalist has joined #openstack-ironic | 15:15 | |
| lbragstad | oh - i think i'm following no | 15:15 |
| lbragstad | now* | 15:15 |
| lbragstad | yeah - nova implements this in middleware using oslo.context | 15:15 |
| lbragstad | which pulls the token attributes from ksm based on the headers it sets in the request environment | 15:16 |
| lbragstad | the context object is created from those headers and then stuff back into the request environment for the nova api to fetch later | 15:16 |
| dansmith | yeah | 15:16 |
| lbragstad | fwiw - i just tested with a domain-admin and the same 500 exists | 15:16 |
| dansmith | so is nova doing req.get('project', None) or passing req to oslo something? | 15:17 |
| lbragstad | https://github.com/openstack/nova/blob/master/nova/api/auth.py#L90 | 15:18 |
| dansmith | yeah, that's what I was expecting :) | 15:18 |
| lbragstad | so - iiuc, nova api code is expecting to see ctx.project_id to use it for ownership, and it's failing | 15:19 |
| dansmith | oh, that's nova_context not oslo_context (yet) | 15:19 |
| lbragstad | i think nova subclasses the oslo.context request context object | 15:19 |
| dansmith | yeah | 15:19 |
| lbragstad | i don't see a from_environ subclass in nova's implementation https://github.com/openstack/nova/blob/master/nova/context.py | 15:20 |
| dansmith | ack, so, | 15:21 |
| dansmith | what I was getting at before is, | 15:21 |
| lbragstad | full nova log http://paste.openstack.org/raw/803516/ | 15:21 |
| dansmith | I would have expected that early on before system-scope, project_id was always required, so something in that from_environ() would object if it didn't find it, and that was preventing us from ever getting this prior to the system-scope stuff? | 15:22 |
| lbragstad | yeah - potentially | 15:23 |
| lbragstad | fwiw - that log was using a domain-scoped token, which have been around for years | 15:23 |
| lbragstad | system-scoped tokens are just susceptible to the same problem | 15:24 |
| dansmith | yeah, so that's nova's RPC layer being more strict than the REST one | 15:24 |
| * dansmith takes pride | 15:24 | |
| dansmith | ack | 15:24 |
| lbragstad | should the api layer be handling that exception and returning a 400? | 15:24 |
| lbragstad | or - an alternative would be to short-circuit earlier in the API layer so that validation it's doing is consistent with the RPC layer? | 15:25 |
| dansmith | no, I think it probably needs to explicitly check the project_id earlier and never get that far | 15:25 |
| lbragstad | yeah, ok | 15:25 |
| dansmith | this isn't even getting to RPC, it's trying to write to the DB, but the same object is used for that before it makes the rpc call | 15:26 |
| lbragstad | ah | 15:26 |
| *** ociuhandu has quit IRC | 15:32 | |
| *** eagereagle1 has quit IRC | 15:43 | |
| *** ociuhandu has joined #openstack-ironic | 15:47 | |
| *** ociuhandu has quit IRC | 15:53 | |
| *** k_mouza has quit IRC | 15:58 | |
| *** ociuhandu has joined #openstack-ironic | 16:07 | |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add import system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/770614 | 16:11 |
| *** k_mouza has joined #openstack-ironic | 16:11 | |
| *** dhellmann has quit IRC | 16:12 | |
| *** dhellmann has joined #openstack-ironic | 16:13 | |
| arne_wiebalck | dtantsur: from reading the "how to release" docs I think there is not much left to do for me for the stable ironic-lib releases than to wait for good things to happen ... or do I miss anything? | 16:14 |
| dtantsur | this is correct | 16:14 |
| arne_wiebalck | dtantsur: nice, thanks! | 16:16 |
| *** ociuhandu has quit IRC | 16:17 | |
| TheJulia | arne_wiebalck: thanks! | 16:18 |
| openstackgerrit | OpenStack Release Bot proposed openstack/sushy-cli stable/wallaby: Update .gitreview for stable/wallaby https://review.opendev.org/c/openstack/sushy-cli/+/780289 | 16:32 |
| openstackgerrit | OpenStack Release Bot proposed openstack/sushy-cli stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby https://review.opendev.org/c/openstack/sushy-cli/+/780290 | 16:32 |
| openstackgerrit | OpenStack Release Bot proposed openstack/sushy-cli master: Update master for stable/wallaby https://review.opendev.org/c/openstack/sushy-cli/+/780291 | 16:32 |
| openstackgerrit | OpenStack Release Bot proposed openstack/sushy-cli master: Add Python3 xena unit tests https://review.opendev.org/c/openstack/sushy-cli/+/780292 | 16:33 |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Update export system configuration https://review.opendev.org/c/x/sushy-oem-idrac/+/759425 | 16:38 |
| openstackgerrit | OpenStack Release Bot proposed openstack/python-ironic-inspector-client stable/wallaby: Update .gitreview for stable/wallaby https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/780294 | 16:39 |
| openstackgerrit | OpenStack Release Bot proposed openstack/python-ironic-inspector-client stable/wallaby: Update TOX_CONSTRAINTS_FILE for stable/wallaby https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/780295 | 16:39 |
| openstackgerrit | OpenStack Release Bot proposed openstack/python-ironic-inspector-client master: Update master for stable/wallaby https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/780296 | 16:40 |
| openstackgerrit | OpenStack Release Bot proposed openstack/python-ironic-inspector-client master: Add Python3 xena unit tests https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/780298 | 16:40 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Allocation support for project scoped RBAC https://review.opendev.org/c/openstack/ironic/+/778340 | 16:47 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP Scoped RBAC Devstack Plugin support https://review.opendev.org/c/openstack/ironic/+/778957 | 16:47 |
| TheJulia | now to see *if* tempest will install :) | 16:48 |
| JayF | anyone coming to spuc in 10m? | 16:49 |
| dtantsur | I can if anyone is interested | 16:50 |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add get PXE port macs for BIOS mode https://review.opendev.org/c/x/sushy-oem-idrac/+/770904 | 16:50 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Trivial: add a missing argument to an exception https://review.opendev.org/c/openstack/ironic/+/780270 | 16:50 |
| JayF | I'll be there :) | 16:50 |
| TheJulia | yeah, I can join in | 16:52 |
| *** lucasagomes has quit IRC | 16:57 | |
| TheJulia | spuc? | 17:00 |
| dtantsur | omw | 17:01 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Deprecate legacy policies, update project scoped docs https://review.opendev.org/c/openstack/ironic/+/778767 | 17:09 |
| openstackgerrit | Richard G. Pioso proposed x/sushy-oem-idrac master: Add get PXE port MACs for BIOS mode https://review.opendev.org/c/x/sushy-oem-idrac/+/770904 | 17:09 |
| *** ociuhandu has joined #openstack-ironic | 17:11 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Increment API version for Secure RBAC https://review.opendev.org/c/openstack/ironic/+/778947 | 17:11 |
| openstackgerrit | Merged x/sushy-oem-idrac master: Add get PXE port MACs for BIOS mode https://review.opendev.org/c/x/sushy-oem-idrac/+/770904 | 17:38 |
| *** ociuhandu has quit IRC | 17:43 | |
| *** ociuhandu has joined #openstack-ironic | 17:44 | |
| *** k_mouza has quit IRC | 17:48 | |
| *** ociuhandu has quit IRC | 17:49 | |
| openstackgerrit | Aija Jauntēva proposed x/sushy-oem-idrac master: Add resources to put iDRAC in known good state https://review.opendev.org/c/x/sushy-oem-idrac/+/721596 | 17:53 |
| *** gyee has joined #openstack-ironic | 17:59 | |
| dtantsur | TheJulia, do we need a release with https://opendev.org/openstack/sushy/commit/f52aac46cd0a77f9282591f4d888a241b6eb0a39 now? we're near the freeze, I guess we'll need an exception | 18:04 |
| TheJulia | ohh yes | 18:05 |
| *** derekh has quit IRC | 18:07 | |
| *** dtantsur is now known as dtantsur|afk | 18:10 | |
| dtantsur|afk | o/ | 18:10 |
| *** ociuhandu has joined #openstack-ironic | 18:10 | |
| TheJulia | lbragstad: oh, I think I see what is going on with glance. We're enforcing scope in the job config and then tempest tries to check the image and still can't see them because it is apparently trying something $else | 18:14 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP Scoped RBAC Devstack Plugin support https://review.opendev.org/c/openstack/ironic/+/778957 | 18:14 |
| TheJulia | oh The service catalog is empty. | 18:23 |
| *** ociuhandu has quit IRC | 18:41 | |
| *** ociuhandu has joined #openstack-ironic | 18:43 | |
| *** diurnalist has quit IRC | 18:43 | |
| *** ociuhandu has quit IRC | 18:47 | |
| *** diurnalist has joined #openstack-ironic | 18:59 | |
| *** ociuhandu has joined #openstack-ironic | 19:12 | |
| *** ociuhandu has quit IRC | 19:17 | |
| TheJulia | oh, its because we unset env vars for project | 19:18 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP Scoped RBAC Devstack Plugin support https://review.opendev.org/c/openstack/ironic/+/778957 | 19:25 |
| *** jamesdenton has quit IRC | 19:26 | |
| *** jamesdenton has joined #openstack-ironic | 19:26 | |
| *** zzzeek has quit IRC | 19:45 | |
| *** ociuhandu has joined #openstack-ironic | 19:46 | |
| *** zzzeek has joined #openstack-ironic | 19:46 | |
| *** mraineri has joined #openstack-ironic | 19:47 | |
| *** k_mouza has joined #openstack-ironic | 19:49 | |
| mraineri | TheJulia: I have a topic I'd like to bring up at a future meeting; the Redfish Forum has a public facing forum to take feedback, answer questions, etc. I'd like to socialize it so it can be used by anyone to get in touch with folks in the DMTF. | 19:54 |
| *** k_mouza has quit IRC | 19:54 | |
| TheJulia | mraineri: that or possibly the BareMetal SIG could work? | 19:55 |
| mraineri | Yeah, I think that should work | 19:55 |
| rpioso | mraineri, TheJulia: How about both? | 19:56 |
| mraineri | That works too | 19:56 |
| TheJulia | This could also work! | 19:56 |
| rpioso | TheJulia: May we add it to Monday's agenda? | 19:57 |
| TheJulia | sure | 20:00 |
| *** ociuhandu has quit IRC | 20:01 | |
| *** ociuhandu has joined #openstack-ironic | 20:01 | |
| zer0c00l | If i have to add a new library for requirements.txt , is there a process/tool? | 20:02 |
| rpioso | TheJulia: Thank you :-) | 20:02 |
| zer0c00l | the library in question is pycdlib - LGPLv2 (according to docs, this license is fine) | 20:02 |
| zer0c00l | https://pypi.org/project/pycdlib/ | 20:02 |
| zer0c00l | At the top requirements.txt says - the order of things in there matter. | 20:03 |
| iurygregory | zer0c00l, you need to check if the library is in openstack/requirements (if it's not you need to add there before adding in Ironic) | 20:07 |
| zer0c00l | iurygregory: thanks | 20:08 |
| zer0c00l | iurygregory: https://github.com/openstack/requirements/blob/master/global-requirements.txt? | 20:08 |
| iurygregory | zer0c00l, https://docs.openstack.org/project-team-guide/dependency-management.html#adding-a-new-dependency this doc would explain the process | 20:09 |
| iurygregory | yeah | 20:10 |
| openstackgerrit | Merged openstack/ironic master: Allow users to configure priority for {create,delete}_configuration https://review.opendev.org/c/openstack/ironic/+/778145 | 20:16 |
| rpioso | TheJulia: May we remove the stuffs presently on the Ironic Project Team Meeting agenda? I believe it's from this past Monday's meeting. | 20:25 |
| openstackgerrit | Merged openstack/networking-generic-switch stable/train: Remove grenade jobs from old stable branches https://review.opendev.org/c/openstack/networking-generic-switch/+/773335 | 20:26 |
| iurygregory | mnaser, now you can celebrate =) | 20:46 |
| *** ociuhandu has quit IRC | 20:47 | |
| *** ociuhandu has joined #openstack-ironic | 20:48 | |
| rpioso | TheJulia: I removed the old agenda content and mraineri added a discussion topic about the Redfish public forum. | 20:50 |
| iurygregory | question about redfish, it's possible that /redfish/v1/Systems will return 404 or 0 members? =) | 20:51 |
| *** ociuhandu has quit IRC | 20:53 | |
| mraineri | Depending on the type of product, it's very possible | 20:54 |
| mraineri | Things like rack PDUs that implement Redfish for management would return 404 for /redfish/v1/Systems | 20:54 |
| mraineri | Composable infrastructures might start off with an empty collection, members are populated when clients request systems to be created | 20:54 |
| mraineri | If you're focused on more of a general server, then no, I would not expect that to happen | 20:56 |
| iurygregory | tks =) | 20:57 |
| *** zzzeek has quit IRC | 21:04 | |
| *** zzzeek has joined #openstack-ironic | 21:05 | |
| JayF | zer0c00l: there already exists code in Ironic to create ISO images | 21:14 |
| JayF | zer0c00l: I wonder if it | 21:14 |
| JayF | **if it's possible to use that code as a library instead of introducing a new lib | 21:14 |
| zer0c00l | JayF: can you give me pointers towards it? | 21:15 |
| zer0c00l | AFAIK it directly uses genisoimage (utils.execute) | 21:16 |
| JayF | probably somewhere in virtual media code | 21:16 |
| JayF | yeah; if that's what it does why shouldn't you do the same? | 21:16 |
| zer0c00l | the code i am writing actually "reads" iso image | 21:16 |
| rpioso | mraineri: What would the value of the Systems property in the service root resource be for those types of products? | 21:16 |
| JayF | zer0c00l: oooh | 21:16 |
| JayF | zer0c00l: in that case, perhaps ignore my input? lol. I don't think we do much ISO *reading* | 21:16 |
| zer0c00l | basically i am looking at https://github.com/clalancette/pycdlib | 21:17 |
| JayF | I will warn you not to underestimate the amounto f effort it can take to get something added to g-r | 21:17 |
| zer0c00l | looks well maintained and LGPLv2 | 21:17 |
| zer0c00l | gosh i know | 21:17 |
| zer0c00l | :( | 21:17 |
| zer0c00l | i don't know if i have a choice | 21:17 |
| rpioso | mraineri: /redfish/v1 > Systems | 21:17 |
| JayF | What is the "x" for this? Why do you want to read iso images? | 21:18 |
| *** ociuhandu has joined #openstack-ironic | 21:19 | |
| mraineri | rpioso: In the cases where a product doesn't support /redfish/v1/Systems, I would expect the "Systems" property in service root to not be present | 21:21 |
| mraineri | Generally speaking, properties are not implemented if the functionality doesn't exist | 21:21 |
| rpioso | mraineri: Thank you :-) | 21:21 |
| rpioso | iurygregory: ^^^ | 21:22 |
| iurygregory | oh I see! | 21:22 |
| iurygregory | so redfish/v1/ wouldn't have the info sor redfish/v1/Systems | 21:23 |
| mraineri | Bingo! | 21:23 |
| rpioso | iurygregory: Generally for things like rack PDUs :) | 21:23 |
| zer0c00l | https://docs.openstack.org/project-team-guide/dependency-management.html#adding-a-new-dependency is this generate-constraints command right? | 21:23 |
| *** ociuhandu has quit IRC | 21:23 | |
| zer0c00l | anyone ran this recently? | 21:23 |
| iurygregory | rpioso, yeah, I'm adding some tests for gofish client in metal3 since we will check if virtual media is available for the nodes =) | 21:24 |
| JayF | the inclusion of only py2.7/py3.6 indicates to me there's no way that command is right | 21:24 |
| iurygregory | https://github.com/metal3-io/baremetal-operator/pull/720/ | 21:24 |
| mraineri | iurygregory: I actually added some virtual media stuff to the Tacklebox project in Redfish; it's been working well against the different systems I have available to me | 21:25 |
| mraineri | https://github.com/DMTF/Redfish-Tacklebox | 21:25 |
| mraineri | Check out scripts/rf_virtual_media.py | 21:26 |
| JayF | zer0c00l: I'd try running it against py3.6/3.7/3.8 | 21:26 |
| JayF | zer0c00l: or I think there's an IRC for just the requirements project? | 21:26 |
| mraineri | At the very least it might give you some ideas for how I thought about solving some day to day virtual media tasks | 21:26 |
| JayF | yep, #openstack-requirements | 21:26 |
| zer0c00l | thanks JayF | 21:26 |
| iurygregory | mraineri, tks for sharing! | 21:27 |
| *** diurnali1t has joined #openstack-ironic | 21:27 | |
| *** diurnalist has quit IRC | 21:27 | |
| *** ociuhandu has joined #openstack-ironic | 21:46 | |
| *** hoonetorg has joined #openstack-ironic | 21:49 | |
| *** ociuhandu has quit IRC | 21:50 | |
| *** ociuhandu has joined #openstack-ironic | 21:54 | |
| *** ociuhandu has quit IRC | 21:59 | |
| *** ociuhandu has joined #openstack-ironic | 22:05 | |
| *** ociuhandu has quit IRC | 22:09 | |
| *** ociuhandu has joined #openstack-ironic | 22:22 | |
| *** ociuhandu has quit IRC | 22:36 | |
| TheJulia | rpioso: Thanks, I'll put back the appropriate reminders later today or over the weekend | 22:55 |
| rpioso | TheJulia: Thanks! | 22:55 |
| * TheJulia goes and starts dinner since it needs about 1.5 hours to cook | 22:56 | |
| *** diurnali1t has quit IRC | 23:05 | |
| zer0c00l | what? | 23:16 |
| zer0c00l | 1.5 hrs? | 23:16 |
| zer0c00l | what are you cooking? | 23:16 |
| TheJulia | zer0c00l: manicotti | 23:33 |
| *** tzumainn has quit IRC | 23:36 | |
| openstackgerrit | Jay Faulkner proposed openstack/ironic-specs master: No Conductor to IPA Communication spec https://review.opendev.org/c/openstack/ironic-specs/+/777172 | 23:37 |
| *** rloo has quit IRC | 23:39 | |
| JayF | that would sound good TheJulia if I wasn't hoping the answer was one of your pizzas I get to eat with my eyes on FB all the time lol | 23:42 |
| TheJulia | If the restaurant business was not brutal, I'd go into it. | 23:50 |
| *** diurnalist has joined #openstack-ironic | 23:52 | |
| *** diurnalist has quit IRC | 23:56 | |
| *** bfournie has quit IRC | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!