Wednesday, 2021-11-17

opendevreviewSteve Baker proposed openstack/ironic stable/train: Fix redfish-virtual-media file permission  https://review.opendev.org/c/openstack/ironic/+/81818301:50
hgy__Good afternoon, ironic!06:53
arne_wiebalckGood morning, hgy__ and Ironic!07:07
rpittaugood morning ironic! o/07:33
rpittauif anyone has a moment I'd like an opinion on this https://review.opendev.org/c/openstack/ironic-python-agent/+/775391 since I rebased it already a couple of times :)08:26
opendevreviewMerged openstack/ironic-python-agent master: Move manage_uefi from the image extension to a public location  https://review.opendev.org/c/openstack/ironic-python-agent/+/81565110:14
dtantsurgood morning folks10:50
rpittaugood morning dtantsur :)10:50
dtantsurmy eventlet fix has been merged \o/10:56
rpittauyep, that's great10:56
dtantsurand even got into 0.33.0 \o/10:56
rpittauyeah, so hopefully we can just skip 0.32.010:58
dtantsuryep, updating my requirements patch now10:58
dtantsuralso a partial python 3.10 support is in the new release10:59
rpittauyeah, saw that, was going to test later10:59
rpittauwell at least unit tests in ironic and ironic-python-agent look kind of ok on python 3.10 with eventlet 0.33.0, I mean no errors related to eventlet as before11:08
dtantsursweet!11:08
dtantsurrpittau: I think I've already asked you whether centos 7 is fine with --json in lsblk?11:11
rpittaudtantsur: heh... not with the basic util-linux (version 2.23) unfortunately, same story as blkid, I originally tested with the version from virt7 (2.29), so I guess we'll have to wait11:26
dtantsurle sigh11:27
dtantsuruntil 2024, I assume? :)11:27
rpittauthat or we convince centos people that having json output is a mission critical bug fix11:30
dtantsurfor 7? extremely unlikely11:30
rpittauyeah, I'll set a reminder for July 1st 2024 :)11:31
opendevreviewMerged openstack/ironic master: Reduce the number of small functions in pxe_utils  https://review.opendev.org/c/openstack/ironic/+/81799111:36
opendevreviewDmitry Tantsur proposed openstack/bifrost master: Revert "Explicitly trap on ERR" and fix exit code hanlding  https://review.opendev.org/c/openstack/bifrost/+/81797811:48
muellerbehello ironic, hello @all12:35
rpittauhello muellerbe 13:05
muellerbehello rpittau13:09
dtantsurokay, I'm puzzled. https://zuul.opendev.org/t/openstack/build/a43fb0464e60421094020ace2ac60da7/log/job-output.txt#23716 and then https://zuul.opendev.org/t/openstack/build/a43fb0464e60421094020ace2ac60da7/log/job-output.txt#2371713:25
dtantsurhow it is okay after "exit 2"? what the hell is going on?13:25
rpittausomething odd on the script controlling that13:27
dtantsuryep, it's because of | tee in ansible13:28
dtantsurfixing now13:28
rpittauah!13:29
opendevreviewDmitry Tantsur proposed openstack/bifrost master: Revert "Explicitly trap on ERR" and fix exit code hanlding  https://review.opendev.org/c/openstack/bifrost/+/81797813:33
TheJuliaIt is good to be home14:10
dtantsurTheJulia: welcome!14:10
rpittauhey TheJulia :)14:14
* TheJulia caffinates14:14
TheJuliaproperly, with an espresso based beverage14:14
TheJuliaSo has there been any consensus or thought on tinycore 10.x?14:40
iurygregoryOpenInfra Live: Keynotes will start in 3 minutes14:57
TheJuliahttps://www.youtube.com/watch?v=sXkpjvX54ug <-- The link on eventbrite just sends us to youtube. And it is publicly viewable if you search it up.14:59
TheJuliaso... I guess we *could* update the ussuri devstack plugin to sed edit the tinycore build script15:11
dtantsurokay, https://review.opendev.org/c/openstack/bifrost/+/817978 now works and shows the correct CI status, i.e. completely red15:12
dtantsurwho do we bribe to land https://review.opendev.org/c/openstack/requirements/+/818062 ?15:13
TheJuliahmmmmm15:18
TheJuliathe release team?15:18
TheJuliawell, requirements is a team on it's own15:19
dtantsurI've pinged #openstack-requirements already15:19
TheJuliaYeah15:26
TheJulia:(15:26
TheJuliaCould everyone be watching youtube?15:26
opendevreviewJulia Kreger proposed openstack/ironic stable/ussuri: CI: Get tinyipa build working for CI usage  https://review.opendev.org/c/openstack/ironic/+/81824115:35
TheJuliadtantsur: ^^ a little more expansive, but includes a reno warning which should get rendered in15:35
TheJuliaand the reno builds, so that is a goodsignâ„¢15:38
dtantsurk, +215:52
dtantsurneed to go now, see you tomorrow15:52
TheJuliao/15:52
dtantsuroh, the requirements patch has just been approved! please recheck it if needed15:52
TheJuliadtantsur: sure15:53
NobodyCamGood Morning Ironic'er, Happy Hump Day!16:52
NobodyCamerr, *Ironic'ers16:53
NobodyCam;p16:53
rpittauheeey NobodyCam :)16:55
NobodyCamo/ rpittau 16:59
TheJuliagah, looks like entirely unrelated failures for my last change17:14
arne_wiebalckhey NobodyCam o/17:21
arne_wiebalckrpioso: I have some new hardware I am trying to use with redfish17:22
NobodyCammorning TheJulia and arne_wiebalck 17:22
arne_wiebalckrpioso: I ran into some issues and now I am testing the profiles to see if they would flag issues17:22
arne_wiebalckrpioso: and they do, e.g. the endpoint does not provide allowable values for Reset (so Ironic has to guess) and the interop profile reports this as well \o/17:23
arne_wiebalckrpioso: I knew the endpoint does not provide this since sushy issues a warning "Could not figure out the allowed values for the reset system action  ..."17:24
arne_wiebalckrpioso: one question, though: when I run the interop validator for the power profile, for instance, it seems like it is checking all kinds of things, not only the profile (which is short), it runs for a few minutes in fact ... is that correct?17:26
rpittaugood night! o/17:31
arne_wiebalckrpioso: it seems it is :)17:38
TheJuliagood morning NobodyCam 17:40
NobodyCamO/ 17:40
opendevreviewMerged openstack/ironic-python-agent bugfix/8.1: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81747017:44
arne_wiebalckbye everyone, see you tomorrow o/17:45
TheJuliaDetails: {'code': 403, 'message': 'Quota exceeded for ram: Requested 333096, but already used 0 of 51200 ram'} <-- wut17:48
rpiosoarne_wiebalck: Apologies for my delayed response ... If memory serves, it does a significant amount of preparatory processing before analyzing the profile, including GETs of the Redfish resources offered by the service/BMC. Time for a server upgrade? I have recommendations :-)19:40
rpiosoarne_wiebalck: Glad to hear it flagged the issue. Consistency can be good.19:41
stevebaker[m]Good morning!19:47
TheJuliagood morning stevebaker[m] 19:47
stevebaker[m]TheJulia: welcome back to civilization19:50
TheJuliaThanks19:51
arne_wiebalckHey stevebaker[m] , thanks for the SIG video !20:06
stevebaker[m]arne_wiebalck: you're welcome :)20:11
arne_wiebalckrpioso: I tried different profiles from the proposed patch with the validator and they come back with different results, so seems all good. Quite pleased to see it found the ComputerSystem.Reset issue! Runtime is several minutes per profile (which I did not remember).20:12
arne_wiebalckrpioso: I guess we should try and move the interop profiles forward, i.e. get the patch merged. Or is there something fundamental missing?20:13
arne_wiebalckstevebaker[m]: almost 170 views already, seems popular! :)20:19
stevebaker[m]nice :)20:22
stevebaker[m]Why is there blockchain in this openinfra keynote, grrr20:54
admiyoan openstack command to list bm nodes by flavor would not be amiss22:43
TheJuliaIronic, and ultimately even the command are unaware of flavors or understanding them really22:43
TheJuliaflavors match data supplied to the node properties and resource_class field22:43
TheJuliaI believe you *can* filter by resource class, fwiw22:44
admiyoTHat would be nice22:45
admiyoI am sure this is all due to a typo I made, but dang if I can see where22:46
TheJuliagive me a couple of minutes and I'll look too22:46
admiyoWe have two types of baremetal nodes here.  jades and mystiques (internal names)22:48
admiyothe jades work fine22:48
admiyoI suspect mostly due to dumb luck22:48
admiyomy call to openstack baremetal node create has --resource-class $ironic_resource_name \22:49
admiyoand for the mystique it is22:49
admiyobaremetal-mystique22:49
admiyohowever, on the bm node itslef I see22:49
admiyo$ openstack baremetal node show mystique01-r097 -f json | jq '.resource_class'22:50
admiyo"98:03:9B:AD:49:0D"22:50
TheJuliadoes the flavor just not work22:50
* TheJulia blinks22:50
TheJuliathat seems horribly wrong22:50
TheJuliais that *really* what is in the field?22:50
admiyolet me see what a Jade that works has....22:50
admiyoAyuh22:50
TheJuliacould the import somehow have had a json error or something?22:50
TheJuliabecause resource class shouldn't be a MAC address22:51
admiyo$ openstack baremetal node show jade11-r097 -f json | jq '.resource_class'22:51
admiyo"baremetal-jade2"22:51
admiyoI bet a command switch got ignored...22:51
admiyomy typo I bet I bet I bet22:51
TheJuliaif I remember correctly, you can change it22:51
admiyoCounting by adam.  0, 1,2,522:52
admiyo$ openstack baremetal node show mystique01-r097 -f json | jq '.resource_class'22:54
admiyo"baremetal-mystique"22:54
admiyomuch goooder22:54
TheJuliagive it ~2-3 minutes and I guess try to see if your flavor will match it with nova22:55
admiyoI still think it would be nice to be able to do a baremetal node list --flavor bm.mystique22:58
* TheJulia grumbles about RBAC tests22:59
TheJuliathat requires ironic to be fully flavor aware, and do in-house filter22:59
admiyoRBAC?  That is like my favorite thing!22:59
TheJuliaand there is no guarentee it will match like the placement service, so it would be an unreliable indicator I guess22:59
TheJuliaI'm adding a new role today because of indecision and scope creep23:00
TheJuliaso as such, I hate the world23:00
admiyoWell, it isnot goingto do worse than Error 500: No valid host was found. That my user is getting23:00
admiyoA new role?23:00
TheJuliaa manager role23:00
TheJuliahttps://review.opendev.org/c/openstack/governance/+/815158/14/goals/proposed/consistent-and-secure-rbac.rst23:00
admiyoSO less than admin, more then end user?23:00
TheJuliayeah23:01
admiyoscoped to a project or no?23:01
TheJuliaproject scoped23:01
TheJuliain this case, in ironic, it will likely be the same as owner/lessee admin23:01
admiyoGod that sounds so yuck23:02
TheJuliato update, yeah23:02
admiyo"if an operator uses a system-scoped token to create an instance23:02
admiyofor a user in a specific project"23:02
TheJuliamainly all in test code23:02
admiyoDont23:02
TheJuliaadmiyo: ++23:02
TheJuliayeah, don't. refuse23:02
admiyoIt is a mistake23:02
TheJuliathat, specifically yeah23:02
TheJuliathere are so many different nuances23:03
opendevreviewMerged openstack/ironic stable/ussuri: CI: Get tinyipa build working for CI usage  https://review.opendev.org/c/openstack/ironic/+/81824123:03
* TheJulia dances23:03
admiyoSystem scoped tokens maybe hould be able to clean up other people's messes, but they should not be allocating new resources in projects23:03
opendevreviewJulia Kreger proposed openstack/ironic stable/train: CI: Get tinyipa build working for CI usage  https://review.opendev.org/c/openstack/ironic/+/81816223:03
admiyoTheJulia, I'd pocket veto that role If I were you23:04
TheJuliamanager on it's own, in project scope is harmless to be honest. Just lots of test busywork23:05
TheJulia*anyway* there is a way to query placement, but I don't know how23:05
admiyoTHat is what a project admin is supposed to be23:05
admiyothis smells alot like "We didn't really fix 968696" to me23:05
TheJuliaadmiyo: indeed :(23:06
TheJuliaor people want to be able to abuse system scope23:06
TheJuliaand keep project admin unchanged23:06
TheJuliawhich is insane, but hey23:06
admiyoAnyway, if operators need to do that kind of operation, they should do something like this: https://adam.younglogic.com/2018/02/openstack-hmt-cloudforms/23:06
TheJuliaI only work here some days23:06
admiyoSo you treat the top level domain as a project, and use project scoped tokens to do project scoped work23:07
admiyolook at it this way, Kerberos doens't let you use a TGT to authenticate to a service, you need to get a service ticket for that, no matter who you are.23:08
TheJuliaI also think some operators fear changing anything, preferably people just use ironic with system scoped tokens except in limited cases, but Ironic is a weird case where the whole model is kind of flipped upside down23:08
TheJulia"Hi, we have actual *real* things here"23:08
admiyoNope23:11
admiyoMetal is just another resource23:11
TheJuliaI mean, we basically had amodel which was the system scoped use model23:11
admiyoWhat openstack lacks is the ability to have one project own a resource, and lend it to another project23:11
TheJuliaso for us, it was about formalizing real project support while adding explicit support of the system scope23:12
TheJuliaadmiyo: so, we actaully have the capability for that, but it has to be delegated through action23:12
TheJuliawhich doubles the tests :\23:12
admiyoI mean, it is a one of for each resoure in each of the various services23:13
TheJuliahmm, and delegate out from there23:13
TheJuliaso you build a tree23:13
TheJuliaof sorts23:13
admiyoit should be like inodes and dentries.  If I mount the same inode in two directires, I can make one readonly, one read write23:14
admiyosame thing should be true of any openstack resource, but we don't put project names into the URLs we use to access the resources, only global identifiers23:14
admiyoand the project name is then read off the resource.  23:15
admiyowell, the id, not even the name23:15
TheJuliawell, the couple instances where project-ids were used in urls, it has been basically a total disaster for them to adopt the system scope23:15
admiyoThat means it is working23:15
TheJuliabecause, well, project-id has to be known23:15
admiyoand this is why I am not longer excited about RBAC and it is no longer my favorite thing23:15
admiyoI blame termie23:16
TheJuliaunfortunately, the huge operators needing system scoped readers for accounting/auditing/support desks don't want/need to be project aware and scope themselves into a project to pull a list23:16
TheJulialol23:16
admiyoHa...such PTSD from the termie years I actually scrolled to make sure he wasnt' in the room23:16
TheJuliaCute puppies. It is the oly way23:16
TheJuliaonly way23:16
TheJuliaor kittens.23:16
TheJuliaKittens work as well, especially when they are a bonded pair and they are playing.23:17
admiyoThen make non-project scoped APIs for them.  Make the API react differently to a project scoped and a system scoped token. 23:17
admiyoI mean, I am kindof in that boat myself, and I only have 2 projects23:17
TheJulia++23:17
admiyoBaremetal nodes are treated like hypervisors AND like end resources23:18
TheJuliayes, in part, except we don't record a node owner explicitly23:18
TheJuliaor a lessee23:18
* TheJulia needs to put that patch in23:19
admiyoBut for the RH undercloud/overcloud split, the undercloud probably should just be one big project23:19
TheJulia++23:19
admiyoand...that is probably whey they want this, because they really do want to treat baremetal nodes as system resources, not project23:19
admiyoHrm...so, Adam, <MR. Smarty pants...which should they be?23:20
TheJuliathem, nah23:20
admiyoBM nodes?  Probably system scoped, right?23:21
TheJuliaI don't think the manager role idea came at all from rh but seems to have come from community interactions23:21
TheJuliaadmiyo: originally, the right way was to create a dedicated baremetal project and grant explicit baremetal_admin or baremetal_observer roles23:21
admiyoA project owning a node would be too restrictive, but I bet that is a common pattern, where different organization want the API to manage their HW, but want to own it themselves23:21
TheJuliabut in ironic, operating mode wise, it is like the system owns everything, unless access has been explicitly permitted 23:22
TheJuliaeasiest way to describe it23:22
admiyoI have not yet implemented quota in my tiny test cluster, but I am so tempted to.  I have one guy that just grabs up all of the nodes23:22
admiyoand breaks them23:22
admiyodamn firmware testeers23:22
TheJuliaugh :(23:22
TheJuliaI will glady review/approve quota support23:23
admiyoIt is not the right mechanism, though23:23
admiyoI should be able to make BM node a global resource, and then assign them to projects23:23
admiyothat way, if he reprovisions, he does'nt lose it23:23
JayFYou can implement something-ish like that by having a project designated system23:23
JayFand leasing nodes out to the tenant projects23:23
admiyotenant projects?23:24
JayFjust a made-up term for "a project that has a bm node leased to it"23:24
admiyois that like Dollar Yens?23:24
admiyoShekel Rupees23:24
TheJuliaadmiyo: this is why we have owner, and lessee with differing level of access23:24
TheJuliaowners "own" the nodes forever, lesses just have them on loan23:25
admiyowhat are valid values for owner? 23:25
TheJuliaowners can rip the nodes away if needed23:25
TheJuliaproject_id23:26
admiyoAcha.  That is exactly what I want23:26
TheJuliaI'm almost... done with this patch for the day23:26
TheJuliait might semi-click if you look at it once I post it23:27
TheJuliaIts wrong, I need to revise it some, I found some tests I need to fix where I did stupid human things when I conjured them originally23:27
JayFadmiyo: I just got your joke. I forgot that "projects" in openstack used to be a synonym for "tenant"23:27
admiyoI'm old23:27
JayFAs are we all :D 23:28
admiyoand I was the one responsible for implementing the code changes in Keystone for that23:28
JayFI think I started on Ironic back in Icehouse. I don't really work on it anymore though, which TBH has been a little refreshing.23:28
JayFAlthough I miss the people so I hang out in here, and it looked extra spicy this afternoon so I unlurked :D 23:28
TheJuliaCan we just start a "troublemaking stackers who need coffee support group" ?23:28
admiyoI think we already have, and you just named it.23:28
TheJuliaToo early for spicy beverages!23:29
TheJulia;)23:29
admiyoNot where I sit it isn't23:29
TheJuliaheh23:29
admiyohttps://pics.me.me/half-the-day-i-wonder-if-its-too-late-for-coffee-51534164.png23:29
JayFadmiyo: who are you?23:30
admiyothe artist formerly known as ayoung23:30
JayFaha; makes sense23:30
admiyoBut I blew up my laptop right before leaving Red Hat, and decided to go with the old Nickname when I set up nickserv23:30
TheJuliaheh23:30
TheJuliaadmiyo: was this like a.... literal detonation of the laptop?23:31
JayFWhat/where do you do now?23:31
TheJuliaInquiring minds want to know if so and if there is high speed video23:31
admiyoAnd by blew up, I mean I meant to write to an SD card and instead wrote to the NVME device and wiped, amoung other things, my key file23:31
TheJuliaadmiyo: doh :(23:31
JayFThere are benefits to the new kernel /dev/nvme0[snip] device names for nvme for sure :D23:31
admiyoIts kinda like burning down your house when you move so you don't need to pack23:31
JayFmakes writing USB images a lot less scary23:31
admiyoOh, it was namde nvme.  I just was brain dead23:32
JayF😱23:32
admiyoI am now at Ampere23:32
admiyoI'm on the software team, and I am setting up systems for Dev Ops type functioning.  I inherited an OpenStack cluster, and ,well, I need to learn how to admin now.  I am a poor admin.23:33
JayFGood luck :D 23:34
opendevreviewJulia Kreger proposed openstack/ironic master: Fix some of the SRBAC tests  https://review.opendev.org/c/openstack/ironic/+/81829823:34
opendevreviewJulia Kreger proposed openstack/ironic master: WIP: project scoped manager support  https://review.opendev.org/c/openstack/ironic/+/81829923:34
admiyoI can probalby use reservations to test the flavor node matching 23:34
* TheJulia found more project scoped rbac tests to double/triple/quadrouple test23:34
TheJuliasoooo many lines23:35
admiyoThis is why I wanted to pull all of the RBAC out of the projects and enforce it in middleware23:35
TheJuliaI'm just glad we do it all in our API because having to do some in API and some in the conductor/past rpc code path is bonkers23:38
admiyoIt really is two distinct checks:  does the role on the token match the role for the API, and does the project on the token match the project on the resource. The first part can be done in MIddleware, the second needs the object from the Database for read/mod/delete23:40
TheJuliayup23:45
admiyoWhat is the command to create a lease?23:45
admiyoreservation: null23:47
admiyoOK, so if I modify a node to set the project_id as the owner, then a user needs a reservation in order to be able to access it?23:47

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!