| rpittau | good morning ironic! o/ | 08:52 |
|---|---|---|
| masghar | Good morning! | 09:15 |
| songwenping_ | tkajinam, iurygregory, morning :), i am sorry i always offline, but like this case with min_microversion 1.28: ironic_tempest_plugin.tests.api.admin.test_nodes.TestNodesVif#test_vif_on_portgroup, is it skipped for victroy branch job? | 10:14 |
| iurygregory | good morning Ironic | 11:47 |
| iurygregory | songwenping_, do you have a link so I can take a look? | 11:47 |
| iurygregory | songwenping_, so, `tests.api` are only run in the ironic-tempest-functional-python3 jobs, by victory branch I think you wanted to say victoria right? (We don't run functional tests in this release since is Unmaintained) | 12:08 |
| dtantsur | iurygregory: hi, could you maybe check https://review.opendev.org/c/openstack/bifrost/+/896925/ today? | 13:34 |
| iurygregory | dtantsur, it's on my list =) | 13:34 |
| iurygregory | I will prioritize this one | 13:35 |
| dtantsur | thx | 13:37 |
| iurygregory | tks for the review in the tests, I totally forgot to update to use the constant /facepalm | 13:38 |
| opendevreview | Iury Gregory Melo Ferreira proposed openstack/ironic master: RedfishFirmwareInterface - Unit Tests & More logs https://review.opendev.org/c/openstack/ironic/+/903379 | 13:45 |
| dtantsur | TheJulia: JFYI (no urgency) https://github.com/metal3-io/metal3-docs/pull/373/files | 14:20 |
| opendevreview | Julia Kreger proposed openstack/ironic-tempest-plugin master: DNM/WIP: Detect misconfig and navigate https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/906432 | 15:08 |
| opendevreview | Verification of a change to openstack/bifrost master failed: Configure the new inspection implementation https://review.opendev.org/c/openstack/bifrost/+/896925 | 15:14 |
| dtantsur | TheJulia: hey, is it possible that your rbac changes broke bifrost? https://c040fcb6b9079c7be14d-364fef5d0294fafc2826805b949e4cc6.ssl.cf5.rackcdn.com/896925/6/gate/bifrost-integration-tinyipa-keystone-ubuntu-jammy/bc22acd/job-output.txt | 15:16 |
| dtantsur | "\"baremetal:conductor:get\": \"(role:reader and system_scope:all) or (role:service and system_scope:all)\" requires a scope of ['system'], request was made with project scope. (HTTP 500)" | 15:16 |
| TheJulia | yup, bifrost needs to use system scope to get insight into conductors | 15:17 |
| dtantsur | any ideas on how to fix that? | 15:17 |
| dtantsur | rbac is the most foreign part of ironic for me :) | 15:17 |
| TheJulia | yeah, uhhhh | 15:17 |
| TheJulia | do you know where we write out cloud config? | 15:17 |
| dtantsur | can find, one sec | 15:18 |
| dtantsur | TheJulia: you mean https://opendev.org/openstack/bifrost/src/branch/master/playbooks/roles/bifrost-keystone-client-config/templates/clouds.yaml.j2 ? | 15:18 |
| TheJulia | https://opendev.org/openstack/bifrost/src/branch/master/playbooks/roles/bifrost-keystone-client-config/templates/clouds.yaml.j2#L11-L14 needs to be changed so it is not using a project | 15:19 |
| TheJulia | in other words add "system_scope: all", and remove the project/domain settings | 15:20 |
| dtantsur | TheJulia: just drop project name and domain_id? or there is more to that? | 15:20 |
| dtantsur | ah gotcha | 15:20 |
| TheJulia | yeah, that *should* do it as long as the system scope password is the same and I think bifrost uses the same password for that bootstrap | 15:20 |
| * dtantsur looking | 15:21 | |
| dtantsur | yeah, seems so | 15:21 |
| TheJulia | if your up for it to just author a quick patch, by all means, if not I'm going to finish writing this bug first. | 15:21 |
| dtantsur | TheJulia: what's the environment variable? OS_SYSTEM_SCOPE? | 15:25 |
| TheJulia | I believe so | 15:26 |
| TheJulia | although env vars really shouldn't be used these days | 15:26 |
| dtantsur | yeah, but we do.. | 15:26 |
| TheJulia | yes, OS_SYSTEM_SCOPE | 15:26 |
| dtantsur | We also use the service user, I wonder how well THAT works | 15:27 |
| TheJulia | service role? | 15:27 |
| dtantsur | wellll... however the 'ironic' user is created | 15:28 |
| TheJulia | eh, that doesn't really matter as much | 15:28 |
| TheJulia | https://opendev.org/openstack/bifrost/src/branch/master/playbooks/roles/bifrost-keystone-client-config/templates/openrc.j2#L15-L17 needs to be changed as well | 15:29 |
| dtantsur | true | 15:29 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 15:31 |
| dtantsur | TheJulia: something along these lines ^^? | 15:31 |
| TheJulia | you can't have the user_domain_id | 15:32 |
| dtantsur | so, it's even across domains? | 15:32 |
| TheJulia | commented on the change, in that state it will fail because it will continue to attempt to use project scope | 15:33 |
| TheJulia | system scope is the system itself | 15:33 |
| TheJulia | it is stacked as: System | Domain | Project | 15:33 |
| dtantsur | Okay. I hope we create users in a compatible way... | 15:34 |
| TheJulia | looking | 15:34 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 15:34 |
| TheJulia | bootstrap looks right | 15:36 |
| TheJulia | hmmmmm | 15:39 |
| TheJulia | so the user created *should* only give a user credentials to access ironic, ideally it has a service role which grants it elevated access, but I'm not sure there is a way to set it and we don't setup the service with admin | 15:40 |
| TheJulia | oh, nevmd | 15:40 |
| TheJulia | it is later down | 15:40 |
| TheJulia | ideally https://opendev.org/openstack/bifrost/src/branch/master/playbooks/roles/bifrost-ironic-install/tasks/keystone_setup.yml#L72-L80 would also be done for the service role, but as is, I suspect everything should just work without it | 15:41 |
| dtantsur | okay, let's see | 15:43 |
| TheJulia | the direction pivot of openstack didn't really help us in general since we didn't loosen everything across our API, we kept a lot of stuff tightly restricted | 15:43 |
| TheJulia | in large part because most of those endpoints are either "blissfully unaware" *OR* users really shouldn't be leveraging in any context outside of interacting with the system itself. | 15:44 |
| * TheJulia wants to become blissfully unaware of rbac | 15:45 | |
| dtantsur | :D | 15:45 |
| TheJulia | (and for us to get this far and keep *all* of the issues to relatively minor, is still damn impressive) | 15:46 |
| dtantsur | ++ | 15:46 |
| TheJulia | I do need to fix our default state for tempest tests | 15:46 |
| TheJulia | ... should have changed them a long time ago :( | 15:47 |
| opendevreview | Julia Kreger proposed openstack/ironic-tempest-plugin master: Ensure scope logic is enforced https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/906434 | 15:51 |
| TheJulia | related! | 15:51 |
| dtantsur | TheJulia: "Expecting to find domain in user. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-fb6be639-e8f0-44ec-8d87-e15af99b8d1e)" | 15:53 |
| TheJulia | ouch | 15:53 |
| dtantsur | https://b81053bdfdc4dbc88f2c-c7d993eea60b4734415017f4886af138.ssl.cf1.rackcdn.com/906528/2/check/bifrost-keystone-collections-src/b658855/job-output.txt | 15:53 |
| TheJulia | where did that... | 15:53 |
| * TheJulia looks | 15:53 | |
| TheJulia | keystone validation error | 15:57 |
| TheJulia | hmmmmmmm | 15:57 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP try reordering things https://review.opendev.org/c/openstack/bifrost/+/906534 | 15:58 |
| dtantsur | a bit hacky, may work ^^^ | 15:58 |
| TheJulia | oh, you've got project and region in that one | 15:59 |
| TheJulia | what was done with the env vars *should* have worked, unless the module has an issue with system scope auth or starts to make weird assumptions | 16:00 |
| * TheJulia looks at the keystone code and blinks | 16:02 | |
| TheJulia | this is bizzare | 16:05 |
| TheJulia | it has to be something on the input | 16:05 |
| TheJulia | so, worst comes to worst, I'm fine reverting the change on ironic while we try and get bifrost fully switched over, my worry right now is something bifrost is dependent upon doesn't comprehend or makes assumptions about use/access | 16:06 |
| dtantsur | yeah, unfortunately it may come to a revert.. but it's not critical yet, so we can keep looking | 16:06 |
| dtantsur | if we need to fix openstacksdk or ansible-collections, then a revert may be unavoidable | 16:07 |
| TheJulia | I almost want to sniff what the request is between the ansible module and keystone at this point because something is very not right | 16:08 |
| TheJulia | what is the best OS to run bifrost on for dev/test these days? | 16:09 |
| dtantsur | TheJulia: I'm using CS9 | 16:09 |
| TheJulia | eeek, a 9GB ISO image | 16:09 |
| dtantsur | I use a script that I wrote for myself https://github.com/dtantsur/config/blob/master/virt-install.sh | 16:11 |
| * dtantsur is rebuilding his 2nd bifrost VM | 16:12 | |
| TheJulia | I'm going to build an environment to try and reproduce and dig further locally since we never saw anything like this with local commands being executed, which makes me *very* worried | 16:12 |
| TheJulia | somehow we're falling into a keystone code path where it assumes there is a domain | 16:13 |
| TheJulia | which is not right | 16:13 |
| TheJulia | but if the input says "you have it", then obviously it is not going to go right | 16:13 |
| TheJulia | at least the ISO is downloading with an okay speed | 16:14 |
| TheJulia | ~20% | 16:15 |
| * dtantsur ./bifrost-cli install --testenv --develop --enable-keystone | 16:15 | |
| * TheJulia waits while ISO downloads, and wonders why the 3d printer is off | 16:16 | |
| TheJulia | 40% \o/ | 16:22 |
| * dtantsur stuck on "Download via GIT" | 16:26 | |
| opendevreview | Kyrylo Romanenko proposed openstack/python-ironicclient master: Add functional tests for VIFs in OSC plugin https://review.opendev.org/c/openstack/python-ironicclient/+/430904 | 16:27 |
| TheJulia | 75% | 16:29 |
| dtantsur | aha, I've reproduced the failure | 16:35 |
| * dtantsur reruns with --debug | 16:46 | |
| TheJulia | \o/ | 16:47 |
| TheJulia | installing now | 16:47 |
| TheJulia | well, centos | 16:47 |
| dtantsur | heh, that's why I always keep 2 bifrost VMs, one main and one stand-by for cases like this | 16:49 |
| TheJulia | ... I oom'ed since an ubuntu VM I have refused to shut down | 16:50 |
| dtantsur | :( | 16:50 |
| TheJulia | .... interestingly enough, the ATSC tuner it was connected to is still initialized. | 16:50 |
| dtantsur | TheJulia: http://192.168.122.1:5000/v3/auth/tokens {'json': {'auth': {'identity': {'methods': ['password'], 'password': {'user': {'password': 'HCMBd5pSX3LSVy470PIE', 'name': 'ironic'}}}}}, 'headers': | 16:56 |
| dtantsur | {'Accept': 'application/json'}, 'authenticated': False, 'log': False} | 16:56 |
| dtantsur | no traces of scope | 16:57 |
| TheJulia | and well, that would do it | 16:57 |
| dtantsur | hold on, my bad | 16:57 |
| dtantsur | TheJulia: {'json': {'auth': {'identity': {'methods': ['password'], 'password': {'user': {'password': 'HCMBd5pSX3LSVy470PIE', 'name': 'ironic'}}}, 'scope': {'sy | 16:57 |
| dtantsur | stem': {'all': True}}}}, 'headers': {'Accept': 'application/json'}, 'authenticated': False, 'log': False} | 16:57 |
| dtantsur | so yeah, scope included, still HTTP 400 | 16:57 |
| TheJulia | umm.. that seems like a weird definition of it | 16:58 |
| TheJulia | but I'd need to consult the api | 16:58 |
| dtantsur | The only difference with the example is user ID vs name | 16:59 |
| dtantsur | Tried with a user ID, got HTTP 401 instead \o/ | 17:01 |
| dtantsur | TheJulia: User fcdfaeef4ef041d79b6a5d78503ca7fa has no access to the system | 17:01 |
| dtantsur | so yeah, we need to use user ID (bad, we don't have access to that) and probably not the service user too | 17:01 |
| TheJulia | that actually makes sense | 17:01 |
| TheJulia | because we create the user in a project today | 17:02 |
| TheJulia | and that user account is being re-used for other activities | 17:02 |
| dtantsur | That's the lesser of my concerns. I don't know what to do about the ID | 17:04 |
| dtantsur | The requirements of using an ID is quite bad (also outside of bifrost) | 17:04 |
| dtantsur | TheJulia: hah, I think you got it wrong: a user domain ID must be provided. Then stuff works. | 17:06 |
| TheJulia | That, itself, doesn't make sense unless the request formatting is doing something weird, because we've successfully passed names in for cross-service communication | 17:06 |
| dtantsur | OS_SYSTEM_SCOPE=all OS_AUTH_TYPE=password OS_AUTH_URL=http://192.168.122.1:5000/v3 OS_USERNAME=admin OS_PASSWORD=$(cat ~/.config/bifrost/admin_password) OS_USER_DOMAIN_ID=default baremetal --debug conductor list | 17:06 |
| dtantsur | this works ^^^ | 17:06 |
| TheJulia | ... | 17:06 |
| * TheJulia wonders if there is a bug which is requiring it somewhere | 17:07 | |
| TheJulia | anyway, I guess time to retry with the domain id set | 17:07 |
| rpittau | good night! o/ | 17:07 |
| JayF | o/ | 17:07 |
| TheJulia | goodnight | 17:08 |
| TheJulia | good morning JayF | 17:08 |
| JayF | I've been reading with some interest | 17:08 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 17:08 |
| dtantsur | retrying ^^ | 17:08 |
| JayF | while asynchronously plotting with arne_wiebalck to have a bm sig at cern in june :) | 17:08 |
| dtantsur | hi JayF | 17:08 |
| TheJulia | I guess I should consider discussing with the wife if she is up for a trip to the EU in June | 17:09 |
| JayF | to be clear, it's pretty much 100% that I'm going end of may / early June | 17:10 |
| JayF | UK and CERN | 17:10 |
| JayF | maybe a weekend in Paris in the middle | 17:10 |
| iurygregory | nice JayF =D | 17:10 |
| dtantsur | If you end up publishing a nice information page with a rough schedule, it may help us a lot | 17:10 |
| dtantsur | at least those of us in Europe | 17:10 |
| JayF | dtantsur: that was one of the things I included in the email; that getting something more detailed online was a crucial step in allowing upstream folks to get travel approved | 17:11 |
| iurygregory | I can join virtual if possible lol =P | 17:11 |
| dtantsur | ++ | 17:11 |
| JayF | (including about the OIF days themselves) | 17:11 |
| dtantsur | I'm puzzled why OIF is so low key about these days | 17:12 |
| JayF | Kristin, the marketing director, got laid off at the top of the year. | 17:13 |
| JayF | I imagine some of the low-noise is possibly as a result of that. | 17:13 |
| dtantsur | that would do it :( | 17:13 |
| TheJulia | There dynamic/model has also changed and there is some expectation that local folks would be helping craft/drive the messaging | 17:13 |
| dtantsur | I mean, for the event in Berlin they'll probably get enough people anyway.. but if they want them *regional*, it's not that easy any more | 17:14 |
| TheJulia | downside of smaller more local events, is the further away you are, the far less info there also is | 17:14 |
| JayF | Yeah, like for instance: this CERN trip *is my openinfra travel* for 2024 | 17:14 |
| JayF | I will not be at the regional summit in SK | 17:14 |
| JayF | (GR-OSS may send someone; but it won't be me) | 17:15 |
| dtantsur | My kubecon talk was not accepted, so I may have a reason to ask for another travel :) | 17:21 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 17:45 |
| TheJulia | hmmm | 17:54 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 18:15 |
| TheJulia | heh, nice to see you just posted what I just saved locally | 18:16 |
| dtantsur | lol | 18:16 |
| dtantsur | I'll see if it works tomorrow - have a good night | 18:16 |
| TheJulia | goodnight! | 18:17 |
| TheJulia | yeah, we're going to need to combine things, basically we're trying to use the service catalog credentials to query but we can't authenticate with that | 18:24 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 18:45 |
| TheJulia | another try \o/ | 18:45 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 19:08 |
| TheJulia | that *should* do it | 19:09 |
| * TheJulia crossses fingers | 19:10 | |
| TheJulia | nope | 19:20 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: WIP fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 19:24 |
| opendevreview | Julia Kreger proposed openstack/bifrost master: fix keystone auth scope https://review.opendev.org/c/openstack/bifrost/+/906528 | 21:09 |
| TheJulia | I tagged https://review.opendev.org/c/openstack/bifrost/+/906528 as ironic-week-prio. It appears good at this point. | 21:10 |
| JayF | rpittau: re: 890408; I've never seen `if which commandname` used as a construct before in a shell script. That's neat. | 21:18 |
| * JayF usually does something more like CMD=`which commandname`; if [[ -x $CMD ]] or similar, which is more explicit but less cool | 21:19 | |
| iurygregory | JayF, great catch in https://review.opendev.org/c/openstack/ironic/+/903379/7/ironic/drivers/modules/redfish/utils.py =) (bad copy/paste XD) | 21:38 |
| JayF | I'm not a human rubber stamp, and I need those comments, what am I supposed to do, read the code to figure out what it does?! That's what I pay the python interpreter to do for me! /s :D | 21:39 |
| iurygregory | :D | 21:40 |
| opendevreview | Iury Gregory Melo Ferreira proposed openstack/ironic master: RedfishFirmwareInterface - Unit Tests & More logs https://review.opendev.org/c/openstack/ironic/+/903379 | 21:41 |
| iurygregory | done =) | 21:41 |
| TheJulia | https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/906434 <-- a quick review would be awesome | 21:47 |
| iurygregory | TheJulia, will look after finish reviewing https://review.opendev.org/c/openstack/bifrost/+/906528 =) | 21:47 |
| TheJulia | ack, thanks | 21:48 |
| TheJulia | JayF: at some point, if you want to discuss self-service-templates, I'm available | 21:49 |
| JayF | lets just put time on a calendar in the future | 21:50 |
| iurygregory | indeed is a quick review lol two edits | 21:50 |
| TheJulia | ok | 21:50 |
| JayF | I am trying to work harder to plan my work out and have fewer interrupts (to very little success so far) because longer-form projects keep getting bumped by a thousand tiny things | 21:50 |
| JayF | TheJulia: tomorrow or fri afternoon? | 21:51 |
| TheJulia | Friday if that works for you | 21:53 |
| JayF | sure, noon-1ish would be ideal | 21:54 |
| JayF | but anytime between 11a-4p is doable (4p is my scheduled end-of-day) | 21:54 |
| TheJulia | That works for me | 21:54 |
| TheJulia | noon-1ish on Friday | 21:55 |
| JayF | invite heading your way | 21:55 |
| JayF | to the RH email? | 21:55 |
| TheJulia | cool cool | 21:55 |
| JayF | apparently don't have the RH email in my gr-oss.io email, so sent it to your gmail | 21:56 |
| TheJulia | this is fine | 21:56 |
| opendevreview | Kyrylo Romanenko proposed openstack/python-ironicclient master: Add functional tests for VIFs in OSC plugin https://review.opendev.org/c/openstack/python-ironicclient/+/430904 | 22:06 |
| opendevreview | Verification of a change to openstack/bifrost master failed: Collect lshw output in json format https://review.opendev.org/c/openstack/bifrost/+/890408 | 22:27 |
| opendevreview | Merged openstack/ironic-tempest-plugin master: Add negative tests for VIF attach/detach operations https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/906078 | 22:52 |
| opendevreview | Merged openstack/ironic-tempest-plugin master: Ensure scope logic is enforced https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/906434 | 23:13 |
| opendevreview | Kyrylo Romanenko proposed openstack/python-ironicclient master: Tests for OSC baremetal node create command with options https://review.opendev.org/c/openstack/python-ironicclient/+/382352 | 23:24 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!