| *** Lucas_Gray has quit IRC | 00:27 | |
| *** Lucas_Gray has joined #openstack-ironic | 00:31 | |
| *** dking_desktop has joined #openstack-ironic | 01:38 | |
| *** Lucas_Gray has quit IRC | 02:03 | |
| *** ociuhandu has joined #openstack-ironic | 02:20 | |
| *** ociuhandu has quit IRC | 02:30 | |
| *** ociuhandu has joined #openstack-ironic | 02:31 | |
| *** ociuhandu has quit IRC | 02:36 | |
| *** mkrai has joined #openstack-ironic | 03:08 | |
| *** mmethot_ has joined #openstack-ironic | 03:20 | |
| *** mmethot has quit IRC | 03:22 | |
| *** uzumaki has quit IRC | 03:29 | |
| *** _mmethot_ has joined #openstack-ironic | 03:31 | |
| *** mmethot_ has quit IRC | 03:34 | |
| *** chihhsin has quit IRC | 03:34 | |
| *** chihhsin has joined #openstack-ironic | 03:35 | |
| *** ricolin_ has joined #openstack-ironic | 03:46 | |
| *** mkrai has quit IRC | 03:49 | |
| *** mkrai has joined #openstack-ironic | 03:49 | |
| *** ricolin_ has quit IRC | 04:17 | |
| *** yaawang has quit IRC | 04:40 | |
| *** yaawang has joined #openstack-ironic | 04:40 | |
| *** ociuhandu has joined #openstack-ironic | 05:14 | |
| *** ociuhandu has quit IRC | 05:19 | |
| *** mkrai has quit IRC | 05:22 | |
| *** mkrai has joined #openstack-ironic | 05:24 | |
| *** dking_desktop has quit IRC | 05:26 | |
| *** mkrai has quit IRC | 05:32 | |
| *** mkrai has joined #openstack-ironic | 05:32 | |
| openstackgerrit | Kaifeng Wang proposed openstack/networking-generic-switch master: [Trivial] doc fix https://review.opendev.org/714045 | 05:49 |
|---|---|---|
| arne_wiebalck | Good morning, ironic! | 06:44 |
| *** mkrai has quit IRC | 07:03 | |
| openstackgerrit | Kaifeng Wang proposed openstack/ironic-python-agent master: Adds poll mode deployment support https://review.opendev.org/694461 | 07:05 |
| *** mkrai has joined #openstack-ironic | 07:08 | |
| openstackgerrit | Kaifeng Wang proposed openstack/ironic-python-agent master: Adds poll mode deployment support https://review.opendev.org/694461 | 07:10 |
| *** k_mouza has joined #openstack-ironic | 07:12 | |
| *** k_mouza has quit IRC | 07:16 | |
| *** dtantsur|afk is now known as dtantsur | 07:18 | |
| dtantsur | morning ironic | 07:18 |
| arne_wiebalck | dtantsur: gm o/ | 07:23 |
| etingof | o/ | 07:27 |
| *** yaawang has quit IRC | 07:29 | |
| *** yaawang has joined #openstack-ironic | 07:30 | |
| *** jawad_axd has joined #openstack-ironic | 07:49 | |
| *** dougsz has joined #openstack-ironic | 07:56 | |
| *** ociuhandu has joined #openstack-ironic | 07:56 | |
| *** yaawang has quit IRC | 07:58 | |
| *** dougsz has quit IRC | 08:00 | |
| *** mbeierl has quit IRC | 08:04 | |
| *** mbeierl1 has joined #openstack-ironic | 08:04 | |
| *** mbeierl1 is now known as mbeierl | 08:06 | |
| iurygregory | good morning dtantsur etingof arne_wiebalck and Ironic o/ | 08:08 |
| arne_wiebalck | iurygregory: o/ | 08:08 |
| *** tkajinam has quit IRC | 08:15 | |
| *** dougsz has joined #openstack-ironic | 08:20 | |
| *** belmoreira has joined #openstack-ironic | 08:23 | |
| *** amoralej|off is now known as amoralej | 08:41 | |
| *** ktibi has joined #openstack-ironic | 08:48 | |
| *** ociuhandu has quit IRC | 08:49 | |
| *** ociuhandu has joined #openstack-ironic | 08:50 | |
| *** rpittau|afk is now known as rpittau | 08:52 | |
| rpittau | good morning ironic! o/ | 08:52 |
| iurygregory | morning rpittau o/ | 08:54 |
| rpittau | hey iurygregory :) | 08:54 |
| *** ociuhandu has quit IRC | 08:55 | |
| *** lucasagomes has joined #openstack-ironic | 09:01 | |
| *** diga has joined #openstack-ironic | 09:32 | |
| diga | Hi Ironic | 09:34 |
| diga | some zuul gates are failing on this patch, not sure reason behind it because patch changes are not impacting to any of these gates - https://review.opendev.org/#/c/693328 | 09:35 |
| patchbot | patch 693328 - ironic - DRAC : idrac-redfish inspect updates pxe port - 6 patch sets | 09:35 |
| *** ociuhandu has joined #openstack-ironic | 09:37 | |
| rpittau | diga: the failing test is actually the one you modified in the patch | 09:37 |
| diga | okay | 09:37 |
| rpittau | orr well, not you, the one that is modified in the patch :) | 09:38 |
| diga | :) No problem | 09:38 |
| diga | I will talk to Mahendra | 09:38 |
| *** ociuhandu has quit IRC | 09:41 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Use openstacksdk for ironic module https://review.opendev.org/672179 | 09:42 |
| *** derekh has joined #openstack-ironic | 09:43 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: [WIP] Use tinycore 11 https://review.opendev.org/711647 | 09:44 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: [WIP] Use tinycore 11 https://review.opendev.org/711647 | 09:44 |
| *** mkrai has quit IRC | 09:52 | |
| *** ktibi_ has joined #openstack-ironic | 09:57 | |
| *** ktibi has quit IRC | 10:00 | |
| *** k_mouza has joined #openstack-ironic | 10:00 | |
| *** alexmcleod has joined #openstack-ironic | 10:04 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/networking-generic-switch master: Disable ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode https://review.opendev.org/714410 | 10:11 |
| *** mkrai has joined #openstack-ironic | 10:13 | |
| *** dougsz has quit IRC | 10:19 | |
| *** hoonetorg has quit IRC | 10:24 | |
| *** dougsz has joined #openstack-ironic | 10:32 | |
| *** hoonetorg has joined #openstack-ironic | 10:37 | |
| * dtantsur sighs at ^^^ | 10:37 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-lib master: Explicitly set ramdisk type https://review.opendev.org/714418 | 10:54 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Explicitly set ramdisk type https://review.opendev.org/714419 | 10:56 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Explicitly set ramdisk type https://review.opendev.org/714419 | 10:57 |
| yolanda | hi, good morning... i'm continuing testing ironic/pxe on supermicros, and i'm getting this error now: No partition with UUID 0x00000000 found on device /dev/sda . I have an sda disk, looking at inspector logs it detects it properly (but it has partitions as well). What can be the problem? | 11:01 |
| yolanda | can it be because it is already partitioned? | 11:01 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: Add release note for CentOS 8 and Ansible 2.8 https://review.opendev.org/714420 | 11:04 |
| mgoddard | morning | 11:04 |
| mgoddard | dtantsur, rpittau: Would like to get ^ merged then make a bifrost train release | 11:05 |
| dtantsur | looking | 11:05 |
| dtantsur | mgoddard: thanks. we may need to release it as a minor (not patch) version because of the impact. | 11:06 |
| dtantsur | I think the version number is still free since we haven't released Ussuri. | 11:06 |
| openstackgerrit | Riccardo Pittau proposed openstack/networking-baremetal master: Explicitly set ramdisk type https://review.opendev.org/714421 | 11:07 |
| yolanda | when i look at inspection of the node, in driver_internal_info, i see 'root_uuid_or_disk_id': '0x00000000' , that's a result of inspection? | 11:11 |
| dtantsur | yolanda: nope, it's a temporary thing from, presumably, a previous deployment? | 11:12 |
| yolanda | i started deployment from scratch, removing all ironic bits | 11:12 |
| dtantsur | I don't remember for sure, but ironic-inspector cannot even modify driver_internal_info | 11:12 |
| dtantsur | maybe it's an artifact of some internal logic in the boot interface?.. | 11:13 |
| dtantsur | does it break you anyhow? | 11:13 |
| yolanda | yep, i cannot deploy ... when it tries to copy the deployment image, i see that "No partition with UUID 0x000 found on device /dev/sda" . So the image is not copied to hard disk, and it just reboots on IPA again | 11:13 |
| dtantsur | mgoddard: and forgot to say: good job with centos 8 support on train! it was an important gap to close. | 11:14 |
| dtantsur | yolanda: okay, it's a runtime problem (unrelated to inspection). is it a whole disk image? | 11:14 |
| yolanda | yep, deploying in the context of baremetal IPI | 11:14 |
| mgoddard | dtantsur: thanks :) the patch didn't end up as bad as I'd expected | 11:14 |
| dtantsur | me too :) | 11:15 |
| dtantsur | yolanda: any differences from your previous deployments? changed image? | 11:15 |
| yolanda | i changed machines... i am testing on my local supermicros | 11:16 |
| yolanda | so this is a new deployment on an internal lab | 11:16 |
| dtantsur | it's not the first time I hear about root_uuid_or_disk_id being zero.. | 11:16 |
| yolanda | specific with hardware? | 11:17 |
| dtantsur | I need to dive into the code to update myself. iurygregory or arne_wiebalck do either of you remember anything about ^^? | 11:17 |
| dtantsur | yolanda: should not be. more likely to the image somehow. | 11:17 |
| iurygregory | dtantsur, reading | 11:17 |
| yolanda | i can send you logs of inspector/conductor if they are useful | 11:18 |
| dtantsur | yolanda: inspector is likely unrelated. ramdisk logs would be most useful probably.. | 11:18 |
| iurygregory | you trying wholedisk image or partition image? | 11:19 |
| yolanda | iurygregory, whole disk, it's a baremetal IPI | 11:19 |
| yolanda | and first time that i try on those machines | 11:19 |
| iurygregory | uefi boot right? | 11:19 |
| yolanda | yep | 11:20 |
| iurygregory | can you share the ipa logs ? | 11:20 |
| yolanda | i see in conductor, some logs... None"; prepare_image: result "{'result': 'prepare_image: image (rhcos-44.81.202003062006-0-compressed.x86_64.qcow2) written to device /dev/sda root_uuid=0x00000000'}", 2020-03-23 10:39:17.742 1 DEBUG ironic.drivers.modules.agent_base [-] Installing the bootloader for node ae7c756c-3b19-4d47-a203-fae69d2f065e on partition 0x00000000, 2020-03-23 10:39:17.903 1 DEBUG ironic.drivers.modules.agent_client [-] Agent | 11:21 |
| yolanda | command image.install_bootloader for node ae7c756c-3b19-4d47-a203-fae69d2f065e returned result None, error {'message': 'Error finding the disk or partition device to deploy the image onto: No partition with UUID 0x00000000 found on device /dev/sda', 'code': 404, 'type': 'DeviceNotFound', 'details': 'No partition with UUID 0x00000000 found on device /dev/sda'}, | 11:21 |
| yolanda | let me paste ipa log now | 11:21 |
| dtantsur | note for myself: root_uuid_or_disk_id comes from IPA after writing the image | 11:22 |
| *** ricolin_ has joined #openstack-ironic | 11:23 | |
| arne_wiebalck | sorry, connectivity brownout | 11:24 |
| openstackgerrit | Merged openstack/bifrost stable/train: Add release note for CentOS 8 and Ansible 2.8 https://review.opendev.org/714420 | 11:25 |
| arne_wiebalck | yolanda: this sounds like the bootloader installation, so I guess the image was installed, but now the IPA cannot find the partition with 0x000 ... this breaks the deployment, the node still boots from the network and hence into the IPA ... does that make sense? | 11:26 |
| arne_wiebalck | yolanda: no, it does not :) | 11:27 |
| dtantsur | okay, so this value (in case of whole disk images) comes from hexdump | 11:27 |
| arne_wiebalck | yolanda: since the log clearly says it cannot find the disk to deploy the image onto | 11:27 |
| dtantsur | yolanda: is it UEFI or legacy boot? | 11:28 |
| * arne_wiebalck has to leave, is back in 2 hrs | 11:28 | |
| dtantsur | does your image use GTP or MBR? | 11:28 |
| yolanda | dtantsur, UEFI | 11:29 |
| dtantsur | * GPT | 11:29 |
| iurygregory | MBR is normally for SW Raid | 11:30 |
| dtantsur | I don't think yolanda is using software RAID | 11:30 |
| iurygregory | yeah | 11:30 |
| yolanda | i need to check but i'd say GPT | 11:31 |
| dtantsur | my question is whether the IPI images use MBR or GPT inside of them | 11:31 |
| yolanda | let me check | 11:31 |
| iurygregory | should be GPT I would say | 11:31 |
| dtantsur | I've seen an exactly similar thing in another case with UEFI | 11:31 |
| dtantsur | I think the solution was to make ironic use efibootmgr | 11:32 |
| dtantsur | the ramdisk logs can probably help more | 11:32 |
| yolanda | trying to upload logs | 11:32 |
| iurygregory | yeah | 11:32 |
| iurygregory | the ipa logs would tell how the check for efi went | 11:33 |
| dtantsur | but in any case, we need to fix the 0x00.. issue | 11:33 |
| * dtantsur wonders if get_disk_identifier works corrrectly for GPT | 11:33 | |
| dtantsur | it mentions MBR quite explicitly | 11:34 |
| yolanda | https://paste.centos.org/view/891091aa | 11:35 |
| yolanda | ramdisk of one of the machines | 11:35 |
| dtantsur | yolanda: it's from inspection, I think | 11:36 |
| rpittau | yolanda: what's the version of the base os ? | 11:36 |
| rpittau | looks like centos 7 ? | 11:37 |
| yolanda | centos 7 yes | 11:37 |
| iurygregory | python2.7... | 11:37 |
| rpittau | yeah | 11:37 |
| iurygregory | =O | 11:37 |
| yolanda | i mean, IPA is centos7 | 11:38 |
| rpittau | would be curious about the ipa version there | 11:38 |
| rpittau | yep | 11:38 |
| yolanda | the image i want to install is RHCOS | 11:38 |
| rpittau | yeah but the problem might be the version of ipa on that ipa-ramdisk | 11:38 |
| iurygregory | maybe it's an old one without the efi changes.. | 11:39 |
| rpittau | for example | 11:39 |
| dtantsur | probably | 11:40 |
| dtantsur | honestly, I'm staring at the code, and I cannot understand why it's even supposed to work.. | 11:40 |
| dtantsur | the zero ID is a thing in the CI as well: https://zuul.opendev.org/t/openstack/build/5a609496cadf449fa7a13eafdebb28d1/log/controller/logs/ironic-bm-logs/node-0_no_ansi_2020-03-17-15:54:33_log.txt#1479 | 11:41 |
| yolanda | let me check where i get ipa from... should be rdo | 11:41 |
| etingof | TheJulia, re ipmi indicators - so far I do not see where that unknown argument error comes from. I will debug your patch to figure that. | 11:43 |
| dtantsur | rpittau, iurygregory, we don't have a CI job with UEFI and whole disk images, do we? | 11:43 |
| yolanda | yep, https://images.rdoproject.org/train/rdo_trunk/ | 11:43 |
| yolanda | current-tripleo folder | 11:44 |
| dtantsur | ugh. folks, I think this code path is hopelessly broken. | 11:44 |
| rpittau | dtantsur: no, we don't | 11:44 |
| dtantsur | I wonder if anyone has ever tested a GPT whole disk image with local boot.. | 11:44 |
| iurygregory | I know bfournie testes wholedisk image + efi while I did the changes in ipa | 11:45 |
| iurygregory | tested* | 11:45 |
| yolanda | i tested these images in HPE machines last week, worked fine for me | 11:45 |
| rpittau | dtantsur: I think the wholedisk+uefi is in the todo list for the CI | 11:46 |
| yolanda | it was same scenario... uefi boot, pxe boot | 11:46 |
| iurygregory | good to hear that | 11:46 |
| *** rpittau is now known as rpittau|bbl | 11:47 | |
| *** mkrai has quit IRC | 11:48 | |
| *** mkrai_ has joined #openstack-ironic | 11:48 | |
| * iurygregory goes and start working in add a job with wholedisk+uefi | 11:49 | |
| dtantsur | yolanda: mmm, so it used to work? | 11:50 |
| dtantsur | I'm putting the findings in https://storyboard.openstack.org/#!/story/2007455 | 11:50 |
| yolanda | dtantsur, first time that i test on this hardware | 11:50 |
| yolanda | but used to work in HPE, i was testing that scenario last week | 11:50 |
| iurygregory | I know people used uefi+wholedisk and it was working | 11:50 |
| dtantsur | okay, good. I still don't see how it can possibly work (see the story), but it helps.. a bit. | 11:51 |
| *** rh-jelabarre has joined #openstack-ironic | 11:54 | |
| dtantsur | yolanda: I'd be really curious to see the ramdisk logs for deployment | 11:55 |
| dtantsur | ideally, a successful deployment as well | 11:55 |
| dtantsur | to compare | 11:55 |
| bfournie | dtantsur, iurygregory; yep, I created a uefi whole disk image using "openstack overcloud image build" and installed on a Dell set for uefi. I don't have the logs any more though | 11:56 |
| yolanda | ramdisk for HPE boxe you mean? | 11:56 |
| *** rh-jelabarre has quit IRC | 11:56 | |
| dtantsur | yolanda: I definitely the logs from the failing box, but ideally from HPE too | 11:57 |
| yolanda | i paste ramdisk from the failing box | 11:57 |
| yolanda | https://paste.centos.org/view/891091aa | 11:57 |
| dtantsur | yolanda: it's from inspection, we need deployment. | 11:57 |
| dtantsur | on a per-hipster world it would be from /var/log/ironic/deploy | 11:58 |
| dtantsur | * pre-hipster | 11:58 |
| yolanda | ok let me paste that | 11:59 |
| *** rh-jelabarre has joined #openstack-ironic | 12:00 | |
| *** rh-jelabarre has quit IRC | 12:00 | |
| yolanda | https://paste.centos.org/view/90b457fa | 12:02 |
| *** ociuhandu has joined #openstack-ironic | 12:02 | |
| dtantsur | yolanda: thanks! there's also an lsblk file in the same archive, could you paste it? | 12:03 |
| * iurygregory looks | 12:03 | |
| dtantsur | actually, there is an lsblk dump in the logs | 12:04 |
| yolanda | in ironic? i cannot find any lsblk | 12:04 |
| yolanda | ps, lshw, iptables, ip_addr, df, journal | 12:05 |
| dtantsur | yolanda: weird, but anyway. I've found what I was looking for. | 12:05 |
| yolanda | seems there are some errors related to partitions there... | 12:06 |
| dtantsur | well, I still don't understand how this all could possibly work before... | 12:08 |
| dtantsur | yolanda: are you using the same IPA and instance image as before? | 12:10 |
| yolanda | dtantsur, yes, i always collected ipa from there | 12:11 |
| yolanda | there is an ipa-downloader container that takes care of downloading ipa from rdo | 12:12 |
| dtantsur | Can someone please read my findings on https://storyboard.openstack.org/#!/story/2007455 and tell me what I'm missing? | 12:12 |
| dtantsur | yolanda: you seem to be missing the iurygregory's efibootmgr patches. they might fix your situation, even though the bug stays.. | 12:15 |
| yolanda | dtantsur, where can i grab those images then? i am using current-tripleo | 12:17 |
| yolanda | https://github.com/metal3-io/ironic-ipa-downloader/blob/master/get-resource.sh#L10 | 12:18 |
| dtantsur | https://images.rdoproject.org/train/rdo_trunk/current-tripleo/ should be recent enough, dunno why the patch is not there.. | 12:18 |
| dtantsur | although things take time to propagate there | 12:19 |
| dtantsur | yolanda: can you switch to https://tarballs.opendev.org/openstack/ironic-python-agent/dib/files/ temporary (skipping inspection if you can)? | 12:19 |
| yolanda | sure, let me try | 12:20 |
| yolanda | centos8 ipa? | 12:21 |
| *** ociuhandu has quit IRC | 12:21 | |
| arne_wiebalck | dtantsur: so, the issue is around installing the bootloader? | 12:21 |
| dtantsur | TheJulia: re ipa-builder and checkout: there are DIB args for that. I agree that an argument could be easier.. | 12:21 |
| *** khansa has joined #openstack-ironic | 12:21 | |
| dtantsur | yolanda: yep | 12:21 |
| dtantsur | arne_wiebalck: tl;dr https://storyboard.openstack.org/#!/story/2007455 | 12:22 |
| arne_wiebalck | dtantsur: from what I remember, the uuid was only passed for partition images initially | 12:22 |
| arne_wiebalck | dtantsur: when I added this for RAID, I asked where the ID comes from | 12:22 |
| arne_wiebalck | dtantsur: for RAID, this is now part of the image and taken from there | 12:22 |
| arne_wiebalck | dtantsur: I never understood where the uuid would have come from for other cases | 12:23 |
| dtantsur | arne_wiebalck: see the story. I feels like it's all wrong.. but people report it working for a while. | 12:23 |
| dtantsur | I need to talk to TheJulia when she's online. She may have an idea why this all is happening.. | 12:24 |
| openstackgerrit | Merged openstack/ironic master: Add indicators REST API endpoints https://review.opendev.org/651785 | 12:29 |
| etingof | wow! ^ | 12:37 |
| *** rh-jelabarre has joined #openstack-ironic | 12:40 | |
| dtantsur | congrats :) | 12:54 |
| *** ociuhandu has joined #openstack-ironic | 12:54 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Support burning configdrive into boot ISO https://review.opendev.org/687250 | 12:55 |
| *** ociuhandu has quit IRC | 12:58 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Follow-up releasenote use_secrets https://review.opendev.org/714435 | 13:03 |
| dtantsur | rpittau|bbl: ehmm, we have no logs for https://fc8933f96044d5a5cb6e-2093f8081c337530c67183f7eaa6e193.ssl.cf2.rackcdn.com/672179/58/check/openstack-tox-functional-py36/3dc98ca/job-output.txt | 13:04 |
| *** ociuhandu has joined #openstack-ironic | 13:04 | |
| dtantsur | rpittau|bbl: more importantly, we seem to be breaking bifrost: https://zuul.opendev.org/t/openstack/build/d5e038dd2f754275ba4ff163bb5d68e6/log/logs/ironic-inspector.log#573 | 13:05 |
| dtantsur | I may have a fix for that, hold on | 13:08 |
| *** ociuhandu has quit IRC | 13:08 | |
| *** rpittau|bbl is now known as rpittau | 13:08 | |
| rpittau | dtantsur: bifrost is fixed by starting inspector after ironic-api | 13:08 |
| rpittau | dtantsur: https://review.opendev.org/713608 | 13:09 |
| patchbot | patch 713608 - bifrost - [DNM] Test run inspector after ironic-api - 1 patch set | 13:09 |
| dtantsur | rpittau: we cannot rely on the ordering, we need to fix inspector | 13:09 |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add `network_data` field to ironic node object https://review.opendev.org/687910 | 13:09 |
| *** amoralej is now known as amoralej|lunch | 13:09 | |
| rpittau | dtantsur: yep, I'm not saying that, that's why I did only a test patch while still working on the openstacksdk migration | 13:10 |
| *** ociuhandu has joined #openstack-ironic | 13:11 | |
| dtantsur | rpittau: try adding something like http://paste.openstack.org/show/791023/ to your patch | 13:11 |
| dtantsur | it may also fix the functional tests | 13:11 |
| rpittau | dtantsur: the error on bifrost is the same issue that is breaking the functional test btw | 13:11 |
| *** ociuhandu has quit IRC | 13:13 | |
| *** ociuhandu has joined #openstack-ironic | 13:14 | |
| rpittau | dtantsur: that worked | 13:15 |
| dtantsur | neat | 13:15 |
| dtantsur | I guess the SDK does more on initialization than ironicclient.. | 13:15 |
| *** ociuhandu has quit IRC | 13:16 | |
| rpittau | dtantsur: yeah, I was doing a test and I saw the connection to ironic during the initialization, that's why I thought of an issue on the discovery as I was saying on the sdks channel | 13:16 |
| *** ociuhandu has joined #openstack-ironic | 13:16 | |
| *** ociuhandu has quit IRC | 13:16 | |
| *** ociuhandu has joined #openstack-ironic | 13:18 | |
| *** rloo has joined #openstack-ironic | 13:18 | |
| yolanda | dtantsur, i started to use centos 8 ipa... but seems it's not doing anything. It loads centos8 but sits there, and no activity on logs | 13:20 |
| dtantsur | weird | 13:21 |
| yolanda | https://tarballs.opendev.org/openstack/ironic-python-agent/dib/files/ - centos8 one | 13:21 |
| yolanda | https://imgur.com/a/VhesOyH | 13:22 |
| yolanda | just sits there | 13:22 |
| dtantsur | are you sure it's not doing anything? maybe just the output is silenced? | 13:23 |
| yolanda | no logs | 13:23 |
| yolanda | anything on ironic-inspector | 13:24 |
| dtantsur | yolanda: I'd suggest skipping inspection, it may not work in our configuration | 13:24 |
| dtantsur | python-hardware is not in these ramdisks | 13:24 |
| yolanda | how can i disable that? removing inspector interfaces? | 13:25 |
| *** gudrutis2 has joined #openstack-ironic | 13:25 | |
| dtantsur | I'm not sure how to do it in metal3, to be honest | 13:25 |
| gudrutis2 | And I am back. Hi Ironic, how have you been? | 13:25 |
| yolanda | dtantsur, in ironic.conf? | 13:26 |
| dtantsur | yolanda: I mean.. it's not ironic that starts inspection, it's metal3 in your case | 13:26 |
| dtantsur | hi gudrutis2, surviving successfully | 13:26 |
| dtantsur | yolanda: you can try setting the default_inspect_interface to fake, but I suspect metal3 may not like it | 13:26 |
| iurygregory | hi gudrutis2 welcome back o/ doing good so far, and you? | 13:26 |
| rpittau | hey gudrutis2 , fighting for toilet paper as the rest of the world :) | 13:27 |
| yolanda | ok let me check | 13:27 |
| iurygregory | rpittau, I can try to send some for you (here in CZ np atm) | 13:27 |
| rpittau | iurygregory: just kidding, fortunately, but thanks :) | 13:27 |
| gudrutis2 | dtantsur: in germany you doing pretty well with infection/death ratio, so I am not surprised :) | 13:27 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Use openstacksdk for ironic module https://review.opendev.org/672179 | 13:28 |
| iurygregory | rpittau, good \o/ | 13:28 |
| dtantsur | gudrutis2: yeah, they seem to have it in check.. for now | 13:30 |
| *** tzumainn has joined #openstack-ironic | 13:31 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/networking-generic-switch master: [WIP] Fix CI https://review.opendev.org/714410 | 13:34 |
| *** ociuhandu has quit IRC | 13:36 | |
| *** ociuhandu_ has joined #openstack-ironic | 13:36 | |
| iurygregory | ngs seems broken let's see how it goes | 13:38 |
| openstackgerrit | Riccardo Pittau proposed openstack/python-ironicclient master: Explicitly set ramdisk type https://review.opendev.org/714443 | 13:44 |
| rpittau | iurygregory: s/seems/is , I was also testing it :) | 13:46 |
| *** ociuhandu_ has quit IRC | 13:46 | |
| *** ociuhandu has joined #openstack-ironic | 13:46 | |
| iurygregory | rpittau, gotcha | 13:47 |
| openstackgerrit | Tzu-Mainn Chen proposed openstack/ironic master: Add node lessee field https://review.opendev.org/706864 | 13:49 |
| openstackgerrit | Riccardo Pittau proposed openstack/sushy master: Explicitly set ramdisk type https://review.opendev.org/714447 | 13:49 |
| * TheJulia wipes sleep from h er eyes | 13:50 | |
| openstackgerrit | Riccardo Pittau proposed openstack/sushy-tools master: Explicitly set ramdisk type https://review.opendev.org/714448 | 13:53 |
| rpittau | good morning TheJulia :) | 13:53 |
| TheJulia | dtantsur: or maybe we should just document the example argument pass-through | 13:54 |
| openstackgerrit | Riccardo Pittau proposed openstack/virtualbmc master: Explicitly set ramdisk type https://review.opendev.org/714449 | 13:56 |
| dtantsur | morning TheJulia | 14:05 |
| dtantsur | I bet you won't like seeing https://storyboard.openstack.org/#!/story/2007455 | 14:06 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost master: Don't install python openwsman package https://review.opendev.org/714451 | 14:06 |
| *** amoralej|lunch is now known as amoralej | 14:06 | |
| iurygregory | good morning TheJulia | 14:09 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Wholedisk+Uefi job https://review.opendev.org/714453 | 14:09 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Wholedisk+Uefi job https://review.opendev.org/714453 | 14:11 |
| TheJulia | yolanda: could you get a partition list from that machine somehow? | 14:13 |
| TheJulia | specifically wondering what the uefi boot partition's contents look like | 14:13 |
| yolanda | i'm testing with downstream image now, that shows different logs. Let me paste | 14:14 |
| TheJulia | yolanda: looks like the contents of sda2 is what we're going to need to look at | 14:15 |
| TheJulia | and then sda1 | 14:15 |
| yolanda | that's with downstream... https://paste.centos.org/view/53694b0d | 14:17 |
| yolanda | in this case it doesnt' seem to throw errors on that | 14:18 |
| TheJulia | The agent thinks it is in bios mode or is intended for bios mode | 14:22 |
| *** ociuhandu_ has joined #openstack-ironic | 14:22 | |
| TheJulia | it sees a bios boot header on the image and combined with bios mode it goes "okay, don't need to do anything here" | 14:22 |
| yolanda | i have the machines set to UEFI | 14:24 |
| *** ociuhand_ has joined #openstack-ironic | 14:24 | |
| yolanda | need to go for lunch but i'll continue debugging later | 14:25 |
| *** ociuhandu has quit IRC | 14:25 | |
| TheJulia | so _is_bootloader_booted doesn't check the running boot mode :( | 14:26 |
| * TheJulia starts on a fix | 14:26 | |
| *** ociuhand_ has quit IRC | 14:27 | |
| *** ociuhandu_ has quit IRC | 14:27 | |
| *** ociuhandu has joined #openstack-ironic | 14:27 | |
| *** ociuhandu_ has joined #openstack-ironic | 14:30 | |
| *** ociuhand_ has joined #openstack-ironic | 14:32 | |
| *** ociuhandu has quit IRC | 14:33 | |
| TheJulia | dtantsur: yolanda: is the partition pre-encrypted? | 14:35 |
| *** ociuhandu_ has quit IRC | 14:36 | |
| *** mkrai_ has quit IRC | 14:37 | |
| *** ociuhand_ has quit IRC | 14:43 | |
| dtantsur | TheJulia: I don't know, but seems so | 14:44 |
| dtantsur | yeah, in BIOS mode we won't go into the bootloader | 14:44 |
| *** cdearborn has joined #openstack-ironic | 14:44 | |
| dtantsur | what I don't understand how it even works for UEFI when we don't use efibootmgr | 14:44 |
| etingof | opinions wanted: what's the best place for network_data validation - boot interface or node ORM object? | 14:44 |
| iurygregory | rpittau, wondering about "maybe put this phrase after the next one ?" my idea was, the information in the begin would highlight more that it's necessary the config. | 14:46 |
| dtantsur | etingof: option #3: API layer? | 14:46 |
| iurygregory | dtantsur, well it can go to grub2 in some cases... | 14:46 |
| dtantsur | iurygregory: right, and I see now way for that "going to grub2" to work.. but people report it does. | 14:47 |
| dtantsur | I'm confused as heck | 14:47 |
| iurygregory | yeah | 14:47 |
| rpittau | iurygregory: ok, mmm then maybe rephrase the entire paragraph as there is some repetition | 14:50 |
| TheJulia | dtantsur: we should be able to find the bootloader in the existing partitions.. *should* | 14:50 |
| TheJulia | yolanda: patch inbound, just writing a reno | 14:50 |
| iurygregory | rpittau, ok let me think a little bit here =) | 14:50 |
| dtantsur | TheJulia: I think it also only works for legacy boot | 14:50 |
| dtantsur | for UEFI we seem to always require efibootmgr or grub | 14:50 |
| dtantsur | in the latter case, we're trying to find a root partition via hexdump that is arguably MBR-specific | 14:51 |
| dtantsur | see other findings in https://storyboard.openstack.org/#!/story/2007455 | 14:51 |
| dtantsur | I do suspect that I'm missing something still since it's reported to work.. | 14:51 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Return false for MBR bootloader check on UEFI machines https://review.opendev.org/714468 | 14:53 |
| TheJulia | dtantsur: likely bios specific behavior is occuring as well in terms of the machine working after reboot. We've seen tons of that before as well | 14:54 |
| etingof | we can do that at the API level as well, however I see for properties (for example) we do it at the DB level and for raid at the interface level... | 14:56 |
| * arne_wiebalck is wondering how he could boot UEFI nodes with dual-mode images | 14:59 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Follow-up releasenote use_secrets https://review.opendev.org/714435 | 15:00 |
| arne_wiebalck | TheJulia: this is only when the boot mode is not set on the node? | 15:00 |
| dtantsur | arne_wiebalck: there was an attempt to implement that, I don't think it has been finished | 15:00 |
| dtantsur | mmm, meeting time? | 15:00 |
| TheJulia | #startmeeting ironic | 15:00 |
| openstack | Meeting started Mon Mar 23 15:00:25 2020 UTC and is due to finish in 60 minutes. The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** kaifeng_ has joined #openstack-ironic | 15:00 | |
| *** openstack changes topic to " (Meeting topic: ironic)" | 15:00 | |
| openstack | The meeting name has been set to 'ironic' | 15:00 |
| arne_wiebalck | dtantsur: to implement what, sorry? | 15:00 |
| dtantsur | arne_wiebalck: support for dual-boot images | 15:00 |
| kaifeng_ | o/ | 15:00 |
| rajinir | o/ | 15:00 |
| dtantsur | o/ | 15:00 |
| rpioso | \o | 15:00 |
| etingof | \o | 15:00 |
| TheJulia | dtantsur: I _suspect_ if we can't use uefibootmgr, we're dead in the water | 15:00 |
| cdearborn | o/ | 15:00 |
| iurygregory | o/o/ | 15:00 |
| rpittau | o/ | 15:00 |
| erbarr | o/ | 15:00 |
| TheJulia | honestly | 15:00 |
| TheJulia | Anyway! | 15:00 |
| arne_wiebalck | dtantsur: it seems to work for us :) | 15:00 |
| dtantsur | good :) | 15:00 |
| TheJulia | Good morning everyone! It is time for another meeting if the ironic minds! | 15:01 |
| rloo | o/ | 15:01 |
| arne_wiebalck | o/ | 15:01 |
| *** ricolin_ has quit IRC | 15:01 | |
| TheJulia | Our agenda is on the wiki! | 15:01 |
| TheJulia | #link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting | 15:01 |
| TheJulia | Seems we have a few topics so we should go ahead and get started! | 15:01 |
| *** ricolin_ has joined #openstack-ironic | 15:01 | |
| TheJulia | #topic Announcements / Reminders | 15:01 |
| *** openstack changes topic to "Announcements / Reminders (Meeting topic: ironic)" | 15:02 | |
| mgoddard | \o | 15:02 |
| ajya | o/ | 15:02 |
| TheJulia | #info We are engering the end of the Ussuri cycle, and as such deadlines are coming up. | 15:02 |
| *** ociuhandu has joined #openstack-ironic | 15:02 | |
| TheJulia | #info Extra-ATC submissions must take place by March 26th, or R-7 | 15:03 |
| TheJulia | #info Non-client library freeze is April 2nd. (ironic-lib, sushy) | 15:03 |
| openstackgerrit | Merged openstack/virtualbmc master: Fix stop command by using default SIGTERM handler in children https://review.opendev.org/713038 | 15:03 |
| TheJulia | #info Client library freeze and the Ussuri-3 milestone for the openstack community is April 9th. | 15:04 |
| rloo | does that extra-atc submission mean anything, now that it is going virtual? | 15:04 |
| TheJulia | #info Cycle Highlightes are also due on the 9th. | 15:04 |
| dtantsur | rloo: ATC also participate in elections, for example | 15:04 |
| TheJulia | rloo: extra-atc can be granted for people that aid discussion, or that help others in irc. | 15:04 |
| rloo | ah, then yes, it means something. thx! | 15:05 |
| TheJulia | Extra-ATC is also valid for ?11? months | 15:05 |
| TheJulia | so in theory, 2 election rounds. | 15:05 |
| TheJulia | Does anyone have anything to raise or remind us of this week? | 15:05 |
| * dtantsur reminds to stay safe | 15:06 | |
| *** ociuhandu has quit IRC | 15:06 | |
| TheJulia | Yes, everyone, please stay safe! | 15:06 |
| TheJulia | Also, everyone! Reviews! | 15:07 |
| TheJulia | Shall we carry on? | 15:07 |
| iurygregory | ++ | 15:07 |
| rpittau | let's | 15:07 |
| TheJulia | #topic Review action items from previous meeting | 15:08 |
| *** openstack changes topic to "Review action items from previous meeting (Meeting topic: ironic)" | 15:08 | |
| TheJulia | iurygregory: you had an action item to document supported branch policy, I believe you posted it? | 15:08 |
| iurygregory | TheJulia, https://review.opendev.org/713660 | 15:08 |
| patchbot | patch 713660 - ironic - Document our policies for stable branches - 2 patch sets | 15:08 |
| TheJulia | most excellent | 15:08 |
| iurygregory | we just need to reach a consensus on the text | 15:09 |
| iurygregory | =) | 15:09 |
| * TheJulia adds to the list of patches for the week | 15:09 | |
| TheJulia | Moving on! | 15:10 |
| TheJulia | #topic Review subteam status reports | 15:10 |
| *** openstack changes topic to "Review subteam status reports (Meeting topic: ironic)" | 15:10 | |
| TheJulia | #link https://etherpad.openstack.org/p/IronicWhiteBoard | 15:10 |
| TheJulia | Starting at line 251 | 15:11 |
| TheJulia | arne_wiebalck: thanks for the scale testing notes! | 15:11 |
| iurygregory | rpittau, are we tracking the CI work in the status report? | 15:11 |
| arne_wiebalck | TheJulia: will add more data as it comes in :) | 15:11 |
| arne_wiebalck | TheJulia: looks good so far | 15:11 |
| TheJulia | Regarding the Bare Metal SIG whitepaper, would it help if we did some scheduled "editing the doc times"? | 15:11 |
| arne_wiebalck | I think so ... this is dangling since so long now. | 15:12 |
| rpittau | iurygregory: no, I'm adding a section now as I have the story as well | 15:12 |
| TheJulia | arne_wiebalck: yeah :( | 15:12 |
| arne_wiebalck | TheJulia: let's just schedule sth and finish it | 15:12 |
| TheJulia | k | 15:13 |
| * arne_wiebalck is naive maybe | 15:13 | |
| TheJulia | Anyone have any ideal time windows? | 15:13 |
| TheJulia | or days? | 15:13 |
| TheJulia | Looks like multitenancy patches are still in review | 15:13 |
| dtantsur | TheJulia: start a doodle? | 15:13 |
| TheJulia | #action TheJulia to send out a doodle for whitepaper | 15:14 |
| arne_wiebalck | thanks | 15:14 |
| TheJulia | I saw network_data discussion in irc earlier, so that is a good sign | 15:14 |
| TheJulia | dtantsur: can you add your disk device selection patch to priorites to review if they are not already there | 15:15 |
| * etingof is working on dhcpless full-throttle now days | 15:15 | |
| TheJulia | etingof: <3 | 15:15 |
| dtantsur | TheJulia: added earlier today | 15:15 |
| TheJulia | <3 | 15:15 |
| *** ociuhandu has joined #openstack-ironic | 15:16 | |
| TheJulia | Looks like iurygregory has made good progress on retooling some tempest jobs for simplification of CI load :) | 15:16 |
| *** ociuhandu has quit IRC | 15:16 | |
| iurygregory | it's working with redfish | 15:16 |
| TheJulia | \o/ | 15:16 |
| iurygregory | I would appreciate feedback to see if this is the way we want to move or not | 15:16 |
| TheJulia | iurygregory: maybe put the patch(s) on the list to review? | 15:17 |
| *** TxGirlGeek has joined #openstack-ironic | 15:17 | |
| iurygregory | TheJulia, sure | 15:17 |
| TheJulia | Updates look good to me, shall we carry on to priorities for the week? | 15:17 |
| iurygregory | ++ | 15:18 |
| rpittau | yup | 15:18 |
| TheJulia | #topic Deciding on priorities for the coming week | 15:18 |
| *** openstack changes topic to "Deciding on priorities for the coming week (Meeting topic: ironic)" | 15:18 | |
| TheJulia | #link https://etherpad.openstack.org/p/IronicWhiteBoard | 15:19 |
| TheJulia | Line 122 | 15:19 |
| etingof | should we review pending sushy patches? | 15:20 |
| TheJulia | etingof: yes, very much so given the short time we'll ahve on them | 15:20 |
| * TheJulia is marking out merged patches | 15:20 | |
| * TheJulia is amused by the "race lottery" | 15:21 | |
| * etingof can gather sushy patches pending reviews on the whiteboard | 15:21 | |
| TheJulia | sounds good | 15:23 |
| TheJulia | there are a lot of things on the board today. I'm going to delete the merged items now | 15:23 |
| TheJulia | Anyone interested in making the pxe module smarter about what format messages to return for neutron integrated stacks? | 15:24 |
| iurygregory | TheJulia, I can | 15:24 |
| TheJulia | https://review.opendev.org/#/c/714212/ | 15:24 |
| patchbot | patch 714212 - ironic - "dual stack" support for PXE/iPXE - 7 patch sets | 15:25 |
| TheJulia | Already did it :) | 15:25 |
| rpittau | I'm too slow :( | 15:25 |
| iurygregory | oh =) | 15:25 |
| iurygregory | rpittau, no you are not slow | 15:25 |
| TheJulia | I can add it to the list if people are interested | 15:25 |
| dtantsur | let's try | 15:27 |
| *** gyee has joined #openstack-ironic | 15:27 | |
| TheJulia | done! | 15:28 |
| * TheJulia moves the 3 sushy items up | 15:28 | |
| TheJulia | The list looks good to me, hopefully this many of us will be able to focus on reviews and trying to get some code merged as we're starting to run out of time in the cycle. | 15:29 |
| *** khansa has quit IRC | 15:29 | |
| TheJulia | Shall we move on to Discussion topics? | 15:29 |
| iurygregory | ++ | 15:29 |
| TheJulia | #topic Discussion | 15:30 |
| *** openstack changes topic to "Discussion (Meeting topic: ironic)" | 15:30 | |
| TheJulia | The first item I believe was added by iurygregory noticing that HP's 3rd party CI logs were inaccessible and they were now attempting to run python2 jobs | 15:30 |
| TheJulia | unfortunately, I don't see anyone from HP here this morning. | 15:31 |
| iurygregory | yeah =( | 15:31 |
| TheJulia | iurygregory: Would you be able to draft an email to stendulker and copy me? | 15:31 |
| iurygregory | I can't access any job logs from HPE CI | 15:31 |
| iurygregory | TheJulia, sure! | 15:31 |
| iurygregory | you can put as action item | 15:31 |
| TheJulia | iurygregory: if you don't have his email address, just let me know | 15:31 |
| TheJulia | #action iurygregory to reach out to HPE regarding CI status | 15:32 |
| iurygregory | I'm also wondering if other 3rd party CI have py2 jobs running | 15:32 |
| iurygregory | I hope not =) | 15:32 |
| TheJulia | I am 98% sure I already talked to Dell folks about changing all of their jobs to python3 | 15:32 |
| rajinir | All Dell CI jobs are Python3 now | 15:32 |
| iurygregory | rajinir, tks! | 15:32 |
| TheJulia | rajinir: Thanks! | 15:32 |
| TheJulia | Okay, next topic then! | 15:32 |
| TheJulia | Migrate from oslo.rootwrap to oslo.privsep. I don't know who added this, but it seems like a thing | 15:33 |
| rpittau | TheJulia: I did | 15:33 |
| iurygregory | I saw the thread on openstack-discuss | 15:33 |
| rpittau | there was a topic in the ML about that, it's a proposed goal for V release | 15:33 |
| iurygregory | rpittau, ++ | 15:33 |
| * TheJulia hands the microphone over | 15:33 | |
| rpittau | maybe we can start having a look at it since other projects (nove) moved there already | 15:34 |
| TheJulia | I guess that is reasonable, although I've been hoping cycle goals get re-thought | 15:34 |
| TheJulia | Oh, that is good and does help a lot | 15:34 |
| iurygregory | rpittau, I've started looking at it =) | 15:34 |
| rpittau | I believe it's a good idea in general, not a super high priority | 15:34 |
| rpittau | iurygregory: heh ok, nvm then :) | 15:34 |
| TheJulia | rpittau: My impression from this discussion is the same | 15:34 |
| TheJulia | iurygregory: awesome! | 15:34 |
| iurygregory | I will work on a etherpad with the projects we need to update | 15:35 |
| TheJulia | Well we have nothing listed for Baremetal SIG and I already have an action tiem for that. And no listed RFE's for review. One worth noting, I did start updating the old kexec RFE with some thoughts and comments becuase I'm trying to sort through how to make it all work in the machenery | 15:36 |
| TheJulia | I'm also hacking it out and will hopefully have it working during one of the quarentine days ahead. | 15:36 |
| TheJulia | iurygregory: awesome | 15:36 |
| TheJulia | So I guess we're good to proceed to Open Discussion? | 15:36 |
| iurygregory | ++ | 15:37 |
| dtantsur | yeah | 15:37 |
| TheJulia | #topic Open Discussion | 15:37 |
| *** openstack changes topic to "Open Discussion (Meeting topic: ironic)" | 15:37 | |
| TheJulia | so is everyone doing okay? | 15:37 |
| dtantsur | Do we need a virtual ironic (un)conference? :) | 15:37 |
| TheJulia | I think it could be a good idea. Maybe have a rule for everyone to wear crazy hats? | 15:38 |
| dtantsur | We could even have a post-conference beer-n-chat | 15:38 |
| * iurygregory will use a pikachu hat | 15:38 | |
| TheJulia | ++ | 15:38 |
| dtantsur | iurygregory: I need to see that! | 15:38 |
| iurygregory | dtantsur, hahahaha you can check on instagram =) | 15:39 |
| * dtantsur will | 15:39 | |
| rpittau | I still have my goofy hat from disneyland, can that work ? | 15:39 |
| TheJulia | Shall I put together a "un-conference video call, and bring your best crazy hat, and then enjoy a pint?" | 15:39 |
| TheJulia | err, a doodle | 15:39 |
| iurygregory | TheJulia, ++ | 15:39 |
| TheJulia | Excellent! | 15:39 |
| openstackgerrit | Merged openstack/sushy-cli master: Update commands to generate docs, renos and pdf https://review.opendev.org/706826 | 15:40 |
| TheJulia | I like the idea of just trying to do it Wednesday or Thursday, personally | 15:40 |
| dtantsur | I'd take Friday, fewer meetings for me | 15:40 |
| dtantsur | but we can doodle it | 15:41 |
| TheJulia | k | 15:41 |
| TheJulia | Wheee I have a jury summons coming in the mail :( | 15:41 |
| dtantsur | great time for that | 15:41 |
| TheJulia | iurygregory: by the way, if you logout and log back into gerrit, you'll have some more rights on a few repos. | 15:42 |
| etingof | btw, we have quite a lot of pending sushy-cli patches | 15:42 |
| dtantsur | TheJulia: I think refreshing a page is enough for iurygregory :) | 15:42 |
| iurygregory | TheJulia, \o/ | 15:42 |
| iurygregory | etingof, I'll take a look | 15:42 |
| TheJulia | dtantsur: not for group memberships if memory serves | 15:42 |
| dtantsur | mmm, maybe | 15:42 |
| dtantsur | it's easy to check | 15:42 |
| TheJulia | yeah | 15:42 |
| iurygregory | I can see after a f5 | 15:43 |
| TheJulia | Anyway, is there anything else to dsicuss today? | 15:44 |
| iurygregory | thanks for the opportunity, I will do my best o/ | 15:44 |
| TheJulia | iurygregory: Thank you for being an active reviewer! | 15:45 |
| rloo | ++++ | 15:45 |
| iurygregory | =) | 15:46 |
| TheJulia | for our unconference, shall I just send out a bluejeans link or something? | 15:46 |
| *** dking_desktop has joined #openstack-ironic | 15:46 | |
| dtantsur | works for me | 15:47 |
| TheJulia | "The SPUC - Sanity Preservation Un-Conference" ? | 15:48 |
| iurygregory | ++ | 15:48 |
| * TheJulia feels that is an ironic title if there ever was one | 15:48 | |
| dtantsur | lol, like it | 15:48 |
| dtantsur | shall we announce it on the ML/twitter/etc or keep it more private? | 15:48 |
| TheJulia | I was going to post it to the ML | 15:48 |
| dtantsur | ++ | 15:50 |
| TheJulia | Anyway, I think we're done for the day! Thanks everyone and happy reviewing changes! | 15:50 |
| TheJulia | #endmeeting | 15:51 |
| *** openstack changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/ironic/ | Bugs: https://storyboard.openstack.org/#!/project_group/75 | Contributors are generally present between 6 AM and 12 AM UTC, If we do not answer, please feel free to pose questions to openstack-discuss mailing list." | 15:51 | |
| openstack | Meeting ended Mon Mar 23 15:51:50 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:51 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-03-23-15.00.html | 15:51 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-03-23-15.00.txt | 15:51 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-03-23-15.00.log.html | 15:51 |
| openstackgerrit | Merged openstack/bifrost master: Don't install python openwsman package https://review.opendev.org/714451 | 15:52 |
| *** ijw_ has joined #openstack-ironic | 15:53 | |
| *** dsneddon has joined #openstack-ironic | 15:56 | |
| *** ijw has quit IRC | 15:57 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add `network_data` field to ironic node object https://review.opendev.org/687910 | 16:08 |
| *** ociuhandu has joined #openstack-ironic | 16:11 | |
| etingof | a thought of network_data validation: the benefit of doing that within the boot interface is that we can have different validation schemata per node... | 16:18 |
| *** ociuhandu has quit IRC | 16:18 | |
| TheJulia | well, driver class which could be overriden I guess: ) | 16:18 |
| dtantsur | etingof: boot interface is certainly not the place, I guess you mean the deploy interface | 16:19 |
| dtantsur | mmm, I can see where you're coming from re boot interface.. | 16:19 |
| dtantsur | there is a bit of ambiguouty here. | 16:20 |
| dtantsur | boot interface is where you process the file, but it's agnostic to the exact ramdisk | 16:20 |
| etingof | I meant this -- https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/redfish/boot.py#L584 | 16:20 |
| TheJulia | there is, but we've got precedence for "as more information" is populated validation checking | 16:20 |
| TheJulia | This is also true about the ramdisk | 16:20 |
| dtantsur | the deploy interface is what is aware of IPA | 16:20 |
| dtantsur | but it's not the code that actually touches network_data | 16:20 |
| * dtantsur will probably just rely on etingof's best judgement | 16:21 | |
| TheJulia | realistically, there is no way to validate in advance because the machine would need to be up | 16:21 |
| TheJulia | true as well :\ | 16:21 |
| dtantsur | yeah, it's still about a sanity check | 16:21 |
| TheJulia | That is quite a conundrum too | 16:21 |
| dtantsur | I'd stick it in the API for the sake of simplicity.. but I'm not going to -1 other reasonable approaches. | 16:21 |
| TheJulia | yeah, implementer is likely going going to have to find the best happy place | 16:22 |
| dtantsur | etingof: we've just delegated the decision to you :) | 16:22 |
| TheJulia | etingof: congrats! | 16:22 |
| openstackgerrit | Merged openstack/ironic master: Follow-up releasenote use_secrets https://review.opendev.org/714435 | 16:22 |
| etingof | does it make sense to account for different nodes using different ramdisks using different network data schemata? | 16:23 |
| dtantsur | not much, really | 16:24 |
| etingof | if we do validation at the DB or API levels, then that ^ is impossible | 16:24 |
| dtantsur | it's not impossible, but chances are very high it will never happen | 16:24 |
| dtantsur | now, I have an argument for doing it in the API layer | 16:24 |
| dtantsur | and a quite decisive with my API SIG hat on | 16:24 |
| dtantsur | if we do it there, we can version the schema | 16:24 |
| dtantsur | although.. now that I think about it, it's unlikely to ever change as well | 16:24 |
| * dtantsur needs to grab a snack | 16:25 | |
| *** mbeierl has quit IRC | 16:27 | |
| TheJulia | Baremetal SIG doodle: https://doodle.com/poll/k4wnmuay34mvh94v | 16:27 |
| *** mbeierl has joined #openstack-ironic | 16:30 | |
| * etingof is not certain how to version a JSON schema within ironic API considering that the schema (1) evolves independently and (2) is not versioned by itself... | 16:31 | |
| TheJulia | do we version instance_info? | 16:33 |
| TheJulia | nope | 16:33 |
| TheJulia | While not "pefect", it is practical not to | 16:33 |
| *** riuzen has joined #openstack-ironic | 16:33 | |
| TheJulia | perfect | 16:33 |
| * TheJulia remembers the debates over this like 5 years ago | 16:33 | |
| TheJulia | or version driver_info? Which we CAN'T because 3rd party driver parameters | 16:33 |
| riuzen | Guys, what is the difference between flat and neutron when we define enabled_network_interfaces= ? Any reference I can read? | 16:34 |
| TheJulia | riuzen: https://docs.openstack.org/ironic/latest/install/configure-tenant-networks.html talks about it a little bit. the TL;DR flat is for preconfigured static networks where neutron just manages DHCP. neutron is for interface plugging/more dynamic networks where neutron actually manages switch configurations. | 16:37 |
| openstackgerrit | Christopher Dearborn proposed openstack/ironic master: Fix issue where server fails to reboot https://review.opendev.org/714491 | 16:41 |
| *** belmoreira has quit IRC | 16:43 | |
| etingof | I think your decision to delegate this decision to me has been a questionable decision | 16:43 |
| etingof | I lean towards boot/deploy interface approach because it seems most flexible | 16:44 |
| etingof | somehow I suspect that dtantsur does not share my gut feeling on this ;) | 16:45 |
| riuzen | TheJulia: So, I should choose neutron as network interface if I use openstack provider network? | 16:46 |
| TheJulia | riuzen: if it is static and preconfigured switch side and everything is just attached to the provider network, then I believe you'll want flat. | 16:47 |
| iurygregory | rpittau, good news we have a fix for the ngs | 16:48 |
| iurygregory | https://review.opendev.org/#/c/714410/ | 16:49 |
| patchbot | patch 714410 - networking-generic-switch - [WIP] Fix CI - 2 patch sets | 16:49 |
| iurygregory | will change the commit message =) | 16:49 |
| iurygregory | kaifeng_, your patch will be good after this one merges https://review.opendev.org/#/c/714410/ | 16:49 |
| patchbot | patch 714410 - networking-generic-switch - [WIP] Fix CI - 2 patch sets | 16:49 |
| * etingof goes for a walk in a gas mask | 16:50 | |
| * iurygregory wonders where etingof found masks to buy.... | 16:50 | |
| * etingof read on the news that they set up vending machines in some cities | 16:51 | |
| * TheJulia is hoping 3d printer stuffs will arrive soon to begin making masks | 16:51 | |
| iurygregory | etingof, WUT?! | 16:52 |
| rpittau | iurygregory: great news :) | 16:52 |
| iurygregory | vending machines with masks ++ | 16:52 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/networking-generic-switch master: Fix CI https://review.opendev.org/714410 | 16:53 |
| rpittau | a bit late for carneval and a bit early for halloween for the masks | 16:53 |
| iurygregory | rpittau, here it's mandatory, you can only go out if you are wearing a mask (need to have mouth and nose covered) | 16:54 |
| JayF | in the US they're telling people wearing a mask only helps if you're symptomatic, and doesn't help prevent transmission TO you | 16:55 |
| JayF | which isn't really accurate, but they're lying to try and save the masks for the medicial folks | 16:55 |
| TheJulia | so much propganda... | 16:55 |
| iurygregory | JayF, yeah, we can't use the medical ones here | 16:55 |
| iurygregory | basically you can make your own mask using cotton clothes etc | 16:56 |
| TheJulia | Does anyone have filter effiency data for at home things like coffee filters, cotten rounds? | 16:56 |
| riuzen | TheJulia : Ok I'll try it. Thank you. I simulate it on top of KVM btw. | 16:56 |
| TheJulia | I spent a few hours trying to find a 3d editor that didn't crash and didn't drive me crazy because I want to modify the face mask STL design that the ?PLACTIVE? folks published. Mainly also cartridge defect in that changing it requires lots of work. | 16:57 |
| JayF | I've just been staying inside and clorox wiping anything that gets delivered. | 16:58 |
| iurygregory | https://twitter.com/BrnoNewsCTV/status/1242102734179926017 | 16:59 |
| TheJulia | nice! | 16:59 |
| TheJulia | JayF: Reasonable, but we've been struggling to get certain things like eggs/milk delivered so tactical missions to obtain supplies are going to become a thing if the supply lines still seem crazy. | 17:00 |
| JayF | Well, we'll go out to shop. Trying not to use grocery delivery services here as they are booked up, and Vanessa+I are not in the high risk group if we caught the virus anyway. | 17:01 |
| JayF | (like, grocery shop, occassionally, not going out all the time) | 17:01 |
| TheJulia | https://doodle.com/poll/krtn9p4z7xx9pa29 <-- For the Sanity Preservation Un-Conference | 17:07 |
| JayF | What is the purpose of this? Conversation and sanification of those who need it? Actual Ironic work? | 17:08 |
| dking_desktop | When trying "server create...", I get an error in Ironic conductor about "Could not find these security groups (specified via ironic config) in neutron", but the security group does exist in "openstack security group show". What are some things to check? | 17:08 |
| rpittau | good night! o/ | 17:08 |
| *** rpittau is now known as rpittau|afk | 17:08 | |
| dtantsur | JayF: both! | 17:09 |
| *** lucasagomes has quit IRC | 17:10 | |
| iurygregory | dking_desktop, the security group is set in the config? | 17:10 |
| *** diga has quit IRC | 17:10 | |
| TheJulia | JayF: I just sent an email to openstack-discuss on the topic :) | 17:11 |
| dking_desktop | iurygregory: Yes, it's set in the ironic.conf for the ironic-conductor as the cleaning and provisioning network security group, as well as using it with the "server create" command. | 17:12 |
| JayF | I'm not on the openstack list these days, not enough time to drink in that firehose | 17:12 |
| TheJulia | Oh, its not a firehose... | 17:12 |
| TheJulia | dking_desktop: Sounds like your using the neturon network_interface and your missing the security group settings | 17:12 |
| TheJulia | Okay, way more messages than I thought so still firehose-ey | 17:13 |
| TheJulia | http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013521.html | 17:13 |
| dking_desktop | TheJulia: yes. Where would I set those? | 17:13 |
| TheJulia | dking_desktop: who owns the security groups? | 17:13 |
| dking_desktop | It should be admin. I haven't setup any users manually yet. | 17:14 |
| TheJulia | cleaning_network_security_groups, provisioning_network_security_groups should both have UUIDs of the groups, and ensure that the ironic service user can see the security groups. | 17:15 |
| *** jawad_axd has quit IRC | 17:17 | |
| dking_desktop | Ah, they need the UUIDs, and not the group names? | 17:17 |
| TheJulia | I believe so | 17:17 |
| TheJulia | names are not unique in neutron | 17:17 |
| dtantsur | yeah, we support names for networks, but it's a bit risky | 17:18 |
| dking_desktop | That seems frustrating as I won't know the UUIDs before deploy, and I deploy Ironic along with the rest of the services. So, I'd have to go back and edit the config after. | 17:18 |
| TheJulia | dking_desktop: oh.. | 17:18 |
| TheJulia | no | 17:18 |
| TheJulia | oh! your trying to do the ironic deploy dynamically? | 17:19 |
| TheJulia | yeah, you'll need to create the networks and populate them into the config | 17:19 |
| TheJulia | that being said, really double check access because names should work, just are not the best way to handle it | 17:19 |
| dking_desktop | Yeah, that, too. I'm using kolla-ansible for the deploy. | 17:19 |
| dtantsur | the same chicken-and-egg problem that we had with tripleo | 17:21 |
| dtantsur | hence names for networks | 17:21 |
| dking_desktop | Yeah. Eventually, I'd like to be able to use names for the deploy_kernel and deploy_ramdisk, too, for the same reason. | 17:22 |
| dtantsur | I hit this all the time, for some reason I assume we support them.. | 17:23 |
| iurygregory | and pray that the user won't use the same name again... | 17:23 |
| *** gregwork has joined #openstack-ironic | 17:24 | |
| dking_desktop | TheJulia: How would I check the access? I suppose shell into the ironic-conductor container and attempt to connect to neutron from there? Also, would it be different than when it deploy with network_interface=flat ? Because that works. | 17:25 |
| TheJulia | dking_desktop: use the same credentials in the ironic.conf to run `openstack network list` | 17:26 |
| TheJulia | dking_desktop: flat doesn't use security groups, so that is why it works | 17:26 |
| TheJulia | dking_desktop: you might want to skim through https://docs.openstack.org/ironic/latest/install/configure-tenant-networks.html if you haven't already | 17:32 |
| dking_desktop | And yes, with the creds from the ironic-conductor ironic.conf [neutron] section, I can connect and see the desired entries both in "openstack network list" and "openstack security group list" | 17:33 |
| dking_desktop | Thank you. That's the guide where I saw to set those values. | 17:34 |
| dking_desktop | I might try to use the id instead, just in case, but the names are unique. | 17:35 |
| *** dtantsur is now known as dtantsur|afk | 17:36 | |
| dtantsur|afk | g'night | 17:36 |
| *** ktibi_ has quit IRC | 17:40 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Return false for MBR bootloader check on UEFI machines https://review.opendev.org/714468 | 17:41 |
| *** mmethot_ has joined #openstack-ironic | 17:44 | |
| *** _mmethot_ has quit IRC | 17:46 | |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic-python-agent master: [trivial] Fix comment for Software RAID restart https://review.opendev.org/714503 | 17:56 |
| *** riuzen has quit IRC | 17:56 | |
| openstackgerrit | Merged openstack/sushy-cli master: Add system/manager/chassis inventory show command https://review.opendev.org/706005 | 17:57 |
| arne_wiebalck | bye everyone o/ | 18:01 |
| *** derekh has quit IRC | 18:02 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Make deploy step failure logging indicate the error https://review.opendev.org/714511 | 18:19 |
| TheJulia | o/ | 18:19 |
| *** naved001 has quit IRC | 18:21 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Add info on how to enable ironic-tempest-plugin https://review.opendev.org/712901 | 18:23 |
| etingof | should we ship network_data schema with ironic? or leave that to the operator to take care about? | 18:24 |
| etingof | the schema is (kind of) published on docs.openstack by now | 18:25 |
| dking_desktop | TheJulia: So, changing to the UUID instead of the name certainly did something. It got much further this time, and I fixed an error I found with the port. However, It is not telling me, in slightly different error message, that "Security group <UUID> does not exist". | 18:26 |
| *** naved001 has joined #openstack-ironic | 18:27 | |
| dking_desktop | That is in the ironic-conductor logs. But again, I checked with the creds from its ironic.conf under [neutron], and I can connect, and I can even show that UUID. | 18:27 |
| TheJulia | etingof: define define | 18:29 |
| TheJulia | err | 18:29 |
| TheJulia | sorry, define ship | 18:29 |
| etingof | repo-wise | 18:29 |
| etingof | and package-wise later on | 18:29 |
| TheJulia | etingof: an example in docs maybe | 18:29 |
| etingof | example of the schema? nova schema is huge | 18:29 |
| TheJulia | etingof: maybe simplified? | 18:30 |
| TheJulia | minimal fields kind of thing? | 18:30 |
| *** alexmcleod has quit IRC | 18:30 | |
| etingof | not really, hold on | 18:31 |
| TheJulia | etingof: Maybe take the example data from bifrost? | 18:31 |
| TheJulia | I'm thinking purely to help someone create their own, not everything possible | 18:31 |
| *** k_mouza has quit IRC | 18:32 | |
| etingof | here it is -- https://github.com/openstack/nova/blob/master/doc/api_schemas/network_data.json | 18:32 |
| TheJulia | dking_desktop: is networking-baremetal running in neutron as well? | 18:32 |
| etingof | why would one want to create a schema while it's already done and functionally hardwired into glean/cloud-init | 18:32 |
| etingof | if I were an operator, I would just curl it down and use without even looking inside | 18:33 |
| etingof | that schema is not feature-reach, it's just declarative syntax that takes up so much bytes... | 18:34 |
| etingof | *rich | 18:34 |
| dking_desktop | TheJulia: How do I check for networking-baremetal? | 18:34 |
| TheJulia | dking_desktop: it is another ml2 plugin | 18:35 |
| dking_desktop | TheJulia: Also, I just checked in neutron-server.log, and I can see the request coming in for the security group, and the log shows a success, and the DEBUG entry even shows all the whole security group info being sent back! | 18:35 |
| TheJulia | etingof: I'm just thinking a practical operator example ought to be in documentation | 18:35 |
| * etingof is confused | 18:36 | |
| TheJulia | dking_desktop: where in the entire process is provisioning breaking down? I'm wondering if your vif might not be valid and if it is right at the end of the deploy process | 18:37 |
| dking_desktop | Oh, no, that's not the security group info. That's just the info for the port. But the response is a 200. | 18:37 |
| etingof | I imagine the majority of the operators just take ironic ramdisk and go ahead... why would they be interested in messing with nova metadata schema? | 18:37 |
| TheJulia | etingof: they need to provide schema compliant data to manually deploy | 18:37 |
| TheJulia | so our friends in metal3 will need to create a json document that can be leveraged | 18:37 |
| etingof | well, for that I'd just hlink nova docs | 18:38 |
| TheJulia | dking_desktop: so the port you have set as the vif is also viewable buy the account ironic is operating as? | 18:39 |
| etingof | e.g. https://docs.openstack.org/nova/latest/user/metadata.html#nova-metadata | 18:39 |
| dking_desktop | I'm not positive. I run "openstack server create ...", and I'm watching the ironic-conductor logs. I see that the conductor attaches to the node, and then moves the state to deploying, checks the power state, then unbinds instance ports, adds provisioning network, and then it requests the security group and neutron logs show a 200 response, and then ironic-conductor reports that the security group does not ex | 18:39 |
| dking_desktop | ...unless the 200 response could include a not found response, I suppose. | 18:40 |
| etingof | they have cut&paste network_data.json to serve as a prototype... | 18:40 |
| TheJulia | dk, it can | 18:40 |
| TheJulia | dking_desktop: 200 with an empty list | 18:41 |
| etingof | however the schema is something I'd never expect any sane person to ever look into | 18:41 |
| dking_desktop | The port that I'm seeing is the "openstack baremetal port" entry. I did have another such port, but I removed it to just work with one. | 18:41 |
| TheJulia | etingof: Yeah.... I was going to say something similar and then decided I needed more coffee | 18:41 |
| TheJulia | I'e already demonstrated a lack of sanity | 18:41 |
| *** k_mouza has joined #openstack-ironic | 18:41 | |
| etingof | coffee! I forgot my coffeepot on the oven! | 18:42 |
| TheJulia | DOH! | 18:42 |
| TheJulia | Hopefully it will still return "418 I'm a teapot" | 18:44 |
| dking_desktop | Oh, yes! I see it. The same server logs show the connection from ironic-conductor and also from my deploy host's "openstack" query. The successful one has a len of 2603, but the one from ironic has a len of 267. So, that's possibly where it fails. | 18:44 |
| *** amoralej is now known as amoralej|off | 18:44 | |
| TheJulia | dking_desktop: your on the right track! | 18:44 |
| etingof | with the emergence of smart freezers, we need a new RFC for that | 18:44 |
| TheJulia | feel free to post patches with more debug logging | 18:45 |
| etingof | *fridges | 18:45 |
| dking_desktop | The query is a bit different, but I don't think that matters. Success: "GET /v2.0/security-groups/547caee8-1139-49b7-bd45-9549c34e30f9 HTTP/1.1" Failed: "GET /v2.0/security-groups?fields=id&id=547caee8-1139-49b7-bd45-9549c34e30f9 HTTP/1.1" | 18:46 |
| TheJulia | etingof: it is never too late to submit an April Fools RFC... just might not be posted until next year. | 18:46 |
| *** k_mouza has quit IRC | 18:46 | |
| etingof | anyway, my uncertainty is about the schema file (which is highly reusable), not network_data.json (which is indeed site-specific) | 18:46 |
| TheJulia | dking_desktop: interesting, we have to do the elongated query because some oeprators have huge lists of groups and the neutron query count was compounding to something absurd | 18:47 |
| dking_desktop | Both give a status of 200. I'm assuming that rules out authentication issues. | 18:47 |
| etingof | this schema, should we borrow it from nova and ship with ironic (in battery included spirit)? | 18:47 |
| TheJulia | but not authorization to see | 18:47 |
| TheJulia | etingof: likely a good idea | 18:47 |
| etingof | or rather point the operator to nova to grab from? | 18:47 |
| dking_desktop | Yeah, I suspected that it was just limiting the response fields. | 18:47 |
| dking_desktop | I could try a tcpdump to see what's in the packets. | 18:48 |
| *** kaifeng_ has quit IRC | 18:49 | |
| TheJulia | etingof: we don't control nova's docs, so likely best to copy and cross-link | 18:55 |
| etingof | will etc/ironic be a reasonable location for schema file? | 18:56 |
| TheJulia | likely | 18:58 |
| etingof | 10x | 18:58 |
| TheJulia | hjensas: does it make sense to block https://review.opendev.org/#/c/700002 on an ironic-inspector patch? or just note it? I know many don't re-inspect after starting out, but *shrug* | 19:02 |
| patchbot | patch 700002 - ironic - Additional IP addresses to IPv6 stateful ports - 15 patch sets | 19:02 |
| TheJulia | hjensas: of course, ironic-inspector would need a patch for that too | 19:03 |
| *** ricolin_ has quit IRC | 19:24 | |
| TheJulia | tzumainn: hey, you around? | 19:25 |
| dking_desktop | TheJulia: So, I checked with tcpdump. I see that when Ironic checks, there's two requests sent to Neutron, with the same X-OpenStack-Request-ID. Shortly after that, I see two incomplete 200 responses going back to their respective ports. I see several incomplete packets. Then, finally, I see two complete 404 responses going back the the respective ports. | 19:45 |
| dking_desktop | I can't find the logs for the 404s. neutron-server logs only seem to show the 200s. I'm using kolla-ansible, so I'm using haproxy on the ports, but I don't know of anything else that would be causing a problem. | 19:47 |
| TheJulia | dking_desktop: if they are full length packet captures, it might be good to post because something is afoot :( | 19:48 |
| *** jawad_axd has joined #openstack-ironic | 19:48 | |
| tzumainn | TheJulia, yep! what's up? | 19:50 |
| TheJulia | With regards to 708678, that is just that the bind profile can't be updated by a non-admin user correct? | 19:54 |
| TheJulia | err, mac address I thought | 19:54 |
| TheJulia | tonyb: Have you heard of anyone trying to do software raid on a POWER machine with ironic? | 19:57 |
| tzumainn | TheJulia, I think so, yes - that's the one command that I updated in the patch, right? | 19:58 |
| tzumainn | ohhhh should I be separating that out into two commands? one for the mac? | 19:59 |
| TheJulia | tzumainn: Eh, I _think_ so, differing rights and all. The huge concern I have is if a unprivilged user happens to know a vifid on another network, and requests their machine's port be bound to that other network. | 19:59 |
| tzumainn | TheJulia, yeah, that makes a ton of sense - okay, I have to re-setup my environment a bit, but I'll try and get an updated PR out soon | 20:00 |
| TheJulia | tzumainn: sorry :( | 20:00 |
| tzumainn | thanks for pointing that out! | 20:00 |
| tzumainn | oh, not at all | 20:00 |
| tzumainn | better to get it right :) | 20:00 |
| tonyb | TheJulia: no but it's just linux at that point and I can't think of much in the I/O subsystem that'd make it any harder | 20:00 |
| tonyb | TheJulia: why do you ask? | 20:01 |
| TheJulia | tzumainn: yeah, I'd rather not have a CVE filing against ironic :) | 20:01 |
| TheJulia | tonyb: https://review.opendev.org/#/c/696156/ | 20:01 |
| patchbot | patch 696156 - ironic-python-agent - Software RAID: Add UEFI support - 8 patch sets | 20:01 |
| tonyb | TheJulia: I'll have a quick look I have a mild panic when thinking about GPT and power and /boot but I also think that's mostly history and the situation is okay now | 20:03 |
| TheJulia | tonyb: eek, didn't mean to cause mild panic | 20:04 |
| TheJulia | well, I'm sure it is mild over other panics | 20:04 |
| tonyb | TheJulia: I'll probably have some fairly "basic" and strange questions after I review it | 20:04 |
| * TheJulia reloads the stats dashboard | 20:04 | |
| TheJulia | tonyb: no worries | 20:04 |
| * TheJulia clsoes the stats dashboard | 20:04 | |
| * tonyb had to write the initial GPT on power support in grub | 20:04 | |
| TheJulia | tonyb: Well, I guess it is good I raised the question... | 20:05 |
| tonyb | TheJulia: yes it is :) Thank you | 20:07 |
| dking_desktop | TheJulia: http://paste.openstack.org/show/791047/ Sorry about all that info. | 20:08 |
| dking_desktop | The security group ID is 547caee8-1139-49b7-bd45-9549c34e30f9 | 20:09 |
| *** dougsz has quit IRC | 20:14 | |
| TheJulia | dking_desktop: look at the post with the binding profile update "pology" | 20:16 |
| TheJulia | hmm, looks like console garbage from the output | 20:17 |
| TheJulia | dking_desktop: so binding_profile validation is failing somewhere deep inside of neutron and it is telling us the security group doesn't exist in that process :( | 20:18 |
| dking_desktop | Well, I captured it with tee. | 20:18 |
| dking_desktop | Oh, that sounds interesting. | 20:19 |
| JayF | Have you ensured the Neutron SG shows up when doing an `openstack security group list` as *the exact user that Ironic is connecting to Neutron as*? | 20:19 |
| TheJulia | JayF: they have, the packet captures between neutron and ironic show ironic sees the group | 20:19 |
| JayF | damn :( | 20:19 |
| TheJulia | something in port binding is just going sideways on validation :( | 20:20 |
| TheJulia | I suspect time to fire up neutron in debug mode and see if it dumps out the bnding data. | 20:20 |
| JayF | You could add a subscriber, in neutron, to the port crud events and log what it's seeing | 20:20 |
| TheJulia | and the process of enumerating through validations | 20:20 |
| JayF | (if neutron in debug doesn't give it away) | 20:20 |
| dking_desktop | I've already had neutron-server with debug=True. I think some of the logs I've shown had the DEBUG lines in them. | 20:21 |
| dking_desktop | Of course, those logs have a lot in them, so that's why I've been having to filter through it. | 20:21 |
| TheJulia | dking_desktop: what do you get if you grep for "req-fff2e8c2-d510-4f98-8d2b-4c18442283f0" | 20:22 |
| dking_desktop | I'm sure I've misconfigured something, since I've just been playing hit or miss with most of it. But at least I can say that things work well for the flat network. | 20:22 |
| dking_desktop | About 9 lines, which I can post if necessary, but I did see these relevant entries at the end: "default default] POST failed (client error): The resource could not be found." and "default default] No notification will be sent due to unsuccessful status code: 404 after /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79", and then there's the 404 for "POST /v2.0/ports" | 20:24 |
| *** jawad_axd has quit IRC | 20:28 | |
| *** jawad_axd has joined #openstack-ironic | 20:29 | |
| *** jawad_axd has quit IRC | 20:33 | |
| TheJulia | rpioso: etingof: I've posted a couple comments on https://review.opendev.org/#/c/710751 and I'm wondering what the path forward is since I'm really confused about the current -1, as I suspect etingof may be as well. | 20:44 |
| patchbot | patch 710751 - sushy - Add `set_system_boot_options` method - 4 patch sets | 20:44 |
| TheJulia | dking_desktop: I suspect it is time to ask in #openstack-neutron for insight :( | 20:45 |
| dking_desktop | Oh, good point! Thank you so much for the help with this one so far. I didn't even really know where to begin when I started. | 20:46 |
| TheJulia | dking_desktop: I feel bad that I feel like we haven't been as much help as we could be, but the fact the exception is coming from deep inside neutron is kind of... worrisome. | 20:55 |
| TheJulia | etingof: rpioso: I noticed an x/sushy-opem-idrac patch was added to the vendor priorities. Just FYI, ironic-core doesn't have privileges on that repo so I _suspect_ you both may need to take a look at it. | 20:55 |
| dking_desktop | Yeah. I'm totally sure that I've just horribly set up some option. I just don't know how to find it from that end. | 20:55 |
| dking_desktop | Thanks for the help. | 20:56 |
| TheJulia | dtantsur|afk: I kind of wonder if you should rebase your ipa change on top of the uefi software raid change in order to save headaches in the long run and just get the patches stacked for landing in rapid succession | 20:59 |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic master: [doc] Add documentation for retirement support https://review.opendev.org/713410 | 21:19 |
| openstackgerrit | Merged openstack/bifrost master: Fix dhcp job https://review.opendev.org/712304 | 21:22 |
| JayF | ^^ I knew nothing about retirement, this is amazing! | 21:28 |
| arne_wiebalck | JayF: Thanks. Relatively simple change, but hopefully helpful for operators :) | 21:29 |
| JayF | I'm writing up an internal doc that is in that category of things, so it's just super timely. | 21:29 |
| arne_wiebalck | category of things == life cycle management? | 21:30 |
| JayF | yes, exactly | 21:34 |
| JayF | and I've often wanted a flag like that in the past -- for instnace, in OnMetal, we once had a customer report a hardware issue on an active machine, but they didn't want to reprovision to fix (it was a missing ram stick?) | 21:35 |
| JayF | so we could've flagged it retired, made sure it didn't get provisioned to anyone else, and fixed once torn down | 21:35 |
| arne_wiebalck | yes, there was a discussion about a "take-out-of-prod-for-a-while" flag ... in the end we decided this is different (so we called this one retirement), but it could be used of course :) | 21:37 |
| JayF | I mean, I can use the flag for whatever, no matter what it's called for | 21:37 |
| JayF | I sure as hell have put nodes in maintenance for things other than maintenancing them :P | 21:37 |
| arne_wiebalck | there are also scoped introspection rules in the pipeline, this allows to store rules for different hardware types and avoids flushing all the rules in between (which is handy if you have a heterogeneous fleet to manage) | 21:37 |
| arne_wiebalck | oh, yes :-D | 21:37 |
| arne_wiebalck | https://review.opendev.org/#/c/702215/ in case you're interested | 21:39 |
| patchbot | patch 702215 - ironic-inspector - Added scope to introspection rules. - 14 patch sets | 21:39 |
| JayF | I don't know enough about inspector (and never really have) to have an informed opinion on what that even means | 21:42 |
| JayF | but I'm sure it's awesome :D | 21:43 |
| arne_wiebalck | lol | 21:43 |
| openstackgerrit | Merged openstack/sushy-cli master: Add systems, managers and chassis listing commands https://review.opendev.org/706461 | 21:56 |
| openstackgerrit | Merged openstack/sushy-cli master: Add system boot device/mode commands https://review.opendev.org/706392 | 21:56 |
| openstackgerrit | Merged openstack/sushy-cli master: Add `--insecure` and `--tls-certificates` options https://review.opendev.org/712058 | 21:56 |
| * etingof noticed a merge party is unfolding \o/ | 22:00 | |
| openstackgerrit | Merged openstack/ironic-python-agent master: Explicitly set ramdisk type https://review.opendev.org/714419 | 22:01 |
| openstackgerrit | Verification of a change to openstack/ironic-python-agent failed: [trivial] Fix comment for Software RAID restart https://review.opendev.org/714503 | 22:01 |
| openstackgerrit | Merged openstack/ironic-prometheus-exporter stable/train: DevStack support and Redfish job https://review.opendev.org/711402 | 22:05 |
| TheJulia | yeessss... a merge party indeed | 22:05 |
| TheJulia | er, yaaasss | 22:05 |
| TheJulia | avoiding doing frustrating things, I did code review | 22:07 |
| openstackgerrit | Merged openstack/ironic-prometheus-exporter master: Add documentation https://review.opendev.org/713274 | 22:07 |
| TheJulia | hjensas: fyi, tagged you on https://review.opendev.org/#/c/714212/ | 22:08 |
| patchbot | patch 714212 - ironic - "dual stack" support for PXE/iPXE - 7 patch sets | 22:08 |
| etingof | codetherapy it is | 22:09 |
| rpioso | TheJulia: Ack re: x/sushy-oem-drac patch. | 22:23 |
| *** mbeierl has quit IRC | 22:23 | |
| *** mbeierl has joined #openstack-ironic | 22:32 | |
| *** ktibi_ has joined #openstack-ironic | 22:39 | |
| *** ktibi_ has quit IRC | 22:44 | |
| *** persia has quit IRC | 22:46 | |
| *** persia_ has joined #openstack-ironic | 22:47 | |
| hjensas | TheJulia: regarding - https://review.opendev.org/#/c/700002 on an ironic-inspector patch? | 22:56 |
| patchbot | patch 700002 - ironic - Additional IP addresses to IPv6 stateful ports - 15 patch sets | 22:56 |
| hjensas | TheJulia: I don't think we need anything in inspector. When inspection uses neutron networks the same code path adding multiple addresses in ironic is used. And in the other case we have a dynamic pool of addresses in the non-neutron dhcp server for inspection. | 22:57 |
| *** tkajinam has joined #openstack-ironic | 22:57 | |
| TheJulia | hjensas: the network_type field? | 23:31 |
| hjensas | TheJulia: if I get it right inspector does a "patch" when adding local_link_connection. So it should'nt overwrite the 'networ_type' field? | 23:33 |
| hjensas | TheJulia: I may be wrong there. | 23:33 |
| TheJulia | hjensas: I may have misunderstood your comment then | 23:34 |
| hjensas | TheJulia: my first iteration did'nt allow mixing network_type with a switch_id and port_id. I changed that so network_type can be mixed with other data. | 23:36 |
| hjensas | TheJulia: I will take another look at Inspector to check if it will overwrite the entire field, or just update/add the relevant keys. | 23:38 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!