| *** rloo has quit IRC | 00:19 | |
| *** diurnalist has quit IRC | 00:20 | |
| openstackgerrit | Merged openstack/ironic master: Use IRONIC_VM_SPECS_RAM from ironic-base in ipv6 job https://review.opendev.org/738653 | 00:28 |
|---|---|---|
| *** diurnalist has joined #openstack-ironic | 00:46 | |
| *** sean-k-mooney1 has joined #openstack-ironic | 00:59 | |
| *** sean-k-mooney has quit IRC | 01:00 | |
| *** Goneri has quit IRC | 01:02 | |
| *** diurnalist has quit IRC | 01:08 | |
| *** gyee has quit IRC | 01:15 | |
| *** ricolin has joined #openstack-ironic | 01:54 | |
| *** diurnalist has joined #openstack-ironic | 02:13 | |
| openstackgerrit | Steve Baker proposed openstack/metalsmith master: metalsmith_deployment role switch to metalsmith_instances https://review.opendev.org/737888 | 02:17 |
| openstackgerrit | Steve Baker proposed openstack/metalsmith master: Document metalsmith_instances https://review.opendev.org/738131 | 02:17 |
| openstackgerrit | Steve Baker proposed openstack/metalsmith master: DNM debug image variables https://review.opendev.org/738949 | 02:17 |
| *** Lucas_Gray has quit IRC | 02:23 | |
| *** diurnalist has quit IRC | 02:27 | |
| *** ericlei has joined #openstack-ironic | 02:28 | |
| *** ericlei has quit IRC | 02:29 | |
| *** ericlei has joined #openstack-ironic | 02:30 | |
| *** ericlei has quit IRC | 02:46 | |
| *** mugsie has quit IRC | 02:59 | |
| *** mugsie has joined #openstack-ironic | 03:02 | |
| *** tzumainn has quit IRC | 03:09 | |
| *** Goneri has joined #openstack-ironic | 03:28 | |
| *** Goneri has quit IRC | 03:53 | |
| openstackgerrit | Steve Baker proposed openstack/metalsmith master: metalsmith_deployment role switch to metalsmith_instances https://review.opendev.org/737888 | 03:54 |
| openstackgerrit | Steve Baker proposed openstack/metalsmith master: Document metalsmith_instances https://review.opendev.org/738131 | 03:54 |
| *** ddurst has joined #openstack-ironic | 04:03 | |
| *** diurnalist has joined #openstack-ironic | 05:01 | |
| *** abdysn has joined #openstack-ironic | 05:06 | |
| *** diurnalist has quit IRC | 05:06 | |
| *** priteau has joined #openstack-ironic | 05:16 | |
| arne_wiebalck | Good morning, ironic! | 05:37 |
| *** akahat is now known as akahat|pto | 05:58 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 06:03 | |
| iurygregory | good morning arne_wiebalck and Ironic o/ | 06:04 |
| arne_wiebalck | hey iurygregory o/ | 06:04 |
| *** yaawang has quit IRC | 06:38 | |
| *** yaawang has joined #openstack-ironic | 06:44 | |
| rpittau | good morning ironic! o/ | 06:44 |
| iurygregory | hey rpittau o/ | 06:49 |
| rpittau | hey iurygregory :) | 06:50 |
| openstackgerrit | Richard G. Pioso proposed openstack/ironic master: WIP: Fix idrac HW type DevStack plug-in support https://review.opendev.org/738973 | 06:53 |
| openstackgerrit | Merged openstack/ironic master: Fix fast track following managed inspection https://review.opendev.org/738864 | 07:06 |
| arne_wiebalck | Good morning rpittau and rpioso o/ | 07:07 |
| rpittau | hey arne_wiebalck :) | 07:08 |
| rpioso | Good morning, arne_wiebalck and rpittau :-) And good nite o/ | 07:09 |
| arne_wiebalck | rpioso: that's what I thought when I saw your commit :) | 07:09 |
| rpioso | arne_wiebalck: Yeah, I haven't traveled far from home :-) | 07:09 |
| rpittau | good night rpioso :) | 07:10 |
| rpioso | rpittau: Hope you a great day! | 07:11 |
| rpittau | thanks! | 07:11 |
| rpioso | s/you/you have/ | 07:11 |
| *** yaawang has quit IRC | 07:17 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Prepare release notes and docs for release 15.1 https://review.opendev.org/738900 | 07:28 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Prepare release notes and docs for release 15.1 https://review.opendev.org/738900 | 07:28 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Prepare release notes and docs for release 15.1 https://review.opendev.org/738900 | 07:29 |
| *** dougsz has joined #openstack-ironic | 07:29 | |
| *** yaawang has joined #openstack-ironic | 07:31 | |
| *** Lucas_Gray has joined #openstack-ironic | 07:40 | |
| *** rcernin has quit IRC | 07:45 | |
| *** rh-jlabarre has joined #openstack-ironic | 07:47 | |
| *** dtantsur|afk is now known as dtantsur | 07:47 | |
| dtantsur | morning ironic | 07:48 |
| *** rh-jelabarre has quit IRC | 07:48 | |
| *** Lucas_Gray has quit IRC | 07:50 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent stable/train: Add timeout operations to try and prevent hang on read() https://review.opendev.org/738532 | 07:50 |
| openstackgerrit | Merged openstack/ironic master: Untie the ramdisk deploy from AgentDeploy https://review.opendev.org/737724 | 07:54 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent stable/train: Add timeout operations to try and prevent hang on read() https://review.opendev.org/738532 | 07:54 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent stable/train: Add full download retries https://review.opendev.org/738533 | 07:57 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent stable/train: Extend retries to 9, 10 seconds apart. https://review.opendev.org/738534 | 07:57 |
| dtantsur | rpittau: I hope these ^^ are *finally* ready | 07:57 |
| iurygregory | morning dtantsur | 07:58 |
| * iurygregory brb need to go to the office | 07:58 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/ussuri: Fix fast track following managed inspection https://review.opendev.org/738977 | 07:58 |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic-inspector master: Add leader election for periodic sync with ironic https://review.opendev.org/735527 | 08:01 |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic-inspector master: Add leader election for periodic sync with ironic https://review.opendev.org/735527 | 08:02 |
| *** Lucas_Gray has joined #openstack-ironic | 08:07 | |
| *** lucasagomes has joined #openstack-ironic | 08:07 | |
| openstackgerrit | Merged openstack/ironic master: Use getfullargspec to inspect functions https://review.opendev.org/738868 | 08:12 |
| *** diurnalist has joined #openstack-ironic | 08:14 | |
| *** diurnalist has quit IRC | 08:18 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Prepare release notes and docs for release 15.1 https://review.opendev.org/738900 | 08:19 |
| *** k_mouza has joined #openstack-ironic | 08:22 | |
| *** ociuhandu has quit IRC | 08:22 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: agent_base: support inserting in-band deploy steps https://review.opendev.org/737695 | 08:29 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Decompose the core deploy step of the direct deploy https://review.opendev.org/698309 | 08:31 |
| *** ericlei has joined #openstack-ironic | 08:38 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Decompose the core deploy step on iscsi and ansible deploy https://review.opendev.org/731180 | 08:41 |
| *** hjensas|afk is now known as hjensas | 08:46 | |
| *** derekh has joined #openstack-ironic | 08:49 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-specs master: Amend the in-band deploy steps spec https://review.opendev.org/738991 | 08:49 |
| *** ociuhandu has joined #openstack-ironic | 08:50 | |
| *** k_mouza has quit IRC | 08:53 | |
| openstackgerrit | Merged openstack/python-ironicclient master: Allow to pass global request id in port and volume https://review.opendev.org/725930 | 08:54 |
| *** k_mouza has joined #openstack-ironic | 08:57 | |
| openstackgerrit | Merged openstack/sushy-tools master: vmedia: fix TypeError when accessing a URL without content-disposition https://review.opendev.org/738638 | 08:58 |
| *** yedongcan has joined #openstack-ironic | 09:02 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Prepare release notes and docs for release 15.1 https://review.opendev.org/738900 | 09:03 |
| *** ericlei has quit IRC | 09:07 | |
| *** tkajinam has quit IRC | 09:09 | |
| * arne_wiebalck made Zuul happy | 09:26 | |
| dtantsur | good job arne_wiebalck :) | 09:27 |
| arne_wiebalck | dtantsur: :) | 09:43 |
| arne_wiebalck | dtantsur: this is still the leader election, I tried this again this morning with the two inspectors in our qa env and it seems to be working as expected | 09:44 |
| dtantsur | woohoo | 09:44 |
| arne_wiebalck | as soon as someone else had a look I can test^H^H^H^Hmove it to prod (which has 12 inspectors) | 09:46 |
| * arne_wiebalck does not test in prod | 09:47 | |
| openstackgerrit | Merged openstack/ironic master: Add ironic 15.1 release mapping https://review.opendev.org/738904 | 09:51 |
| iurygregory | Im back o/ | 09:53 |
| *** priteau has quit IRC | 09:58 | |
| dtantsur | arne_wiebalck: just a couple of minor things | 09:58 |
| arne_wiebalck | dtantsur: thanks, I'll have a look | 09:59 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: Fix serializing ironic-lib exceptions https://review.opendev.org/739005 | 10:19 |
| dtantsur | rpittau: since I've updated https://review.opendev.org/#/c/738533/ I'd like a 2nd pair of eyes on it please | 10:23 |
| patchbot | patch 738533 - ironic-python-agent (stable/train) - Add full download retries - 4 patch sets | 10:23 |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic-inspector master: Add leader election for periodic sync with ironic https://review.opendev.org/735527 | 10:24 |
| *** ricolin has quit IRC | 10:30 | |
| *** k_mouza has quit IRC | 10:31 | |
| openstackgerrit | Merged openstack/ironic-python-agent master: Mount all vfat partitions before calling grub2 https://review.opendev.org/725628 | 10:37 |
| *** k_mouza has joined #openstack-ironic | 10:41 | |
| ftarasenko | arne_wiebalck: Hi! Can you help me with building of provision image for using with software raid and uefi? How do you create it? I found that ironic-python-agent-builder does not put efi packages into the image, so I get error while trying to put grub2 to the device. Now trying to rebuild image with required packages, just wondering, maybe there is easier way doing this) | 10:44 |
| *** alexmcleod has quit IRC | 10:44 | |
| iurygregory | O.o | 10:46 |
| iurygregory | wut | 10:46 |
| *** alexmcleod has joined #openstack-ironic | 10:50 | |
| *** sean-k-mooney1 is now known as sean--k-mooney | 10:52 | |
| dtantsur | ftarasenko: an important thing to note: ironic uses grub2 packages from the *user image*, not from IPA | 10:55 |
| *** sean--k-mooney is now known as sean-k-mooney | 10:55 | |
| ftarasenko | dtantsur: So next error can be related to my image? Intersting point, thank you. I'll try to rebuild it also. | 10:56 |
| ftarasenko | grub2-install: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist. Please specify --target or --directory.\n" | 10:56 |
| dtantsur | ftarasenko: yep, all grub2 stuff is taken from your image (IPA chroots into it) | 10:59 |
| ftarasenko | I see chroot before this command, so you must be right | 10:59 |
| * dtantsur wonders if arne_wiebalck has more experience | 10:59 | |
| ftarasenko | dtantsur: thank you! | 11:00 |
| *** rajinir has quit IRC | 11:00 | |
| ftarasenko | do you have any good experience how to get rootfs_uuid from target image? without boot of course) | 11:01 |
| *** yedongcan has quit IRC | 11:02 | |
| *** uzumaki has joined #openstack-ironic | 11:10 | |
| dtantsur | ftarasenko: something like https://github.com/openstack/metalsmith/blob/master/playbooks/integration/centos-image.yaml#L37 ? | 11:15 |
| *** yedongcan has joined #openstack-ironic | 11:21 | |
| dtantsur | TheJulia: when you have a minute: https://doodle.com/poll/3r9tbhmniattkty8 | 11:23 |
| *** yedongcan has quit IRC | 11:25 | |
| arne_wiebalck | ftarasenko: hi! | 11:36 |
| * arne_wiebalck is reading the backlog ... | 11:39 | |
| arne_wiebalck | ftarasenko: I had a similar issue with running grub: some of the files were not accessible since the ESP was not mounted *inside* the user image. | 11:40 |
| arne_wiebalck | ftarasenko: https://review.opendev.org/725628 has just been merged to fix this. Not sure you have the same issue, though. | 11:41 |
| patchbot | patch 725628 - ironic-python-agent - Mount all vfat partitions before calling grub2 (MERGED) - 4 patch sets | 11:41 |
| arne_wiebalck | ftarasenko: what I suggest is to put a break point right before the chroot (a while loop looking for a non-existing file, for instance), and do the chroot and grub-install by hand | 11:42 |
| arne_wiebalck | ftarasenko: what is your user image based on? | 11:43 |
| arne_wiebalck | ftarasenko: which OS I mean (since apparently, things also changed from CentOS7 to CentOS8, for instance) | 11:43 |
| *** uzumaki has quit IRC | 11:44 | |
| arne_wiebalck | ftarasenko: the file missing in your case is coming from grub2-efi-x64-modules | 11:46 |
| ftarasenko | arne_wiebalck: I have Centos8. But I think, I do not install grub2-efi-modules | 11:46 |
| arne_wiebalck | ftarasenko: is this installed in the user image? | 11:46 |
| arne_wiebalck | ftarasenko: :-) | 11:46 |
| arne_wiebalck | ftarasenko: https://techblog.web.cern.ch/techblog/post/bios_uefi_cloud_image/ | 11:47 |
| arne_wiebalck | ftarasenko: this post summarises what we did | 11:47 |
| ftarasenko | arne_wiebalck: It's not installed, I'm rebuilding image. A bit strange that I had successful deploys of this image to UEFI systems. | 11:47 |
| ftarasenko | arne_wiebalck: Thank you! | 11:47 |
| arne_wiebalck | ftarasenko: successful with software RAID? | 11:47 |
| arne_wiebalck | ftarasenko: (the post also mentions grub2-efi-x64 and grub2-efi-x64-modules) | 11:48 |
| ftarasenko | arne_wiebalck: No, without software raid. UEFI only. I've added grub2-efi-modules to package-installs, other packages seems to be dependencies (they are also installed). Now I'll check deployment. | 11:49 |
| arne_wiebalck | ftarasenko: right, w/o software RAID there is no need for these packages since the bootloader is not reinstalled | 11:50 |
| arne_wiebalck | ftarasenko: this is why your previous deploys worked | 11:50 |
| dtantsur | arne_wiebalck: do we need to backport https://review.opendev.org/#/c/725628/ ? | 11:51 |
| patchbot | patch 725628 - ironic-python-agent - Mount all vfat partitions before calling grub2 (MERGED) - 4 patch sets | 11:51 |
| ftarasenko | arne_wiebalck: do you have any good ideas how to get uuid from built image? or guestfish is the best way? | 11:51 |
| arne_wiebalck | dtantsur: I think this is only for the special case of trying to install the BIOS bootloader on am image that is UEFI capable ... not sure many people have this | 11:52 |
| arne_wiebalck | ftarasenko: guestfish is one way, mounting it on a loopback device another I think | 11:53 |
| *** ricolin has joined #openstack-ironic | 11:53 | |
| * arne_wiebalck thinks we should add sth about this to the docs | 11:54 | |
| dtantsur | a good idea | 11:59 |
| arne_wiebalck | ftarasenko: yes, mounting loopback and then running blkid will give you the uuid | 11:59 |
| ftarasenko | qemu-img convert -p -O raw ./hw-centos7-v2.qcow2 ./hw-centos7-v2.raw | 12:02 |
| ftarasenko | sorry, i've succeded with next commands: | 12:03 |
| ftarasenko | qemu-img convert -p -O raw ./hw-centos7-v2.qcow2 ./hw-centos7-v2.raw | 12:03 |
| ftarasenko | losetup /dev/loop0 hw-centos7-v2.raw | 12:03 |
| ftarasenko | kpartx -a /dev/loop0 | 12:03 |
| ftarasenko | blkid | 12:03 |
| openstackgerrit | Verification of a change to openstack/ironic failed: Fix fast track following managed inspection https://review.opendev.org/738977 | 12:04 |
| arne_wiebalck | ftarasenko: thanks, I'll add this to the docs | 12:11 |
| openstackgerrit | Merged openstack/ironic master: ironic-standalone, use http basic auth for json-rpc https://review.opendev.org/736398 | 12:12 |
| openstackgerrit | Merged openstack/ironic-python-agent stable/train: Add timeout operations to try and prevent hang on read() https://review.opendev.org/738532 | 12:12 |
| *** rh-jlabarre has quit IRC | 12:17 | |
| *** rh-jelabarre has joined #openstack-ironic | 12:18 | |
| openstackgerrit | Richard G. Pioso proposed openstack/ironic master: WIP: Fix idrac HW type DevStack plug-in support https://review.opendev.org/738973 | 12:24 |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic master: [doc] Describe how to extract an fs UUID from an image https://review.opendev.org/739027 | 12:25 |
| arne_wiebalck | ftarasenko: dtantsur : ^^ | 12:26 |
| iurygregory | arne_wiebalck, ++ | 12:29 |
| ftarasenko | arne_wiebalck: I think, we need to call losetup -f at first, to have /dev/loop0 available for mount. | 12:39 |
| arne_wiebalck | ftarasenko: works for me without -f | 12:42 |
| arne_wiebalck | ftarasenko: but I used loop5 ... maybe this is needed on the very first one? | 12:43 |
| ftarasenko | arne_wiebalck: by default you do not have loop devices in system. losetup -f creates first one | 12:43 |
| arne_wiebalck | ftarasenko: or it prints the first one available, yes | 12:44 |
| arne_wiebalck | ftarasenko: so, did it print loop0 for you? | 12:44 |
| arne_wiebalck | ftarasenko: I did this "manually" be checking the device list :-D | 12:45 |
| arne_wiebalck | ftarasenko: I will add it to the patch | 12:45 |
| ftarasenko | arne_wiebalck: it prints /dev/loop1 for me, but it's ok, if there is no loop device, losetup creates one | 12:45 |
| arne_wiebalck | ftarasenko: yes, now, since you have loop0 | 12:46 |
| arne_wiebalck | ftarasenko: it prints the next one free | 12:46 |
| openstackgerrit | Arne Wiebalck proposed openstack/ironic master: [doc] Describe how to extract an fs UUID from an image https://review.opendev.org/739027 | 12:47 |
| ftarasenko | arne_wiebalck: so it cat tell us, which device to use. I think, you may add this to patch | 12:47 |
| arne_wiebalck | ftarasenko: done | 12:47 |
| ftarasenko | arne_wiebalck: tnx! | 12:47 |
| *** Abdallahyas has joined #openstack-ironic | 12:50 | |
| *** abdysn has quit IRC | 12:50 | |
| arne_wiebalck | ftarasenko: thank you! | 12:50 |
| *** Lucas_Gray has quit IRC | 12:57 | |
| *** Abdallahyas has quit IRC | 13:01 | |
| *** abdysn has joined #openstack-ironic | 13:01 | |
| *** rloo has joined #openstack-ironic | 13:06 | |
| *** Lucas_Gray has joined #openstack-ironic | 13:07 | |
| *** eric_lei has joined #openstack-ironic | 13:13 | |
| TheJulia | good morning | 13:25 |
| rpittau | good morning TheJulia :) | 13:26 |
| *** hoonetorg has quit IRC | 13:26 | |
| dtantsur | morning TheJulia | 13:30 |
| *** hoonetorg has joined #openstack-ironic | 13:31 | |
| openstackgerrit | Merged openstack/ironic-python-agent stable/train: Add full download retries https://review.opendev.org/738533 | 13:32 |
| dtantsur | desperately looking for reviews on https://review.opendev.org/#/c/737695/ | 13:34 |
| patchbot | patch 737695 - ironic - agent_base: support inserting in-band deploy steps - 4 patch sets | 13:34 |
| dtantsur | it's not as hard as it may seem | 13:34 |
| iurygregory | good morning TheJulia =) | 13:34 |
| openstackgerrit | Merged openstack/ironic-python-agent stable/train: Extend retries to 9, 10 seconds apart. https://review.opendev.org/738534 | 13:36 |
| TheJulia | dtantsur: responded | 13:37 |
| dtantsur | thx! | 13:37 |
| *** eric_lei has quit IRC | 13:38 | |
| dtantsur | gah, we don't seem to have a way to include stevebaker :( | 13:38 |
| TheJulia | dtantsur: you realize on https://review.opendev.org/#/c/738874/ that your asking me to write test framework for code that never had it before... right? | 13:40 |
| patchbot | patch 738874 - sushy-tools - Support non-ide virtual media buses - 4 patch sets | 13:40 |
| dtantsur | TheJulia: I *think* I checked to see that tests existed for that one | 13:41 |
| TheJulia | it exists, only tests like the tiniest fraction of the code | 13:41 |
| TheJulia | and by it, I mean the file | 13:41 |
| *** tzumainn has joined #openstack-ironic | 13:42 | |
| dtantsur | well, there are tests for this function at least https://opendev.org/openstack/sushy-tools/src/branch/master/sushy_tools/tests/unit/emulator/resources/systems/test_libvirt.py#L496 | 13:43 |
| dtantsur | although... it doesn't check the resulting XML \o/ | 13:43 |
| iurygregory | hehehe | 13:43 |
| TheJulia | yeeeahh | 13:44 |
| * dtantsur rolls eyes | 13:44 | |
| dtantsur | okay, fine, sorry, I missed that | 13:44 |
| TheJulia | I can do it, just I've not had much luck with throughput with interrupts recently | 13:44 |
| dtantsur | the reason I asked is because I've just fixed a bug in sushy-tools that was easily preventable by a unit test | 13:44 |
| dtantsur | TheJulia: here or in a follow-up - up to you | 13:44 |
| dtantsur | and please tell me you did test it :) | 13:45 |
| iurygregory | it was tested | 13:45 |
| TheJulia | dtantsur: let me see if I can get something done today, I think I only have one meeting today \o/ | 13:45 |
| iurygregory | our fault XD | 13:46 |
| dtantsur | I can approve if it's blocking people (I guess it may be blocking stbenjam) | 13:46 |
| TheJulia | yeah, stbenjamin fired it up and it worked like a charm once we figured out that the xml returned a string, and the list tracking was a number | 13:46 |
| iurygregory | dtantsur, you correct =) | 13:46 |
| dtantsur | I left my review under assumption that these unit tests actually test something | 13:46 |
| TheJulia | no worries | 13:46 |
| * TheJulia drinks coffee and attempts to like.. really wake up | 13:47 | |
| dtantsur | so please let me know if you're prefer me to approve it now or wait for tests | 13:47 |
| TheJulia | dtantsur: let me see what I can get done today | 13:47 |
| dtantsur | k | 13:47 |
| TheJulia | for a test, because while it is simple, i'd still prefer to avoid senseless issues | 13:47 |
| iurygregory | I will keep my +2 in that case | 13:48 |
| stbenjam | Yea it is blocking me from making progress on some work to remove the provisioning network in OSP -- would greatly appreciate it if it could land soon :) | 13:49 |
| TheJulia | o/ stbenjam | 13:50 |
| stbenjam | \o | 13:51 |
| stbenjam | s/OSP/OCP/ | 13:52 |
| stbenjam | Too many acronyms | 13:53 |
| iurygregory | hehehe | 13:53 |
| *** belmoreira has joined #openstack-ironic | 13:58 | |
| *** abdysn has quit IRC | 13:59 | |
| *** Goneri has joined #openstack-ironic | 14:01 | |
| *** belmorei_ has joined #openstack-ironic | 14:05 | |
| *** belmore__ has joined #openstack-ironic | 14:06 | |
| *** belmoreira has quit IRC | 14:07 | |
| *** belmorei_ has quit IRC | 14:09 | |
| *** diurnalist has joined #openstack-ironic | 14:21 | |
| stbenjam | What's needed to get https://review.opendev.org/#/c/738874/ in? It says missing workflow label? | 14:22 |
| patchbot | patch 738874 - sushy-tools - Support non-ide virtual media buses - 4 patch sets | 14:22 |
| TheJulia | stbenjam: 2x core reviewers to +2 and push it through. I'm going to see if I can get some additional unit testing once I'm able to get out of discussion threads and questions | 14:23 |
| TheJulia | (not saying this is is an issue or interruption, it just so happens I'm reaching the end of the queue it seems | 14:23 |
| iurygregory | stbenjam, it will probably merge today/tomorrow morning | 14:26 |
| TheJulia | I suspect we could let it move forward with just a single core review on tests | 14:27 |
| TheJulia | We may want to just finally explicitly revise the 2x core "rules" | 14:27 |
| iurygregory | makes sense to me | 14:28 |
| TheJulia | ugh, my desktop does not want the libvirt python stuff installed | 14:36 |
| dtantsur | wow | 14:37 |
| TheJulia | debian is full of lolcat's going "nope" | 14:38 |
| * TheJulia ats breakfast and will go power-up the at home lab machines that I know have the packages already | 14:42 | |
| *** mkrai has joined #openstack-ironic | 14:43 | |
| *** priteau has joined #openstack-ironic | 14:51 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Enforce autospec in test_manager https://review.opendev.org/738848 | 14:55 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Follow up of enforce autospec in test_manager https://review.opendev.org/739063 | 14:55 |
| *** sshnaidm|ruck is now known as sshnaidm|mtg | 14:57 | |
| *** belmore__ has quit IRC | 15:08 | |
| *** rh-jlabarre has joined #openstack-ironic | 15:10 | |
| *** sean-k-mooney1 has joined #openstack-ironic | 15:12 | |
| *** vesper has joined #openstack-ironic | 15:13 | |
| *** mkrai has quit IRC | 15:13 | |
| *** gudrutis1 has joined #openstack-ironic | 15:13 | |
| *** rh-jelabarre has quit IRC | 15:14 | |
| *** vesper11 has quit IRC | 15:14 | |
| *** Goneri has quit IRC | 15:14 | |
| *** irclogbot_0 has quit IRC | 15:14 | |
| *** mnasiadka has quit IRC | 15:14 | |
| *** mnasiadka_ has joined #openstack-ironic | 15:14 | |
| *** JasonF has joined #openstack-ironic | 15:14 | |
| *** Goneri has joined #openstack-ironic | 15:14 | |
| *** sean-k-mooney has quit IRC | 15:14 | |
| *** JayF has quit IRC | 15:14 | |
| *** dougsz has quit IRC | 15:14 | |
| arne_wiebalck | dtantsur: leadership election seems also to be working fine with 12 inspectors (yes, in prod :-); I reduced the clean_up_period now by a factor 10 again and see no impact on the response time; I also confirmed that at least one of them is indeed still syncing :) | 15:14 |
| *** JasonF is now known as JayF | 15:14 | |
| *** gudrutis has quit IRC | 15:14 | |
| *** gudrutis1 is now known as gudrutis | 15:14 | |
| *** lifeless has quit IRC | 15:14 | |
| *** tonyb has quit IRC | 15:15 | |
| *** jamesdenton has quit IRC | 15:15 | |
| dtantsur | cool | 15:15 |
| dtantsur | arne_wiebalck: talked to ftarasenko in private, apparently our assumption that ESP fits into 128MiB is overly optimistic | 15:15 |
| *** dougsz has joined #openstack-ironic | 15:15 | |
| dtantsur | specifically, this https://opendev.org/openstack/ironic-python-agent/src/branch/master/ironic_python_agent/raid_utils.py#L92 is incorrect | 15:16 |
| dtantsur | esp. after https://github.com/openstack/diskimage-builder/blob/7f469e3e83c45089d2f66c91ec3ed4eec236d7b4/releasenotes/notes/gpt-esp-partition-size-200568cb6fcc8788.yaml :) | 15:16 |
| rpittau | lol | 15:16 |
| *** tonyb has joined #openstack-ironic | 15:16 | |
| *** jtomasek_ has quit IRC | 15:17 | |
| dtantsur | https://superuser.com/a/1310938 attributes 550 to the author of gdisk | 15:17 |
| dtantsur | arne_wiebalck: do you have any concerns about s/128/550/ in IPA? | 15:17 |
| * arne_wiebalck is still reading | 15:18 | |
| *** irclogbot_1 has joined #openstack-ironic | 15:18 | |
| *** jamesdenton has joined #openstack-ironic | 15:18 | |
| *** lifeless has joined #openstack-ironic | 15:20 | |
| arne_wiebalck | dtantsur: no ... but I was pretty sure I had read 128 somewhere ... | 15:20 |
| dtantsur | maybe it used to be the guidance | 15:20 |
| rpittau | dtantsur, arne_wiebalck, the only configuration parameter we have for coordination is the coordination backend_url ? | 15:20 |
| dtantsur | rpittau: I think so? | 15:21 |
| arne_wiebalck | rpittau: there is now also the leader_election_interval | 15:21 |
| arne_wiebalck | but before the patch, yes | 15:21 |
| rpittau | I was thinking of a possible scenario where we pass from an unsupported to a supported backend and we end up with keeping self.supports_election False, but that change requires a restart, so that should not happen | 15:22 |
| * dtantsur prepares a patch | 15:22 | |
| dtantsur | yeah, I don't think we can change the backend without restarting | 15:22 |
| rpittau | ok | 15:23 |
| TheJulia | woot, lab machine had the packages already *dances* | 15:24 |
| *** sean-k-mooney1 is now known as sean-k-mooney | 15:26 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: Increase the ESP partition size to 550 MiB when using software RAID https://review.opendev.org/739075 | 15:30 |
| dtantsur | arne_wiebalck: ^^ | 15:30 |
| arne_wiebalck | dtantsur: the centos image uses ~500M | 15:31 |
| dtantsur | okay, so 550 should be enough | 15:31 |
| arne_wiebalck | dtantsur: so we should be good with a 550M partition | 15:31 |
| arne_wiebalck | :) | 15:31 |
| arne_wiebalck | I am looking at https://github.com/CentOS/sig-cloud-instance-build/blob/master/cloudimg/CentOS-8-x86_64-Azure.ks | 15:32 |
| arne_wiebalck | seems like 600MB is the max | 15:38 |
| dtantsur | rpittau, iurygregory, if either of you is by chance in the mood for more release note fixing, could you check IPA, bifrost and inspector? | 15:48 |
| rpittau | dtantsur: I was looking at inspector | 15:48 |
| dtantsur | awesome, thanks | 15:48 |
| *** gyee has joined #openstack-ironic | 15:55 | |
| *** gyee has quit IRC | 15:56 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Prepare release notes and docs for release 10.2 https://review.opendev.org/739081 | 15:58 |
| *** k_mouza has quit IRC | 15:59 | |
| *** dmellado has joined #openstack-ironic | 16:00 | |
| * TheJulia comes to the conlusion mock is insane | 16:00 | |
| rpittau | and tries very hard to bring everyone in its insane world | 16:01 |
| rpittau | dtantsur, TheJulia, checked IPA renos, didn't see anything to be changedk | 16:02 |
| rpittau | -k | 16:02 |
| TheJulia | thanks | 16:03 |
| rpittau | since I'm here, I'm checking bifrost too | 16:03 |
| TheJulia | thanks :) | 16:03 |
| TheJulia | gaH!!!!! mock | 16:03 |
| *** lucasagomes has quit IRC | 16:07 | |
| dtantsur | TheJulia: trying to backport something? :) | 16:11 |
| TheJulia | no, luckilly ont | 16:11 |
| TheJulia | should have unit tests upf or that patch in about five minutes | 16:11 |
| dtantsur | ah, sushy-tools | 16:12 |
| dtantsur | nice | 16:12 |
| * dtantsur is debugging fast-track nowadays | 16:12 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-inspector master: Prepare release notes for release 10.2 https://review.opendev.org/739081 | 16:12 |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Prepare release notes for release 8.2 https://review.opendev.org/739089 | 16:14 |
| *** gyee has joined #openstack-ironic | 16:14 | |
| openstackgerrit | Julia Kreger proposed openstack/sushy-tools master: Support non-ide virtual media buses https://review.opendev.org/738874 | 16:18 |
| dtantsur | TheJulia: so, an interesting problem with fast-track (I think we're hitting it in metal3). Imagine, you run inspection and then immediately cleaning. It may easily happen that IPA does not have a chance to heartbeat, so ironic assumes the ramdisk is down. | 16:19 |
| dtantsur | I wonder how we can cleanly solve it | 16:19 |
| dtantsur | for one, we take too long to start heartbeating | 16:20 |
| *** dmellado has quit IRC | 16:20 | |
| dtantsur | tomorrow I'd like to try caching hardware information so that we don't reload it so often | 16:21 |
| *** dmellado has joined #openstack-ironic | 16:22 | |
| TheJulia | dtantsur: hmmm | 16:24 |
| TheJulia | So basically we're hitting a case where the ramdisk may not have heartbeated at all? | 16:25 |
| TheJulia | or ??? | 16:25 |
| TheJulia | that is such a tight window... | 16:25 |
| dtantsur | It did not have a chance to finish lookup+heartbeat by the time cleaning starts | 16:25 |
| TheJulia | wow | 16:25 |
| dtantsur | it's not very tight, we seem to be reloading hardware inventory before lookup | 16:26 |
| TheJulia | gah | 16:26 |
| dtantsur | presumably to get the MAC addresses | 16:26 |
| TheJulia | hmm | 16:26 |
| dtantsur | metal3 switches from inspection to cleaning in 30 seconds | 16:26 |
| dtantsur | it's not always enough, as it seems | 16:26 |
| TheJulia | I would have thought we would have gotten a heartbeat in that window | 16:26 |
| TheJulia | but yeah, I guess not if we're reloading the hardware | 16:26 |
| TheJulia | yeouch | 16:26 |
| TheJulia | hmm | 16:27 |
| dtantsur | on this positive note I'll go grab dinner | 16:28 |
| *** dtantsur is now known as dtantsur|afk | 16:28 | |
| dtantsur|afk | g'night | 16:28 |
| TheJulia | goodnight | 16:29 |
| TheJulia | We can't really unify the operations. Ironic also has no way to really know if the node is going to be up shortly. Metal3 also doesn't really know when the machine is up and heartbeating | 16:30 |
| TheJulia | gah | 16:30 |
| openstackgerrit | Merged openstack/ironic master: Explicitly use python3 for ironic_lib_prefix https://review.opendev.org/738604 | 16:30 |
| *** ociuhandu_ has joined #openstack-ironic | 16:33 | |
| TheJulia | dtantsur|afk: if we're going to fix it and close it, it likely should be in ipa where we do lookup/heartbeat before we ever post back to inspector and/or use the same hardware information. I'm worried either of those changes are super invasive though | 16:34 |
| *** derekh has quit IRC | 16:34 | |
| TheJulia | hardware once is likely the least and at least closes the window signifigantly | 16:34 |
| *** dougsz has quit IRC | 16:35 | |
| *** ociuhandu has quit IRC | 16:36 | |
| openstackgerrit | Richard G. Pioso proposed openstack/ironic master: WIP: Fix idrac HW type DevStack plug-in support https://review.opendev.org/738973 | 16:36 |
| *** ociuhandu_ has quit IRC | 16:38 | |
| *** Lucas_Gray has quit IRC | 16:40 | |
| TheJulia | rpittau: fyi, on the sushy-tools patch, the machine type doesn't mater, because scsi/sata are both available for PC platforms | 16:52 |
| TheJulia | dtantsur|afk: maybe move _wait_for_interface up in the entire sequence? It seems logical based upon the duality of the ramdisk | 16:58 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Fix uefi jobs with native ubuntu ovmf package https://review.opendev.org/716889 | 17:10 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Limit Inspection->Lookup->Heartbeat lag https://review.opendev.org/739120 | 17:16 |
| *** k_mouza has joined #openstack-ironic | 17:17 | |
| TheJulia | dtantsur|afk: I suspect ^^^ should remove most of the lag. The other thing we could do is record a lookup as an early heartbeat, but that seems... problematic as well | 17:18 |
| *** k_mouza has quit IRC | 17:23 | |
| *** ricolin has quit IRC | 17:28 | |
| JayF | I'm not up to speed on the fast track stuff, but it seems like if we're able to start cleaning before heartbeats are recieved there's a flawed assumption somewhere -- IPA sends back a callback url in the heartbeat -- isn't it possible the machine has changed networks, and therefore IPs, between Inspection and Cleaning? | 17:47 |
| TheJulia | so plus side of ramdisk iso tempest test. It passes \o// Downside because the ISO being doesn't log to to console... it may be a PITA to troubleshoot one day. | 17:48 |
| * JayF assumes, but doesn't know for sure, that we have an inspection_network along with cleaning_network and provisiong_network | 17:48 | |
| openstackgerrit | Merged openstack/ironic stable/ussuri: Fix fast track following managed inspection https://review.opendev.org/738977 | 17:49 |
| TheJulia | JayF: the idea is we don't try and reboot the machine in fast-track mode if we've gotten a heartbeat already. I _think_ what dmitry is investigating is that we've not gotten that heartbeat so we spin killing the power, resetting the ramdisk potentially, and then restarting the deployment all which takes a bunch of extra time | 17:49 |
| TheJulia | well, definitely resetting the ramdisk if it is an virtual media iso | 17:50 |
| JayF | So do we not support changing from inspection -> cleaning network if fast-track is available? | 17:50 |
| JayF | (still working under my unverified assumption) | 17:50 |
| JayF | s/available/enabled/ | 17:50 |
| TheJulia | it is not a direct transition | 17:50 |
| TheJulia | it is more inspection->manageable->provide->cleaning | 17:51 |
| JayF | I'm convinced I don't know enough to be in this conversation. Just wanted to make sure you all had considered that if it's an issue, I trust you to know if it is or not actually :D | 17:51 |
| TheJulia | fast track is really modeled around non-neutron networks, standalone operators or operators not using separate networks because their posture explicitly allows long running ramdisks at that point | 17:51 |
| TheJulia | if we need to swap networks... that is likely a whole lot of nope | 17:52 |
| JayF | ack, disregard the following :D TheJulia: Yeah, but in that transition, the node could change networks, right? Which would trigger a re-DHCP because the interface would go up/down? Which would render the callback_url that Ironic was using to restart cleaning rapidly. | 17:52 |
| TheJulia | heh | 17:52 |
| * TheJulia wonders why we has no lolcats in documents | 17:52 | |
| JayF | I suspect that gap is not as huge as you'd think, to add network-switching to fast track, if you made sure you got a new heartbeat before each step | 17:52 |
| JayF | most OSes will act sanely when it sees the network change on the far side of a network cable | 17:53 |
| JayF | I guess I'm assuming there that a switch would cycle a port down/up to change networks, which may not be true everywhere :shrug: | 17:53 |
| TheJulia | That about switchports, and neutron generally takes a really long time to get the dhcp records updated | 17:53 |
| TheJulia | Although the bug that was making it fail quietly or take even longer seems to have largely been fixed | 17:54 |
| * TheJulia knocks on wood | 17:54 | |
| JayF | What? The real world operating around baremetal is messy? Never! /s :D | 17:54 |
| TheJulia | heh | 17:54 |
| JayF | thanks for entertaining that thought though, it's been nagging at me since I read that this morning, and I just nailed down why it did :D | 17:54 |
| TheJulia | Always happy to | 17:55 |
| TheJulia | I suspect if we added network switching we'd want to auto-retune the heartbeat interval to something really low so it just keeps heartbeating and also keep the window fairly tight. realistically as long as the network doesn't actually change, then it should work if explicitly enabled | 17:57 |
| * TheJulia wonders if we should do some sort of re-dhcp yourself now flag | 17:57 | |
| JayF | I'm not certain but I *think* the knobs for that might also exist for that, e.g. making heartbeats retry more rapidly if they fail for a specific reason | 17:57 |
| JayF | yep https://opendev.org/openstack/ironic-python-agent/src/branch/master/ironic_python_agent/agent.py#L82 | 18:01 |
| TheJulia | Yeah, we'd likely have to smooth that out or somethign | 18:04 |
| TheJulia | i dunno, I'm not sure the overlap is that huge for users | 18:05 |
| JayF | I mean, I don't have a use case for it, or care that it happens generally. Just noting it :D | 18:07 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Guard conductor from consuming all of the ram https://review.opendev.org/726483 | 18:18 |
| *** ociuhandu has joined #openstack-ironic | 18:34 | |
| *** ociuhandu has quit IRC | 18:39 | |
| TheJulia | wheeeee for making changes deep in validation code | 18:41 |
| openstackgerrit | Merged openstack/ironic-python-agent master: Increase the ESP partition size to 550 MiB when using software RAID https://review.opendev.org/739075 | 19:12 |
| *** sshnaidm|mtg is now known as sshnaidm|afk | 20:27 | |
| rpioso | TheJulia: When you have a moment, please take a look at https://review.opendev.org/#/c/737892/. I asked a couple of questions I expect you could readily answer. | 20:34 |
| patchbot | patch 737892 - ironic - Correct Redfish boot once fallback conditional - 1 patch set | 20:34 |
| *** ociuhandu has joined #openstack-ironic | 20:35 | |
| *** ociuhandu has quit IRC | 20:40 | |
| *** priteau has quit IRC | 20:44 | |
| *** ociuhandu has joined #openstack-ironic | 20:45 | |
| iurygregory | stbenjam, https://review.opendev.org/#/c/738874/ it's on gate now =) | 20:45 |
| patchbot | patch 738874 - sushy-tools - Support non-ide virtual media buses - 5 patch sets | 20:45 |
| TheJulia | <3 | 20:45 |
| TheJulia | rpioso: okay, will try to look once I wrap up the patch updates I'm working on | 20:46 |
| *** ociuhandu has quit IRC | 20:49 | |
| stbenjam | iurygregory: sweet thanks! I'm headed out for the weekend... will retry on Monday o/ | 20:49 |
| TheJulia | stbenjam: have a wonderful weekend | 20:51 |
| stbenjam | Metal3 reviewers actually want a pinned sha: https://github.com/metal3-io/metal3-dev-env/pull/362, I'll try to pop by and update the PR a bit later. How long do the gates take? | 20:51 |
| *** priteau has joined #openstack-ironic | 20:52 | |
| *** priteau has quit IRC | 20:52 | |
| TheJulia | stbenjam: depends on the job, give me a minute and I can give you an idea | 20:53 |
| * TheJulia loads le status page | 20:53 | |
| TheJulia | stbenjam: should be through the gate and merged in ~45 mintues | 20:53 |
| TheJulia | that is... as long as there are no failures. | 20:54 |
| *** Lucas_Gray has joined #openstack-ironic | 20:57 | |
| rpioso | TheJulia: Thank you! | 21:13 |
| *** ociuhandu has joined #openstack-ironic | 21:14 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Iso booting via redfish virtual media https://review.opendev.org/727480 | 21:18 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: iPXE ISO Ramdisk booting https://review.opendev.org/729464 | 21:18 |
| *** ociuhandu has quit IRC | 21:20 | |
| TheJulia | rpioso: adding a task is likely the best thing to do since it is a follow-up | 21:30 |
| TheJulia | of sorts | 21:30 |
| TheJulia | anyway, stepping away for a little while since I need CI to run tests | 21:30 |
| rpioso | TheJulia: That's what I was thinking, too :-) I'll create one, submit a new patch set, and remove the WIP marking. Thank you! Hope you have a great _Independence_ Day weekend :-) | 21:32 |
| *** mkarpiarz has quit IRC | 21:38 | |
| openstackgerrit | Merged openstack/ironic master: Prepare release notes and docs for release 15.1 https://review.opendev.org/738900 | 21:45 |
| openstackgerrit | Merged openstack/ironic master: Enforce autospec in test_manager https://review.opendev.org/738848 | 21:45 |
| openstackgerrit | Merged openstack/ironic master: Add a bug reporting and triaging guide https://review.opendev.org/737483 | 21:45 |
| openstackgerrit | Merged openstack/sushy-tools master: Support non-ide virtual media buses https://review.opendev.org/738874 | 22:00 |
| TheJulia | rpioso: awesome | 22:04 |
| TheJulia | stbenjam: merged | 22:04 |
| *** gyee has quit IRC | 22:21 | |
| *** rcernin has joined #openstack-ironic | 22:42 | |
| *** tkajinam has joined #openstack-ironic | 22:55 | |
| *** Wryhder has joined #openstack-ironic | 23:02 | |
| *** Lucas_Gray has quit IRC | 23:03 | |
| *** Wryhder is now known as Lucas_Gray | 23:03 | |
| *** diurnalist has quit IRC | 23:08 | |
| *** Goneri has quit IRC | 23:08 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: ISO ramdisk virtual media test enablement https://review.opendev.org/736383 | 23:19 |
| *** diurnalist has joined #openstack-ironic | 23:31 | |
| *** diurnalist has quit IRC | 23:48 | |
| *** rloo has quit IRC | 23:56 | |
| *** penick has joined #openstack-ironic | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!