| opendevreview | Steve Baker proposed openstack/bifrost master: Use safe shim binary paths on redhat family https://review.opendev.org/c/openstack/bifrost/+/806658 | 01:13 |
|---|---|---|
| *** pmannidi is now known as pmannidi|Lunch | 04:16 | |
| opendevreview | Jacob Anders proposed openstack/ironic master: Split node verification code out of manager.py https://review.opendev.org/c/openstack/ironic/+/806675 | 05:17 |
| iurygregory | good morning Ironic o/ | 05:18 |
| opendevreview | Jacob Anders proposed openstack/ironic master: Split node verification code out of manager.py https://review.opendev.org/c/openstack/ironic/+/806675 | 05:37 |
| janders_ | good morning iurygregory o/ | 05:48 |
| iurygregory | hey janders_ o/ | 05:48 |
| *** janders_ is now known as janders | 05:49 | |
| arne_wiebalck | Good morning, janders_ iurygregory and Ironic! | 05:49 |
| janders | hey arne_wiebalck o/ | 05:49 |
| iurygregory | arne_wiebalck, o/ | 05:50 |
| janders | question: I split https://review.opendev.org/c/openstack/ironic/+/806675 out of https://review.opendev.org/c/openstack/ironic/+/800001 to make it more of an "atomic" change. Do I need 806675 to merge prior to rebasing 800001, or is there a more clever way to do this? | 05:51 |
| janders | (gerrit GUI rebase won't work, as there will be some manual merging required) | 05:51 |
| iurygregory | janders, so https://review.opendev.org/c/openstack/ironic/+/806675 should be your first patch in chain | 05:52 |
| iurygregory | locally when you move to the branch that has https://review.opendev.org/c/openstack/ironic/+/800001 | 05:52 |
| iurygregory | you need to do git rebase -i <branch that contains 806675> | 05:53 |
| iurygregory | to use 806675 as head for 800001 | 05:53 |
| iurygregory | and push 800001 again to gerrit | 05:53 |
| iurygregory | ofc you need to solve merge conflicts etc | 05:53 |
| janders | makes sense! thank you iurygregory! :) | 05:54 |
| janders | it's a funny one this one - it's the second time I need to split something off this patch | 05:54 |
| janders | but last time it was quite a simple one so gerrit could do that easily | 05:54 |
| opendevreview | Nisha Agarwal proposed openstack/ironic master: Clean step to remove CA certificates from iLO https://review.opendev.org/c/openstack/ironic/+/799562 | 05:55 |
| iurygregory | just wondering if it would be really required to split | 05:55 |
| iurygregory | to me 800001 contains all required changes we need to start having the basic support for verify steps | 05:55 |
| janders | iurygregory: there is a stage where I split off existing code from manager.py to verify.py and the same for tests | 05:57 |
| opendevreview | Nisha Agarwal proposed openstack/ironic master: Clean step to remove CA certificates from iLO https://review.opendev.org/c/openstack/ironic/+/799562 | 05:57 |
| janders | dtantsur hinted it may be worth splitting this off if it's not too hard - but maybe let's confirm if I got that right before going further :) | 05:58 |
| janders | it was a little more fiddly thank I thought, on the first pass I missed a couple things and things started breaking | 05:58 |
| janders | now it's passing local tests, let's see what CI thinks but if you think I may be overcomplicating things maybe let's get more eyes on this | 05:59 |
| janders | thanks for looking into this iurygregory | 05:59 |
| janders | heading off for a quick bike ride, back in an hour and a bit | 06:00 |
| iurygregory | oh ok now I see there is still some difference between the verify in 806675 | 06:00 |
| iurygregory | 806675 is just moving code arround before adding the real support | 06:00 |
| iurygregory | yeah I think it makes sense, when I started looking for some reason I thought there was some missing bits in 806675 for verify steps hehe | 06:02 |
| * arne_wiebalck has sent the SIG mail and tweet | 06:11 | |
| iurygregory | tks arne_wiebalck ! | 06:12 |
| dtantsur | morning ironic | 06:21 |
| iurygregory | good morning dtantsur | 06:22 |
| arne_wiebalck | Hey dtantsur, good morning! Feeling better today? | 06:26 |
| dtantsur | still waking up, but seems better so far. thank you! | 06:27 |
| janders | that's great dtantsur! good morning! | 07:13 |
| janders | looks like the post-vaccine slowness let me go, back to normal speeds on the bike | 07:13 |
| dtantsur | \o/ | 07:13 |
| *** pmannidi|Lunch is now known as pmannidi|AFK | 07:16 | |
| janders | dtantsur I had a go at splitting-up LC/iDRAC reset patch further. Here is just the part splitting out the node verification from manager.py. When you have time let me know if I got this right and I will re-base the rest. Thank you! :) | 07:22 |
| janders | https://review.opendev.org/c/openstack/ironic/+/806675 | 07:22 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove manager param for iDRAC OEM calls https://review.opendev.org/c/openstack/ironic/+/792200 | 07:55 |
| ajya | any idea why this is failing tox-py36 https://review.opendev.org/c/openstack/ironic/+/792200? Smth to do with oslo.db? | 08:23 |
| dtantsur | ajya: a transient connection issue | 08:24 |
| dtantsur | these problems have been haunting us for years, probably since the very beginning | 08:24 |
| ajya | it failed twice | 08:24 |
| ajya | but I can give another try | 08:24 |
| ajya | or I mixed up smth by seeing that message in IRC half hour ago, that's another bug? | 08:27 |
| ajya | it seems 2nd try is still running | 08:27 |
| iurygregory | yeah, the failure rate is not that bad based on https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-py36&project=openstack%2Fironic&branch=master | 08:27 |
| iurygregory | dtantsur, quick question, I'm adding a check to create_subscription to see if there are headers in the args https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/redfish/vendor.py#L165 do we consider this as a feature or can be considered a fix ? | 08:33 |
| dtantsur | iurygregory: hard to tell without more context, but nothing you've said sounds immediately inappropriate | 08:34 |
| dtantsur | I guess depends on why and how you check it | 08:35 |
| iurygregory | yeah, basically subscriptions can accept HttpHeaders (after I added the simple support without headers I discovered that downstream people will need) | 08:37 |
| iurygregory | so it would be just an if to check if the user is sending HttpHeaders in the kwargs, if yes we send in the payload to sushy | 08:37 |
| iurygregory | to me it sounds like a fix, since we are not very strict with what people will send in the vendor passthru | 08:38 |
| dtantsur | It does sound like a feature, to be honest | 08:38 |
| iurygregory | ack =) | 08:38 |
| dtantsur | if it's a completely new behavior that did not exist and was not expected to exist before | 08:38 |
| dtantsur | (I've also heard that headers don't work reliably) | 08:39 |
| iurygregory | well, if people try to send HttpHeaders they will be just ignored and sent in the payload to sushy | 08:39 |
| iurygregory | I will push the patch considering feature (since we don't need this in the current downstream release) | 08:40 |
| iurygregory | not going to hurt =) | 08:40 |
| dtantsur | yep | 08:40 |
| iurygregory | tks! | 08:41 |
| * dtantsur -> jogging | 08:46 | |
| opendevreview | Merged openstack/bifrost master: os_ironic_node_info: fix TLS and potentially other issues https://review.opendev.org/c/openstack/bifrost/+/806001 | 09:57 |
| opendevreview | Jacob Anders proposed openstack/ironic master: Add support for verify steps https://review.opendev.org/c/openstack/ironic/+/800001 | 10:03 |
| opendevreview | Jacob Anders proposed openstack/ironic master: Add support for verify steps https://review.opendev.org/c/openstack/ironic/+/800001 | 10:51 |
| opendevreview | Jacob Anders proposed openstack/ironic master: Make iDRAC management steps verify steps https://review.opendev.org/c/openstack/ironic/+/804032 | 10:55 |
| opendevreview | Merged openstack/ironic master: Split node verification code out of manager.py https://review.opendev.org/c/openstack/ironic/+/806675 | 11:02 |
| janders | ^ allright, now back to adding more tests :) | 11:05 |
| janders | thanks for reviews and assistance dtantsur iurygregory stendulker | 11:06 |
| iurygregory | janders, yw =) | 11:06 |
| opendevreview | Merged openstack/ironic master: Remove manager param for iDRAC OEM calls https://review.opendev.org/c/openstack/ironic/+/792200 | 11:08 |
| opendevreview | Aija Jauntēva proposed openstack/ironic master: Add RAID mode conversion for idrac-redfish https://review.opendev.org/c/openstack/ironic/+/804112 | 11:13 |
| opendevreview | Aija Jauntēva proposed openstack/ironic master: Add clear foreign config to idrac-redfish RAID https://review.opendev.org/c/openstack/ironic/+/804848 | 11:14 |
| janders | see you tomorrow Ironic o/ | 12:02 |
| iurygregory | bye janders o/ | 12:07 |
| opendevreview | Merged openstack/ironic master: Add iDRAC configuration mold docs https://review.opendev.org/c/openstack/ironic/+/790997 | 12:26 |
| TheJulia | good morning | 13:12 |
| dtantsur | morning TheJulia | 13:15 |
| iurygregory | good morning TheJulia =) | 13:16 |
| opendevreview | Aija Jauntēva proposed openstack/ironic master: Redfish RAID: Use identity instead of durable_name https://review.opendev.org/c/openstack/ironic/+/805866 | 13:35 |
| TheJulia | is lower constraints failing on ironic on all jobs now? | 13:44 |
| TheJulia | hmm, apparently not | 13:45 |
| opendevreview | Leo McGann proposed openstack/ironic master: Keylime attestation interface https://review.opendev.org/c/openstack/ironic/+/806751 | 13:47 |
| iurygregory | the failure rate in ironic doesn't look bad https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-lower-constraints&project=openstack%2Fironic&branch=master | 13:48 |
| TheJulia | hmm, interesting/odd failures on one of the integration jobs :\ | 13:50 |
| TheJulia | are we review jamming? | 14:00 |
| TheJulia | Insert all the review jamming dance music into code reivew | 14:01 |
| TheJulia | lmcgann: iurygregory: dtantsur ^ | 14:01 |
| iurygregory | I can join | 14:01 |
| TheJulia | iurygregory: it would be good | 14:04 |
| opendevreview | Aija Jauntēva proposed openstack/ironic master: Add RAID mode conversion for idrac-redfish https://review.opendev.org/c/openstack/ironic/+/804112 | 14:16 |
| tosky | TheJulia: hi, can you please vote to definitely put ocata to sleep? (I think it needs the current PTL): https://review.opendev.org/c/openstack/releases/+/804453 | 14:35 |
| TheJulia | tosky: done | 14:56 |
| TheJulia | tosky: anything else ? :) | 14:56 |
| * arne_wiebalck has sent out mails with invitations for next week's call to several Ironic users/operators and sites considering Ironic | 15:00 | |
| TheJulia | arne_wiebalck: thanks | 15:00 |
| TheJulia | I'll tweet the archive link out shortly | 15:00 |
| iurygregory | I've helped with the RT on twitter \o/ | 15:00 |
| * arne_wiebalck counts on others to go through their address books :) | 15:01 | |
| iurygregory | does anyone know the the ops meetup has a ML? maybe we can send an email? | 15:01 |
| arne_wiebalck | TheJulia: iurygregory; thanks! | 15:01 |
| TheJulia | err, I wonder if neutron is going to break us http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024568.html | 15:01 |
| iurygregory | oh WOW | 15:02 |
| tosky | TheJulia: thanks, no - unless you want to EOL any other version :D | 15:03 |
| TheJulia | okay, looks like we might be okay as long as we don't ship newlines in any of the extra options | 15:03 |
| TheJulia | yeah, we ship a list | 15:04 |
| TheJulia | so it should be okay | 15:04 |
| iurygregory | \o/ | 15:06 |
| TheJulia | NobodyCam: http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024554.html | 15:39 |
| NobodyCam | oh | 15:40 |
| iurygregory | guilhermesp, ^ | 15:40 |
| iurygregory | :D | 15:40 |
| TheJulia | timeu: http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024554.html :) | 15:40 |
| * dtantsur sent to the Russian ops chat | 15:42 | |
| TheJulia | I sent the same to some of our internal folks including solution engineering :) | 15:42 |
| TheJulia | And also posted to the global openstack facebook group | 16:09 |
| dtantsur | someone in a chat pointed me at https://github.com/dchirikov/luna and their impressive provisioning times | 16:14 |
| dtantsur | I'm trying to find someone to work on the torrent spec, but with very moderate success so far | 16:14 |
| TheJulia | I guess the translation down to the block layer makes it a little... difficult | 16:18 |
| dtantsur | you mean, we'd have to cache the image in RAM? | 16:18 |
| TheJulia | I don't remember the original bittorrent code/mechanism well enough | 16:19 |
| TheJulia | that was like... 2002 and I was working 3rd shift | 16:19 |
| TheJulia | and bram stopped joining IRC around then too | 16:19 |
| TheJulia | at least, the IRC channel I was in | 16:19 |
| dtantsur | what's the conundrum, whether we'll be able to stream to disk? | 16:19 |
| dtantsur | we already don't do it for qcow2 images fwiw | 16:20 |
| TheJulia | Well, you don't get everything in order | 16:20 |
| TheJulia | you could be missing the first 64kb until the very end | 16:20 |
| TheJulia | so it would have to be cached in ram, but if we could directly use the block device..... | 16:21 |
| TheJulia | that would be so much better with buffer cached IO operations | 16:21 |
| dtantsur | for raw images - yes, it's interesting | 16:21 |
| dtantsur | in theory, block devices should support seeking | 16:21 |
| TheJulia | oh, they do | 16:21 |
| dtantsur | so, again in theory, I don't see what an image cannot be streamed on a device, even if the last 64kb comes in the end | 16:21 |
| TheJulia | Maybe some funky code on file size handling from a starting image to a write out on a deice, but yeah | 16:21 |
| dtantsur | my client had an option whether to allocate the file in advance or not | 16:22 |
| TheJulia | dtantsur: when your saying streamed, I'm thinking streamed sequentially | 16:22 |
| dtantsur | ah, no, sorry | 16:22 |
| TheJulia | yeah, I just realized that | 16:22 |
| dtantsur | I'm reusing the same term we already have, but it's really about a sequential streaming there | 16:22 |
| TheJulia | So yeah, we could to disk | 16:22 |
| dtantsur | meh, any google search with torrent is complete garbage | 16:23 |
| TheJulia | yeah | 16:23 |
| TheJulia | if memory serves original bittorrent was only kike 3k lines of python | 16:23 |
| dtantsur | it's probably easier for someone to try than to find the answer | 16:24 |
| TheJulia | https://www.bittorrent.org/beps/bep_0000.html | 16:25 |
| dtantsur | YAY https://blog.project-insanity.org/2018/01/08/download-an-installation-medium-directly-to-your-flash-drive/ | 16:29 |
| dtantsur | when googling about torrent, avoid the word "stream" and use quotes around "block device" | 16:29 |
| TheJulia | nice | 16:30 |
| * TheJulia senses dmitry will be proposing a new torrent interface when he collects his next batch of coding spoons | 16:30 | |
| dtantsur | when I collect some free time :) | 16:32 |
| dtantsur | iurygregory: can that be a part of our intern's work? | 16:33 |
| dtantsur | probably a bit too much for the next outreachy project :) | 16:33 |
| * iurygregory looks | 16:34 | |
| TheJulia | That would be really awesome | 16:36 |
| iurygregory | if I recall our goal with the intern was https://storyboard.openstack.org/#!/story/2006910 | 16:37 |
| TheJulia | "yo, I heard you wanted to deploy 10,000 machines, but not using ramdisks" | 16:37 |
| iurygregory | but we can probably chat in our downstream meeting tomorrow about this dtantsur =) | 16:38 |
| dtantsur | I'm conflicted | 16:39 |
| dtantsur | On one hand, deployment API is needed for consistency and ease of use. | 16:39 |
| dtantsur | On the other - torrent provisioning may be a real improvement for deployments that happen in spikes. | 16:39 |
| dtantsur | "So many snacks, so little time" (c) | 16:39 |
| iurygregory | if we have a spec/rfe for the torrent we can let him choose I would say | 16:40 |
| TheJulia | I'm kind of conflicted in that consistency is also something that is sometimes a great goal but not always possible when differences *are* required. Also I'd kind of prefer people with deep ironic experience work through something that intertwined and possibly performance impacting in the code, where as torrent stuff seems fairly isolated depending on how far (or not) we take it | 16:41 |
| TheJulia | since torrent could just be "your image source is a torrent url" | 16:41 |
| dtantsur | yeah, maybe the deployment API will wait for me.. | 16:42 |
| iurygregory | got it, that makes sense to me | 16:42 |
| TheJulia | dtantsur: the last time I glanced at that, it felt like we were going to need to do something with the db and joins too, which makes me nervous, but maybe I shouldn't be | 16:43 |
| dtantsur | re deployment API? there were no such plans at that point. maybe there should be. will see. | 16:43 |
| dtantsur | iurygregory: https://review.opendev.org/c/openstack/ironic-specs/+/311091 is the abandoned spec | 16:43 |
| iurygregory | dtantsur, ack | 16:44 |
| TheJulia | I know with allocation we need to join-ize the view... at some point | 16:45 |
| TheJulia | by view, I mean web api, not db view | 16:49 |
| dtantsur | dinner time, see you tomorrow folks | 16:59 |
| opendevreview | Leo McGann proposed openstack/ironic master: Adds Attestation Interface to Node https://review.opendev.org/c/openstack/ironic/+/755836 | 17:10 |
| janders | good morning Ironic o/ | 20:04 |
| iurygregory | morning janders o/ | 20:23 |
| janders | hey iurygregory o/ | 20:23 |
| janders | just read about torrent-driven image transfer, sounds like a really cool idea dtantsur TheJulia iurygregory | 20:24 |
| TheJulia | It makes a lot of sense for the super huge deployments needing to deploy huge clusters | 20:25 |
| janders | 100% | 20:25 |
| TheJulia | some of the scientific folks I've talked to have mentioned using bittorrent or similar things to copy the base data or tooling into a ramdisk to run a thing | 20:25 |
| TheJulia | like.. imagine having a container that runs your simulation, but you download the container via bittorrent in your cluster environment on start-up and it all runs in a ramdisk shipping updates periodically to a central service | 20:26 |
| janders | back in the day people used to either run a provisioning server per rack - or temporarily repurpose some computes into provisioning servers to improve startup speeds for huge clusters | 20:26 |
| janders | using something like bittorrent between "ordinary" nodes is much slicker | 20:27 |
| TheJulia | i know some folks who still do it | 20:27 |
| TheJulia | deploy like the first node and then daisy chain the rest of the rack off of it | 20:27 |
| TheJulia | stevebaker: good question w/r/t tripleo and capabilities. Looks like the capabilities works | 20:31 |
| TheJulia | I only had to go look it up and the underlying behavior of setcap | 20:31 |
| stevebaker | TheJulia: yeah going full privileged needs a good reason | 20:58 |
| TheJulia | yeah, dtantsur ^^ fyi you can just launch dnsmasq with just three capabilities instead of privileged in metal3 | 20:59 |
| opendevreview | Steve Baker proposed openstack/ironic master: WIP use packaged grub efi for network boot https://review.opendev.org/c/openstack/ironic/+/806808 | 21:16 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/ironic-python-agent-builder master: Add configs for CentOS 9 https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/806809 | 21:25 |
| sshnaidm | stevebaker, ^^ | 21:25 |
| stevebaker | sshnaidm: cool, thanks | 21:25 |
| iurygregory | CentOS9? WOOT?! | 21:26 |
| * iurygregory wondering if is C9S or C9... | 21:26 | |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/ironic-python-agent-builder master: Add centos9 job https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/806812 | 21:32 |
| stevebaker | oh look, 1 day old 9-stream images http://cloud.centos.org/centos/9-stream/x86_64/images/ | 21:34 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/ironic-python-agent-builder master: Add configs for CentOS 9 https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/806809 | 21:38 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/ironic-python-agent-builder master: Add centos9 job https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/806812 | 21:39 |
| TheJulia | stevebaker: yup :( I noticed an rpm package search was listing a 9 version with brand new grubs | 22:04 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP: Facilitate asset copy for bootloader ops https://review.opendev.org/c/openstack/ironic/+/806815 | 22:19 |
| TheJulia | stevebaker: ^^ maybe that might be ideal....so distro specific knowledge doesn't need to be really maintained and it becomes... optional! | 22:20 |
| stevebaker | TheJulia: what is the distro specific part there? the last part of loader_file_paths would be an absolute file path? | 22:30 |
| TheJulia | stevebaker: yeah, the last, since that is also what we've seen evolve on distros over time | 22:34 |
| stevebaker | TheJulia: yeah that could be useful. But sometimes the destination filename will need to be something specific because the shim is loading its hard-coded paths? | 22:40 |
| TheJulia | hmm | 22:41 |
| TheJulia | good point | 22:41 |
| TheJulia | what if the file was a key too and the destination filename was a value? | 22:42 |
| stevebaker | TheJulia: yes and if there is no value, use the source filename | 22:46 |
| TheJulia | distros could ship additional config that matches the paths out pretty easy | 22:47 |
| stevebaker | TheJulia: oh and what if the default config value *did* have all distro specific knowledge, and the copy is skipped if the source file is missing. That could be too much | 22:48 |
| TheJulia | maybe | 22:50 |
| TheJulia | maybe not | 22:50 |
| opendevreview | Shnaidman Sagi (Sergey) proposed openstack/ironic-python-agent-builder master: Add centos9 job https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/806812 | 23:26 |
| stevebaker | TheJulia: progress report, the node fetched bootx64.efi, then fetched grubx64.efi, and now the console is stuck on "Fetching Netboot Image" | 23:34 |
| *** sshnaidm is now known as sshnaidm|afk | 23:34 | |
| *** pmannidi|AFK is now known as pmannidi | 23:41 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!