| opendevreview | Steve Baker proposed openstack/ironic master: Set postgress password encryption for FIPS compliance https://review.opendev.org/c/openstack/ironic/+/803128 | 01:18 |
|---|---|---|
| opendevreview | Steve Baker proposed openstack/ironic master: DNM/WIP - Add FIPS jobs https://review.opendev.org/c/openstack/ironic/+/797739 | 01:18 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Implements node history: database https://review.opendev.org/c/openstack/ironic/+/768009 | 01:21 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP: Record Errors in history https://review.opendev.org/c/openstack/ironic/+/803292 | 01:21 |
| TheJulia | wheeeee no internet for a couple hours | 01:22 |
| TheJulia | dtantsur: NobodyCam: arne_wiebalck: I could use opinions on batch sizes for cleaning up node history records, wrt ^^^. The spec called for 300 entries by default and to batch a maximum of a thousand nodes per day. Operationally I guess that might be okay, but then again churn might be higher in some environments. I've not written tests around the db size of the work, but if they work it should keep things fairly clean. | 01:24 |
| opendevreview | ZhouHao proposed openstack/ironic master: update irmc document https://review.opendev.org/c/openstack/ironic/+/803412 | 02:06 |
| opendevreview | ZhouHao proposed openstack/ironic master: update irmc document https://review.opendev.org/c/openstack/ironic/+/803412 | 02:57 |
| opendevreview | ZhouHao proposed openstack/ironic master: update irmc document https://review.opendev.org/c/openstack/ironic/+/803412 | 03:30 |
| opendevreview | cenne proposed openstack/ironic master: Add api endpoints for changing boot_mode and secure_boot state https://review.opendev.org/c/openstack/ironic/+/800084 | 04:46 |
| *** pmannidi is now known as pmannidi|Lunch | 05:48 | |
| *** pmannidi|Lunch is now known as pmannidi | 06:46 | |
| iurygregory | good morning Ironic o/ | 07:04 |
| janders | good morning iurygregory o/ | 07:12 |
| iurygregory | hey janders o/ | 07:12 |
| *** rpittau|afk is now known as rpittau | 07:13 | |
| rpittau | good morning ironic! o/ | 07:14 |
| iurygregory | morning rpittau o/ | 07:14 |
| rpittau | hey iurygregory :) | 07:15 |
| janders | hey rpittau o/ | 07:25 |
| rpittau | hey janders :) | 07:26 |
| janders | do you know if Lenovo servers may need a separate vMedia license? | 07:37 |
| janders | hitting this: https://paste.opendev.org/show/807875/ | 07:37 |
| janders | I can't see vMedia specific URLs while poking around with curl (and Lenovo doco references these) | 07:37 |
| janders | will go poking around in the GUI now but if you have any pointers that would be awesome | 07:37 |
| rpittau | I would not exclude that or some kind of difference in the path, not sure though | 07:38 |
| iurygregory | janders, are you using the latest FW? | 07:38 |
| janders | iurygregory: likely not - but having said that I never flashed a Lenovo machine | 07:39 |
| iurygregory | or maybe they have the resource under a different name (I don't think this would be the case) | 07:39 |
| rpittau | would be worth giving a try updating the FW | 07:39 |
| janders | agreed | 07:39 |
| janders | I will see how easy that is | 07:39 |
| opendevreview | Aija Jauntēva proposed x/sushy-oem-idrac master: Add Flake8 W503 ignore https://review.opendev.org/c/x/sushy-oem-idrac/+/803425 | 07:39 |
| iurygregory | janders, yeah, the only person I know that tested vmedia with Lenovo was timeu =) | 07:40 |
| rpittau | on a different note, I disabled the ovn services but they get installed anyway \o/ | 07:41 |
| iurygregory | \o/ | 07:41 |
| iurygregory | yay! | 07:41 |
| rpittau | if anyone has a moment please check https://review.opendev.org/c/openstack/ironic/+/801350 thanks! | 07:46 |
| iurygregory | rpittau, looking now | 07:47 |
| opendevreview | Riccardo Pittau proposed openstack/networking-generic-switch master: Fix tempest based job https://review.opendev.org/c/openstack/networking-generic-switch/+/803320 | 07:53 |
| cenne | Good morning ironic | 08:40 |
| cenne | hey iurygregory, janders, rpittau | 08:41 |
| rpittau | hey cenne :) | 08:41 |
| opendevreview | Riccardo Pittau proposed x/sushy-oem-idrac master: Increase version of hacking and pycodestyle https://review.opendev.org/c/x/sushy-oem-idrac/+/803428 | 08:41 |
| iurygregory | hey cenne o/ | 08:42 |
| opendevreview | Merged openstack/ironic stable/victoria: Use shim-signed on Ubuntu, shim is empty now https://review.opendev.org/c/openstack/ironic/+/803338 | 09:04 |
| opendevreview | Merged openstack/ironic master: Add lower-constraints job to current development branch https://review.opendev.org/c/openstack/ironic/+/801350 | 09:12 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent master: Add lower-constraints job to current development branch https://review.opendev.org/c/openstack/ironic-python-agent/+/803453 | 09:37 |
| opendevreview | Riccardo Pittau proposed openstack/networking-generic-switch master: Fix tempest based job https://review.opendev.org/c/openstack/networking-generic-switch/+/803320 | 10:06 |
| opendevreview | Merged openstack/sushy stable/wallaby: Removing optional fields from insert_media payload https://review.opendev.org/c/openstack/sushy/+/803197 | 10:36 |
| janders | \o/ | 10:41 |
| janders | dtantsur https://review.opendev.org/c/openstack/ironic/+/803327 did fix things up for my patch indeed, thank you | 10:42 |
| janders | now - trying to test out WriteProtected on Lenovo, but I'm still yet to get vMedia working | 10:42 |
| janders | BIOS upgrades done (but didn't help) figuring out IMM upgrade (which is the one that should matter more I guess) | 10:42 |
| dtantsur | morning/afternoon ironic | 10:54 |
| dtantsur | TheJulia: I'd start with slow and conservative. 1000 nodes per day does surprise me: it seems too low | 10:56 |
| janders | hey dtantsur o/ | 10:56 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic bugfix/18.1: Use shim-signed on Ubuntu, shim is empty now https://review.opendev.org/c/openstack/ironic/+/803460 | 10:59 |
| janders | timeu in order to be able to configure vMedia via RedFish on Lenovo, did you need a separate license? | 11:24 |
| janders | I've got an X3550 and it refuses to cooperate vMedia-wise. Redfish paths that are supposed to be used for vMedia config are missing. I wonder if it may be a license problem... | 11:25 |
| janders | jungleboyj I heard you may also have some insights into ^ | 11:26 |
| janders | thanks in advance! | 11:26 |
| cenne | https://review.opendev.org/c/openstack/ironic/+/800084 | 11:35 |
| cenne | Hi ironic, if you are feeling like doing a review ^ please :) | 11:36 |
| janders | rpittau from the Lenovo repo it does seem there should be a VirtualMedia key under the manager: https://github.com/lenovo/python-redfish-lenovo/blob/master/examples/lenovo_mount_virtual_media.py#L100 | 11:37 |
| janders | and it's not there (which is what is breaking sushy) | 11:37 |
| janders | so I am more leaning towards the missing license theory | 11:38 |
| janders | thank you for pointing me to this repo, it's very useful | 11:38 |
| iurygregory | I'm starting to have the feeling we need a license :D | 11:39 |
| iurygregory | or maybe the model doesn't support vmedia at all? .-. | 11:40 |
| janders | funny thing is there is a GUI field for it | 11:40 |
| janders | heck, I will try to configure it :D | 11:40 |
| iurygregory | hehehehe | 11:40 |
| iurygregory | Hardware World <3 | 11:40 |
| janders | "it's called hardware, cause it's HARD..." | 11:41 |
| janders | and compulsory reference to https://xkcd.com/927/ | 11:41 |
| opendevreview | cenne proposed openstack/ironic master: Add api endpoints for changing boot_mode and secure_boot state https://review.opendev.org/c/openstack/ironic/+/800084 | 11:42 |
| iurygregory | after reading this I went back to the time I was in the university hearing computer science jokes :D | 11:42 |
| iurygregory | janders, not sure if heard this before, "How to know the problem my computer has?" "If you are yelling at your computer the problem is software, if you are hitting your computer the problem is in the hardware" | 11:46 |
| janders | LOL! | 11:46 |
| janders | and what if it's both? | 11:46 |
| janders | a hybrid one? | 11:46 |
| iurygregory | you are doing both at the same time LOL | 11:46 |
| janders | w/r/t Lenovo vMedia via GUI - it reports success | 11:47 |
| iurygregory | \o/ | 11:47 |
| janders | so my suspicion is it does require a license for vMedia config over RedFish | 11:47 |
| janders | TheJulia: I think I remember you talking about something like this - have you seen this before? (it's a Lenovo x3550 M5) | 11:48 |
| *** pmannidi is now known as pmannidi|Gone | 12:14 | |
| *** pmannidi|Gone is now known as pmannidi|AFK | 12:14 | |
| rpittau | janders: glad it was helpful, in some way :) | 12:17 |
| jungleboyj | janders: I don't think that the x3650s support Redfish. I ran into that problem recently in one of my labs. The older IMMs only support IPMI. You need to be on a newer system with xClarity Controller (XCC) for the BMC to get Redfish support. | 12:24 |
| * dtantsur suspects janders has already left for a day | 12:30 | |
| janders | dtantsur: I probably should have, but still hanging around :) | 12:41 |
| janders | thank you jungleboyj | 12:41 |
| dtantsur | :D | 12:41 |
| janders | I see *some* RF support, but the vMedia URLs are missing | 12:41 |
| janders | (e.g. I was only able to get system's serial number to get BIOS/IMM upgrades via RF queries :) ) | 12:42 |
| jungleboyj | janders: Interesting. | 13:13 |
| TheJulia | dtantsur: it would only be what it manages though, 1000 nodes that exceed 300 messages | 13:21 |
| TheJulia | also, good morning everyone | 13:22 |
| dtantsur | morning TheJulia | 13:22 |
| dtantsur | honestly, I don't have a lot of opinions on numbers | 13:22 |
| TheJulia | I just kind of feel like it might be too much | 13:22 |
| TheJulia | but a larger operator may have feelings on their churn | 13:22 |
| dtantsur | what's your biggest worry, locking the table? | 13:23 |
| TheJulia | yeah, locking writes out | 13:23 |
| jungleboyj | So, I talked to my guy who knows interacting with the BMCs well and he did confirm that RedFish on those systems was not the most fully functional thing. | 13:24 |
| TheJulia | but the writes will retry | 13:24 |
| TheJulia | the deletes won't based upon the code, which is fine, its about eventually clean up | 13:24 |
| jungleboyj | I can confirm it works well on our SRxxx line of systems. :-) | 13:24 |
| dtantsur | TheJulia: they you need small batches, but it matters less how many of them | 13:24 |
| dtantsur | s/they/then/ | 13:24 |
| TheJulia | yeah, it *should* be lots of small batches to begin with, it does it by node | 13:25 |
| * TheJulia might just be over thinking it | 13:25 | |
| janders | see you tomorrow Ironic o/ | 13:26 |
| dtantsur | see you janders | 13:26 |
| dtantsur | TheJulia: so, I'd drop the requirement on the nodes per day. Instead, I'd use batches of, say, 100 per run of a periodic task (so with a delay between each 100). | 13:26 |
| TheJulia | hmm yeah | 13:27 |
| TheJulia | the by node approach makes it super convenient because we can easily extract how many | 13:28 |
| TheJulia | we *could* run a query to build a list of all of the ids and just limit that at execution time | 13:29 |
| TheJulia | just requires an extra db query most likely | 13:29 |
| dtantsur | TheJulia: the delete question should return the number of rows affected | 13:29 |
| dtantsur | so in theory you can proceed to the next node if your 100 are not yet used up | 13:29 |
| TheJulia | yeah | 13:30 |
| dtantsur | which is nice if you have many nodes with not so many events for each | 13:31 |
| dtantsur | (maybe have a reasonable limit for nodes per iteration too) | 13:31 |
| TheJulia | that hmm, I'd have to think about how to make the delete queries to keep the number of records under the threshold properly if I'm not pre-determining how many rows to delete for each node based on it's overage | 13:32 |
| dtantsur | TheJulia: I think we have more or less this code for online migrations | 13:33 |
| TheJulia | that just uses a limit | 13:33 |
| dtantsur | are there any issues with a limit? | 13:33 |
| TheJulia | the online migrations also works through everything, which is a thing I think we need to avoid because this table could become huge in some of the operators. My thought here was more tactically identify and delete where applicable | 13:34 |
| dtantsur | it may be faster to do it on sql level though | 13:35 |
| TheJulia | which is what I did | 13:35 |
| TheJulia | just in two steps | 13:35 |
| dtantsur | I think we also do it in two steps for online migrations, I don't remember why | 13:35 |
| TheJulia | I'd need to go look at the migration code again and have coffee and not have neck pain | 13:36 |
| dtantsur | :( | 13:36 |
| dtantsur | at least fix the coffee | 13:36 |
| opendevreview | ZhouHao proposed openstack/ironic master: update irmc document https://review.opendev.org/c/openstack/ironic/+/803412 | 13:55 |
| TheJulia | And my router has died... Like sparks inside the case died | 14:15 |
| rpittau | oO | 14:16 |
| dtantsur | Oo | 14:23 |
| iurygregory | wow =O | 14:33 |
| JayF | At least you know it's bad when it puts up the magic smoke | 14:34 |
| JayF | as opposed to dropping every 64th packet or w/e | 14:35 |
| iurygregory | this one is a bit more hard to debug :D ^ | 14:35 |
| opendevreview | Merged openstack/ironic bugfix/18.1: Use shim-signed on Ubuntu, shim is empty now https://review.opendev.org/c/openstack/ironic/+/803460 | 14:48 |
| dtantsur | I'm pondering a mode of deployment where all database access goes through a conductor. Thoughts? | 14:49 |
| TheJulia | That would be a huge bottleneck | 14:51 |
| TheJulia | We, at a minimum, need to support read slave connections because many more conductors would be needed | 14:52 |
| TheJulia | We also don't have the methods RPC remotes, but I *think* it is just a decorator | 14:52 |
| dtantsur | I'm interested in standalone case mostly | 14:53 |
| dtantsur | so, I don't think it should be the default mode of operation | 14:53 |
| dtantsur | but it may be interesting in two cases (both coming from metal3 background - disclosure): | 14:53 |
| dtantsur | 1) Ephemeral database. We'd like to use sqlite, but accessing one database from two processes is a recipe for problems (someone is trying it in production as we speak - fingers crossed for them) | 14:54 |
| TheJulia | Okay, I do think it makes some sense as long as it is not overly complex to leverage. May just be a this or that logic thing that is needed | 14:54 |
| dtantsur | 2) Very distributed cluster. Essentially make database local to the conductor group, while the control plane is stateless (from the ironic pov) | 14:54 |
| TheJulia | Those both make lots of sense | 14:54 |
| TheJulia | I still think we need reader connections :). For history read queries that would be perfect. | 14:55 |
| dtantsur | could you elaborate on reader connections? | 14:55 |
| TheJulia | So oslo.db has wiring for a database replication slave target connection for reads but it is not exposed | 14:56 |
| dtantsur | this seems somewhat orthogonal to my proposal? | 14:56 |
| TheJulia | Yes it is orthogonal | 14:57 |
| dtantsur | ah, gotcha | 14:57 |
| TheJulia | I was thinking it because you'd need more consuctors | 14:57 |
| TheJulia | But in metal3 or small scale standalone it shouldn't be an issue | 14:57 |
| dtantsur | yeah, we need read connections, I agree. I'm not sure we even update anything from the API layer any more. | 14:57 |
| dtantsur | mmm, yeah, we patch chassis and probably others | 14:58 |
| dtantsur | and allocations | 14:59 |
| * dtantsur needs to stop planning future features and concentrate on the presentation he does tomorrow | 15:00 | |
| TheJulia | ++ | 15:02 |
| * TheJulia needs to check store inventory for routers | 15:03 | |
| dtantsur | I've just realized that my perfect plan doesn't account for hash rings... | 15:25 |
| NobodyCam | Good Morning Ironic folks.... Happy hump day | 15:44 |
| rpittau | well looks like I fixed at the 4th tentative https://review.opendev.org/c/openstack/networking-generic-switch/+/803320 | 16:00 |
| opendevreview | Riccardo Pittau proposed openstack/networking-generic-switch master: Increase version of hacking and pycodestyle https://review.opendev.org/c/openstack/networking-generic-switch/+/803213 | 16:00 |
| dtantsur | rpittau: nice job figuring it out! | 16:02 |
| rpittau | thanks :) | 16:02 |
| * dtantsur is on slide 23, and the end is not near | 16:02 | |
| rpittau | good night! o/ | 16:14 |
| *** rpittau is now known as rpittau|afk | 16:14 | |
| dtantsur | o/ | 17:18 |
| JayF | o/ | 17:22 |
| *** ricolin_ is now known as ricolin | 18:02 | |
| opendevreview | Eric Barrera proposed x/sushy-oem-idrac stable/wallaby: Enable coverage HTML output https://review.opendev.org/c/x/sushy-oem-idrac/+/803447 | 18:46 |
| opendevreview | Eric Barrera proposed x/sushy-oem-idrac stable/victoria: Enable coverage HTML output https://review.opendev.org/c/x/sushy-oem-idrac/+/803448 | 18:47 |
| opendevreview | Danni Shi proposed openstack/ironic-python-agent master: Add an attestation extension https://review.opendev.org/c/openstack/ironic-python-agent/+/803510 | 19:37 |
| NobodyCam | crazy question does anyone happen to have a check to ensure that a ironic node is able to be provisioned through nova.. | 19:37 |
| * TheJulia screams into the technology void | 19:37 | |
| NobodyCam | LOL | 19:37 |
| TheJulia | able to be provisioned? I mean, nova's classifer is is available and not in maintenance state | 19:37 |
| TheJulia | nova does call validate before hand, but that just validates the information supplied, it typically becomes a gigo problem at that point | 19:38 |
| TheJulia | unless there are other known things like if cleaning is disabled/bypassed and that always fails | 19:38 |
| NobodyCam | and placement record is really updated | 19:39 |
| NobodyCam | inventory has been recorded as available | 19:39 |
| opendevreview | Bob Fournier proposed openstack/python-ironicclient master: Include BIOS registry fields in bios setting list command https://review.opendev.org/c/openstack/python-ironicclient/+/803332 | 19:44 |
| TheJulia | I think your hitting upon a gap in the mechanics of nova and nova-compute as it relates to using baremetal | 20:15 |
| stevebaker | morning | 20:21 |
| JayF | NobodyCam: TheJulia: TBH, I see that a little like asking libvirt if it has VMs available to provision according to nova's criterion... we just don't have that data at the ironic layer | 20:23 |
| TheJulia | well, we have no insight into nova's inner processes | 20:24 |
| JayF | I mean, just by design we don't know what nova has configured, for instance for flavors | 20:24 |
| JayF | we just present resources and it's up to nova to categorize and filter them | 20:24 |
| JayF | I think it's a proper antifeature (e.g. a good separation to have) that you can't ask Ironic the question NobodyCam wants to ask it :| | 20:25 |
| NobodyCam | :) | 20:34 |
| TheJulia | Okay. New mesh base station ordered | 20:35 |
| TheJulia | router... seeming to be okay | 20:35 |
| TheJulia | new router, total garbage | 20:35 |
| TheJulia | Hows your day? | 20:35 |
| NobodyCam | placement-y heheheh | 20:35 |
| JayF | What brand did you get TheJulia? | 20:37 |
| TheJulia | for which part :) We keep a very complex network here :) | 20:38 |
| JayF | I've had orbi (like the 1st or 2nd gen one), and Eero is what I use now | 20:38 |
| JayF | for AP / mesh wifi | 20:38 |
| JayF | I run my own router but still use an off-the-shelf mesh wifi for my APs | 20:38 |
| TheJulia | We have the first generation orbi | 20:38 |
| TheJulia | Same | 20:38 |
| JayF | and you... like it? | 20:38 |
| JayF | maybe just as an AP it's less terrible | 20:38 |
| TheJulia | as just an AP, they rock | 20:38 |
| JayF | if I enabled v6 on that, when using it as router+ap, it just absolutely was terrible | 20:39 |
| TheJulia | yeah, found the same thing with the repalcement router, it couldn't grok v6 properly and wasn't discovering it on the ISP network | 20:39 |
| JayF | I have an embedded linux box as my router | 20:40 |
| TheJulia | so back to the Linksys ?EA8500? with openwrt router using its antennas which have poor coverage/performance on the far side of the house | 20:40 |
| JayF | running gentoo linux :) | 20:40 |
| JayF | it's fun to do a `ps aux` on your router and have it fit on one screen | 20:40 |
| TheJulia | ++ | 20:40 |
| JayF | it did take me hours and a ritual sacrifice to get DHCPv6 delegation working in systemd-network lol | 20:40 |
| TheJulia | lol | 20:41 |
| TheJulia | stevebaker: o/ | 20:41 |
| stevebaker | today I will learn about postgresql authentication | 20:43 |
| TheJulia | heh | 20:43 |
| TheJulia | enjoy? | 20:43 |
| stevebaker | it'll just be running bits of test-setup.sh manually and figuring out why it isn't working https://review.opendev.org/c/openstack/ironic/+/803128 | 20:50 |
| stevebaker | TheJulia: what do our jobs actually use postgres for? | 20:50 |
| TheJulia | stevebaker: unit tests and we have one integration test | 20:50 |
| TheJulia | we're one of the *few* projects taht run it afaik | 20:50 |
| TheJulia | so are the VMs even rebooting? | 20:50 |
| opendevreview | Steve Baker proposed openstack/ironic master: Set postgress password encryption for FIPS compliance https://review.opendev.org/c/openstack/ironic/+/803128 | 20:52 |
| opendevreview | Steve Baker proposed openstack/ironic master: DNM/WIP - Add FIPS jobs https://review.opendev.org/c/openstack/ironic/+/797739 | 20:52 |
| stevebaker | TheJulia: I haven't got to that yet, but I've touched base with ade in #tripleo. I see keystone have a working job which runs the enable-fips role, so it works for them | 20:53 |
| TheJulia | weird... | 20:57 |
| opendevreview | Leo McGann proposed openstack/ironic master: Keylime attestation interface https://review.opendev.org/c/openstack/ironic/+/803517 | 21:04 |
| opendevreview | Steve Baker proposed openstack/ironic master: Set postgress password encryption for FIPS compliance https://review.opendev.org/c/openstack/ironic/+/803128 | 21:28 |
| opendevreview | Steve Baker proposed openstack/ironic master: DNM/WIP - Add FIPS jobs https://review.opendev.org/c/openstack/ironic/+/797739 | 21:28 |
| opendevreview | Merged openstack/networking-generic-switch master: Fix tempest based job https://review.opendev.org/c/openstack/networking-generic-switch/+/803320 | 22:53 |
| stevebaker | TheJulia: the reboot is no longer causing an issue for the enable-fips job, its now failing in install_apache_uwsgi | 23:02 |
| TheJulia | freaky | 23:04 |
| *** pmannidi|AFK is now known as pmannidi | 23:25 | |
| stevebaker | I've done no commits to devstack for 6 years, and now 2. TWO | 23:46 |
| opendevreview | Steve Baker proposed openstack/ironic master: Set postgresql password encryption for FIPS compliance https://review.opendev.org/c/openstack/ironic/+/803128 | 23:47 |
| opendevreview | Steve Baker proposed openstack/ironic master: DNM/WIP - Add FIPS jobs https://review.opendev.org/c/openstack/ironic/+/797739 | 23:47 |
| stevebaker | TheJulia: the postgres change is happy now | 23:48 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!