| vanou | good morning Ironic | 00:34 |
|---|---|---|
| opendevreview | Merged openstack/ironic bugfix/18.1: Do not reboot into nowhere after BIOS settings with fast-track https://review.opendev.org/c/openstack/ironic/+/858468 | 00:42 |
| *** dmellado_ is now known as dmellado | 01:39 | |
| *** dking is now known as Guest1515 | 02:15 | |
| opendevreview | OpenStack Proposal Bot proposed openstack/ironic master: Imported Translations from Zanata https://review.opendev.org/c/openstack/ironic/+/859182 | 03:15 |
| shukun | JayF: After read your comment about backporting to bugfix branch, I have create a patch for bugfix/20.2 yesterday, but it seems that this should be done by the community release team (did not know that), so should I abandon that change? | 04:54 |
| TheJulia | shukun: anyone can create a backport to a branch. I doubt you would want to abandon a change. | 05:00 |
| * TheJulia goes to sleep | 05:00 | |
| shukun | TheJulia: thanks for the answer, I will keep that change open. | 05:02 |
| arne_wiebalck | Good morning, Ironic! | 06:30 |
| rpittau | good morning ironic! o/ | 07:34 |
| stevebaker[m] | good evening | 07:34 |
| stevebaker[m] | arne_wiebalck: SIG video is live :) https://www.youtube.com/watch?v=mHrj-ae9qwI | 07:35 |
| arne_wiebalck | awesome, thanks stevebaker[m] ! | 07:51 |
| opendevreview | Merged openstack/ironic-ui master: Imported Translations from Zanata https://review.opendev.org/c/openstack/ironic-ui/+/859184 | 08:30 |
| opendevreview | Nisha Agarwal proposed openstack/ironic master: Fix the anaconda deploy for the ISO mounted https://review.opendev.org/c/openstack/ironic/+/859397 | 09:38 |
| opendevreview | Merged openstack/ironic stable/wallaby: Do not reboot into nowhere after BIOS settings with fast-track https://review.opendev.org/c/openstack/ironic/+/859234 | 09:47 |
| opendevreview | Merged openstack/ironic master: Imported Translations from Zanata https://review.opendev.org/c/openstack/ironic/+/859182 | 09:47 |
| Nisha_ | iurygregory, ping | 10:01 |
| Nisha_ | TheJulia, i raised the patch for fixing anaconda deploy when ISO is mounted manually | 10:03 |
| TheJulia | Nisha_: I’ll look when I am in later today | 10:09 |
| Nisha_ | TheJulia, sure | 10:09 |
| rpittau | Nisha_: probably worth a release note | 10:11 |
| TheJulia | Nisha_: well, that is a problem because that breaks the other use mode detection for paths… so why does your mounted iso match that for a file? | 10:11 |
| TheJulia | If your image source is a file, but extension ends with gz… why is content-type text in your case | 10:12 |
| TheJulia | I think we need more info | 10:12 |
| Nisha_ | No its just the ISO mounted there | 10:12 |
| TheJulia | What was the instance_info you were using there? | 10:13 |
| Nisha_ | say its the CENTOS9 STREAM DVD mounted at the location http://172.17.1.29:8080/centos9mnt/ | 10:13 |
| Nisha_ | same what i have given in the story details | 10:13 |
| TheJulia | Ahhhhhhh | 10:13 |
| Nisha_ | i tried giving stage2 url as squashfs.img, but that doesnt work | 10:14 |
| TheJulia | So you are giving it a path | 10:14 |
| Nisha_ | Yes | 10:14 |
| Nisha_ | i am giving webserver url which contains the path | 10:14 |
| TheJulia | So… stilll why. Is it not working for you. | 10:15 |
| Nisha_ | actually the deploy works if the kickstart has url --url http://172.17.1.29:8080/centos9mnt/ directive | 10:15 |
| TheJulia | Yeah, it finds the assets what way. It needs to return True | 10:15 |
| TheJulia | Yet… that check should return it. I think we still need more info because I’m struggling to understand | 10:16 |
| Nisha_ | if the kickstart has liveimg --url <url> it doesnt work | 10:16 |
| TheJulia | Eh, that might just be the wrong file in instance_info | 10:17 |
| kubajj | TheJulia: when you mentioned that I should use SQLalchemy 2 instead of what's there for the inspector inventory API, was this for the inspector or the ironic side? | 10:17 |
| TheJulia | Because stage2 != image_source | 10:17 |
| Nisha_ | In this current case the info has both Content-Length and Content-Type and hence the current code condition always return false | 10:18 |
| TheJulia | kubajj: ironic side, use an db api model. I have a patch I’m revising on ironic-inspector to change the model to be like ironic before I clean up the calls to be sqlalachemy 2 friendly | 10:18 |
| kubajj | TheJulia: and where does ironic write to the db? | 10:19 |
| TheJulia | Db/salalchemy/api.oh | 10:19 |
| kubajj | TheJulia: ok, thanks | 10:20 |
| TheJulia | Nisha_: unit test please, because it should fall through to the next I think | 10:20 |
| TheJulia | Anyway, to be airpor! | 10:20 |
| kubajj | I think it will take me a little bit more time to get familiar with ironic. IPA was much more straightforward, ngl | 10:21 |
| TheJulia | kubajj: we can do a walkthrough at some point soon to help | 10:21 |
| kubajj | TheJulia: that would be lovely | 10:22 |
| Nisha_ | TheJulia, It doesnt falls to next condition because it satisfies the first condition as well i.e. it has "Content-Length" also | 10:23 |
| TheJulia | Nisha_: hmm, interesting | 10:25 |
| TheJulia | Not what I would have expected. A unit test would be good anyhow since I got some push back on just using the path and looking for a trailing / | 10:26 |
| Nisha_ | ok | 10:26 |
| Nisha_ | let me add that | 10:27 |
| Nisha_ | I could deploy Rhel 8.5 and Centos 9Stream this way within 10 min | 10:27 |
| Nisha_ | TheJulia, could u tell me if i give stage2 url as squashfs.img, then what shud be there in image_source | 10:30 |
| iurygregory | good morning Ironic | 11:37 |
| rpittau | ehmmm why the ci jobs didn't run on https://review.opendev.org/c/openstack/ironic-python-agent/+/859255 ? or well except the tox ones.... | 12:20 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent bugfix/8.6: Pin bugfix 8.6 ci jobs to zed https://review.opendev.org/c/openstack/ironic-python-agent/+/859253 | 12:30 |
| opendevreview | Jakob Meng proposed openstack/bifrost master: [DNM] Debugging calls to openstack.cloud modules https://review.opendev.org/c/openstack/bifrost/+/859430 | 12:33 |
| opendevreview | Jacob Anders proposed openstack/sushy master: [WIP] Retry BootSourceOverride request when SettingsURI is read-only https://review.opendev.org/c/openstack/sushy/+/856597 | 12:40 |
| opendevreview | Jacob Anders proposed openstack/sushy master: Retry BootSourceOverride request when SettingsURI is read-only https://review.opendev.org/c/openstack/sushy/+/856597 | 12:52 |
| TheJulia | Nisha_: a filesystem image | 14:02 |
| TheJulia | Okay, good morning everyone | 14:02 |
| opendevreview | Julia Kreger proposed openstack/ironic-inspector master: WIP: Use declarative reader/writer separation https://review.opendev.org/c/openstack/ironic-inspector/+/463768 | 14:08 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/victoria: Do not reboot into nowhere after BIOS settings with fast-track https://review.opendev.org/c/openstack/ironic/+/859411 | 14:23 |
| Nisha_Agarwal | TheJulia, I just now saw the msg | 14:24 |
| Nisha_Agarwal | TheJulia, by ur statement does it mean that a filesystem image part is broken with proposed changes | 14:24 |
| Nisha_Agarwal | ? | 14:24 |
| TheJulia | so, I think the issue is the filesystem image has to be created if your going to use a base liveimg on the template, as opposed to a url stock kickstart install | 14:26 |
| JayF | TheJulia: you're talking about kickstart, not ironic limitiations, right? | 14:27 |
| TheJulia | I am | 14:27 |
| JayF | Yeah, that's what I thought :) | 14:28 |
| * TheJulia has yet to reproduce that race condition locally, going to try a few more times before digging blindlly | 14:28 | |
| JayF | I bet it only shows on slow test runs or something | 14:29 |
| TheJulia | In other words: have you tried streaming some HD video *and* running unit tests | 14:29 |
| TheJulia | no, not yet :) | 14:29 |
| JayF | TheJulia: more like "good luck finding a machine as slow as the bottom .1% of the machines that run our CI" lol | 14:30 |
| JayF | step 1: get a bunch of noisy neighbors | 14:30 |
| TheJulia | noisy neighbors as in web browser processes with HD video... | 14:43 |
| dtantsur | rpittau: CI jobs are skipped when some required projects cannot be resolved. | 14:46 |
| rpittau | dtantsur: thanks | 14:46 |
| dtantsur | you probably may need another override for $something, but $something is hard to figure out | 14:46 |
| dtantsur | rpittau: here goes a guess. requirements doesn't seem to have stable/zed yet | 14:47 |
| rpittau | thanks, I was looking at that | 14:47 |
| rpittau | although it seems working in https://review.opendev.org/c/openstack/ironic-python-agent/+/859253 | 14:47 |
| dtantsur | rpittau: you don't override requirements there? | 14:48 |
| Nisha_Agarwal | TheJulia, liveimg directive in the kickstart requires the image to be mountable by the anaconda installer which the mount point url is not. | 14:48 |
| rpittau | I do | 14:48 |
| dtantsur | ah, in some jobs | 14:48 |
| TheJulia | JayF: well, the temp in the room is starting to rise... | 14:49 |
| dtantsur | dunno. this is not my area of expertise | 14:49 |
| TheJulia | Nisha_Agarwal: indeed | 14:49 |
| rpittau | the 2 patches are basically identical since both 8.6 and 9.0 bugfix branches come from zed | 14:49 |
| dtantsur | rpittau: then it could be something else. there is a way to enable debug output in zuul, but I never remember it. | 14:49 |
| TheJulia | JayF: I'm semi-wondering if this is going to be totally random given we don't do synchronous locking on the db since we want to keep it lightweight. | 14:50 |
| JayF | TheJulia: so like, first question would be -- what do we model this problem as? A missing lock somewhere? A possible deadlock? | 14:50 |
| JayF | TheJulia: like, do we have a theory about what could be causing it | 14:50 |
| Nisha_Agarwal | TheJulia, even the mirror url doesnt work for us as our test systems are behind the proxy(no direct access to internet) and the kickstart "url" directive is not appended with --proxy parameter. Currently the generated kickstart has "url --url http://ip/..." while it shud even have --proxy in this command line | 14:51 |
| TheJulia | I need to look at the code again, hoping to establish a fail rate/frequency/odds of failure | 14:51 |
| JayF | Nisha_Agarwal: you can create your own kickstart template, based off the one shipped by Ironic, that includes a proxy directive. | 14:51 |
| TheJulia | I was about to suggest the same thing | 14:51 |
| opendevreview | Jakob Meng proposed openstack/bifrost master: [DNM] Debugging calls to openstack.cloud modules https://review.opendev.org/c/openstack/bifrost/+/859430 | 14:53 |
| TheJulia | Hmm, no dice on that unit test so far | 14:54 |
| JayF | I don't wanna say it because it seems like it'd be a curse but ... is it possible it's not a bug? Or it's such an unlikely random failure we're never going to repro reliably enough to fix it? | 14:57 |
| TheJulia | Oh, we've had these curses before | 14:57 |
| JayF | How did they end? | 14:58 |
| TheJulia | I'm giving it *one last* try now | 14:58 |
| JayF | ack | 14:58 |
| TheJulia | Typically us acknowledging the issue, either changing the test, or going "do we *really* need to test this that way" | 14:58 |
| TheJulia | hmmmm | 14:59 |
| TheJulia | okay, we can remove the test. It is counting the number of times sleep gets called, which is race condition filled insanity since that all gets rooted down in time | 15:03 |
| TheJulia | well, the check in the test | 15:04 |
| TheJulia | amongs the things we've seen before! | 15:04 |
| TheJulia | err, among | 15:04 |
| JayF | yeah, I was wondering about that when I saw what the test was doing | 15:04 |
| JayF | I am happy to reverse my -1 if we remove the test as awkward/not-testing-what-we-think-it-is | 15:05 |
| TheJulia | I'll rev it and make a note of it | 15:05 |
| knikolla[m] | JayF: TheJulia : This https://paste.opendev.org/show/bvCJ0DOLqdOflcw3CP63/ is the working test to https://review.opendev.org/c/openstack/keystoneauth/+/841169 | 15:07 |
| knikolla[m] | posting here since there's the keystone weekly meeting ongoing and din't want to cause noise. | 15:07 |
| JayF | knikolla[m]: you wanna solve our ironic unit test problem while you're here, too ;) S | 15:08 |
| JayF | * /s | 15:08 |
| JayF | thanks! I'll update that patch right now | 15:08 |
| JayF | knikolla[m]: actually, that is super nice that you linked that up here, I appreciate it o/ | 15:09 |
| * JayF running the local tests now | 15:09 | |
| JayF | TheJulia: knikolla[m]: https://review.opendev.org/c/openstack/keystoneauth/+/841169 pushed; passing tests locally \o/ | 15:13 |
| knikolla[m] | fixing the world, one unit test at a time | 15:14 |
| TheJulia | \o/ | 15:14 |
| JayF | TheJulia; others: what would you all think about bringing back the SPUC, except maybe inviting anyone openstack-y to join? | 15:17 |
| JayF | (or even just doing it with ironic folks again) | 15:17 |
| TheJulia | eh... maybe? It pretty much puttered out due to pandemic burnout | 15:22 |
| TheJulia | But cross team/wider context would be beneficial | 15:23 |
| knikolla[m] | how was your experience with the SPUCs that did happen before? | 15:27 |
| JayF | I mean, it's hard to say how well it'd scale to the rest of OpenStack | 15:27 |
| JayF | most of the participation was from the "super core" Ironic groups | 15:27 |
| JayF | and we've known each other, in some cases outside of OpenStack, for years and years | 15:27 |
| JayF | so it was nice when it was happening, but I think it got to a point where just "having an hour carved out on a calendar" was consuming too many spoons, even if it wasn't for tech work | 15:28 |
| dtantsur | yeah, this ^^ was the case for me when I was organizing it | 15:29 |
| JayF | having a larger pool of folks attending/organizing it could make it more of a drop-in thing which might reduce that effect some | 15:30 |
| knikolla[m] | in my experience it's really hard competing with people instead spending that time disconnected from a screen and with family/friends/pets/tv/etc. | 15:34 |
| JayF | That's a reasonable take, I don't think that's true for everyone but might represent enough of the opinion to quash an attempt at rebuilding this | 15:36 |
| knikolla[m] | i bring it up because I want it to survive attempts at being quashed. I think we badly need more community building events in the broader openstack community. | 15:39 |
| knikolla[m] | the Ironic having a little more experience with them being a great opportunity to learn what went well and what didn't. | 15:40 |
| knikolla[m] | the ironic team* | 15:40 |
| knikolla[m] | (sorry for hijacking the discussion :) was just passing by.) | 15:41 |
| JayF | knikolla[m]: you're welcome to hang out and hijack whatever discussions you want lol | 15:42 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Phase 1 - SQLAlchemy 2.0 Compatability https://review.opendev.org/c/openstack/ironic/+/856336 | 15:43 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Phase 2 - SQLAlchemy 2.0 Compatability https://review.opendev.org/c/openstack/ironic/+/857516 | 15:43 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Phase 3 - SQLAlchemy 2.0 Compatability https://review.opendev.org/c/openstack/ironic/+/857932 | 15:43 |
| TheJulia | okay... I hope that has it all working | 15:45 |
| TheJulia | JayF: knikolla[m]: We did get some irregular attendance from folks that were not cores as well, which was awesome and it tended to turn into more technical/problem solving, at least in the ironic community | 15:46 |
| JayF | maybe I should just put it up as "office hours" | 15:46 |
| JayF | e.g. I'll be in this video chat from X-Y, working on openstack stuff, stop by to say hello, ask for help, literally anything | 15:47 |
| TheJulia | ++ | 15:47 |
| knikolla[m] | In your experience, what worked better? Having an agenda/structured meeting or not? | 15:48 |
| JayF | I don't think we've tried video meetings with an agenda, and bluntly, our structured meeting is not a large part of our intraproject communications. | 15:49 |
| JayF | We also I think are pretty darn close to 50/50 in terms of timezones | 15:50 |
| JayF | so async is the default, which means PR comments and the like | 15:50 |
| rpittau | bye everyone, have a great evening! o/ | 15:53 |
| knikolla[m] | ah, timezones. makes sense. | 15:55 |
| TheJulia | our structured meeting is much more a sync-up point than anything else | 16:07 |
| TheJulia | and not everyone is able to attend becaues time zones | 16:08 |
| JayF | Honestly, Ironic might just be special in that we don't have that many new independent contributors | 16:15 |
| JayF | we have new humans cycling in and out, but usually from/around places or people who have contributed before | 16:15 |
| JayF | which means we really are pretty self-organizing | 16:16 |
| TheJulia | sign, test_migrations is sadness | 17:08 |
| TheJulia | err, sigh | 17:08 |
| opendevreview | Jakob Meng proposed openstack/bifrost master: [DNM] Debugging calls to openstack.cloud modules https://review.opendev.org/c/openstack/bifrost/+/859430 | 17:39 |
| iurygregory | arne_wiebalck, JayF since both of you are now part of the TC can you review https://review.opendev.org/c/openstack/governance/+/859464 and also ask other TC members to look at it? Ty! | 17:40 |
| JayF | iurygregory: I voted; those are usually brought up in the weekly TC meeting so it shouldn't get lost | 18:04 |
| iurygregory | JayF, tks! | 18:06 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Phase 3 - SQLAlchemy 2.0 Compatability https://review.opendev.org/c/openstack/ironic/+/857932 | 18:36 |
| TheJulia | so... yeah. Migrations are fun, raw sql text == happier julia | 19:06 |
| stevebaker[m] | good morning | 19:14 |
| TheJulia | good morning | 19:16 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/18.0: Remove ironic-tempest-bfv due to consisent failures https://review.opendev.org/c/openstack/ironic/+/859474 | 19:28 |
| JayF | TheJulia: ^ re: our conversation about CI in bugfix/18.0 | 19:28 |
| TheJulia | you beat me! | 19:29 |
| TheJulia | except, I thought htat was 19.0 | 19:29 |
| JayF | <.< >.> | 19:29 |
| opendevreview | Julia Kreger proposed openstack/ironic bugfix/19.0: CI: disable bfv testing on bugfix/19.0 https://review.opendev.org/c/openstack/ironic/+/859475 | 19:30 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/19.0: Remove ironic-tempest-bfv due to consisent failures https://review.opendev.org/c/openstack/ironic/+/859476 | 19:30 |
| JayF | TheJulia: damn, by :19 | 19:30 |
| JayF | TheJulia: I kinda thing our changes are each half-complete | 19:30 |
| JayF | TheJulia: and we need both things, right? | 19:31 |
| JayF | no need to define the bfv test if we don't use it? | 19:31 |
| TheJulia | we don't need to remove the definition | 19:31 |
| TheJulia | but project.yaml is what runs | 19:31 |
| JayF | ack | 19:31 |
| JayF | TheJulia: +2, wouldn't mind someone else looking at it since we basically talked thru the change in DM, feels like it'd be landing my own change at least to some small % | 19:32 |
| opendevreview | Jacob Anders proposed openstack/sushy master: Retry BootSourceOverride request when SettingsURI is read-only https://review.opendev.org/c/openstack/sushy/+/856597 | 20:35 |
| TheJulia | woot the sqlalchemy patch 3 passes | 20:42 |
| TheJulia | \o/ | 20:42 |
| opendevreview | Julia Kreger proposed openstack/ironic-tempest-plugin master: Tempest test for anaconda deploy https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/854031 | 20:44 |
| vanou | good morning Ironic | 22:26 |
| vanou | Regarding CI failure of https://review.opendev.org/c/openstack/ironic/+/858896 (backport patch to bugfix/19.0), this is based on backport patch of stable/yoga but not stable/xena. I think this is one of cause of CI failure. | 22:28 |
| JayF | so stable/yoga is right before bugfix/19.0 so that would be correct | 22:36 |
| JayF | Julia and I looked into the CI failures; we have a patch above to remove the broken test | 22:36 |
| JayF | and are waiting additional cores to review and land it, since we worked on it together | 22:36 |
| opendevreview | Vanou Ishii proposed openstack/ironic bugfix/19.0: Fix iRMC driver to use certification file in HTTPS https://review.opendev.org/c/openstack/ironic/+/858896 | 22:38 |
| opendevreview | Vanou Ishii proposed openstack/ironic bugfix/18.1: Fix iRMC driver to use certification file in HTTPS https://review.opendev.org/c/openstack/ironic/+/859482 | 22:43 |
| vanou | ah. Sorry I'm wrong bugfix/19.0 should be based on stable/yoga. Thanks JayF | 22:46 |
| JayF | vanou: https://etherpad.opendev.org/p/IronicWhiteBoard so if you go tto line 73, basically, the listed lettered version is the one that comes after | 22:47 |
| JayF | vanou: so if I were backporting a patch today; I'd go: stable/zed, bugfix/21.0, bugfix/20.2, stable/yoga, bugfix/19.0, stable/xena, bugfix/18.1, stable/{w,v,u} | 22:48 |
| JayF | I know it's confusing; I keep a separate note personally just to help keep the ordering straight | 22:48 |
| vanou | I see. Confusing... But now I understand what etherpad says. | 22:50 |
| vanou | Sorry. My post against bugfix/19.0 affects another patch https://review.opendev.org/c/openstack/ironic/+/858896 (I didn't intend to so). I should revert it | 22:51 |
| JayF | vanou: if you look at that etherpad now, just the preamble and the ironic portion, does it make more sense? | 22:52 |
| JayF | vanou: Sadly there's no push-button revert. | 22:52 |
| * JayF edits the rest of that section in that model | 22:53 | |
| vanou | JayF: Thanks a lot! It's now nice :) However it's bit strange for me that bugfix/21.0 and bugfix/20.2 are based on stable/zed | 22:55 |
| JayF | There's no limit on intermediate bugfix releases, it's mainly done when an Ironic operator or downstream packager requests a bugfix release be cut. | 22:56 |
| JayF | (and are generally considered only useful for consumption with standalone Ironic; not with an integrated OpenStack install) | 22:56 |
| vanou | I see. With your explanation, I have good grasp with it. | 22:58 |
| vanou | Thank you. | 22:59 |
| JayF | Thanks for the chat; it's good the whiteboard is improved around this, and I should make a note to see if we need to cut a patch release of any of those bugfix releases or stable branches still under support. | 22:59 |
| opendevreview | Vanou Ishii proposed openstack/ironic bugfix/19.0: Fix iRMC driver to use certification file in HTTPS https://review.opendev.org/c/openstack/ironic/+/858896 | 23:13 |
| opendevreview | Vanou Ishii proposed openstack/ironic bugfix/19.0: Fix iRMC driver to use certification file in HTTPS https://review.opendev.org/c/openstack/ironic/+/858896 | 23:20 |
| vanou | Now I revert what I modified by mistake for bugfix/19.0 backport patch. | 23:21 |
| opendevreview | Vanou Ishii proposed openstack/ironic bugfix/18.1: Fix iRMC driver to use certification file in HTTPS https://review.opendev.org/c/openstack/ironic/+/859482 | 23:33 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!