Monday, 2020-12-07

*** JamesBenson has joined #openstack-ironic00:04
*** JamesBenson has quit IRC00:08
*** zzzeek has quit IRC00:12
*** zzzeek has joined #openstack-ironic00:16
*** tosky has quit IRC00:20
*** ociuhandu has joined #openstack-ironic00:22
*** ociuhandu has quit IRC00:26
*** JamesBenson has joined #openstack-ironic00:36
*** JamesBenson has quit IRC00:40
*** jamesdenton has quit IRC00:48
*** jamesden_ has joined #openstack-ironic00:49
*** cp- has joined #openstack-ironic00:52
*** paras333 has quit IRC00:56
*** paras333 has joined #openstack-ironic01:03
*** paras333 has joined #openstack-ironic01:04
*** paras333 has joined #openstack-ironic01:05
*** johnsag has quit IRC01:09
*** paras333 has quit IRC01:19
*** paras333 has joined #openstack-ironic01:27
*** paras333 has quit IRC01:31
openstackgerritSteve Baker proposed openstack/ironic master: Complete the REST API POST documentation  https://review.opendev.org/c/openstack/ironic/+/75735301:44
*** ociuhandu has joined #openstack-ironic02:10
*** ociuhandu has quit IRC02:15
*** Qianbiao has joined #openstack-ironic02:35
*** jslater has quit IRC02:57
*** JamesBenson has joined #openstack-ironic03:12
*** JamesBenson has quit IRC03:17
*** ociuhandu has joined #openstack-ironic03:24
*** ociuhandu has quit IRC03:29
*** vesper11 has quit IRC03:34
*** vesper11 has joined #openstack-ironic03:35
*** JamesBenson has joined #openstack-ironic03:51
*** JamesBenson has quit IRC03:55
*** jamesden_ has quit IRC04:34
*** jamesdenton has joined #openstack-ironic04:35
*** vesper has joined #openstack-ironic04:36
*** vesper11 has quit IRC04:37
*** ociuhandu has joined #openstack-ironic05:12
*** ociuhandu has quit IRC05:17
openstackgerritlikui proposed openstack/ironic master: Update doc8 version  https://review.opendev.org/c/openstack/ironic/+/76572205:21
*** ricolin has joined #openstack-ironic05:37
*** QianbiaoNG has joined #openstack-ironic06:17
*** Qianbiao has quit IRC06:20
*** HagunKim has joined #openstack-ironic06:56
*** anuradha1904 has joined #openstack-ironic06:57
*** ociuhandu has joined #openstack-ironic07:01
*** ociuhandu has quit IRC07:07
*** ociuhandu has joined #openstack-ironic07:21
*** ociuhandu has quit IRC07:28
arne_wiebalck Good morning, ironic!07:51
*** rcernin has quit IRC07:55
*** tosin has joined #openstack-ironic07:58
*** rcernin has joined #openstack-ironic08:25
*** rcernin has quit IRC08:30
*** fdegir has quit IRC08:34
*** maelk has quit IRC08:34
*** fdegir has joined #openstack-ironic08:34
*** maelk has joined #openstack-ironic08:34
jandersgood morning arne_wiebalck08:38
*** rpittau|afk is now known as rpittau08:46
rpittaugood morning ironic! o/08:46
arne_wiebalckhey janders and rpittau o/08:52
rpittauhey arne_wiebalck :)08:52
*** ociuhandu has joined #openstack-ironic08:53
*** ociuhandu has quit IRC08:53
*** lucasagomes has joined #openstack-ironic08:54
*** dougsz has joined #openstack-ironic08:55
*** derekh has joined #openstack-ironic09:04
*** HagunKim has quit IRC09:05
*** ociuhandu has joined #openstack-ironic09:15
*** zzzeek has quit IRC09:19
*** tosin has quit IRC09:20
*** zzzeek has joined #openstack-ironic09:20
*** mgoddard has joined #openstack-ironic09:20
*** rcernin has joined #openstack-ironic09:25
*** ociuhandu has quit IRC09:28
*** rcernin has quit IRC09:30
*** dtantsur|afk is now known as dtantsur10:07
dtantsurmorning ironic10:08
dtantsurjanders: sorry for missing SPUC, I was submitting some papers here10:09
*** rcernin has joined #openstack-ironic10:21
*** rcernin has quit IRC10:21
*** ociuhandu has joined #openstack-ironic10:30
*** mgoddard has quit IRC10:33
openstackgerritVerification of a change to openstack/ironic-python-agent-builder failed: Support using lzma for initramfs compression  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/76437110:39
*** mgoddard has joined #openstack-ironic10:43
*** mgoddard has quit IRC10:49
*** mgoddard has joined #openstack-ironic10:49
*** mgoddard has quit IRC10:52
*** mgoddard has joined #openstack-ironic10:52
dtantsurbroken requirements, yay10:57
rpittauis there anything NOT broken today ?11:01
dtantsurdunno11:14
dtantsurhttps://review.opendev.org/c/openstack/requirements/+/76578511:14
*** tosky has joined #openstack-ironic11:22
iurygregoryit's monday so it's fine =)11:37
iurygregorygood morning Ironic11:38
openstackgerritlikui proposed openstack/python-ironicclient master: Update doc8 version  https://review.opendev.org/c/openstack/python-ironicclient/+/76579311:52
*** zzzeek has quit IRC12:03
*** johnsag has joined #openstack-ironic12:03
*** zzzeek has joined #openstack-ironic12:05
*** ociuhandu has quit IRC12:09
*** uzumaki has joined #openstack-ironic12:11
*** JamesBenson has joined #openstack-ironic12:11
*** zzzeek has quit IRC12:38
*** zzzeek has joined #openstack-ironic12:39
*** priteau has quit IRC12:40
*** ociuhandu has joined #openstack-ironic12:40
*** ociuhandu has quit IRC12:49
*** alexmcleod_ has quit IRC12:59
*** alexmcleod_ has joined #openstack-ironic12:59
*** uzumaki has quit IRC13:04
*** ociuhandu has joined #openstack-ironic13:12
*** ociuhandu has quit IRC13:20
*** priteau has joined #openstack-ironic13:27
*** paras333_ has joined #openstack-ironic13:45
*** Goneri has joined #openstack-ironic13:48
*** ociuhandu has joined #openstack-ironic13:53
*** ociuhandu has quit IRC13:53
*** ociuhandu has joined #openstack-ironic13:54
*** sdanni has joined #openstack-ironic13:58
TheJuliagood morning14:04
*** ociuhandu has quit IRC14:04
*** ociuhandu_ has joined #openstack-ironic14:04
*** rloo has joined #openstack-ironic14:04
iurygregorygood morning TheJulia =)14:07
*** uzumaki has joined #openstack-ironic14:07
TheJuliaso everything broken?14:12
iurygregoryyup \o/14:15
*** ociuhandu_ has quit IRC14:19
*** k_mouza has joined #openstack-ironic14:21
TheJuliasweet14:21
TheJuliaI should have stayed in bed14:21
*** k_mouza has quit IRC14:22
*** k_mouza_ has joined #openstack-ironic14:23
dtantsurmorning TheJulia14:24
TheJuliadtantsur: do you want to continue to track https://review.opendev.org/c/openstack/bifrost/+/760791 on the weekly review priorities14:32
dtantsurnot necessary, assuming somebody will actually approve it..14:32
* TheJulia looks at a calendar14:33
TheJulianext week is likely the last week worth of trying to have a meeting this year14:33
*** ociuhandu has joined #openstack-ironic14:34
iurygregorylooking now14:36
rpittauargh, I have that in my prio list, but I have a very long prio lost :/14:37
rpittauI can test it this week14:37
iurygregoryif I do remember dtantsur did a lot of testing while trying to make it work hehe14:39
dtantsurwell, it worked for me at least14:39
rpittauI think I tested it as well once at some point, but forgot about it :(14:40
iurygregoryit works without devstack =P14:40
*** tzumainn has joined #openstack-ironic14:42
*** kaifeng has joined #openstack-ironic14:43
*** ociuhandu has quit IRC14:46
*** ociuhandu has joined #openstack-ironic14:49
TheJuliaIf anyone has anythign to add to potential review priorities or their subteam status updates: https://etherpad.opendev.org/p/IronicWhiteBoard14:52
*** ociuhandu_ has joined #openstack-ironic14:59
*** k_mouza_ has quit IRC14:59
TheJulia#startmeeting ironic15:00
openstackMeeting started Mon Dec  7 15:00:17 2020 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
TheJuliao/15:00
*** openstack changes topic to " (Meeting topic: ironic)"15:00
openstackThe meeting name has been set to 'ironic'15:00
iurygregoryo/15:00
*** stendulker has joined #openstack-ironic15:00
dtantsuro/15:00
TheJuliaGood morning everyone!15:00
rpioso\o15:00
stendulkero/15:00
bdoddo/15:00
kaifengo/15:00
TheJuliaIt is time for our weekly meeting of baremetal cloud irony!15:00
rlooo/15:00
TheJuliaOur agenda can be found on the wiki today.15:01
TheJulia#link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting15:01
TheJuliaLooksl ike only one discussion item, so maybe today will go quick?!?15:01
TheJulia#topic Announcements / Reminders15:01
*** openstack changes topic to "Announcements / Reminders (Meeting topic: ironic)"15:01
rpittauo/15:02
TheJulia#info We are approaching end of year. Numerous contributors are expected to be out the last half of December. Please remember to provide review feedback on sprint-1 and early sprint 2 items if applicable and present. If an existing item won't make it, now would be a good time to signal so.15:02
dtantsurwe have releases due this week, right?15:03
TheJuliaumm... good question!15:03
TheJuliachecking15:03
bfournieo/15:03
JayFo/15:03
*** ociuhandu has quit IRC15:03
TheJuliadtantsur: next week15:03
ajyao/15:03
*** tosin has joined #openstack-ironic15:03
dtantsurokay, cool. let's try to do it rather earlier than later15:03
TheJulia++15:04
rpittauI can help with the releases if needed15:04
TheJuliaI can always put in the actual change for the releases team later in the week, but items need reviews.15:04
dtantsurI'll be here next week, but only Mon-Wed (same the week after that)15:05
TheJuliak15:06
TheJuliaAnyway, shall we proceed?15:06
rpittauLet's15:06
dtantsuryep15:06
TheJulia#topic Review action items from the previous meeting15:07
*** openstack changes topic to "Review action items from the previous meeting (Meeting topic: ironic)"15:07
*** tosin has quit IRC15:09
TheJuliaWe had one action item which was me to sync with the stable team regarding skipping backports on branches in the "extended maintenance phase". Basically they feel it is a stable policy violation, and then discussion kind of devolved because their view switched to the community as a whole and not as a project or as a contributor. The resulting consensus seemed to be we need to do what is reasonable for the15:09
TheJuliasituation that exists at that time.15:09
*** tosin has joined #openstack-ironic15:09
TheJuliarloo: ^^15:09
TheJuliaMoving on!15:09
TheJulia#topic Review subteam status reports15:10
*** openstack changes topic to "Review subteam status reports (Meeting topic: ironic)"15:10
TheJulia#link https://etherpad.opendev.org/p/IronicWhiteBoard15:10
dtantsurThis makes EM quite costy..15:10
*** k_mouza has joined #openstack-ironic15:10
rloothx TheJulia!15:10
rlooi'll update the etherpad...15:10
TheJuliadtantsur: They don't seem to care and we're not really interested in discussing further, I pointed that out and they just let it drop15:10
TheJuliaSo I think our EM path is to kind of just do whatever is reasonable to ourselves.15:11
TheJuliaWhich likely need to include starting to drop extensive testing on older branches15:11
kaifengi wondner if other projects backporting patches to the EM releases15:11
rloo(I think there ought to be some community ... understanding... about this but ...)15:11
TheJuliakaifeng: nova is doing skip patching, so master -> appropriate current stables and then jumping to the queens em that they care about15:12
TheJuliarloo: the Problem is the community, they fallback on the community in the past said they don't want to do this15:12
TheJuliaso their stance was frozen in time and not considering the realities of those that actually have to get stuff in to EM.15:12
TheJuliaAnyway!15:12
TheJulia#link https://etherpad.opendev.org/p/IronicWhiteBoard15:13
TheJuliaLine 26715:13
rloobut if some projects are doing it, i think that ought to be communicated to the community, vs separate projects doing things their own way. having said that, i don't really want to get involved and don't want this to block/delay whatever.15:13
kaifengskip patching doesn't seem upgrade compatible th15:13
kaifengs/th/tbh/15:13
TheJuliakaifeng: depends on the patch I guess, if it is not data model, then there is low risk as long as someone is going to a newer release version and not just going from say Queens -> Rocky15:14
rlookaifeng: good point. i think though, that the stuff that is backported, is to fix bugs. not add features. so it won't prevent upgrades, and if one upgrades enough, they'll get the bug fix.15:14
TheJuliarloo: I think the same is true, they did not want to be in the middle of such a discussion either.15:15
TheJuliadtantsur: rpittau: No luck on nvme secure erase?15:15
rlooTheJulia: to confirm -- we're only talking about backporting patches as per the stable team's rules wrt backporting. we're just not going to backport to 'all' the interim releases.15:15
rloowrt rules, i mean wrt what qualifies to be backported.15:16
rpittauTheJulia: nothing for now15:16
TheJuliarloo: lets continue this in open discussion15:16
rloook15:16
TheJuliarpittau: Could you put a note saying that it is going to not make sprint1 and will likely make sprint 2?15:16
dtantsurTheJulia: nothing from me, and I think janders will only get to it after the holidays15:16
TheJuliak15:16
TheJuliakaifeng: I see you have an approved spec \o/15:17
rpittauTheJulia: will do15:17
TheJuliaw/r/t node history15:17
kaifengTheJulia: yeah, thanks everyone for the review!15:17
TheJuliaajya, bdodd: reminder: redfish raid is presently listed for sprint 2. I see there is some discussion on the change. I'm curious if it can be updated at this point?15:18
TheJuliaiurygregory: same status on oslo.privsep?15:18
iurygregoryTheJulia, going to update things this week15:18
bdoddI'm working on updates to the RAID patches15:18
iurygregoryI was on PTO15:18
TheJulia\o/15:18
TheJuliaOkay, thanks15:18
*** ociuhandu_ has quit IRC15:19
TheJuliazer0c00l: Looks like you have a little discussion on the anaconda patch, I can drop my +2 if you want for now. You'll likely want to consider revising maybe.15:19
TheJuliaEverything looks good to me, are we good to proceed to priorities for the week?15:20
rloo++ for revising15:20
*** tosin has quit IRC15:21
TheJuliavote changed15:21
TheJuliaor... changing15:21
JayFI certainly want my reviews taken into account before it merges :D15:21
* TheJulia needs a gerrit stop-watch15:21
TheJulia#topic Deciding on priorities for the coming week15:22
*** openstack changes topic to "Deciding on priorities for the coming week (Meeting topic: ironic)"15:22
*** tosin has joined #openstack-ironic15:22
TheJulia#link https://etherpad.opendev.org/p/IronicWhiteBoard15:22
rloo(I added a note wrt anaconda revision desired)15:22
TheJuliaStarting at line 12015:23
TheJuliaLooks like we got some stuff merged last week, so I'll clean that up. I added items aaround 19815:23
* arne_wiebalck cannot join the meeting today, just briefly: SIG meeting tmrw with rpioso on Redfish interop profiles, NTR for the SIG otherwise, bye o/15:25
TheJuliaThanks arne_wiebalck15:26
TheJuliaAny objections to the items to add?15:26
JayFlgtm15:27
TheJuliaOkay, I;kk do the shuffling after the meeting15:27
TheJuliaerr I'll15:28
TheJuliaOnward!15:28
* TheJulia hears crickets15:28
TheJulia#topic Discussion15:28
*** openstack changes topic to "Discussion (Meeting topic: ironic)"15:28
TheJuliaOne quick discussion item, should we have a final meeting for the month next week?15:28
dtantsurI'll be likely here for the next 2 Mondays15:29
dtantsurwon't be around on the 4th of January though15:29
TheJuliaI was about to suggest we skip the 21st, 28th, and Janurary 4th15:29
dtantsursounds reasonable to me15:29
TheJuliaWhich is a long gap, but I think a number of us need the heads down time be it code or in our pillows15:30
rloo++ so dec 14 is last meeting of the year!15:30
TheJuliaSeems that way if there are no objections :)15:31
dtantsurprepare your santa hats \o/15:31
TheJuliaOh wait, are we doing that?!?15:31
TheJuliaeek!15:31
TheJulia:)15:31
dtantsurwe also have two more SPUCs planned15:31
TheJulia++15:31
TheJuliaSpucs are good15:31
* dtantsur had to miss last Friday, sorry for that15:31
iurygregoryI don't have santa hats =(15:31
TheJuliaI assume santa hats should now be on SPUCs15:31
rpittauI have a wizard goofy hat, is that ok?15:31
TheJuliamaybe ;)15:31
iurygregoryrpittau, yes!15:32
TheJuliaAnyway, I'll send out the "end of year email" and we'll switch to etherpad updates as we have the last couple of end of year holiday seasons15:32
TheJuliaIf there are any questions or concerns, please feel free to reach out to me.15:32
TheJuliaAnyway! Onward to the Baremetal SIG15:32
TheJulia#topic Baremetal SIG15:32
*** openstack changes topic to "Baremetal SIG (Meeting topic: ironic)"15:32
TheJulia#Info Baremetal SIG session covering Redfish Interop profiles - Tomorrow, December 8th, at 2PM UTC15:33
TheJulia#link https://etherpad.opendev.org/p/bare-metal-sig15:33
TheJuliaarne_wiebalck: one reminder, record the talk portion so we can build some content for youtube :)15:34
rpiosoTheJulia: Ack15:34
TheJuliaSince we have no RFE's on the agenda, we'll go directly to Open Discussion15:34
TheJulia#topic Open Discussion15:34
*** openstack changes topic to "Open Discussion (Meeting topic: ironic)"15:34
TheJuliarloo: where were we?15:35
rlooTheJulia: for myself, I think what I'd like is a clarification of this process, so everyone (in ironicland) knows.15:35
TheJuliaI think we just try and do the best we can, applicable to stable policy, and just start killing EM branch testing that doesn't make sense15:36
rlooif i recall, i was fine with what we discussed at ptg15:36
rloowhat EM branch testing are we doing now?15:36
*** ociuhandu has joined #openstack-ironic15:36
TheJuliaI can write a doc update then15:36
TheJuliarloo: basically trying to keep everything alive15:36
TheJuliawhich is insane15:36
rlooright. i agree. i thought we had already killed some branch testing.15:36
TheJuliaSome, but not tons15:36
TheJuliaWe can likely dial it back by a 50%15:37
JayFDoes OpenStack's branch model fit this concept? e.g. can we put "r" in unmaintained while "q" is in EM?15:37
TheJulias/a//15:37
rloothere were 2 things? 1 kill some branch testing. 2. allow backports to skip some branches. eg if someone wants to backport to rocky but not stein. (we're talking only branches in EM, can't skip branches in M)15:37
TheJuliaJayF: EM is up to anyone wantint to put patches forth to burn the time to get the patch in, but forcing someone to go through a ton of EM branches and keeping them all perfectly alive with full testing, is a huge time sink15:38
rlooand then maybe 3. is it ok to backport to an EM branch were there is no upstream testing.15:38
TheJuliaor limited testing, like we know some scenarios are more likely to fail than others15:38
JayFTheJulia: I don't disagree in concept, I'm just wondering if there's a clear way to communicate that to users. OpenStack not having an "EM ... but more maintained than the other EM" branch type is unfortunate15:39
TheJuliaAt least inCI15:39
dtantsurit won't help to kill "some" branch testing, our jobs tend to break in large groups :)15:39
TheJuliaJayF: unfortunately it is up to those that care about an EM branch to carry that load15:39
rloodtantsur: what do you mean by that? keep all branch testing then? or kill all of them? :)15:40
TheJuliadtantsur: well, huge breakages are generally easy to resolve once we identify the issue. The problem is noise and overall load crushing the ability for the job to actually pass15:40
JayFI'm just thinking, we're going through a process right now to determine which release to upgrade to. As a contributor, I know to look for either Queens or a "M" release or newer. I don't think an external user would easily be able to determine that Queens is a better choice than Rocky, for instance15:40
dtantsurI mean, it's not about removing one job, it's about removing most of them15:40
rloofwiw, stein is EM now.15:40
dtantsurand I'll personally have reservations about +2'ing something that only passed unit tests..15:40
TheJuliaYeah, I'd prefer to some tests remain functional15:41
dtantsurI still want to see a model similar to Linux kernel and some other projects:15:41
TheJuliajust, not 10+ integration jobs15:41
dtantsurwe maintain any branch for a while. then whoever cares can provide support, everything else is killed off.15:42
TheJuliadtantsur: that is kind of where we're headed I think15:42
rloodtantsur: are you including branches in Maint, or only EM?15:42
TheJuliato be totally honest15:42
dtantsurrloo: Maint is the "we maintain any branch" part. EM is "whoever cares" part.15:42
openstackgerritVerification of a change to openstack/bifrost failed: Support testing secure boot  https://review.opendev.org/c/openstack/bifrost/+/76079115:42
rloo(M=maintenance, EM=extended maintenance)15:42
TheJuliaEM's naming is not the best, but I remember when that was a huge fight15:43
rloodtantsur: does upstream community/stable say anything wrt CI for M branches?15:43
JayFdtantsur: TheJulia: ++ the linux kernel did this a couple of years back to solve this exact problem15:43
dtantsurrloo: I don't recall any firm position. I think it's up for us as a project.15:43
rlooshall we take EM first, that seems easiest?15:43
TheJuliarloo: aiui, it is up to the projects.15:43
dtantsurrloo: I think we're only talking about EM now15:44
dtantsurthe main maintenance is not long, and the stable policy requires us to provide a high level of maintenance for them15:44
TheJuliaI'd like to point out some of the major projects also only have 1-3 integrated scenario jobs on older branches. We've got many more15:44
dtantsurone of the reasons I started removing the iscsi deploy :)15:44
TheJulia++15:45
dtantsurbut yeah. our problem is low concurrency of our tests15:45
dtantsurand a relatively large test time15:45
TheJuliaAnd limited CI resources15:45
TheJulia8 GB of ram, no ability to touch swap15:45
dtantsurif Nova can boot a VM with cirros in 30 seconds and 256M of RAM, they're in a much better position15:45
rlooi'm totally good with 0 CI for EM (except unit tests)15:45
TheJuliato quote an infra person, "Swap is the mind or CI job killer"15:45
dtantsurrloo: would you +2 any patch under these conditions?15:46
rlooi would if we agree to that 'policy'. it is a backport.15:46
JayFI think patches exist where I'd be hesitant to +2 without any integration tests15:46
JayFclean backsports of minor fixes? sure, lets forgo the integration tests15:46
dtantsursame for me15:46
iurygregorysame15:46
rloohmm. what if said person tested downstream?15:47
JayFbut I've seen some backports that I'd want to see a deploy work for15:47
dtantsurrloo: there are people who I'd trust in this case, there are people (most of them) who I'd not15:47
JayFThen we go down a path of why isn't "trust me, I tested it" good enough everywhere :|15:47
rlooi just don't know how reasonable it is, to expect upstream to provide CI, given the state of things upstream.15:47
dtantsurright15:47
dtantsurbifrost jobs can potentially have a longer lifetime now that we default to using virtual environments15:48
rlooso if we don't +2, then an alternative is for the person/company to patch downstream. which is also fine with me (that's what we do at verizon media)15:48
dtantsurbecause of their simplicity15:48
TheJuliaPure downstream patching adds a lot of cost as well, but it all becomes what makes the most sense in the situation15:49
rloohonestly, i haven't been contributing upstream. for the folks that have -- what is your preference?15:49
rloowe have a limited number of people working upstream. i think they ought to have a bigger vote wrt where they want to put their time/energy.15:49
JayFrloo: TBH I think our input, contributor or not, is interesting here because we're soon-to-be consumers, likely, of one of these EM branches15:49
TheJuliaIt seems like the action is to kind of write down the overall feeling/perception, and be able to point to that whilst also possibly beginning to nuke some but not all integration jobs *where it makes sense*15:50
rlooright, we aren't the only ones (I hope) that will be consumers of those EM. having said that though, things aren't 'free'.15:50
dtantsurJayF: would you become a "patron" of an EM branch?15:50
JayFI don't care so much what we do w/r/t deciding support. Putting resources where they matter (e.g. targetting queens and not other EMs) makes sense; I'm just concerned about upstream users (vs rh users) realizing that Queens is more maintained than other EMs15:50
dtantsurHow does kernel solve this messaging problem?15:51
* TheJulia likes this idea dtantsur just raised15:51
JayFdtantsur: ^ is my concern, that we communicate it well. I think OpenStack's branch system is almost designed to make this sorta "branch skipping" awkward and hard to communicate15:51
JayFdtantsur: kernel specifically calls out longterm releases on kernel.org15:51
JayFdtantsur: so you get a menu of maintained things to pick from15:51
dtantsurHonestly, the lack of mutual understanding around EM branches is already problematic15:51
rlooso... we introduce a new 'class' of branches. M -> EM, and EM can include .. sponsored M?15:51
dtantsurCan we maybe pull our EM branches from releases.openstack.org (if they're anyhow mentioned there) and only provide this information in our docs?15:52
dtantsurI'm not convinced this issue is solved by adding more concepts15:52
JayFcan we pull EM branches *except queens* from releases?15:52
TheJuliaThe whole idea around EM is that it can "sponsored" or have a "patron" if the resources are supplied by the interested party/org. The thing is they will never ever be released again so patches are merged at the will of the project or the EM maintainers who care15:52
dtantsurI think openstack as a whole should do a better jobs communicating that the exactly quality of EM branches is up to a project15:52
JayFdtantsur: ++15:53
TheJuliaThis may be worth of taking to the TC, in order for them to drive that visibility15:53
dtantsurare you still on TC? ;)15:54
TheJuliasince, in reality, what is needed for that to be documented OR the EM branch model to be re-visited15:54
JayFI think a fair summary is: 1) Nobody has objections to the idea of targetting your resources on a single EM branch and not spreading the love. 2) We need a good way for operators to know that Queens is more maintained than Rocky (and soon Stein), but OpenStack's release system doesn't make that easy15:54
TheJuliaNo, Board until January when the new board is seated, that is if I'm not re-elected.15:54
TheJuliaJayF: that sounds right15:55
JayFI'd suggest taking a summary of what you wanna do, call out that sticking point, and hit the ML15:55
JayFThis is the sorta thing we should have a longer form discussion about  ... but we can have that discussion while effectively implementing the new plan15:56
TheJuliaAgreed15:56
rloo++15:57
TheJuliaOkay, well, we're almost at time. Thanks everyone15:58
dtantsura heads-up15:58
*** jamesdenton has quit IRC15:58
dtantsurCentOS 8.3 is out, watch out for related failures15:58
dtantsur </heads-up>15:59
TheJuliajoy15:59
dtantsurwe have a minor breakage in openshift land15:59
*** jamesdenton has joined #openstack-ironic15:59
TheJulia#info Heads up - Centos 8.3 has been released, keep an eye out for failures.15:59
TheJuliaIf anyplace explodes it will be metalsmith15:59
dtantsuror bifrost. or IPA-builder.15:59
dtantsur:)15:59
TheJuliaYup15:59
TheJuliaAnywya, thanks everyone! Have a wonderful week!15:59
dtantsurthank you!16:00
JayF\o16:00
TheJulia#endmeeting16:01
*** openstack changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/ironic/ | Bugs: https://storyboard.openstack.org/#!/project_group/75 | Contributors are generally present between 6 AM and 12 AM UTC, If we do not answer, please feel free to pose questions to openstack-discuss mailing list."16:01
openstackMeeting ended Mon Dec  7 16:01:26 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-12-07-15.00.html16:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-12-07-15.00.txt16:01
openstackLog:            http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-12-07-15.00.log.html16:01
openstackgerritVerification of a change to openstack/metalsmith failed: Add metalsmith_instances instance option config_drive  https://review.opendev.org/c/openstack/metalsmith/+/76185016:08
openstackgerritAija Jauntēva proposed openstack/ironic-specs master: Pre-defined system hw configuration in single step  https://review.opendev.org/c/openstack/ironic-specs/+/74072116:14
*** lmcgann has joined #openstack-ironic16:17
*** johnsag has quit IRC16:23
*** ociuhandu has quit IRC16:27
*** ociuhandu has joined #openstack-ironic16:32
*** stendulker has quit IRC16:33
*** tosin has quit IRC16:44
rpittaugood night! o/16:58
*** rpittau is now known as rpittau|afk16:58
*** QianbiaoNG has quit IRC17:00
arne_wiebalckTheJulia: Yes, I will record the presentation.17:04
*** dougsz has quit IRC17:28
*** lucasagomes has quit IRC17:33
*** ociuhandu_ has joined #openstack-ironic17:38
*** ociuhandu has quit IRC17:42
*** ociuhandu_ has quit IRC17:43
*** dsneddon has joined #openstack-ironic17:50
*** ociuhandu has joined #openstack-ironic17:51
*** gyee has joined #openstack-ironic17:54
*** ociuhandu has quit IRC17:55
*** k_mouza has quit IRC17:56
*** derekh has quit IRC17:58
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Correctly decode error messages from ironic API  https://review.opendev.org/c/openstack/ironic-python-agent/+/76583817:59
dtantsurI finally got tired of ^^^17:59
dkingIt looks like the "extra_hardware" collector is included by default, but the "hardware-detect" python package upon which it depends isn't included in the IPA Builder element. Am I seeing that correctly?18:03
dtantsurdking: it's included but not enabled18:03
dtantsurthe second part is correct, yes. there an element to add it.18:04
dkingIs there a reason why it's not added, especially if it's a default collector?18:04
dtantsurdking: for the same reason it's not enabled by default: to avoid adding redundant dependencies18:06
dkingIt seems to me that it's a useful tool that would make it easy for end users to gather a lot of extra information. But if the convention is to only use it with custom IPA builds and folks are used to that, then that's fine. We already have to build ours, anyway. I was just curious.18:09
*** uzumaki has quit IRC18:09
dtantsurThere are ramdisks around with everything included, e.g. the one shipped by RDO18:09
dtantsurhttps://images.rdoproject.org/centos8/victoria/rdo_trunk/current-tripleo/ I think is the current URL18:10
dtantsurbut yeah, some folks don't even use inspection or only use OOB inspection18:11
dkingI knew that TripleO was using it. We're building our own, so I'll just add that to our own element.18:11
dtantsurdking: you can depend on this element https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/dib/extra-hardware18:12
dtantsurit's based on an earlier element from StackHPC18:12
*** kaifeng has quit IRC18:13
dkingOh, thanks. I totally missed that. I might just add that element in separately since technically, it's not a requirement of our hardware managers. It's just something that we wanted to use for our introspection rules, which is separate. I'll add that into our build command.18:14
*** dtantsur is now known as dtantsur|afk18:28
dtantsur|afko/18:28
arne_wiebalckbye everyone o/18:33
openstackgerritVladyslav Drok proposed openstack/networking-baremetal master: Fix exception handling when querying ironic ports  https://review.opendev.org/c/openstack/networking-baremetal/+/76585118:36
TheJuliastevebaker: you awake?19:33
*** cmorey has joined #openstack-ironic19:41
dkingIs this doctring correct: https://opendev.org/openstack/ironic-inspector/src/branch/stable/victoria/ironic_inspector/plugins/extra_hardware.py#L3519:41
cmoreyI've got a node stuck in "Provisioning State" : error thanks to an ipmi issue, i've resolved the issue, but i can't seem to reset the provisioning state, any ideas?19:42
JayFwhat is the state it's "stuck" in?19:43
cmorey@error@19:43
cmoreysorry "error"19:43
dkingIt talks about updating Swift, but I see no actual mention of swift or 'hardware_swift_object' mentioned directly in that code.19:44
cmoreylast error was "uring sync_power_state, max retries exceeded for node 9ed7092a-329f-4a79-a8a4-56a9410e9e99, node state None does not match expected state 'power on'. Updating DB state to 'None' Switching node to maintenance mode. Error: IPMI call failed: power status." it's now showing as "power on"19:44
JayFOK; you'll need to do `openstack baremetal node maintenance unset [uuid]`, `openstack baremetal node manage [uuid]`, `openstack baremetal node provide [uuid]`, and I believe that'll get the node back to available for you19:45
cmorey"The requested action "manage" can not be performed on node "9ed7092a-329f-4a79-a8a4-56a9410e9e99" while it is in state "error". (HTTP 400)"19:46
cmoreysorry :(19:46
JayFhmm.19:46
JayFDo you have something deployed on that node?19:47
JayFI bet it needs to be undeployed19:47
JayFbut I don't know the command for that :D19:47
cmoreyJayF, that's what i would have expected, as far as i can tell from the state machine, i cn delete or rebuild19:47
JayFyeah; you need to "delete" the instance, but node delete in ironic is "delete the node from the db"19:48
JayFah, it's just that19:48
cmoreyI don't have an instance ID assigned19:48
JayFbaremetal node undeploy [uuid]19:48
JayFthis will cause any data on the node to be wiped19:48
JayFso if you don't want that, don't do this :D19:48
cmoreytoo late :)19:49
JayF(assuming you have automated cleaning on; which it is by default)19:49
cmoreyI don't, but that's actually my next step19:49
cmoreyo.k. that worked, i'm sure i tried that before,19:49
JayFunsetting maint mattered too :D19:50
cmoreymaintenance was set to false already19:50
cmoreyah, i tried, rebuild, not undeploy19:51
cmoreyundeploy doesn't appear on https://docs.openstack.org/ironic/train/_images/states.svg19:52
JayFthe commands you pass are "verbs", and don't always represent states themselves19:52
JayFdeleted (via API) most closely models the `undeploy` action19:53
cmoreyah,19:53
JayFe.g. the `provide`, I was telling you before, puts a node from manageable->available (making that node 'provide' an extra inventory that can be deployed onto)19:54
cmoreyis there a verbs version of the state map? (rather than the API version)19:56
JayFfrom a user perspective; here are the CLI commands https://docs.openstack.org/python-ironicclient/latest/cli/osc/v1/index.html#baremetal-node19:57
JayFfrom a dev perspective, all the states are here https://opendev.org/openstack/ironic/src/branch/master/ironic/common/states.py#L4119:58
stevebakerTheJulia: hey19:59
zer0c00lTheJulia: Ack. I been meaning to respond to the comments. Will go though the spec and review them this week.20:07
*** jamesdenton has quit IRC20:15
*** jamesdenton has joined #openstack-ironic20:16
cmoreythanks for your help JayF20:24
JayFnp, good luck20:24
spotzHey all quick question on a tripleo validator run pre-introspection should I be concerned about failures on undercloud-neutron-sanity-check and ctlplane-ip-range. I did verify the ctlplane-ip-range is 20 IPs and the nodes themselves did introspect once I corrected a MAC issue20:32
JayFThat might be better for #tripleo20:35
spotzhehe - they sent me here but I did just say I had a pre-introspection question:)20:36
JayFWell IDK the answer :D20:36
spotzif we knew all the answers what would be the fun?:)20:37
JayFI mean, have you ever won a trivia game? Knowing all the answers is great! LOL20:37
*** paras333_ has quit IRC21:12
*** jamesdenton has quit IRC21:18
*** jamesdenton has joined #openstack-ironic21:18
TheJuliaspotz: yeah, that is definitely not a question anyone in this channel would be able to answer21:46
TheJuliawell, non-already-tripleo-specific-code-contributors most likely21:46
*** sshnaidm has quit IRC21:49
spotzthejulia - I'm just going with that error as tripleo doesn't know either. Now looking at debugging metalsmith. So much has changed between train and victoria21:51
TheJuliaMaybe it has validation code burried in there for that21:52
*** k_mouza has joined #openstack-ironic21:57
*** rcernin has joined #openstack-ironic22:00
*** k_mouza has quit IRC22:01
*** tzumainn has quit IRC22:02
*** rloo has quit IRC22:02
*** johnsom has quit IRC22:02
*** andrein has quit IRC22:02
*** lmcgann has quit IRC22:05
*** jhesketh has quit IRC22:06
*** k_mouza has joined #openstack-ironic22:07
*** jhesketh has joined #openstack-ironic22:08
*** sshnaidm has joined #openstack-ironic22:08
spotzTheJulia: The error when I did a node show was SSL Exception which I thought odd22:08
spotzBut I never have 'normal' issues:)22:08
TheJuliacontext required22:08
spotzTheJulia: metalsmith list shows State Error, openstack baremetal allocation list shows state actiive. And if I do openstack baremetal node show - Failed to load in-band deploy steps. Error: Connection to agent failed: Failed to connect to the agent running on node b2c1e125-c2f6-4bd2-9b55-a410335dd27a for invoking command deploy.get_deploy_steps. Error: HTTPSConnectionPool(host='192.168.126.115', port=9999): Max22:10
spotz retries exceeded with url: /v1/commands/?wait=true&agent_token=EpKgQBNspmhL3yBNFrbKHoSTmSZTMmOlSKY0BGLapk0 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))22:10
*** k_mouza has quit IRC22:12
*** maelk has quit IRC22:14
*** fdegir has quit IRC22:14
*** fdegir has joined #openstack-ironic22:15
*** maelk has joined #openstack-ironic22:15
*** johnsom has joined #openstack-ironic22:30
*** andrein has joined #openstack-ironic22:30
*** rloo has joined #openstack-ironic22:30
*** JamesBenson has quit IRC22:41
TheJuliaso so so so so many review comments22:47
TheJuliaspotz: is certificate validation turned on?22:47
TheJuliavictoria ramdisk/ironic or off master?22:48
spotzthejulia - Um don't know? I did build the images per the instructions so I'll assume Victoria22:49
spotzIt was about then I realized things had changed a lot since my last install22:49
TheJuliahmmm22:50
TheJuliaspotz: can you crack open your ironic.conf and pull the [agent] section out for me?22:52
TheJuliaalso, can you get console on the failed machine?22:52
spotzTheJulia: Yep let me get it22:52
spotzdeploy_logs_collect=always22:55
spotzdeploy_logs_storage_backend=local22:55
spotzdeploy_logs_local_path=/var/log/ironic/deploy/22:55
spotzimage_download_source=swift22:56
spotzThat was the one in etc/ironic22:56
TheJuliaopenstack baremetal node show pls :)22:58
TheJuliaThis is a newer feature, so there may be a bug22:59
TheJuliawell, bugs and surely are, but the defaults it should just work22:59
*** akiselev has joined #openstack-ironic23:00
spotzTheJulia: And you know me I do weird stuff and run into things:)23:00
spotzhttp://paste.openstack.org/show/800817/23:00
TheJulialbragstad: still working on that spec :(23:01
spotzSO I found something?23:04
spotzWell a known something23:05
*** hoonetorg has quit IRC23:20
TheJuliaspotz: your conductor version and ironic version might be helpful. something doesn't add up here. :\23:28
TheJuliaerr23:29
TheJuliaconductor log and ironic version23:29
*** akiselev has quit IRC23:29
*** cmorey has quit IRC23:30
spotzthejulia - looks like the ironicclient is from delorean-component-clients   4.4.0-0.20201109164945.2b60460.el823:31
spotzI wonder if that's the issue?23:31
TheJuliaironic itself, not ironicclient23:31
spotzthejulia - that's the weird thing according to this http://paste.openstack.org/show/800818/23:33
spotzopenstack baremetal --version openstack 5.4.023:34
*** hoonetorg has joined #openstack-ironic23:34
TheJuliaHmmm23:34
TheJuliaheh23:34
TheJuliathats annoying23:34
TheJuliait is likely giving you the version for the openstack-cli23:34
TheJuliaspotz: yeah, we're going to need your whole conductor log23:36
TheJuliafor the ironic-conductor23:36
spotzTheJulia: Yeah I know you used to beable to use like nova --version to get a version23:36
spotzcat'ing it now to paste it23:36
*** anuradha1904 has quit IRC23:36
TheJuliaspotz: specifically it will help to see what hte configuration is at the very start23:37
*** stevebaker has quit IRC23:38
TheJuliaalso, anything on the agent side, did you manage to see anything on the failed deployed host or did it power off?23:38
spotzOk this is going to be way too much to paste. Would a head 100 and a tail 100 work or I can email23:40
spotzThejulia So no issues on the introspect once I fixed a mac address issue, the node set went fine. I was on the new overcloud_baremetal_deploy.yaml node provision command23:42
JayFI'd suggest a `grep -C5 [node uuid] conductor.log` or similar23:43
TheJuliaspotz: email please23:43
TheJuliaor that could work, but I also want to see the config so maybe head -n100023:43
TheJuliaas well23:43
spotzscping it off and then I'll email23:43
spotzok doing the head and the grep23:43
TheJuliainteresting...23:44
spotzHere's the head - http://paste.openstack.org/show/800819/23:46
spotzThis grep is HUGE23:53
TheJuliawheeee it rotated :\23:54

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!