Thursday, 2019-09-26

*** tbachman has joined #openstack-nova00:11
*** TxGirlGeek has quit IRC00:27
*** mlavalle has quit IRC00:35
*** spatel has joined #openstack-nova00:41
*** igordc has joined #openstack-nova00:44
*** nweinber_ has joined #openstack-nova01:15
*** LiangFang has joined #openstack-nova01:17
*** nweinber_ has quit IRC01:21
*** Sundar has joined #openstack-nova01:48
*** BjoernT has joined #openstack-nova02:04
*** boxiang has quit IRC02:06
*** boxiang has joined #openstack-nova02:06
openstackgerritBrin Zhang proposed openstack/nova master: Filter migrations by user_id/project_id  https://review.opendev.org/67424302:12
*** jdillaman has quit IRC02:14
*** brault has quit IRC02:19
*** larainema has joined #openstack-nova02:30
*** ricolin has joined #openstack-nova02:31
*** spatel has quit IRC02:46
*** cfriesen has quit IRC02:47
LiangFanghi, could anybody share how to add agenda for next weekly meeting? Is there any etherpad link? thanks02:51
*** dave-mccowan has joined #openstack-nova03:10
*** artom has quit IRC03:15
*** igordc has quit IRC03:18
*** igordc has joined #openstack-nova03:18
*** dave-mccowan has quit IRC03:47
*** tkajinam has quit IRC04:03
*** BjoernT has quit IRC04:07
*** markvoelker has quit IRC04:08
*** udesale has joined #openstack-nova04:12
*** BjoernT has joined #openstack-nova04:14
*** Sundar has quit IRC04:19
*** BjoernT has quit IRC04:29
*** BjoernT has joined #openstack-nova04:31
*** ratailor has joined #openstack-nova04:34
*** janki has joined #openstack-nova04:35
*** markvoelker has joined #openstack-nova05:05
*** dpawlik has joined #openstack-nova05:06
*** markvoelker has quit IRC05:11
*** BjoernT has quit IRC05:17
*** BjoernT_ has joined #openstack-nova05:17
*** BjoernT_ has quit IRC05:17
*** BjoernT has joined #openstack-nova05:18
*** BjoernT has quit IRC05:18
*** BjoernT_ has joined #openstack-nova05:18
*** BjoernT_ has quit IRC05:19
*** BjoernT_ has joined #openstack-nova05:23
*** BjoernT_ has quit IRC05:24
*** BjoernT has joined #openstack-nova05:25
*** BjoernT has quit IRC05:25
*** BjoernT_ has joined #openstack-nova05:26
*** BjoernT_ has quit IRC05:26
*** BjoernT_ has joined #openstack-nova05:28
*** BjoernT has joined #openstack-nova05:29
*** BjoernT has quit IRC05:30
*** BjoernT_ has joined #openstack-nova05:30
*** BjoernT_ has quit IRC05:31
*** BjoernT has joined #openstack-nova05:32
*** BjoernT has quit IRC05:32
*** BjoernT has joined #openstack-nova05:33
*** BjoernT has quit IRC05:33
*** BjoernT has joined #openstack-nova05:34
*** BjoernT has quit IRC05:34
*** BjoernT_ has joined #openstack-nova05:34
*** BjoernT_ has quit IRC05:35
*** BjoernT has joined #openstack-nova05:35
*** BjoernT has quit IRC05:37
gibiLiangFang: add your item to the agend be editing the wiki https://wiki.openstack.org/wiki/Meetings/Nova05:42
*** mkrai_ has joined #openstack-nova06:18
*** slaweq has joined #openstack-nova06:19
openstackgerritzhurong proposed openstack/nova master: [WIP] add cyborg tempest job  https://review.opendev.org/67099906:27
*** ociuhandu has joined #openstack-nova06:29
*** mkrai_ has quit IRC06:30
*** pcaruana has joined #openstack-nova06:30
*** jaosorior has quit IRC06:30
*** mkrai has joined #openstack-nova06:30
*** ociuhandu has quit IRC06:30
*** ociuhandu has joined #openstack-nova06:31
*** eharney has joined #openstack-nova06:35
*** ociuhandu has quit IRC06:37
*** ociuhandu has joined #openstack-nova06:38
*** rchurch has quit IRC06:43
*** ociuhandu has quit IRC06:49
*** damien_r has joined #openstack-nova06:53
*** igordc has quit IRC06:53
*** maciejjozefczyk has joined #openstack-nova06:57
*** ociuhandu has joined #openstack-nova07:00
*** udesale has quit IRC07:01
*** udesale has joined #openstack-nova07:02
*** jaosorior has joined #openstack-nova07:05
*** tesseract has joined #openstack-nova07:05
*** ralonsoh has joined #openstack-nova07:10
*** ociuhandu has quit IRC07:16
*** ivve has joined #openstack-nova07:19
*** ociuhandu has joined #openstack-nova07:22
*** avolkov has joined #openstack-nova07:23
*** ociuhandu has quit IRC07:25
*** ccamacho has joined #openstack-nova07:27
*** brault has joined #openstack-nova07:28
*** brault has quit IRC07:33
*** ociuhandu has joined #openstack-nova07:36
*** boxiang has quit IRC07:36
*** boxiang has joined #openstack-nova07:36
*** ttsiouts has joined #openstack-nova07:38
*** jangutter has joined #openstack-nova07:39
*** udesale has quit IRC07:44
*** udesale has joined #openstack-nova07:44
*** lbragstad has quit IRC07:47
*** lbragstad has joined #openstack-nova07:47
*** bnemec has quit IRC07:49
*** ttsiouts has quit IRC07:51
*** bnemec has joined #openstack-nova07:51
*** FlorianFa has joined #openstack-nova07:52
*** tesseract has quit IRC07:52
*** tesseract has joined #openstack-nova07:53
*** ttsiouts has joined #openstack-nova07:57
*** rchurch has joined #openstack-nova07:59
*** rpittau|afk is now known as rpittau08:12
kashyapozzzo: Hi, just saw a few mentions in the scrollback; /me goes to look for the salient bits08:13
kashyapozzzo: Okay, saw your question: if you're using (a) 'host-passthrough'; and (b) mismatched microcode b/n Compute nodes, then no, you will _not_ be able migrate an instance across the said nodes.08:14
kashyapozzzo: In my talk, I did mention the impact of microcode on live migration, when using 'host-passthrough':08:16
openstackgerritBalazs Gibizer proposed openstack/nova master: Functional reproduction for bug 1845291  https://review.opendev.org/68454508:17
openstackbug 1845291 in OpenStack Compute (nova) "migration is not recheduled if the server originally booted with --availability-zone <az>:<host>" [Medium,In progress] https://launchpad.net/bugs/1845291 - Assigned to Balazs Gibizer (balazs-gibizer)08:17
kashyapozzzo: efried: See slide-18: https://kashyapc.fedorapeople.org/Effective-Virtual-CPU-Configuration-in-Nova-Berlin2018.pdf08:17
openstackgerritBalazs Gibizer proposed openstack/nova master: Reset forced_destination before migration at a proper time  https://review.opendev.org/68454608:17
kashyapozzzo: To quote at note at the end of slide-18 (which talks about 'host-passthrough': "Along with identical CPUs, identical kernel and microcode are a must for VM live migration!"08:18
*** cdent has joined #openstack-nova08:19
kashyapozzzo: I'd also toss in, "identical BIOS settings" b/n the two COmpute nodes, too.08:20
*** yonglihe has joined #openstack-nova08:27
*** derekh has joined #openstack-nova08:37
*** lpetrut has joined #openstack-nova08:49
*** luksky has joined #openstack-nova09:00
*** brault has joined #openstack-nova09:02
*** rcernin has quit IRC09:08
*** markvoelker has joined #openstack-nova09:08
*** dtantsur|afk is now known as dtantsur09:10
*** ricolin has quit IRC09:11
*** markvoelker has quit IRC09:13
*** udesale has quit IRC09:13
*** udesale has joined #openstack-nova09:17
*** jaosorior has quit IRC09:17
*** victor286 has joined #openstack-nova09:30
*** LiangFang has quit IRC09:37
*** ociuhandu has quit IRC09:38
*** rcernin has joined #openstack-nova09:39
victor286hello,so many people!09:45
openstackgerritIvaylo Mitev proposed openstack/nova master: Fix incorrect usages of fake moref in VMware tests  https://review.opendev.org/68502009:55
kashyapozzzo: Also responded on e-mail09:56
*** xek has joined #openstack-nova09:57
*** tesseract has quit IRC09:59
*** tesseract has joined #openstack-nova10:00
*** tbachman has quit IRC10:00
openstackgerritIvaylo Mitev proposed openstack/nova master: Fix incorrect usages of fake moref in VMware tests  https://review.opendev.org/68502010:00
*** ociuhandu has joined #openstack-nova10:09
*** ttsiouts has quit IRC10:16
*** ttsiouts has joined #openstack-nova10:16
*** ttsiouts has quit IRC10:21
*** ociuhandu has quit IRC10:22
*** BjoernT has joined #openstack-nova10:23
*** ociuhandu has joined #openstack-nova10:30
*** pcaruana has quit IRC10:32
*** victor286 has quit IRC10:42
*** mkrai has quit IRC10:45
*** ociuhandu has quit IRC10:47
*** brault has quit IRC10:55
*** pcaruana has joined #openstack-nova10:58
*** jaosorior has joined #openstack-nova11:05
*** ociuhandu has joined #openstack-nova11:13
*** ociuhandu has quit IRC11:13
*** ociuhandu has joined #openstack-nova11:13
*** ociuhandu has quit IRC11:14
*** udesale has quit IRC11:14
*** markvoelker has joined #openstack-nova11:17
*** ociuhandu has joined #openstack-nova11:18
*** ociuhandu has quit IRC11:19
*** derekh has quit IRC11:20
*** ociuhandu has joined #openstack-nova11:21
*** markvoelker has quit IRC11:22
*** BjoernT has quit IRC11:23
*** mkrai has joined #openstack-nova11:26
openstackgerritMerged openstack/nova master: doc: attaching virtual persistent memory to guests  https://review.opendev.org/68030011:27
openstackgerritMerged openstack/nova master: Add a prelude for the Train release  https://review.opendev.org/68332711:28
*** ttsiouts has joined #openstack-nova11:32
openstackgerritSundar Nadathur proposed openstack/nova master: [WIP] add cyborg tempest job  https://review.opendev.org/67099911:41
*** maciejjozefczyk has quit IRC11:41
*** cdent has quit IRC11:45
openstackgerritBalazs Gibizer proposed openstack/nova master: Functional reproduction for bug 1845291  https://review.opendev.org/68454511:46
openstackbug 1845291 in OpenStack Compute (nova) "migration is not recheduled if the server originally booted with --availability-zone <az>:<host>" [Medium,In progress] https://launchpad.net/bugs/1845291 - Assigned to Balazs Gibizer (balazs-gibizer)11:46
openstackgerritBalazs Gibizer proposed openstack/nova master: Reset forced_destination before migration at a proper time  https://review.opendev.org/68454611:46
*** ratailor has quit IRC11:54
*** brault has joined #openstack-nova12:01
*** ociuhandu has quit IRC12:03
*** pcaruana has quit IRC12:05
*** janki has quit IRC12:05
*** ttsiouts has quit IRC12:06
*** tbachman has joined #openstack-nova12:07
*** markvoelker has joined #openstack-nova12:07
*** maciejjozefczyk has joined #openstack-nova12:10
*** ttsiouts has joined #openstack-nova12:14
*** mkrai has quit IRC12:18
*** derekh has joined #openstack-nova12:19
*** pcaruana has joined #openstack-nova12:19
*** ociuhandu has joined #openstack-nova12:19
*** tbachman has quit IRC12:22
*** tbachman_ has joined #openstack-nova12:22
*** derekh has quit IRC12:24
*** derekh has joined #openstack-nova12:25
*** ociuhandu has quit IRC12:30
*** ociuhandu has joined #openstack-nova12:36
*** mkrai has joined #openstack-nova12:39
*** cdent has joined #openstack-nova12:42
*** boxiang has quit IRC12:46
*** boxiang has joined #openstack-nova12:46
*** Garyx has quit IRC12:48
*** jmlowe has quit IRC12:49
*** ccamacho has quit IRC12:51
*** BjoernT has joined #openstack-nova12:51
*** Garyx has joined #openstack-nova12:52
*** ttsiouts has quit IRC13:00
*** ttsiouts has joined #openstack-nova13:01
*** tobberydberg_ is now known as tobberydberg13:01
*** ttsiouts has quit IRC13:01
*** ttsiouts has joined #openstack-nova13:01
*** ricolin has joined #openstack-nova13:05
*** mriedem has joined #openstack-nova13:06
*** dpawlik has quit IRC13:10
*** jmlowe has joined #openstack-nova13:10
mriedemefried: we might want to consider https://bugs.launchpad.net/nova/+bug/1845243 as train-rc-potential for rc2 since the regression was introduced in train, it was also backported and released on stein and rocy13:12
openstackLaunchpad bug 1845243 in OpenStack Compute (nova) stein "Nested 'path' query param in console URL breaks serialproxy" [High,Confirmed]13:12
mriedem*rocky13:12
mriedemso i think we have 2 potential bugs for rc2 https://bugs.launchpad.net/nova/+bugs?field.tag=train-rc-potential but i think we could still release rc1 today13:13
*** udesale has joined #openstack-nova13:16
mriedemi've updated the rc1 release patch https://review.opendev.org/#/c/684086/13:16
mriedemyaawang: were you able to test this fix for the numa live migration bug you reported? https://review.opendev.org/#/c/684409/13:17
*** nweinber_ has joined #openstack-nova13:26
*** dpawlik has joined #openstack-nova13:28
kashyapsean-k-mooney: efried This was my answer to ozzzo, in an "FAQ form" :D - https://kashyapc.fedorapeople.org/CPU-microcode-host-passthrough-and-live-migration.txt13:28
kashyapmriedem: ^  (But you might already know that answer.)13:28
openstackgerritMatt Riedemann proposed openstack/nova stable/stein: Ignore sqla-migrate inspect.getargspec deprecation warnings on py36  https://review.opendev.org/68507213:32
*** ccamacho has joined #openstack-nova13:33
*** ociuhandu has quit IRC13:35
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional recreate test for bug 1841481  https://review.opendev.org/68484013:39
openstackbug 1841481 in OpenStack Compute (nova) "Race during ironic re-balance corrupts local RT ProviderTree and compute_nodes cache" [Medium,In progress] https://launchpad.net/bugs/1841481 - Assigned to Matt Riedemann (mriedem)13:39
mriedemkashyap: lgtm, i remember you talking about microcode in your presentation13:39
openstackgerritMatt Riedemann proposed openstack/nova master: Clear RT.compute_nodes entry if re-balance update fails  https://review.opendev.org/68484913:39
mriedemif a FAQs style thing is useful for that docs page i'm fine with adding one13:39
mriedemwe recently did that for the bdms docs13:39
mriedemhttps://docs.openstack.org/nova/latest/user/block-device-mapping.html#faqs13:40
kashyapAh, interesting13:41
kashyapmriedem: Yeah, I could actually add some more - expand it to cover the other two modes13:41
kashyapBecause, this is going to become a question for more and more13:41
kashyapAs they slowly realize and start upgrading microcodes ...13:41
kashyap... "hey, live migration is broken, what now? :D"13:42
*** ociuhandu has joined #openstack-nova13:44
* kashyap makes a note to add it; once he gets this out: https://review.opendev.org/#/c/684375/213:44
kashyapmriedem: A quick q. on above -- since we're so close up to GA, is changing such defaults frowned upon?  (Have a gander at the commit message / release note)13:45
efriedmriedem: ack, are we waiting on anything else to cut rc1 today?13:45
kashyapIn short: it's a legit issue, also fixed in the kernel.  But relaxing the 'strict' default in Nova will definitely avoid surprises like: https://lkml.org/lkml/2018/7/24/152 ("VM boot failure on nodes not having DMA32 zone")13:47
* kashyap --> bbiab13:47
*** priteau has joined #openstack-nova13:50
mriedemkashyap: i can't really speak to that change but yeah that's pretty non-trivial for the day of RC113:50
mriedemefried: i don't think so13:50
efriedight, I'll go (update and) ack the release patch I guess.13:51
mriedemkashyap: also, "fixed in the kernel" might be something for rhosp but could be years for someone using suse or canonical openstack to pick up i'd think13:51
mriedemefried: i already did13:51
mriedemupdate htat is13:51
mriedemjust needs an ack13:51
efriedcoo13:51
efrieddone13:52
kashyapmriedem: I'm not one of those who think "downstream first" ;-) When I say "fixed in the kernel" - it's in 4.19, and many of those said distros will backport it13:52
kashyapIt's a super small, isloated change13:52
*** mkrai has quit IRC13:54
kashyapmriedem: Okay, ACK - for the RC1.  But FWIW, the change is quite _safe_ and self-contained: and makes Nova instances work in more NUMA setups13:54
efriedkashyap: If you wanted to be super safe, you could make a [workaround] conf opt, defaulting to "prefer", to allow consumers to revert to "strict" if they want.13:55
*** priteau has quit IRC13:55
efrieddeprecate it immediately (assuming we don't want to continue to allow it) and remove it in U.13:56
kashyap(Also, to preempt a thought: no, it's not "urgent for RHOS".  I just think since Train also has NUMA-aware live migration; this change only helps.  I did a crap load background reading on this, to understand the issue)13:56
kashyapefried: Ah, good point on 'workarounds'13:56
*** BjoernT has quit IRC13:57
kashyapI'll mull on it today; stephenfin - might want to have a look at the above, it's in your wheelhouse - https://review.opendev.org/#/c/684375/ ("libvirt: Use the `preferred` memory allocation mode for NUMA13:57
kashyap")13:58
kashyapefried: Thanks for the input.13:58
efriedyahyoubetcha13:58
* kashyap --> meeting13:58
efriedkashyap: in any case I would think you would want a nova lp bug for that guy.13:58
efried(if that hasn't already been said)13:59
kashyapefried: Yes, yes.  Already said; will be filing13:59
mriedem+1 on the workarounds idea,13:59
kashyapI have a v2 sitting; and I wrote a long explanatory comment on _why_ we're doing that change13:59
mriedemthat also allows people to opt out if they dont' have the kernel fix13:59
*** BjoernT has joined #openstack-nova14:00
*** rcernin has quit IRC14:06
dansmithefried: kashyap: I know we've strayed a bit, but I always wanted the workarounds to be booleans that all default to False, opting into different behavior, not multi-value tunables14:07
*** gbarros has joined #openstack-nova14:07
dansmithefried: kashyap: so enable_old_memory_somethingorother=True if they need the old behavior or whatever it is14:08
dansmithsetting some regular-looking conf tunable to some value is less like a workaround and more like ... config14:08
efriedYeah, I'm not totally convinced this should be a [workaround] in the first place -- perhaps it should be an actual conf opt.14:09
dansmithif it's actually a conf thing (setting to some value) then not a workaround, IMHO14:10
dansmithbut if it's "I don't have a modified kernel to fix X" then that can be true/false14:10
efriedEither way, the opt is for someone who understands the issue, and (IMO) it's an unnecessary extra step making them understand how True/False maps to "prefer"/"strict". But meh, not going to put in my mouthguard for this one.14:11
efriedI don't understand enough about the issue to know whether there's a chance this will need to be worked around at all / some nontrivial percentage of the time or for how long.14:11
dansmithsounds like it's config and not really a workaround then, which is fine14:11
*** mkrai has joined #openstack-nova14:12
bauzasgibi: /me scratches his head when he sees a lot of good helper methods https://github.com/openstack/nova/blob/master/nova/tests/functional/integrated_helpers.py#L582 and the likes14:14
bauzasgibi: I'll passing a new revision today or tomorrow for https://review.opendev.org/#/c/670112 but please tell me whether we could use the above $14:15
bauzas^14:15
kashyapdansmith: Hiya, just back.  Reading your / Eric's remarks14:31
kashyapdansmith: Yeah, I agree with your intention on a workaroud being: "a bool that makes it possible to opt-in to behaviour X".14:32
kashyapefried: And yes, I'm still not convinced either to have a workaroud yet; however, making _everything_ configurable is not desirable either.  "Death by a thousand config cuts"14:33
*** mkrai has quit IRC14:35
gibibauzas: I think you can use assertRequestMatchesUsage and the rest of the helpers. There are some limitations documented there about the structure though14:36
kashyapefried: On the issue, the tl;dr is: today, if you boot a Nova instance on a NUMA host without a "DMA32" zone (one of the zones into which the kernel divides the memory) on the NUMA node, then your instance hard-fails to boot.14:36
kashyap(And you can detect this "DMA32" thing by running `grep DMA /proc/zoneinfo` on the host.)14:36
*** ivve has quit IRC14:36
*** mkrai has joined #openstack-nova14:37
kashyapefried: And no, please remove that mouthguard - you _do_ make a good point: for someone to understand the mapping of bool to 'preferred'/'strict', it's needless gory details14:37
kashyapI'll mull on it more and comment on the change.14:38
kashyapThanks for the input, both.14:38
* kashyap wonders how many workarounds today are multi-value "convenience" options and less of True/False booleans14:39
*** victor286 has joined #openstack-nova14:40
mriedemoh donnyd donnyd donnyd14:42
mriedemyou've opened a can of worms14:42
donnydIs it a good or bad can?14:42
donnydlike fresh worms or worms that have been sitting in the sun for 3 weeks14:42
mriedemmoldy worms14:44
mriedemi, of course, can't help but take that bait14:44
donnydLOL14:44
donnydWell they are just observations.. not saying anything is right, wrong.. indifferent..14:45
mriedemit's fun to see how the sausage gets made though huh14:46
donnydits quite interesting from my perspective - i have been building these things for quite a long time14:47
mriedemfwiw every closed source project i worked on before openstack was also a clusterf*ck14:47
mriedemcome release time anyway14:47
donnydopenstack isn't a charlie foxtrot.14:47
donnydI don't think that is the case at all...14:47
mriedemthe thing i mostly enjoy is the foundation marketing that says "20 release ON TIME"14:48
mriedem*releases14:48
donnydI just think humans have a tendency to delay until they cannot anymore14:48
mriedemyup, i noted that in my reply14:48
mriedemand that's our MO here as well14:48
mriedemand the squeeky wheel gets the grease...14:49
donnydthat is a true story in literally all things14:49
mriedemi've left some squeeky hinges in my house go ungreased out of spite14:51
donnydI also think nobody consumes the milestones because they are perceived to be "beta"14:51
mriedem"gdi i *just* greased you"14:51
donnydand maybe changing around the perception would increase the consumption14:52
mriedemat this point in the game i don't think that's likely14:52
mriedemif anything,14:52
mriedempeople have wanted to consume at a much slower pace as time as gone on14:52
mriedemhence all the demands for LTS branches and stuff14:53
donnydhence the 9 month major cycle14:53
donnydor even 12 month14:53
*** gyee has joined #openstack-nova14:53
mriedemblech14:53
mriedemwe'll just wait until the last minute to cram even less stuff in :)14:53
mriedemand,14:53
mriedemit will be double the shit that someone has to consumer and grok from the release notes when they do14:54
*** mlavalle has joined #openstack-nova14:54
mriedem*consume14:54
donnydhow is that different from say an ubuntu or centos release though?14:54
donnydcentos/rhel14:55
donnydAlso thank you for replying14:56
mriedemi can't say how those distro dev cycles are different,14:56
mriedembut i imagine there much more strictly controlled14:56
mriedem*they're14:56
mriedemand when rhel i'm paying a support license so i expect more rigor before the thing is released than just "here it is, here are the release notes, have fun"14:57
mriedem*with rhel14:57
mriedemman i can't type today14:57
*** maciejjozefczyk has quit IRC14:57
* kashyap makes a pedantic nit: s/license/subscription/ - it matters because: a license is (usually?) for a single release; RHEL sub gets your _future_ releases, too14:58
mriedemi'm also not sure how moving from 6 to 12 months cycles would go over with someone like vexxhost that does actually do proper CI/CD and doesn't wait 18 months before upgrading14:58
* kashyap reads the full scrollback, before he opens mouth further14:58
mriedemkashyap: sure, whatever - "pay for play"14:58
kashyapI need to first load up on the topic being discussed.  (I've got the -discuss list open, with Donny's email)14:59
openstackgerritMatt Riedemann proposed openstack/nova stable/stein: Handle legacy request spec dict in ComputeTaskManager._cold_migrate  https://review.opendev.org/68440715:00
*** jmlowe has quit IRC15:01
kashyapdonnyd: That's indeed a barrel of worms; it was discussed to nauseating levels.  See this epic thread from 2015:15:01
kashyapdonnyd: "Re-evaluating the suitability of the 6 month release cycle" -- https://openstack.nimeyo.com/34284/openstack-dev-evaluating-suitability-month-release-cycle15:02
kashyapdonnyd: Before you click that, go fill up your giant mug of tea, I say.15:02
donnydmriedem: but vexxhost could surely make use of more often . releases15:02
kashyapIt's a long and worthwhile read.  (Especially "looking back")15:02
donnydwell to be fair... that was 4 years ago15:03
kashyapdonnyd: To get the tl;dr of suggestion, see "The modest proposal"15:03
kashyap[quote]15:03
kashyapBased on these observations I would suggest that OpenStack aim to15:03
kashyapswitch to a development cycle that is exactly 2 months long.15:03
kashyapie do 6 releases a year.15:03
kashyapThis will have a number of consequences which I'd expect to be15:03
kashyapbeneficial to the project on balance.15:03
kashyap[/quote]15:03
kashyapdonnyd: 4 years ago, but many things wrote there apply perfectly well today.15:04
*** TxGirlGeek has joined #openstack-nova15:05
donnydbut 4 years ago our community was probably 2x the size it is now wasn't it? (i wasn't here, so i can't actually quantify that)15:06
mriedemlyarwood: we have a few open stable/stein changes that we could get in before the next release but i think the biggest ticket one ends at https://review.opendev.org/#/c/684407/ since that resolves an upgrade issue introduced in stien15:06
mriedemdansmith: ^15:06
mriedemdonnyd: yes much bigger and much more diverse15:07
kashyapdonnyd: It _was_ larger; but shortening the cycle length applies even today.15:07
kashyapdonnyd: It reduces the: "hey, our distro is gonna be based on X upstream release, we _must_ shove our feature upstream in time!"15:07
kashyaps/distro/"long-term OpenStack distro"/15:07
kashyap(Among other things)15:07
donnydfor the record.. I am happy either way15:08
*** cfriesen has joined #openstack-nova15:11
*** tbachman_ is now known as tbachman15:12
kashyapdonnyd: Nod; but yes, as both of you said, it's the "human nature" element.  And "planning is guessing".  Go a "six-month plan"?  Call it a "six-month guess".15:13
kashyap(https://m.signalvnoise.com/planning-is-guessing/)15:13
donnydright... "everyone has a plan until they get punched in the mouth"15:14
donnyd- mike tyson15:14
donnydso wouldn't the message there be create smaller plans??15:15
openstackgerritWalter A. Boring IV (hemna) proposed openstack/nova stable/pike: WIP: Avoid redundant initialize_connection on source post live migration  https://review.opendev.org/68300815:16
*** damien_r has quit IRC15:16
mriedemdonnyd: i think that's efried's plan for ussuri15:17
mriedemi.e. only approve blueprints for things that were previously approved and not yet complete15:17
kashyapdonnyd: Yeah, smaller plans are fine - focus more on the unsexy, "boring" stability (don't-fall-apart-if-you-sneeze) and less of "features"15:18
kashyaps/fine/preferred/ (ha, see what I did there :D)15:18
*** mvkr has quit IRC15:20
kashyapdonnyd: Fun fact I just learnt: Tyson actually _lost_ that match, after that quote (which he targeted at his oponnent, when a journalist asked "if you're worried about his plan") :D15:21
donnydWell the cycle time would just be broken up differently.. I wouldn't think the major release cycle time would be able to be shortened... if anything lengthened ....  So the idea is kinda like slow train fast train15:22
kashyap(So the plan _did_ work :D)15:22
donnydkashyap: LOL15:22
kashyapdonnyd: So, yeah, moral: "punchy quotes don't cook rice [or sustain punches / win matches]"15:23
donnydSo if someone wanted to ride the fast train, they would just use the minor release cycle and those who want that "LTSish" release could just use majors15:23
donnydI am still reading the thread from 201515:25
*** maciejjozefczyk has joined #openstack-nova15:28
*** mvkr has joined #openstack-nova15:33
efriedgibi: you still around?15:38
gibiefried: yes15:38
efriedgibi: would you mind injecting some more description into the commit message and test comments/docstring for https://review.opendev.org/#/c/684545/ ?15:39
efriedAs it is I have to go read the bug report to understand what's going on.15:39
gibiefried: you mean describe the bug a bit more in the commit message and the docstring?15:40
efriedYeah. Actually the commit message is probably fine; I just misread it. But in the test case, an up-front docstring/comment describing what we're testing for would be nice.15:41
gibiefried: sure15:41
efriedyou could do it in a fup if you prefer, I can send both of these.15:41
gibiefried: let me add that info quickly to the func reproduce patch15:42
gibibefore I leave15:42
*** dpawlik has quit IRC15:43
*** dpawlik has joined #openstack-nova15:43
efriedgibi: is there a typo in the test name? (just posted comments)15:45
dansmithmriedem: on this: https://review.opendev.org/#/c/684407/315:46
dansmithmriedem: is that really necessary for stein, which should all be sending a later compute rpc version15:46
dansmith?15:46
*** alex_xu has quit IRC15:46
dansmithokay, so there's some conductor interplay there, I guess, but.. the point in backporting to stein is for what, to make stein better handle upgrade scenarios when running split rocky/stein ?15:47
mriedemi'm not sure what you mean.15:47
mriedemyes15:47
mriedemif you're computes are pinned to rocky it's a failure15:47
mriedem*your15:47
mriedemon resize/cold migrate reschedule i mean15:48
*** priteau has joined #openstack-nova15:48
mriedemcompute will pass a request spec legacy dict back up to conductor which won't handle it as a dict and blow up trying to set the flavor attribute15:48
gibiefried: ack15:49
*** gbarros has quit IRC15:49
dansmithmriedem: but case #2 in your commit message wouldn't affect rocky/stein because it requires pinning to 5.0 which is older than even rocky, right?15:49
dansmithI mean, it could if people had their rpcs pinned to pike, but in reality, that's probably not a likely case15:50
mriedemno,15:54
mriedem5.0 is rocky15:54
mriedem5.1 was i think the only compute rpc api bump in stein15:54
*** larainema has quit IRC15:54
openstackgerritBalazs Gibizer proposed openstack/nova master: Functional reproduction for bug 1845291  https://review.opendev.org/68454515:54
openstackbug 1845291 in OpenStack Compute (nova) "migration is not recheduled if the server originally booted with --availability-zone <az>:<host>" [Medium,In progress] https://launchpad.net/bugs/1845291 - Assigned to Balazs Gibizer (balazs-gibizer)15:54
openstackgerritBalazs Gibizer proposed openstack/nova master: Reset forced_destination before migration at a proper time  https://review.opendev.org/68454615:54
*** dtantsur is now known as dtantsur|afk15:54
mriedemhttps://opendev.org/openstack/nova/src/tag/19.0.0/nova/compute/rpcapi.py#L36515:54
*** eharney has quit IRC15:54
gibiefried: respun https://review.opendev.org/#/c/68454515:55
efried...15:55
mriedemif conductor is pinned less than 1.13, conductor's compute task manager won't even get a request spec15:55
dansmithmriedem: oh, you're right, I read the "Version 5.0 is .... Pike compat" in the comment15:55
dansmithyeah, but conductor pinning isn't really a useful thing15:55
dansmithwhich is why you say "technically.." I think15:55
mriedemand because it's so f'ing old15:55
dansmithbut fair enough, I just had it in my head that 5.0 was much older, but I since we didn't bump for any reason in rocky, that makes sense now15:56
efriedgibi: would it be accurate to say "Ensure that re-schedule is possible on migration even if the server is originally booted with forced host."15:56
*** pcaruana has quit IRC15:57
mriedemefried: gibi: remember i did use call_count recently for something like https://review.opendev.org/#/c/684545/3/nova/tests/functional/regressions/test_bug_1845291.py@40 and it was a big hullabaloo15:57
mriedemartom got all stinky about it15:57
*** rpittau is now known as rpittau|afk15:58
gibiefried:yes, your sentence sounds better than mine15:58
openstackgerritEric Fried proposed openstack/nova master: Functional reproduction for bug 1845291  https://review.opendev.org/68454515:58
openstackbug 1845291 in OpenStack Compute (nova) "migration is not recheduled if the server originally booted with --availability-zone <az>:<host>" [Medium,In progress] https://launchpad.net/bugs/1845291 - Assigned to Balazs Gibizer (balazs-gibizer)15:58
openstackgerritEric Fried proposed openstack/nova master: Reset forced_destination before migration at a proper time  https://review.opendev.org/68454615:59
efriedgibi: okay, tweaked a little bit and approved. Thanks.15:59
* efried ==> meeting15:59
gibiefried: thanks15:59
gibimriedem: sorry I did not remember. How did you managed to do a conditional in the fake based on the call count?16:00
mriedemi'm trying to find it16:00
mriedemhttps://review.opendev.org/#/c/682140/1/nova/tests/functional/test_servers.py@681716:00
openstackgerritMerged openstack/nova-specs master: Amend "Configure max number of volumes to attach" spec  https://review.opendev.org/68213616:03
gibimriedem: thanks. now I remember. it is ugly scoping in any case16:04
mriedemhey,16:04
mriedemone man's trash is another man's treasure16:04
*** qqmber has joined #openstack-nova16:05
gibimriedem: to be correct I don't like either of the solutions due to the scoping trick. nothing personal :)16:06
* mriedem removes gibi's name from his list16:07
qqmberHi.. I'm usong Rocky and for the first time I didn't assign a floating IP to an instance... I enter thru another instance (that has floating IP). When I run "yum upgrade" it takes forever and I see a lot of hangs and timeouts... could be related to not having a floating IP assigned? (even I see it has a floating IP anyways)16:07
qqmberI see stuff like this: Sep 26 16:04:43 es-3 dbus[579]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out16:08
* gibi is wondering what mriedem's list means16:10
*** maciejjozefczyk has quit IRC16:10
donnydfloating ips are at the most basic just an inbound NAT done at the network node. Outbound traffic should follow the same flow with or without a floating ip16:10
donnydI would also ask in neutron16:11
donnydqqmber: also i would be looking at a dns issue16:12
donnydtry something like ping -4 google.com and then ping -4 172.217.13.23816:13
donnydif the second command has no issue or delay... you have a dns problem16:13
donnydlikely i should say16:13
*** victor286 has quit IRC16:14
qqmberdonnyd: I'm running the same (yum upgrade) on two instances.. both in the same host.. one with floating IP, one without.. the one without floating IP has the hanging problem... weird...16:14
*** igordc has joined #openstack-nova16:14
*** ttsiouts has quit IRC16:17
*** gbarros has joined #openstack-nova16:17
mriedemnot really a nova development question either way16:17
donnydqqmber: maybe the neutron channel would have more info on ways to t/s16:17
donnydor openstack-operators16:17
qqmberok, thanks!16:18
*** tesseract has quit IRC16:19
*** cdent has quit IRC16:22
*** BjoernT has quit IRC16:26
*** BjoernT has joined #openstack-nova16:32
*** markvoelker has quit IRC16:33
*** mkrai has quit IRC16:34
*** priteau has quit IRC16:37
*** luksky has quit IRC16:41
*** markvoelker has joined #openstack-nova16:41
*** yaawang has quit IRC16:41
*** igordc has quit IRC16:43
*** yaawang has joined #openstack-nova16:43
*** udesale has quit IRC16:51
*** lpetrut has quit IRC16:53
*** dklyle has quit IRC17:01
*** derekh has quit IRC17:01
*** ricolin has quit IRC17:01
*** dklyle has joined #openstack-nova17:01
*** itlinux has quit IRC17:04
*** henriqueof has joined #openstack-nova17:09
*** henriqueof1 has quit IRC17:09
*** manjeets has joined #openstack-nova17:13
*** gbarros has quit IRC17:14
*** tbachman has quit IRC17:15
*** artom has joined #openstack-nova17:19
*** ralonsoh has quit IRC17:27
*** igordc has joined #openstack-nova17:28
*** ociuhandu has quit IRC17:29
*** brault has quit IRC17:32
*** gbarros has joined #openstack-nova17:35
*** jmlowe has joined #openstack-nova17:37
*** TxGirlGeek has quit IRC17:40
*** TxGirlGeek has joined #openstack-nova17:44
melwittmriedem: fyi I'm looking into https://bugs.launchpad.net/nova/+bug/184524317:49
openstackLaunchpad bug 1845243 in OpenStack Compute (nova) stein "Nested 'path' query param in console URL breaks serialproxy" [High,Confirmed]17:49
*** TxGirlGeek has quit IRC17:49
*** ociuhandu has joined #openstack-nova17:52
mriedemack, cool17:52
*** brault has joined #openstack-nova17:53
*** brault has quit IRC17:54
*** dpawlik has quit IRC17:56
*** ociuhandu has quit IRC17:57
*** TxGirlGeek has joined #openstack-nova18:02
*** pcaruana has joined #openstack-nova18:04
*** factor has joined #openstack-nova18:05
*** maciejjozefczyk has joined #openstack-nova18:13
*** avolkov has quit IRC18:23
*** igordc has quit IRC18:29
*** mrch_ has quit IRC18:30
*** mrch_ has joined #openstack-nova18:31
*** igordc has joined #openstack-nova18:32
*** pcaruana has quit IRC18:36
*** luksky has joined #openstack-nova18:47
*** nweinber_ has quit IRC18:54
*** BjoernT has quit IRC19:05
*** BjoernT has joined #openstack-nova19:07
*** ivve has joined #openstack-nova19:14
*** jmlowe has quit IRC19:19
*** dave-mccowan has joined #openstack-nova19:23
*** gbarros has quit IRC19:24
*** maciejjozefczyk has quit IRC19:24
*** crazik has joined #openstack-nova19:25
*** crazik has left #openstack-nova19:26
*** dave-mccowan has quit IRC19:27
*** dave-mccowan has joined #openstack-nova19:29
*** ociuhandu has joined #openstack-nova19:30
openstackgerritDustin Cowles proposed openstack/nova-specs master: Spec: Use OpenStack SDK in Nova (Ussuri)  https://review.opendev.org/68515419:32
*** ociuhandu has quit IRC19:36
*** cfriesen has quit IRC19:38
*** lpetrut has joined #openstack-nova19:47
*** qqmber has quit IRC19:52
*** lpetrut has quit IRC19:58
*** TxGirlGeek has quit IRC19:59
*** TxGirlGeek has joined #openstack-nova20:01
*** jmlowe has joined #openstack-nova20:02
*** maciejjozefczyk has joined #openstack-nova20:02
*** BjoernT_ has joined #openstack-nova20:04
*** TxGirlGeek has quit IRC20:05
*** BjoernT has quit IRC20:06
*** maciejjozefczyk has quit IRC20:07
*** brault has joined #openstack-nova20:46
*** takashin has joined #openstack-nova20:47
openstackgerritMerged openstack/nova-specs master: Spec: Use OpenStack SDK in Nova (Ussuri)  https://review.opendev.org/68515420:49
*** zigo_ is now known as zigo20:49
*** brault has quit IRC20:50
efriedmriedem: wanna update https://etherpad.openstack.org/p/nova-train-release-todo to track RC2 candidates?20:50
*** xek has quit IRC20:51
mriedemmeaning? https://bugs.launchpad.net/nova/+bugs?field.tag=train-rc-potential20:51
efriedmaybe just s/RC1/RC2/ on L13, but I dunno, swhy I'm asking20:53
mriedemi'll let you drive20:53
*** ccamacho has quit IRC20:53
*** TxGirlGeek has joined #openstack-nova20:58
*** xek has joined #openstack-nova21:17
*** luksky has quit IRC21:25
*** brault has joined #openstack-nova21:28
*** brault has quit IRC21:32
*** markvoelker has quit IRC21:34
openstackgerritMatt Riedemann proposed openstack/nova master: Migrate old style volume attachments on nova-compute startup  https://review.opendev.org/54913021:34
melwittmriedem: commented on https://bugs.launchpad.net/nova/+bug/1845243 and would like a sanity check on the fix approach. dansmith too since he was involved in the discussion of the original change. cc efried21:37
openstackLaunchpad bug 1845243 in OpenStack Compute (nova) stein "Nested 'path' query param in console URL breaks serialproxy" [High,Confirmed]21:37
*** slaweq has quit IRC21:37
mriedemcommented21:41
melwittmriedem: thanks. I think it will be simple. kicking myself for not realizing the 'path' thing was novnc specialness21:43
melwittconsoles: 2 me: 021:43
mriedemi wouldn't feel too bad about it21:43
mriedemwe have no integratoin testing for ironic serial consoles so...21:44
mriedemthings are going to break21:44
melwittyeah. just having a dagnabbit moment21:44
melwittI wonder if it might be cleanest to do a full revert and then follow with a replacement change. I'll try that first21:45
melwittbecause the original change was a bunch of every console type's url changing. is just noise once it gets fixed21:46
mriedemyeah maybe21:51
mriedemdo you know how to / have the setup for testing the ironic serial console thing to see if it's fixed?21:51
melwittunfortunately no. I think any other console type would break too (like spice). so I guess that implies we don't have tempest tests for those either? I'll do some looking around to see how we can verify noVNC + other consoles21:52
openstackgerritEric Fried proposed openstack/nova master: Repro bug 1845530: versioned discovery is authed  https://review.opendev.org/68518021:53
openstackbug 1845530 in OpenStack Compute (nova) "Versioned discovery endpoint should not require authentication" [Undecided,New] https://launchpad.net/bugs/184553021:53
openstackgerritEric Fried proposed openstack/nova master: WIP: Allow versioned discovery unauthenticated  https://review.opendev.org/68518121:53
efriedcdent: Help!  ^21:53
*** markvoelker has joined #openstack-nova21:57
*** brault has joined #openstack-nova21:58
*** BjoernT_ has quit IRC22:01
*** brault has quit IRC22:13
*** efried has quit IRC22:15
*** efried has joined #openstack-nova22:18
*** markvoelker has quit IRC22:27
artommriedem, actually, for that NUMA bug, I can probably reproduce in a regression func test (re: meeting ping)22:33
artomI've been rebasing and splitting into smaller patches the main NUMA func test patch, it's taking forever because it's a PITA and I'm procrastinating22:34
*** ivve has quit IRC22:50
*** brault has joined #openstack-nova22:55
*** tkajinam has joined #openstack-nova22:57
*** brault has quit IRC23:00
*** brault has joined #openstack-nova23:16
*** brault has quit IRC23:20
*** rcernin has joined #openstack-nova23:24
*** mriedem has quit IRC23:25
*** mlavalle has quit IRC23:30
openstackgerritMerged openstack/nova master: Functional reproduction for bug 1845291  https://review.opendev.org/68454523:31
openstackbug 1845291 in OpenStack Compute (nova) "migration is not recheduled if the server originally booted with --availability-zone <az>:<host>" [Medium,In progress] https://launchpad.net/bugs/1845291 - Assigned to Balazs Gibizer (balazs-gibizer)23:31
*** efried has quit IRC23:34
*** efried has joined #openstack-nova23:35
*** boxiang has quit IRC23:59
*** TxGirlGeek has quit IRC23:59
*** boxiang has joined #openstack-nova23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!