Friday, 2020-10-30

« Thursday, 2020-10-29 Index Saturday, 2020-10-31 »
*** macz_ has quit IRC00:12
*** tosky has quit IRC00:27
*** spatel has quit IRC00:32
*** sapd1_x has joined #openstack-nova00:59
*** zzzeek has quit IRC01:00
*** zzzeek has joined #openstack-nova01:01
*** dklyle has quit IRC01:10
*** dklyle has joined #openstack-nova01:12
*** martinkennelly has quit IRC01:22
*** sapd1_x has quit IRC01:25
*** Liang__ has joined #openstack-nova01:31
*** k_mouza has joined #openstack-nova01:43
*** k_mouza has quit IRC01:48
*** hemanth_n has quit IRC01:50
*** PrinzElvis has quit IRC01:50
*** csatari has quit IRC01:50
*** dviroel has quit IRC01:50
*** csatari has joined #openstack-nova01:51
*** pas-ha has quit IRC01:51
*** PrinzElvis has joined #openstack-nova01:51
*** hemanth_n has joined #openstack-nova01:51
*** dviroel has joined #openstack-nova01:51
*** pas-ha has joined #openstack-nova01:52
*** knikolla has quit IRC01:52
*** knikolla has joined #openstack-nova01:53
*** LinPeiWen has joined #openstack-nova01:58
*** gyee has quit IRC02:10
*** dave-mccowan has joined #openstack-nova02:28
*** dklyle has quit IRC02:32
*** xinranwang has joined #openstack-nova02:42
*** rcernin has quit IRC02:43
openstackgerritShi Yan proposed openstack/nova master: Remove unused bindir config option  https://review.opendev.org/76033103:05
*** rcernin has joined #openstack-nova03:12
*** psachin has joined #openstack-nova03:19
*** rcernin has quit IRC03:37
openstackgerritwu.shiming proposed openstack/os-vif master: Remove the unused coding style modules  https://review.opendev.org/76049103:52
*** rcernin has joined #openstack-nova04:01
*** sapd1_x has joined #openstack-nova04:01
*** rcernin has quit IRC04:10
*** rcernin has joined #openstack-nova04:10
*** irclogbot_2 has quit IRC04:27
*** lemko6 has joined #openstack-nova04:27
*** lemko has quit IRC04:28
*** lemko6 is now known as lemko04:28
*** irclogbot_1 has joined #openstack-nova04:32
openstackgerritShi Yan proposed openstack/nova master: Remove unused bindir config option  https://review.opendev.org/76033104:33
*** xinranwang has quit IRC04:50
*** brinzhang_ has quit IRC04:52
*** sapd1_x has quit IRC05:00
*** ratailor has joined #openstack-nova05:28
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-nova05:33
*** eharney has quit IRC05:49
*** vishalmanchanda has joined #openstack-nova05:51
*** legochen has quit IRC05:52
*** LinPeiWen has quit IRC05:55
*** eharney has joined #openstack-nova06:01
*** gregwork has quit IRC06:05
*** LinPeiWen has joined #openstack-nova06:07
*** sapd1_x has joined #openstack-nova06:18
*** rcernin has quit IRC06:39
*** sapd1_x has quit IRC06:48
*** sapd1_x has joined #openstack-nova07:22
*** lpetrut has joined #openstack-nova07:27
openstackgerritXinran WANG proposed openstack/nova-specs master: SRIOV Enabled Nic Support Specification  https://review.opendev.org/74278507:30
*** ralonsoh has joined #openstack-nova07:39
*** tesseract has joined #openstack-nova07:54
bauzasgood morning Nova08:00
*** slaweq has joined #openstack-nova08:00
gibigood morning08:10
*** rpittau|afk is now known as rpittau08:15
*** rcernin has joined #openstack-nova08:27
*** rcernin has quit IRC08:45
*** legochen has joined #openstack-nova08:46
*** tosky has joined #openstack-nova08:50
*** Liang__ has quit IRC08:57
*** Liang__ has joined #openstack-nova08:58
*** ociuhandu has joined #openstack-nova09:00
*** rcernin has joined #openstack-nova09:03
*** sapd1_x has quit IRC09:08
*** dtantsur|afk is now known as dtantsur09:08
openstackgerritBalazs Gibizer proposed openstack/nova master: Prevent starting services with older than N-1 computes  https://review.opendev.org/73848209:12
*** martinkennelly has joined #openstack-nova09:15
*** rcernin has quit IRC09:17
gibidansmith: updated for W ^^09:17
*** Liang__ has quit IRC09:23
*** Liang__ has joined #openstack-nova09:28
*** derekh has joined #openstack-nova09:30
*** k_mouza has joined #openstack-nova09:33
openstackgerritBalazs Gibizer proposed openstack/nova master: Add upgrade check about old computes  https://review.opendev.org/76052009:44
gibidansmith: and here is the related upgrade check ^09:46
*** Liang__ has quit IRC09:48
*** dtantsur is now known as dtantsur|afk10:06
*** dtantsur|afk is now known as dtantsur|brb10:06
bauzaslyarwood: interesting, got a publickey perm issue when trying to upload to gerrit after upgrading10:20
bauzaslyarwood: to F33, I mean10:21
bauzasI guess my pubkey was revoked because of the exposure10:21
* bauzas goes looking at his gerrit preferences10:21
bauzasoh my fingerprint probably changeed10:22
openstackgerritMarcin Juszkiewicz proposed openstack/nova master: libvirt: check for AMD SEV only on x86-64  https://review.opendev.org/71442510:22
*** k_mouza has quit IRC10:24
bauzasdebug1: Offering public key: /home/sbauza/.ssh/id_rsa RSA SHA256:<something> agent10:25
bauzasdebug1: send_pubkey_test: no mutual signature algorithm10:25
bauzasmmmmm10:25
bauzaslooks like it's a known issue https://twitter.com/dmsimard/status/103735474195999129610:29
bauzasthat's going to be fun10:29
bauzashuzzah https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/179096310:31
openstackLaunchpad bug 1790963 in openssh (Fedora) "Unable to connect with openssh 7.8 client and certificates" [Undecided,Confirmed]10:31
bauzasmy openssh client is more recent tho (8.4p1)10:32
*** k_mouza has joined #openstack-nova10:34
*** k_mouza has quit IRC10:35
*** k_mouza has joined #openstack-nova10:35
*** iurygregory has quit IRC10:50
*** iurygregory has joined #openstack-nova10:51
*** tbachman has quit IRC10:56
*** dtantsur|brb is now known as dtantsur11:01
*** zzzeek has quit IRC11:15
*** zzzeek has joined #openstack-nova11:16
bauzashmm, better explanation of the issue https://www.reddit.com/r/Fedora/comments/jh9iyi/f33_openssh_no_mutual_signature_algorithm/11:18
bauzaslyarwood: ^11:19
bauzasokay, got the workaround by setting the ssh-rsa keytype with PubkeyAcceptedKeyTypes11:23
bauzaslyarwood: http://paste.openstack.org/show/799536/11:25
bauzashah https://bugzilla.redhat.com/show_bug.cgi?id=188492011:27
openstackbugzilla.redhat.com bug 1884920 in openssh "Cannot ssh into CentOS 6 using ssh key authentication" [Low,Closed: notabug] - Assigned to jjelen11:27
bauzaswhile the move to remove SHA1 hashes is certainly understandable, I think there is a big user impact that I would have preffered to see better in the relnotes11:29
*** ociuhandu_ has joined #openstack-nova11:47
*** ociuhandu has quit IRC11:50
*** legochen has quit IRC11:51
*** ociuhandu_ has quit IRC11:57
*** ociuhandu has joined #openstack-nova11:58
*** ratailor has quit IRC12:09
*** k_mouza has quit IRC12:18
*** k_mouza has joined #openstack-nova12:25
kashyapstephenfin: Hey12:26
kashyapstephenfin: On documentation: In the past you posted some notes somewhere about how Postgres (or is it some other?) has different kinds of documentation guides?12:27
*** k_mouza has quit IRC12:35
*** k_mouza has joined #openstack-nova12:38
*** k_mouza has quit IRC12:43
*** tbachman has joined #openstack-nova12:45
*** k_mouza has joined #openstack-nova12:46
gibinova PTG continues in 10 minutes12:51
*** sapd1_x has joined #openstack-nova12:56
*** nweinber has joined #openstack-nova13:06
*** k_mouza has quit IRC13:07
*** k_mouza has joined #openstack-nova13:08
*** tesseract has quit IRC13:10
toskygibi: in the "next W cycle goals" slot, are you going to talk also about "completing the V cycle goals"? :) (and in case, do I need to be around?)13:10
*** tesseract has joined #openstack-nova13:12
*** derekh has quit IRC13:13
stephenfinlyarwood: noddy nova-net-related patch here https://review.opendev.org/#/c/760331/13:17
stephenfinif you've a sec13:17
*** jangutter has quit IRC13:27
*** jangutter_ has joined #openstack-nova13:27
*** Luzi has joined #openstack-nova13:38
*** jangutter_ is now known as jangutter13:41
*** jamesdenton has joined #openstack-nova13:46
*** david-lyle has joined #openstack-nova13:53
gibitosky: sorry I missed your comment13:58
gibitosky: I can circle back to this after the current topic13:58
bauzassuffered a laptop crash, rejoining14:01
toskygibi: unless you have other topics of course (the list is huuuge); in case, feel free to ping me14:02
*** legochen_ has joined #openstack-nova14:02
gibitosky: if you can join now then I think we can get to this in about 10 minutes14:02
toskygibi: oh, jumping in14:03
gibitosky: ack14:03
kashyapgibi: /me was participating in KVM Forum BoFs and sessions; so been away here14:11
kashyap(Too many things happening at the same time...)14:11
gibikashyap: ack14:11
gibikashyap: I think we haven't encountered any topics from you yet14:13
kashyapgibi: The main thing I have is the long-running Secure Boot spec -- for repproval -- I put it on the back burner due to other things14:14
kashyapSo it should be fairly uncontroversial14:14
kashyapWe have various infra bits in-tree, and in-flight.  (Aside: I have another conflict in an hour or so ... if I'm not around, please leave questions for me in the Etherpad)14:15
kashyap(I'll come back and answer)14:15
gibikashyap: OK. I it good if we bring the secure boot up right after the current break?14:17
kashyapgibi: Current break is how long?  I have a hard-break at 16:20 CET14:20
kashyapI can pay more attention back here again at 17:30 CET14:21
gibiwe are back at 14:26 UTC14:21
gibiso in like 4 minutes14:22
kashyapAh, sure; that works14:23
kashyapgibi: But I feel like I'm cheating by moving a topic up14:23
kashyapSo feel free to say "no"14:23
kashyap(I don't mean, I literally moved it on the Etherpad :))14:24
gibikashyap: not cheating, it is dynamic scheduling of the topic based on availability :)14:25
kashyapHehe, reframing, is it14:25
kashyapThanks a lot!14:25
gibiI moved your topic up in the etherpd14:26
openstackgerritLee Yarwood proposed openstack/nova master: zuul: Merge nova-evacuate into nova-multinode-live-migration  https://review.opendev.org/75530914:27
*** ociuhandu_ has joined #openstack-nova14:30
*** ociuhandu has quit IRC14:33
*** dave-mccowan has quit IRC14:33
*** jmlowe has quit IRC14:34
sean-k-mooneyim having issue with zoom ill try and rejion as soon as i can14:36
*** dave-mccowan has joined #openstack-nova14:36
*** jmlowe has joined #openstack-nova14:37
*** songwenping_ has quit IRC14:41
gibimelwitt, dansmith: fyi, we are delaying the NUMA discussion on the nova PTG so that you can join. let me know if we should not wait14:42
dansmithgibi: I'm triple-booked today14:43
dansmithgibi: this is numa and placement?14:43
gibidansmith: yes14:43
dansmithack, well, I'm  in the camp of doing this in placement of course14:44
dansmithI think melwitt is too, so she can probably just say everything twice to pretend I'm there :)14:44
gibidansmith: OK. thanks.14:44
lyarwoodmgoddard: https://www.openstack.org/ptg/rooms/liberty - about to talk about https://bugs.launchpad.net/nova/+bug/1895848 if you're free btw14:55
openstackLaunchpad bug 1895848 in OpenStack Compute (nova) "Migration and evacuation fails with encrypted volumes" [Medium,Confirmed] - Assigned to Lee Yarwood (lyarwood)14:55
lyarwoodmgoddard: apologies for the late ping, failed at scrolling down.14:56
lyarwoodmgoddard: https://etherpad.opendev.org/p/nova-wallaby-ptg is the pad btw14:56
mgoddardlyarwood: o/14:56
mgoddardjust joined. Thanks for the ping14:56
lyarwoodnp14:56
lyarwoodjust finishing the previous topic14:56
*** bbowen has quit IRC15:01
*** bbowen has joined #openstack-nova15:04
melwittgibi: cool, thanks for doing that! I'm online now15:05
gibimelwitt: o/ cool then we will take NUMA soon15:05
bauzasjohnthetubaguy: https://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py#L12215:06
bauzaswe log a bit of infor15:06
*** derekh has joined #openstack-nova15:16
*** david-lyle has quit IRC15:19
*** dklyle has joined #openstack-nova15:19
bauzasfedora's oomearly--15:20
bauzaszoom-- for eating all my RAM15:20
*** ociuhandu_ has quit IRC15:22
stephenfinmelwitt: you're super echo'y15:23
*** ociuhandu has joined #openstack-nova15:23
melwittstephenfin: argh ok, checking my headset settings15:23
melwitthm looks like it's all set right15:24
lyarwoodactually sounds really cool, I wouldn't worry15:24
melwittsounds cool xD heh15:26
*** vishalmanchanda has quit IRC15:40
*** hamalq has joined #openstack-nova15:42
lyarwoodgmann: https://review.opendev.org/#/c/755525/ - can you remove your -W on this btw, the `fix` landed in stable/victoria15:44
openstackgerritLee Yarwood proposed openstack/nova stable/ussuri: libvirt: Increase incremental and max sleep time during device detach  https://review.opendev.org/75730615:45
gmannlyarwood: done,15:45
lyarwoodgmann: thanks15:53
*** rpittau is now known as rpittau|afk15:54
johnsomHi Nova community. The Octavia team is seeing some strange behavior with libvirtd. https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c77/759973/3/check/octavia-v2-dsvm-scenario/c77fe63/controller/logs/libvirt/libvirtd_log.txt16:19
johnsomWarning, that is a 136MB log file16:19
johnsomIt seems to go into some sort of tight loop:16:20
lyarwoodops, opening that in chrome was a mistake16:20
johnsomhttps://www.irccloud.com/pastebin/c6NcVeeF/16:20
johnsomAlmost 90% of that log file is the above16:20
lyarwoodkashyap: ^ if you have time16:20
lyarwoodring any bells?16:21
*** hamalq has quit IRC16:21
johnsomEventually it seems to finally exit. The issue we have is while it's stuck in this loop other VMs are not going ACTIVE in nova, so causes all sorts of timeouts.16:21
kashyaplyarwood: Just a sec; lost in two other convo threads :-(16:21
lyarwoodkashyap: np16:22
*** hamalq has joined #openstack-nova16:22
kashyapjohnsom: As a quick question - in what scenario do you see that?16:22
lyarwoodjohnsom: might be useful to write that up in a nova bug that kashyap can follow up with16:22
lyarwoodops sorry16:22
kashyapjohnsom: That "End of file" ... simply means libvirt lost connection to QEMU16:22
johnsomIt appears that nova was trying to delete the instance16:22
kashyaplyarwood: No worries; a bug also helps - as a record16:22
johnsomHere is a chunk of the log right before the loop:16:23
johnsomhttps://www.irccloud.com/pastebin/LRErlmPf/16:23
kashyapjohnsom: Yeah, but that doesn't tell us what events led to that (they're in the 130MB log file).  :-) Is this reproducible?16:26
johnsomIt is reproducible, but intermittent. Not every job triggers it.16:27
gibi#nova now refresh connection_info / avoid storing stale connection_info in Nova16:28
johnsomWe are seeing it while we run our scenario test suites.16:28
kashyapjohnsom: A write-up in a bug would be nice to track.  Along w/ a description of a test case that's hitting it.  (I'm almost out of neurons tonight, and my mind now has the "thousand yard stare")16:30
johnsomYeah, I'm writing one up. I will link to the long, but I don't think I should attach it given it's size, so it will expire off the object store.16:31
johnsomI will paste in a few snippets.16:31
*** lpetrut has quit IRC16:31
*** lemko4 has joined #openstack-nova16:31
kashyapYou can xz-compress the log; as the CI one might get cleaned up any moment16:31
*** lemko has quit IRC16:32
*** lemko4 is now known as lemko16:32
kashyapjohnsom: Also, paste-bins expore; please add relevant snippets as a _text_ file.  Because, LaunchPad, annoyingly enough, breaks formatting16:32
kashyapThanks :)16:32
johnsomOk16:33
kashyapjohnsom: (Sorry for being pedanctic.)16:40
johnsomOh no worries. I want to make sure you get what you need to help us out.16:40
johnsomkashyap https://bugs.launchpad.net/nova/+bug/190227616:54
openstackLaunchpad bug 1902276 in OpenStack Compute (nova) "libvirtd going into a tight loop causing instances to not transition to ACTIVE" [Undecided,New]16:54
kashyapjohnsom: Thanks; another question - so the instances are not coming up online?16:59
kashyapjohnsom: Disregard me; you actually say that "eventually things go back to normal"17:02
johnsomYep17:03
*** ociuhandu_ has joined #openstack-nova17:03
*** ociuhandu has quit IRC17:06
*** ociuhandu_ has quit IRC17:08
kashyapjohnsom: I'm a bit baffled with it all :) I asked a couple of libvirt devs to see if the lead in and the end of the loop makes any further sense to them17:10
* kashyap --> back later17:10
johnsomOk, thank you17:10
*** ociuhandu has joined #openstack-nova17:11
*** Luzi has quit IRC17:14
kashyapjohnsom: Haha, you write: "We seen this regularly, but it's intermittent" — now, which is it?17:15
johnsomkashyap We see it on jobs daily, but it's not every run17:16
*** ociuhandu has quit IRC17:16
johnsomMaybe that makes more sense? grin17:16
johnsomIt's been a long week, words are hard. grin17:18
kashyapAh, yes.  That makes more sense17:18
kashyapBut something seems fishy; what changed so that things "go back to normal"17:18
kashyapTo be continued ... next week :)17:18
johnsomYeah, I have no idea17:18
johnsomIt repeats 276713 times in the log, so not a nice even number really.17:20
*** k_mouza has quit IRC17:21
*** mlavalle has joined #openstack-nova17:32
*** tesseract has quit IRC17:42
*** ralonsoh has quit IRC17:55
*** derekh has quit IRC18:05
melwittdansmith: train backport of cherry pick check fix if you get a moment https://review.opendev.org/75911818:10
*** dtantsur is now known as dtantsur|afk18:15
*** psachin has quit IRC18:23
*** penick has joined #openstack-nova18:24
*** penick has quit IRC18:35
*** tbachman has quit IRC19:34
*** tbachman has joined #openstack-nova19:35
*** ociuhandu has joined #openstack-nova19:50
*** ociuhandu has quit IRC19:52
*** dklyle has quit IRC20:26
*** dklyle has joined #openstack-nova20:34
*** slaweq has quit IRC20:38
*** nweinber has quit IRC20:55
*** sapd1_x has quit IRC21:05
*** jamesdenton has quit IRC21:10
*** sapd1_x has joined #openstack-nova21:17
*** recyclehero has quit IRC21:25
*** recyclehero has joined #openstack-nova21:32
*** legochen_ has quit IRC21:38
*** raildo has quit IRC21:40
*** martinkennelly has quit IRC21:47
openstackgerritMerged openstack/nova stable/train: Follow up for cherry-pick check for merge patch  https://review.opendev.org/75911822:02
*** ociuhandu has joined #openstack-nova22:15
*** hack-char has quit IRC22:16
*** rchurch has quit IRC22:16
*** rchurch has joined #openstack-nova22:18
*** k_mouza has joined #openstack-nova22:19
*** ociuhandu has quit IRC22:20
openstackgerritmelanie witt proposed openstack/nova stable/stein: Follow up for cherry-pick check for merge patch  https://review.opendev.org/76067322:22
*** dasp_ has joined #openstack-nova22:26
*** dave-mccowan has quit IRC22:26
*** dasp has quit IRC22:28
*** k_mouza has quit IRC23:10
*** eharney has quit IRC23:11
*** k_mouza has joined #openstack-nova23:12
*** TheJulia is now known as TheJulia_on_PTO23:24
*** mlavalle has quit IRC23:31
*** jamesdenton has joined #openstack-nova23:39
*** jamesdenton has quit IRC23:40
*** k_mouza has quit IRC23:41
*** bbowen has quit IRC23:52
*** bbowen_ has joined #openstack-nova23:52
*** k_mouza has joined #openstack-nova23:53
*** k_mouza has quit IRC23:59
« Thursday, 2020-10-29 Index Saturday, 2020-10-31 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!