| *** tetsuro has joined #openstack-nova | 00:14 | |
| *** tetsuro has quit IRC | 00:18 | |
| *** brinzhang has joined #openstack-nova | 00:20 | |
| *** tetsuro has joined #openstack-nova | 00:29 | |
| *** tetsuro has quit IRC | 00:30 | |
| *** tetsuro has joined #openstack-nova | 00:31 | |
| *** brinzhang_ has joined #openstack-nova | 00:42 | |
| *** brinzhang has quit IRC | 00:44 | |
| *** lbragstad has joined #openstack-nova | 01:11 | |
| *** bbowen has quit IRC | 01:11 | |
| *** zhanglong has joined #openstack-nova | 01:12 | |
| *** Liang__ has joined #openstack-nova | 01:17 | |
| *** bbowen has joined #openstack-nova | 01:23 | |
| *** sapd1_x has joined #openstack-nova | 01:40 | |
| *** lbragstad has quit IRC | 02:14 | |
| openstackgerrit | mashuting proposed openstack/nova master: Revert "Deprecate the vmwareapi driver" https://review.opendev.org/713401 | 02:15 |
|---|---|---|
| *** ociuhandu has joined #openstack-nova | 02:20 | |
| *** prometheanfire has joined #openstack-nova | 02:25 | |
| *** ociuhandu has quit IRC | 02:30 | |
| *** ociuhandu has joined #openstack-nova | 02:31 | |
| *** prometheanfire has quit IRC | 02:32 | |
| *** ociuhandu has quit IRC | 02:36 | |
| *** mkrai has joined #openstack-nova | 03:08 | |
| *** mmethot_ has joined #openstack-nova | 03:20 | |
| *** mmethot has quit IRC | 03:22 | |
| *** psachin has joined #openstack-nova | 03:23 | |
| *** _mmethot_ has joined #openstack-nova | 03:31 | |
| *** mmethot_ has quit IRC | 03:34 | |
| *** prometheanfire has joined #openstack-nova | 03:47 | |
| *** mkrai has quit IRC | 03:49 | |
| *** mkrai has joined #openstack-nova | 03:49 | |
| *** udesale has joined #openstack-nova | 04:13 | |
| *** hongbin has quit IRC | 04:20 | |
| *** yaawang has quit IRC | 04:40 | |
| *** yaawang has joined #openstack-nova | 04:40 | |
| *** prometheanfire has quit IRC | 04:41 | |
| *** links has joined #openstack-nova | 04:50 | |
| *** links has quit IRC | 04:54 | |
| *** vishalmanchanda has joined #openstack-nova | 04:58 | |
| *** ociuhandu has joined #openstack-nova | 05:14 | |
| *** ociuhandu has quit IRC | 05:19 | |
| *** mkrai has quit IRC | 05:22 | |
| *** mkrai has joined #openstack-nova | 05:24 | |
| *** links has joined #openstack-nova | 05:25 | |
| *** mkrai has quit IRC | 05:32 | |
| *** mkrai has joined #openstack-nova | 05:32 | |
| *** prometheanfire has joined #openstack-nova | 05:34 | |
| *** evrardjp has quit IRC | 05:36 | |
| *** evrardjp has joined #openstack-nova | 05:36 | |
| *** prometheanfire has quit IRC | 05:38 | |
| *** belmoreira has joined #openstack-nova | 05:40 | |
| *** belmoreira has quit IRC | 05:45 | |
| *** ratailor has joined #openstack-nova | 05:46 | |
| *** prometheanfire has joined #openstack-nova | 06:08 | |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: support live migration with vpmems https://review.opendev.org/687856 | 06:30 |
| *** slaweq_ has joined #openstack-nova | 06:32 | |
| *** mkrai has quit IRC | 07:03 | |
| *** mkrai has joined #openstack-nova | 07:08 | |
| *** prometheanfire has quit IRC | 07:18 | |
| *** dtantsur|afk is now known as dtantsur | 07:18 | |
| *** yaawang has quit IRC | 07:29 | |
| *** yaawang has joined #openstack-nova | 07:30 | |
| *** tetsuro_ has joined #openstack-nova | 07:41 | |
| *** dpawlik has joined #openstack-nova | 07:44 | |
| *** tetsuro has quit IRC | 07:45 | |
| *** jawad_axd has joined #openstack-nova | 07:49 | |
| *** ociuhandu has joined #openstack-nova | 07:56 | |
| *** ircuser-1 has quit IRC | 07:57 | |
| *** yaawang has quit IRC | 07:58 | |
| *** ralonsoh has joined #openstack-nova | 07:59 | |
| *** maciejjozefczyk has joined #openstack-nova | 08:04 | |
| openstackgerrit | Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494 | 08:09 |
| *** tkajinam has quit IRC | 08:15 | |
| *** xek has joined #openstack-nova | 08:16 | |
| *** tesseract has joined #openstack-nova | 08:20 | |
| *** belmoreira has joined #openstack-nova | 08:24 | |
| *** tosky has joined #openstack-nova | 08:32 | |
| *** tetsuro has joined #openstack-nova | 08:36 | |
| openstackgerrit | Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494 | 08:37 |
| *** tetsuro_ has quit IRC | 08:38 | |
| lyarwood | https://review.opendev.org/#/c/714057/ - gibi / stephenfin / any cores, could someone +W this again now it's passing, I'm going to spend the day looking at gate failures btw | 08:39 |
| gibi | lyarwood: ack, will look | 08:40 |
| gibi | done | 08:40 |
| gibi | thanks for looking into the gate issues | 08:41 |
| *** amoralej|off is now known as amoralej | 08:41 | |
| bauzas | good morning | 08:43 |
| bauzas | today's a bit hectic, I need to help my children more than latest week | 08:43 |
| brinzhang_ | good morning | 08:44 |
| brinzhang_ | bauzas: Good luck and pay attention to protection. | 08:44 |
| *** brinzhang_ has quit IRC | 08:48 | |
| *** brinzhang_ has joined #openstack-nova | 08:48 | |
| *** ociuhandu has quit IRC | 08:49 | |
| *** ociuhandu has joined #openstack-nova | 08:50 | |
| brinzhang_ | gibi: if I remove the NOTE from the nova-runways's bp's link, does need to add history log? | 08:51 |
| *** rpittau|afk is now known as rpittau | 08:52 | |
| *** brinzhang has joined #openstack-nova | 08:53 | |
| *** brinzhang_ has quit IRC | 08:54 | |
| *** ociuhandu has quit IRC | 08:55 | |
| gibi | brinzhang: nope | 08:56 |
| brinzhang | gibi: yeah. will remove, but nova-runways doesn't seem to work well enough. | 08:58 |
| gibi | brinzhang: you are right. | 08:58 |
| *** tetsuro_ has joined #openstack-nova | 09:03 | |
| *** tetsuro has quit IRC | 09:06 | |
| *** zhanglong has quit IRC | 09:11 | |
| aarents | morning, lyarwood: I'm considering implementation of finish_unshelve as you suggested in https://review.opendev.org/#/c/696084 I have a question see my last post when you will have time for that | 09:11 |
| *** slaweq_ is now known as slaweq | 09:13 | |
| *** ccamacho has joined #openstack-nova | 09:14 | |
| brinzhang | gibi, lyarwood: https://review.opendev.org/#/c/682772/ this patch addressed all your comments | 09:25 |
| *** ociuhandu has joined #openstack-nova | 09:37 | |
| *** ociuhandu has quit IRC | 09:41 | |
| *** tesseract has quit IRC | 09:42 | |
| *** derekh has joined #openstack-nova | 09:43 | |
| *** ralonsoh has quit IRC | 09:44 | |
| *** ralonsoh has joined #openstack-nova | 09:44 | |
| *** Liang__ has quit IRC | 09:44 | |
| *** tesseract has joined #openstack-nova | 09:44 | |
| *** dklyle has quit IRC | 09:51 | |
| *** mkrai has quit IRC | 09:52 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/pike: Avoid circular reference during serialization https://review.opendev.org/714148 | 09:57 |
| nightmare_unreal | hello, how can one verify cell-id passed in nova-manage CLI ? | 09:58 |
| nightmare_unreal | basically how to validate passed cellID/UUID | 09:58 |
| *** martinkennelly has joined #openstack-nova | 10:03 | |
| nightmare_unreal | found it | 10:07 |
| *** brinzhang_ has joined #openstack-nova | 10:09 | |
| *** dpawlik has quit IRC | 10:11 | |
| *** brinzhang has quit IRC | 10:12 | |
| *** dpawlik has joined #openstack-nova | 10:13 | |
| *** mkrai has joined #openstack-nova | 10:13 | |
| *** lbragstad has joined #openstack-nova | 10:14 | |
| openstackgerrit | Merged openstack/nova master: nova-live-migration: Only stop n-cpu and q-agt during evacuation testing https://review.opendev.org/714057 | 10:15 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: nova-live-migration: Ensure subnode is fenced during evacuation testing https://review.opendev.org/713961 | 10:22 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: nova-live-migration: Ensure subnode is fenced during evacuation testing https://review.opendev.org/713962 | 10:23 |
| lyarwood | aarents: ack sorry will try to look this afternoon | 10:23 |
| lyarwood | brinzhang_: ack will also try to look this afternoon | 10:23 |
| brinzhang_ | lyarwood: yeah, thanks | 10:23 |
| *** hoonetorg has quit IRC | 10:24 | |
| *** tetsuro has joined #openstack-nova | 10:27 | |
| *** tetsuro_ has quit IRC | 10:30 | |
| *** tetsuro_ has joined #openstack-nova | 10:31 | |
| *** tetsuro has quit IRC | 10:33 | |
| *** hoonetorg has joined #openstack-nova | 10:37 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add MigrationTypeField https://review.opendev.org/706013 | 10:43 |
| *** brinzhang_ has quit IRC | 10:43 | |
| kashyap | lyarwood: Just looking at the Q35 upstream job - https://review.opendev.org/#/c/708701/ | 10:46 |
| kashyap | While I look at the logs; what was the issue you're alluding to there, that worked for you locally? | 10:46 |
| bauzas | gibi: stephenfin: efried_gone told he was +2 on https://review.opendev.org/#/c/589085/7 if I was writing a specific email | 10:47 |
| bauzas | are you okay with this ? | 10:47 |
| * kashyap looks at https://zuul.opendev.org/t/openstack/build/38f567807e3442b5b1c1d5f155be0af4 | 10:47 | |
| bauzas | I haven't written it until I was providing https://review.opendev.org/#/c/712118/2 | 10:47 |
| lyarwood | kashyap: yeah on F30, I can reproduce the failure on Ubuntu | 10:47 |
| lyarwood | kashyap: I haven't had time to circle back to it yet | 10:48 |
| stephenfin | bauzas: sure | 10:48 |
| lyarwood | kashyap: but it looks like the config drive is missing it's LABEL causing the test to fail | 10:48 |
| kashyap | I see; I'll see if I can poke more to see why Zuul complains | 10:49 |
| lyarwood | kashyap: tbh I think it might be a bug with the version of q35 Ubuntu is using | 10:51 |
| lyarwood | kashyap: well QEMU but you get the point | 10:51 |
| kashyap | Hmm, didn't think of it; you mean version of QEMU? | 10:51 |
| kashyap | Bingo | 10:51 |
| lyarwood | yeah | 10:51 |
| kashyap | lyarwood: BTW, be careful with the term "circle back"; Urban Dictionary has a harsh definition of it. Not gonna link it :D | 10:52 |
| * kashyap ducks the rotten eggs lyarwood is about the throw | 10:52 | |
| *** tetsuro has joined #openstack-nova | 10:53 | |
| kashyap | I'll check the QEMU version of Ubuntu; sometimes they miss to backport some bug fixes. FWIW, we've pointed them out before | 10:53 |
| kashyap | (I mean, in a different scenario) | 10:53 |
| *** tetsuro_ has quit IRC | 10:55 | |
| *** tetsuro_ has joined #openstack-nova | 10:59 | |
| *** brinzhang has joined #openstack-nova | 11:01 | |
| *** tetsuro has quit IRC | 11:02 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: libvirt: change UEFI check to handle AArch64 better https://review.opendev.org/714311 | 11:08 |
| hrw | this patch should pass tests (does 'tox -e py38') and better shows issue | 11:09 |
| kashyap | hrw: Hi | 11:10 |
| hrw | hi kashyap | 11:10 |
| hrw | others | 11:10 |
| kashyap | hrw: Good point on the undefine with NVRAM thing | 11:10 |
| hrw | now I have one of those nasty days when I look at nova code... | 11:10 |
| kashyap | Will look post-lunch | 11:10 |
| kashyap | :D | 11:11 |
| hrw | speaking of q35... when it will be finally default? | 11:11 |
| hrw | x86 users will get hit hard with it. pleasure to look would be | 11:12 |
| bauzas | gibi: stephenfin: done http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013507.html | 11:13 |
| gibi | bauzas: thanks | 11:14 |
| *** sapd1_x has quit IRC | 11:23 | |
| hrw | "kernel doesn't support AMD SEV" message is funny | 11:26 |
| hrw | as it gets checked on all archs instead of just x86 | 11:26 |
| *** evrardjp has quit IRC | 11:44 | |
| *** udesale_ has joined #openstack-nova | 11:44 | |
| *** rpittau is now known as rpittau|bbl | 11:47 | |
| *** udesale has quit IRC | 11:47 | |
| *** mkrai has quit IRC | 11:48 | |
| *** mkrai_ has joined #openstack-nova | 11:48 | |
| *** evrardjp has joined #openstack-nova | 11:50 | |
| hrw | I wonder how much it will be rewritten once it will be supported somewhere else than amd cpus | 11:53 |
| stephenfin | gibi: did you ever do a blog post on the bandwidth-aware scheduling feature? | 11:54 |
| stephenfin | I recall there being one floating around before a past PTG but heck if I can find it now | 11:55 |
| gibi | stephenfin: rubasov did one in the early days. let me find the link | 11:55 |
| gibi | stephenfin: is this the one you remember? https://rubasov.github.io/2018/09/21/openstack-qos-min-bw-demo.html | 11:55 |
| sean-k-mooney | stephenfin: there was two summit presentations too | 11:56 |
| stephenfin | gibi: That's the one, yup. Thanks! | 11:56 |
| sean-k-mooney | one descirbiing the poc and then one describing the final feature | 11:56 |
| gibi | sean-k-mooney: I only remember showing the PoC | 11:57 |
| sean-k-mooney | yes showed the poc in vancovour and the final version in denver i think | 11:58 |
| gibi | https://www.youtube.com/watch?v=ls9RXpG6Is0 | 11:58 |
| gibi | there was some PTG presentation, true. The above is a summit presentation | 11:58 |
| sean-k-mooney | sorry yes that is waht im thinking of | 11:59 |
| gibi | I think we don't have any recording, other than the blog post, about the PTG presentation | 11:59 |
| hrw | https://paste.centos.org/view/abe36617 - does it look ok to send for review? skipping AMD SEV check on !x86-64 | 12:01 |
| *** ociuhandu has joined #openstack-nova | 12:02 | |
| hrw | fixed missing import, running tox, will send | 12:09 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/pike: Avoid circular reference during serialization https://review.opendev.org/714148 | 12:11 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: libvirt: check for AMD SEV only on x86-64 https://review.opendev.org/714425 | 12:12 |
| *** mgariepy has quit IRC | 12:15 | |
| kashyap | hrw: Upstream there's a CI job to test it: https://review.opendev.org/#/c/708701/ | 12:18 |
| *** artom has joined #openstack-nova | 12:19 | |
| kashyap | hrw: "it" == q35; and aiming to switch to it in the 'V' release, or early, if possible. | 12:19 |
| hrw | kashyap: cool ;) | 12:20 |
| *** ociuhandu has quit IRC | 12:21 | |
| hrw | kashyap: I start to think how to add aarch64 ci job ;D | 12:21 |
| kashyap | I'm not good with the CI stuff, but probably check with #openstack-infra or other folks here (like Sean) might tell you more. | 12:22 |
| kashyap | It is "usually" take-an-existing-job-and-tweak-it | 12:22 |
| hrw | I know | 12:22 |
| hrw | handled too much of CI in kolla | 12:22 |
| kashyap | Oh, dear. Okay :) | 12:26 |
| *** evrardjp has quit IRC | 12:27 | |
| lyarwood | melwitt / elod: https://review.opendev.org/#/q/topic:bug/1867380+status:open - could you hit the rest of these today please? | 12:28 |
| openstackgerrit | Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494 | 12:29 |
| lyarwood | https://review.opendev.org/#/q/topic:bug/1813789+status:open should also be ready to review | 12:29 |
| *** martinkennelly has quit IRC | 12:30 | |
| hrw | sean-k-mooney: which nova ci job would be best to run on aarch64? | 12:31 |
| *** eharney has joined #openstack-nova | 12:32 | |
| sean-k-mooney | start with i think tempest-integrated-compute? | 12:32 |
| sean-k-mooney | let me see if i can get the correct name | 12:32 |
| sean-k-mooney | ya tempest-integrated-compute | 12:33 |
| sean-k-mooney | if you have that working we disucsed that you really need 2 multi node jobs | 12:33 |
| sean-k-mooney | one with qcow image backedn and cinder with lvm | 12:34 |
| sean-k-mooney | and the second with ceph | 12:34 |
| kevinz | sean-k-mooney: hrw: so the nova CI still rely on devstack CI test finished ? | 12:35 |
| sean-k-mooney | that way you can test live and code migration with both block and volume backed migration in the first job and shared storage in teh second | 12:35 |
| hrw | kevinz: thx | 12:35 |
| sean-k-mooney | kevinz: can you rephrase that? | 12:35 |
| sean-k-mooney | kevinz: the upstream integration ci jobs outside of install projects all use devstack to deploy the code under test | 12:36 |
| sean-k-mooney | and then use tempest to exectue the tests if that is what you were asking | 12:36 |
| kevinz | I mean "tempest-integrated-compute" test need devstack jobs ready? https://review.opendev.org/#/c/708317/ | 12:36 |
| sean-k-mooney | kevinz: ah right well yes unless you are going to do alot more work and port one of the install jobs | 12:37 |
| sean-k-mooney | e.g. create a job based on osa or kolla-ansible or something that can deploy on aarch64 | 12:38 |
| sean-k-mooney | kevinz: although is https://review.opendev.org/#/c/708317/ for aarch64 as arm64 is technically a different architecutre | 12:38 |
| gibi | dansmith: went through cyborg series again, I had only couple of things so I feel we are close | 12:38 |
| * gibi lunches | 12:38 | |
| *** brinzhang has quit IRC | 12:38 | |
| *** evrardjp has joined #openstack-nova | 12:38 | |
| *** brinzhang has joined #openstack-nova | 12:39 | |
| *** brinzhang has quit IRC | 12:39 | |
| kevinz | sean-k-mooney: OK, so based on Devstack CI jobs, some tempest test are not stable...means failed "sometimes" | 12:40 |
| sean-k-mooney | kevinz: well that happens on x86 too you know | 12:41 |
| sean-k-mooney | is it more unstable | 12:41 |
| *** psachin has quit IRC | 12:42 | |
| *** nweinber has joined #openstack-nova | 12:43 | |
| kevinz | hrw: sean-k-mooney: I'm going to enlarge the VM to 16GB to see if it works better... | 12:43 |
| hrw | kevinz: commented on that devstack patch | 12:44 |
| sean-k-mooney | you sould only need 8G but it might be an issue with your concurancy how may cpus cores do you have exposed and what concurancy is the job running with | 12:45 |
| sean-k-mooney | by the way am in the only person who asociates arm64 with arm64_32 and uses AArch64 when refering to 64bit arm v8 compilation target | 12:46 |
| sean-k-mooney | as in arm64_32 form this context https://llvm.org/devmtg/2019-04/slides/TechTalk-Northover-A_tale_of_two_ABIs_ILP32_on_AArch64.pdf | 12:47 |
| *** huaqiang has joined #openstack-nova | 12:47 | |
| hrw | sean-k-mooney: aarch64 is official architecture name :) | 12:47 |
| sean-k-mooney | i guess without the _32 suffix arm64 is used interchangeable with aarch64 | 12:48 |
| hrw | arm64_32 is aarch32 which is 32bit mode on aarch64 cpu. totally optional thing in cpu design, not present in several server processors | 12:48 |
| sean-k-mooney | hrw: yep just i dont see arm64 that often and was not sure if people were shotening arm64_32 or not | 12:48 |
| hrw | sean-k-mooney: exactly | 12:48 |
| hrw | no arm64==aarch64 | 12:48 |
| sean-k-mooney | ok | 12:48 |
| sean-k-mooney | ill keep avoiding using arm64 then and keep using aarch64 | 12:49 |
| hrw | for me arm64_32 is one step from abomination ;D | 12:49 |
| sean-k-mooney | well its better then continuing to use armv7 | 12:49 |
| *** ratailor has quit IRC | 12:50 | |
| sean-k-mooney | hrw: did you see the recent new about that amazon graviton2 chips | 12:50 |
| sean-k-mooney | those look really interesting | 12:50 |
| hrw | sean-k-mooney: aarch32 support in aarch64 cpus is mainly distro builders and android developers nowadays imho. first ones remove armv7 builders and move to aarch32 VM instances on aarch64 servers, latter ones need a way to build/test their arm32 android apps somewhere natively | 12:50 |
| sean-k-mooney | too bad they dont sell them | 12:51 |
| hrw | sean-k-mooney: graviton2 looks great | 12:51 |
| hrw | sean-k-mooney: let me say that way: please do not start aarch64 developers hardware discussion. | 12:51 |
| sean-k-mooney | hrw: well apple are also pussing aarch32 for legacy compatiblity reasons | 12:51 |
| hrw | sean-k-mooney: app store requires 64bit apps upload so it is only a matter of time | 12:52 |
| sean-k-mooney | yep they just wanted it for a transition period | 12:52 |
| hrw | once support for <iphone 5s gets killed they can go 64bit only | 12:52 |
| sean-k-mooney | anyway its fun times as long as you are not intel right now | 12:52 |
| hrw | o yes | 12:52 |
| hrw | unless you are aarch64 porter/developer and have to fight with software written with 'there is only x86' mantra ;D | 12:53 |
| *** ociuhandu has joined #openstack-nova | 12:54 | |
| sean-k-mooney | until there is a standard aarch64 system that runs normal linux that developers can use that is affordable that is going to be the case for a while | 12:54 |
| hrw | use or buy? | 12:54 |
| sean-k-mooney | well i was thinking by but i guess you can now use aws | 12:55 |
| *** Luzi has joined #openstack-nova | 12:55 | |
| sean-k-mooney | e.g. a laptop/desktop that developers can use as a daily driver without going all the way to a full server | 12:55 |
| hrw | sean-k-mooney: https://marcin.juszkiewicz.com.pl/2019/02/23/we-need-arm64-systems-for-developers-again/ | 12:56 |
| hrw | sean-k-mooney: yeah | 12:56 |
| kevinz | hrw: thanks for the comments, I will check | 12:56 |
| *** sapd1_x has joined #openstack-nova | 12:56 | |
| hrw | sean-k-mooney: there are attempts to run linux natively on those snapdragon based win10 laptops | 12:56 |
| sean-k-mooney | hrw: ya basically that :) | 12:57 |
| *** macz_ has joined #openstack-nova | 12:57 | |
| * hrw -> hitting head to the wall or other less painful things rather than discussing aarch64 developers hardware | 12:57 | |
| sean-k-mooney | that could be intersting but do they supprot hardware virtualisation so you could do openstack dev on them | 12:57 |
| hrw | in theory virtualization is mandatory on aarch64 | 12:58 |
| sean-k-mooney | oh good to know | 12:58 |
| *** ociuhandu has quit IRC | 12:58 | |
| sean-k-mooney | i guess even on a mobile/laptop chip it might be useful for sandboxing | 12:58 |
| sean-k-mooney | e.g. for secure contianer technology/linux subsystem for windows | 12:59 |
| openstackgerrit | Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494 | 12:59 |
| *** Luzi_ has joined #openstack-nova | 13:00 | |
| hrw | sean-k-mooney: https://paste.centos.org/view/975095f5 - how many mistakes I made? | 13:00 |
| hrw | sean-k-mooney: CI job | 13:00 |
| hrw | sean-k-mooney: 'check-arm64' is pipeline in zuul - used at same time as 'check' | 13:01 |
| *** Luzi has quit IRC | 13:01 | |
| *** mgariepy has joined #openstack-nova | 13:01 | |
| sean-k-mooney | if the nodeset is already defiend somewhere else you do not need to redefine it. | 13:02 |
| sean-k-mooney | other then that it looks more or less correct | 13:02 |
| hrw | sean-k-mooney: in kolla we created one | 13:03 |
| sean-k-mooney | zuul collects all the definition form all projects | 13:03 |
| sean-k-mooney | so if its already defeined in another repo you can just reuse it | 13:03 |
| hrw | I know, looking at tempest one now | 13:03 |
| hrw | sean-k-mooney: aarch64 is not quite popular in CI so far | 13:04 |
| *** ociuhandu has joined #openstack-nova | 13:04 | |
| sean-k-mooney | if you want to redfine the node set then you might need to give it a new name | 13:05 |
| sean-k-mooney | the other think is the voting:false | 13:05 |
| sean-k-mooney | that should be in the where you use the job under check-arm64 | 13:05 |
| sean-k-mooney | not in the job definition | 13:05 |
| hrw | sean-k-mooney: you use both form ;) | 13:06 |
| hrw | nova-lvm has voting in definition | 13:07 |
| sean-k-mooney | i think if you set it in the defition it will apply to all uses of the job | 13:07 |
| sean-k-mooney | basically chanign the default | 13:07 |
| hrw | ok | 13:07 |
| sean-k-mooney | people dont expect to have to set voting:true | 13:08 |
| *** tetsuro_ has quit IRC | 13:08 | |
| hrw | yep | 13:08 |
| *** ociuhandu has quit IRC | 13:08 | |
| *** rpittau|bbl is now known as rpittau | 13:08 | |
| sean-k-mooney | i would do something like this http://paste.openstack.org/show/791022/ | 13:08 |
| sean-k-mooney | but ya i think way you had would work | 13:08 |
| hrw | on it | 13:09 |
| *** amoralej is now known as amoralej|lunch | 13:09 | |
| alex_xu | dansmith: last question for this https://review.opendev.org/#/c/631244/68/nova/conductor/manager.py@1621, Sundar said you are more clear on this. The question is why we don't fail directly when binding failed at build_instances instead of try next alt host. The build_and_run_instances looks like will fail directly when binding arq failed. | 13:10 |
| hrw | sean-k-mooney: https://paste.centos.org/view/61a2b0ca | 13:10 |
| hrw | sean-k-mooney: openstack-single-node-bionic-aarch64 copied from devstack | 13:11 |
| *** ociuhandu has joined #openstack-nova | 13:11 | |
| hrw | sean-k-mooney: openstack-single-node-bionic-aarch64 copied from devstack openstack-single-node-bionic one | 13:11 |
| hrw | to be exact | 13:11 |
| *** ociuhandu has quit IRC | 13:13 | |
| *** ociuhandu has joined #openstack-nova | 13:14 | |
| *** ociuhandu has quit IRC | 13:16 | |
| *** ociuhandu has joined #openstack-nova | 13:16 | |
| gibi | lyarwood: I saw yet another evacuation problem in the live-migration task on master | 13:16 |
| gibi | lyarwood: VirtualInterfaceCreateException happens during evac | 13:16 |
| gibi | https://zuul.opendev.org/t/openstack/build/2703656a683043a6b741fa4c2283b314/log/job-output.txt#7708 | 13:16 |
| *** ociuhandu has quit IRC | 13:16 | |
| gibi | https://zuul.opendev.org/t/openstack/build/2703656a683043a6b741fa4c2283b314/log/logs/screen-n-cpu.txt?severity=4#7865 | 13:17 |
| gibi | I don't knowi if you already see this | 13:17 |
| *** zhanglong has joined #openstack-nova | 13:17 | |
| *** ociuhandu has joined #openstack-nova | 13:18 | |
| artom | stephenfin, so I've finally gotten around to removing the 2/3 NUMA split in https://review.opendev.org/#/c/672595/ | 13:20 |
| artom | Mind taking a look when you get a chance? I'll revisit your hugepages bugfix in the meantime | 13:20 |
| sean-k-mooney | hrw: yes that woudl be better since you have the group dfinition that are used for the ansible roles | 13:22 |
| sean-k-mooney | using primary may have worked but like would have caused issues | 13:22 |
| hrw | sean-k-mooney: 'primary' was taken from kolla ;D | 13:23 |
| sean-k-mooney | hrw: is there a reason that you are not defineing this in devstack and using a depends-on | 13:23 |
| sean-k-mooney | ah i know that seamed familar | 13:23 |
| sean-k-mooney | they use primary and secondary instead of subnode terminology | 13:24 |
| sean-k-mooney | that we use for devstack | 13:24 |
| hrw | sean-k-mooney: after playing with devstack in past there is no stick long enough to touch it (forgive poor idiom use) | 13:24 |
| sean-k-mooney | hrw: devstack and kolla-ansible are my two favorite installers | 13:25 |
| hrw | ;) | 13:25 |
| sean-k-mooney | depending on what im doing | 13:25 |
| hrw | yoctozepto: ^^ idea? | 13:25 |
| lyarwood | gibi: sorry was out at lunch, no I hadn't seen any issues since my changes landed :/ | 13:26 |
| hrw | sean-k-mooney: I only use k-a for deployment. at least I know that it works and gives me sane setup | 13:26 |
| hrw | sean-k-mooney: when I used devstack the way of dealing with issues was 'restart and hope' | 13:26 |
| sean-k-mooney | hrw: ya i use k-a for any long lived setup and use devstack for everything else | 13:26 |
| sean-k-mooney | devstack is actully pretty reliable you just need to learn how to use it | 13:27 |
| sean-k-mooney | and how to fix common issues | 13:27 |
| sean-k-mooney | normally when it breaks its because of a depency did a new release | 13:28 |
| *** arxcruz|rover has joined #openstack-nova | 13:28 | |
| sean-k-mooney | once you know what to look for fixing it get much simpler | 13:29 |
| hrw | possible | 13:29 |
| gibi | lyarwood: I hope this is just a one time issue. I've rechecked the patch see if it goes now | 13:29 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: [WIP] CI: add tempest-integrated-compute-aarch64 job https://review.opendev.org/714439 | 13:29 |
| hrw | let's see | 13:29 |
| sean-k-mooney | gibi: whats the current state fo the cyborg series | 13:30 |
| sean-k-mooney | actully i have to join a meeting | 13:31 |
| gibi | sean-k-mooney: I've just re-reviewed before lunch | 13:31 |
| gibi | sean-k-mooney: it looks pretty close | 13:31 |
| *** tbachman has joined #openstack-nova | 13:31 | |
| gibi | sean-k-mooney: I have one deeper question about the amount of ARQ delete calls nova makes | 13:31 |
| gibi | the rest is mostly easy fixes | 13:31 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: [WIP] CI: add tempest-integrated-compute-aarch64 job https://review.opendev.org/714439 | 13:31 |
| sean-k-mooney | ok so i would proably be worth my time looking at it again today | 13:32 |
| *** mriedem has joined #openstack-nova | 13:32 | |
| gibi | sean-k-mooney: yes. I also pinged dansmith as I think he can re-review now as well | 13:32 |
| dansmith | yep, I'm looking at alex_xu's query on that | 13:33 |
| *** sapd1_x has quit IRC | 13:35 | |
| *** ociuhandu has quit IRC | 13:36 | |
| dansmith | gibi: will start going over that in a little bit | 13:36 |
| *** ociuhandu_ has joined #openstack-nova | 13:36 | |
| gibi | cool | 13:37 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: [WIP] CI: add tempest-integrated-compute-aarch64 job https://review.opendev.org/714439 | 13:37 |
| *** Luzi_ has quit IRC | 13:44 | |
| lyarwood | kashyap / stephenfin / melwitt ; https://review.opendev.org/#/c/696834/ - can you hit this again (rbd_swap) today if you have time. FWIW with the increase in required versions I'm not going to be able to test this upstream until I get Fedora 31 images built | 13:44 |
| lyarwood | https://review.opendev.org/#/c/708416/ in this change | 13:45 |
| kashyap | lyarwood: Hi, it's on TODO for today. | 13:45 |
| *** ociuhandu_ has quit IRC | 13:46 | |
| *** ociuhandu has joined #openstack-nova | 13:46 | |
| lyarwood | kashyap: thanks | 13:49 |
| *** amoralej|lunch is now known as amoralej | 14:06 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Expose instance action event details out of the API https://review.opendev.org/694430 | 14:08 |
| openstackgerrit | Merged openstack/nova master: Bump python-subunit minimum to 1.4.0 https://review.opendev.org/713611 | 14:09 |
| openstackgerrit | jayaditya gupta proposed openstack/nova master: Support for nova-manage placement heal_allocations --cell https://review.opendev.org/714454 | 14:11 |
| *** hongbin has joined #openstack-nova | 14:19 | |
| openstackgerrit | jayaditya gupta proposed openstack/nova master: Support for nova-manage placement heal_allocations --cell https://review.opendev.org/714459 | 14:21 |
| *** ociuhandu_ has joined #openstack-nova | 14:22 | |
| *** brinzhang has joined #openstack-nova | 14:24 | |
| *** ociuhand_ has joined #openstack-nova | 14:24 | |
| *** ociuhandu has quit IRC | 14:25 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add instance actions v283 samples test https://review.opendev.org/706251 | 14:25 |
| brinzhang | gibi: Done, thanks | 14:26 |
| *** ociuhand_ has quit IRC | 14:27 | |
| *** ociuhandu_ has quit IRC | 14:27 | |
| *** ociuhandu has joined #openstack-nova | 14:27 | |
| *** ociuhandu_ has joined #openstack-nova | 14:30 | |
| *** ociuhand_ has joined #openstack-nova | 14:32 | |
| *** ociuhandu has quit IRC | 14:33 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add instance actions v283 samples test https://review.opendev.org/706251 | 14:33 |
| *** dklyle has joined #openstack-nova | 14:34 | |
| *** ociuhandu_ has quit IRC | 14:36 | |
| *** mkrai_ has quit IRC | 14:37 | |
| *** ociuhand_ has quit IRC | 14:43 | |
| kashyap | lyarwood: Should the blockCopy() change then be dependent on the F31 change - to prove testability? | 14:43 |
| lyarwood | kashyap: I've already proven it with F30 and 5.10/4.3 tbh | 14:48 |
| kashyap | lyarwood: Ah, then your comment here confused me :) | 14:48 |
| sean-k-mooney | lyarwood: whats the minium qemu you need | 14:50 |
| sean-k-mooney | will 4.2 work | 14:50 |
| lyarwood | kashyap: so the new version constrats you suggested to avoid some known issues is what's causing us to be unable to test this prior to F31 | 14:50 |
| kashyap | lyarwood: Hmm; /me taps on the table and thinks ... | 14:50 |
| lyarwood | sean-k-mooney: Libvirt 5.10 and QEMU 4.{3,2} I forget which | 14:50 |
| kashyap | Seems like the diskimage-builder patch is at least being reviewed by the Right Person(tm). | 14:51 |
| sean-k-mooney | lyarwood: i think ubuntu 20.04 will have libvirt 6.0 and qemu 4.2 | 14:52 |
| lyarwood | right | 14:52 |
| *** prometheanfire has joined #openstack-nova | 14:52 | |
| lyarwood | sean-k-mooney: so support was introduced a while ago, kashyap just suggested we use a later version to include some bugfixes for blockCopy. | 14:53 |
| sean-k-mooney | sure but this should not be a blocker right | 14:53 |
| lyarwood | and using that later version is now causing issues testing this somewhere in the gate as the virt-preview repo has only built these packages for F31 | 14:53 |
| sean-k-mooney | we can test with the older versions? | 14:53 |
| lyarwood | we can't if the constraints in code limit us to the newer versions | 14:53 |
| lyarwood | otherwise we switch to the older blockRebase method | 14:54 |
| sean-k-mooney | well the virt preview repo and fedora is not stble anyway | 14:54 |
| * lyarwood hugs Fedora | 14:54 | |
| kashyap | Yeah, FWIW, it's also the libvirt upstream suggested based on our use-case | 14:54 |
| kashyap | sean-k-mooney: The reason for going with the current 6.1 and QEMU 4.2 (at least Ubuntu has this): | 14:54 |
| sean-k-mooney | kashyap: i was tying to config the version did you find them | 14:55 |
| *** mgariepy has quit IRC | 14:55 | |
| * kashyap is logging into Gerrit; had to restart the browser | 14:55 | |
| hrw | Debian is at 5.0 and 3.1 | 14:55 |
| sean-k-mooney | hrw: ya we are not talking about min versions here | 14:56 |
| kashyap | sean-k-mooney: See my second comment on line 286: https://review.opendev.org/#/c/696834/6/nova/virt/libvirt/driver.py@286 | 14:56 |
| kashyap | sean-k-mooney: The recommended version constants we're now using fix a disk image corruption bug. | 14:56 |
| sean-k-mooney | right so we are defaulting to 5.10 for libvirt | 14:56 |
| sean-k-mooney | and what version for qemu | 14:57 |
| lyarwood | sean-k-mooney: we were | 14:57 |
| sean-k-mooney | oh you bumped it to 6.1 | 14:57 |
| lyarwood | sean-k-mooney: now it's 6.1.0 for Libvirt and QEMU 4.3.0 | 14:57 |
| sean-k-mooney | so i dont think 6.1 will be in 20.04 | 14:57 |
| sean-k-mooney | or 4.3 i think they are using 4.2 | 14:57 |
| sean-k-mooney | so im not sure this will be a good base | 14:58 |
| kashyap | lyarwood: I think you got the QEMU version wrong; now it is 4.2.0 | 14:58 |
| *** links has quit IRC | 14:58 | |
| sean-k-mooney | ya i think it should be 4.2 as well | 14:58 |
| lyarwood | ah! | 14:59 |
| sean-k-mooney | can we use 6.0 for libvirt | 15:00 |
| kashyap | :) | 15:00 |
| sean-k-mooney | i also dont think 6.1 will by in ubuntu 20.04 | 15:00 |
| kashyap | sean-k-mooney: Let me check with the libvirt dev... | 15:00 |
| sean-k-mooney | it might be in the cloud archive | 15:00 |
| hrw | ubuntu focal is on 6.0 so it will stay at 6.0 unless uca | 15:01 |
| sean-k-mooney | yep | 15:02 |
| *** ociuhandu has joined #openstack-nova | 15:02 | |
| efried_gone | bauzas: +2 | 15:03 |
| bauzas | efried_gone: with love | 15:03 |
| bauzas | go save the Openshift world | 15:03 |
| *** ociuhandu has quit IRC | 15:06 | |
| *** evrardjp has quit IRC | 15:07 | |
| *** evrardjp has joined #openstack-nova | 15:11 | |
| kashyap | sean-k-mooney: lyarwood: Good news — we can lower the libvirt version from 6.1.0 to 6.0.0, thanks to lyarwood's "provide backing file explicitly" fix: https://opendev.org/openstack/nova/commit/0cfe9c81e3fe4d | 15:12 |
| kashyap | Added a comment in the change to that effect. | 15:13 |
| lyarwood | kashyap: ack thanks | 15:13 |
| sean-k-mooney | cool so we can use libvirt 6.0.0 and qemu 4.2.0 as the minium safely | 15:13 |
| sean-k-mooney | the other way to test this would be via rdo and a centos job | 15:14 |
| sean-k-mooney | centos 8 | 15:14 |
| lyarwood | do we have an AV repo in CentOS 8? | 15:14 |
| sean-k-mooney | we should, im not sure but we definetly should | 15:15 |
| sean-k-mooney | well in centos 8 or in rdo | 15:15 |
| sean-k-mooney | we should not be testing with the defalut qemu and libvirt on centos | 15:15 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Use virDomainBlockCopy to swap volumes when using -blockdev https://review.opendev.org/696834 | 15:15 |
| * lyarwood looks | 15:16 | |
| *** ociuhandu has joined #openstack-nova | 15:16 | |
| *** ociuhandu has quit IRC | 15:16 | |
| *** TxGirlGeek has joined #openstack-nova | 15:17 | |
| *** gyee has joined #openstack-nova | 15:27 | |
| *** zhanglong has quit IRC | 15:30 | |
| *** mgariepy has joined #openstack-nova | 15:35 | |
| *** KeithMnemonic has joined #openstack-nova | 15:42 | |
| *** lbragstad has quit IRC | 15:42 | |
| hrw | 97 tempest failures on aarch64 show that it is too early for that CI job | 15:49 |
| artom | hrw, well, it's a start :) It'll probably remain non-voting and/or experimental for a bit, but those 97 failures can be chipped away at one by one | 15:55 |
| sean-k-mooney | hrw: you could add it to the periodic pipe line | 15:56 |
| hrw | ;D | 15:57 |
| sean-k-mooney | hrw: but ya as artom siad the failures can be adressed one by one | 15:57 |
| sean-k-mooney | its non voting anyway so it wont block the build | 15:57 |
| sean-k-mooney | and its in a seperate pipleline alreday so the check pipepline wont wait for it | 15:57 |
| sean-k-mooney | so it wont slow down the gate | 15:58 |
| sean-k-mooney | it proably makes sense to not merge it untill its closer to working but you can always recheck the patch | 15:58 |
| sean-k-mooney | if you do that however i would suggest disableing the other jobs | 15:58 |
| sean-k-mooney | so as to not was gate resouces | 15:59 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: CI: add tempest-integrated-compute-aarch64 job https://review.opendev.org/714439 | 16:01 |
| hrw | ok, will reedit it then back | 16:02 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: [WIP] CI: add tempest-integrated-compute-aarch64 job https://review.opendev.org/714439 | 16:03 |
| *** klindgren_ has joined #openstack-nova | 16:03 | |
| *** klindgren has quit IRC | 16:03 | |
| hrw | ok, my redeployment finished, can now teach tempest to use it | 16:04 |
| *** ociuhandu has joined #openstack-nova | 16:11 | |
| *** ociuhandu has quit IRC | 16:18 | |
| *** klindgren has joined #openstack-nova | 16:24 | |
| *** klindgren_ has quit IRC | 16:25 | |
| *** Sundar has joined #openstack-nova | 16:26 | |
| *** udesale_ has quit IRC | 16:27 | |
| gibi | /away | 16:34 |
| *** belmoreira has quit IRC | 16:43 | |
| openstackgerrit | John Garbutt proposed openstack/nova master: WIP: update quota apis with keystone limits and usage https://review.opendev.org/713499 | 16:46 |
| *** tesseract has quit IRC | 16:54 | |
| hrw | Took 15.46 seconds to build instance. | 16:55 |
| hrw | now better | 16:55 |
| *** lbragstad has joined #openstack-nova | 16:56 | |
| *** sapd1_x has joined #openstack-nova | 16:58 | |
| hrw | uf. tempest even run with it | 16:58 |
| *** sapd1_x has quit IRC | 17:03 | |
| hrw | Ran: 353 tests in 34.9147 sec. | 17:04 |
| hrw | - Passed: 0 | 17:04 |
| hrw | so back to config | 17:04 |
| dansmith | sean-k-mooney: gibi: we've got +2s pretty far up the stack, aside from one -1 from alex_xu on top of +2s, and then one -1 from gibi | 17:06 |
| dansmith | sean-k-mooney: gibi: What do you think about dropping the -2 on the bottom patch to let some of those start to flow into the gate while Sundar fixes that one -1 from gibi? | 17:07 |
| sean-k-mooney | dansmith: ya im trying to make my way through the stack today. | 17:07 |
| *** lbragstad_ has joined #openstack-nova | 17:07 | |
| dansmith | sean-k-mooney: okay you have +1s on a bunch of them too | 17:07 |
| sean-k-mooney | i think the bottom patches i have looked at so far look sane to me | 17:07 |
| sean-k-mooney | so i would be ok with droping the -2 and starting to merge those | 17:08 |
| sean-k-mooney | i have not made it to the later patches in a while but if you feel comfortable with them then i would not be against droping the -2 | 17:08 |
| *** rpittau is now known as rpittau|afk | 17:08 | |
| *** lbragstad has quit IRC | 17:09 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Replace ansible --sudo with --become in live_migration/hooks scripts https://review.opendev.org/713844 | 17:14 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713845 | 17:14 |
| *** jawad_axd has quit IRC | 17:17 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713845 | 17:18 |
| dansmith | sean-k-mooney: okay let's see what gibi thinks | 17:21 |
| *** gregwork has joined #openstack-nova | 17:24 | |
| gibi | dansmith sean-k-mooney: I think it is in a good enough shape to start merging the bottom | 17:26 |
| dansmith | gibi: ack, will drop and make sure those that can merge are +Wd | 17:26 |
| gibi | dansmith: ack, thanks | 17:27 |
| Sundar | dansmith, sean-k-mooney, gibi: Thanks. FWIW, I have started responding to gibi's -1 comments. | 17:27 |
| gibi | Sundar: ack. thanks | 17:27 |
| dansmith | Sundar: cool, if and when you propose fixes, be sure to use git review -R to avoid rebasing the patches that may be in the gate below | 17:27 |
| dansmith | gibi: alex_xu's concern on the "create and bind" is an existing problem not a new one, AFAICT.. I don't want to override his -1 but I don't think there's going to be anything we can or should do in that patch | 17:28 |
| dansmith | gibi: can you have a look at his concern and my response and see if you agree? https://review.opendev.org/#/c/631244 | 17:29 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713036 | 17:30 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713845 | 17:31 |
| sean-k-mooney | dansmith: the whole multi create process cause extra issues.. if you set like --min 2 --max 4 and 1 fails but you end up with 3 running vms then that fine right | 17:32 |
| gibi | dansmith: will check soon | 17:33 |
| sean-k-mooney | so if we do fail to create a binding im not sure if we should be killing the full multi create or just that vm. ideally we woudl reshdule just the one vm that filed right | 17:33 |
| sean-k-mooney | but you were saying the way the funtion currenly works that is not easy to do | 17:34 |
| *** evrardjp has quit IRC | 17:36 | |
| *** dtantsur is now known as dtantsur|afk | 17:36 | |
| *** evrardjp has joined #openstack-nova | 17:36 | |
| hrw | is there a tool which takes tempest and gives user readable report? "here are tests which worked, here are skipped ones. and here are failed ones with their output" instead of "here you have @%@Y*T@TGWGHWERIGWYT$(@#YTGWEGHYW as a log" | 17:36 |
| sean-k-mooney | dansmith: doing a continue after cleaning up the current instnace i think could work fine | 17:36 |
| sean-k-mooney | well work in a more intuitive way | 17:37 |
| sean-k-mooney | then killing the entire multi create because 1 instnace failed | 17:37 |
| *** dpawlik has quit IRC | 17:37 | |
| sean-k-mooney | hrw: you are using a non zuulv3 native job yes? | 17:38 |
| sean-k-mooney | hrw: im guessing the logs you are looking at are double compressed | 17:38 |
| hrw | sean-k-mooney: running tempest locally | 17:38 |
| sean-k-mooney | if that is the issue then you can fix it by doing "curl <log url> | zcat | <program of choice or file>" | 17:39 |
| sean-k-mooney | oh so its not that issue | 17:39 |
| gibi | dansmith: responded. I think the current code is OK, that loop was never designed to try multiple hosts for a single instance | 17:39 |
| *** lpetrut has joined #openstack-nova | 17:40 | |
| sean-k-mooney | gibi: but should it contiue to the next iteration and try the other instnaces? | 17:40 |
| gibi | sean-k-mooney: honestly I would not try that right now. Separately we can think about such enhancement | 17:40 |
| hrw | sean-k-mooney: I start to think that at the end I will write a python script which with load that 10-20MB json file into memory and parse to provide some sane report | 17:40 |
| gibi | sean-k-mooney: this is now simple and it cleans up properly as far as I see. | 17:41 |
| sean-k-mooney | gibi: ok dansmith raised that question in https://review.opendev.org/#/c/631244/68/nova/conductor/manager.py@1621 | 17:41 |
| gibi | and I responded there | 17:41 |
| sean-k-mooney | hrw: there is a tool upstream that renders a nice html output | 17:42 |
| dansmith | otp, bbl | 17:42 |
| sean-k-mooney | gibi: cool havent reloaded | 17:42 |
| sean-k-mooney | im going to move on to the next patch in the chain | 17:42 |
| gibi | sean-k-mooney: thanks. | 17:42 |
| gibi | I'm going offline :) | 17:42 |
| gibi | o/ | 17:42 |
| *** mmethot_ has joined #openstack-nova | 17:44 | |
| hrw | sean-k-mooney: it would be too good. | 17:44 |
| *** _mmethot_ has quit IRC | 17:46 | |
| hrw | sean-k-mooney: thanks | 18:01 |
| * hrw off | 18:01 | |
| *** dpawlik has joined #openstack-nova | 18:02 | |
| *** derekh has quit IRC | 18:02 | |
| *** prometheanfire has quit IRC | 18:05 | |
| *** prometheanfire has joined #openstack-nova | 18:21 | |
| *** hongbin has quit IRC | 18:22 | |
| *** macz_ has quit IRC | 18:27 | |
| *** macz_ has joined #openstack-nova | 18:28 | |
| *** maciejjozefczyk has quit IRC | 18:28 | |
| *** maciejjozefczyk has joined #openstack-nova | 18:29 | |
| *** amoralej is now known as amoralej|off | 18:44 | |
| *** bbowen has quit IRC | 18:48 | |
| artom | dansmith, got the energy to look at https://review.opendev.org/#/c/714228/ again? | 19:03 |
| openstackgerrit | Merged openstack/nova master: Make serialize_args handle exception messages safely https://review.opendev.org/712697 | 19:04 |
| artom | stephenfin ^^ same, but you're probably done for the day by now | 19:08 |
| *** maciejjozefczyk has quit IRC | 19:12 | |
| *** lpetrut has quit IRC | 19:12 | |
| *** Sundar has quit IRC | 19:13 | |
| *** ralonsoh has quit IRC | 19:16 | |
| *** hongbin has joined #openstack-nova | 19:18 | |
| *** maciejjozefczyk has joined #openstack-nova | 19:23 | |
| openstackgerrit | Merged openstack/nova master: Store instance action event exc_val fault details https://review.opendev.org/694428 | 19:24 |
| openstackgerrit | Merged openstack/nova master: Enable unshelve with qos ports https://review.opendev.org/705475 | 19:24 |
| openstackgerrit | Merged openstack/nova master: Merge qos related renos for Ussuri https://review.opendev.org/706766 | 19:38 |
| *** vishalmanchanda has quit IRC | 19:46 | |
| *** jawad_axd has joined #openstack-nova | 19:48 | |
| *** bbowen has joined #openstack-nova | 19:56 | |
| *** spatel has joined #openstack-nova | 20:04 | |
| *** ccamacho has quit IRC | 20:05 | |
| *** maciejjozefczyk has quit IRC | 20:07 | |
| *** maciejjozefczyk has joined #openstack-nova | 20:07 | |
| spatel | sean-k-mooney: is rcp_thread_pool_size has been replaced with executor_thread_pool_size ? | 20:12 |
| spatel | https://docs.openstack.org/nova/queens/configuration/config.html#executor_thread_pool_size | 20:13 |
| sean-k-mooney | https://docs.openstack.org/nova/queens/configuration/config.html#DEFAULT.executor_thread_pool_size | 20:14 |
| *** maciejjozefczyk has quit IRC | 20:14 | |
| sean-k-mooney | according to the docs yes | 20:14 |
| spatel | cool, i will use executor_thread_pool_size :) | 20:15 |
| *** nightmare_unreal has quit IRC | 20:17 | |
| *** jawad_axd has quit IRC | 20:28 | |
| *** jawad_axd has joined #openstack-nova | 20:29 | |
| spatel | sean-k-mooney: what do you suggest about heal_instance_info_cache_interval value, its ok to keep it little high (default: 60) | 20:32 |
| sean-k-mooney | in theory it should not be needed at all. it is intended to ensure that nova and neutron does not get out of sync if you modify attributes of the neutron port out of band of nova | 20:33 |
| sean-k-mooney | so yes it can be set longer that said it should not be disabled | 20:33 |
| sean-k-mooney | mainly because there is not testing for running without the healing. | 20:33 |
| *** jawad_axd has quit IRC | 20:33 | |
| spatel | i am trying to reduce load on neutron-servers so trying to tune some value | 20:34 |
| *** nweinber has quit IRC | 20:35 | |
| spatel | as VMs grows it put more pressure on neutron | 20:35 |
| sean-k-mooney | you should be able to set it to 300 safely enough | 20:38 |
| spatel | cool | 20:41 |
| *** dpawlik has quit IRC | 20:43 | |
| *** mgariepy has quit IRC | 20:45 | |
| *** dpawlik has joined #openstack-nova | 20:53 | |
| *** Sundar has joined #openstack-nova | 21:03 | |
| sean-k-mooney | dansmith: gibi Sundar just finished reviewing the cyborg series. i noticed a few nits all of which could be adressed in a follow up patch but one thing is definetly missing. i think we all agree that cyborg integration qualifies for a release note but i dont see one in the series | 21:15 |
| sean-k-mooney | we could add it in the followup patch when adressing some fo the nits but strictly speaking i belive it should be added here https://review.opendev.org/#/c/673735/46 and then update by each of the follow on patches to reflect the different capablities | 21:16 |
| sean-k-mooney | that said we can just add it in https://review.opendev.org/#/c/674726/9 which is the last functional patch | 21:16 |
| *** dpawlik has quit IRC | 21:20 | |
| Sundar | sean-k-mooney: Ack. I'll take a look | 21:21 |
| Sundar | Thanks for the reviews. | 21:21 |
| Sundar | Any luck with testing the rescheduling? | 21:21 |
| *** artom has quit IRC | 21:27 | |
| sean-k-mooney | i have not had a chance to test it but i might get to do it tomororow. i will be redeploying the latest version tomorow in anycase | 21:33 |
| sean-k-mooney | i was condiering locally modifying the spawn to fail on one node to then assert the correct behaviro for reschdule | 21:33 |
| *** artom has joined #openstack-nova | 21:38 | |
| *** artom has quit IRC | 21:38 | |
| *** artom has joined #openstack-nova | 21:39 | |
| *** xek has quit IRC | 21:39 | |
| sean-k-mooney | artom: whitebox is breaking thing it should not be able to break... | 21:42 |
| sean-k-mooney | https://zuul.opendev.org/t/openstack/build/bc5fd77019b74363ab3b06478fd386de | 21:42 |
| sean-k-mooney | artom: its causing tempest-tox-plugin-sanity-check to fail | 21:42 |
| sean-k-mooney | . | 21:43 |
| artom | Whoa | 21:43 |
| sean-k-mooney | i have no idea why its checking whitebox either | 21:43 |
| sean-k-mooney | but we should proably fix that | 21:43 |
| sean-k-mooney | its becasue we merged https://opendev.org/x/whitebox-tempest-plugin/commit/c1285626fee890a00f247ff0d0bde1249b90a59f | 21:44 |
| artom | sean-k-mooney, right, I get the error | 21:44 |
| artom | I just have no idea how we ended up in that job | 21:44 |
| artom | TO my knowledge we didn't "register" anywhere as a tempest plugin | 21:44 |
| sean-k-mooney | did your governance patch merge | 21:44 |
| artom | No | 21:44 |
| sean-k-mooney | then ya im not sure either | 21:45 |
| sean-k-mooney | i guess we need to set a default for the config | 21:45 |
| sean-k-mooney | so 0? | 21:45 |
| artom | I was gonna say - that's probably the easiest fix | 21:45 |
| artom | Yeah | 21:45 |
| sean-k-mooney | shoudl we also add that job to whitebox | 21:45 |
| artom | Let's fix it for now | 21:45 |
| sean-k-mooney | ok want me to do it now or will i leave it to you | 21:46 |
| artom | sean-k-mooney, we need each other to review it :) | 21:46 |
| sean-k-mooney | yes but in this case i think we can self approve | 21:46 |
| sean-k-mooney | once the other revies its also trival | 21:46 |
| *** mriedem has left #openstack-nova | 21:49 | |
| *** mgariepy has joined #openstack-nova | 21:50 | |
| artom | sean-k-mooney, https://review.opendev.org/#/c/714539/ | 21:52 |
| sean-k-mooney | artom: hehe you bet me too it | 21:52 |
| artom | sean-k-mooney, ah, I think that tempest job dynamically gets a list of projects that are tempest plugins from gerrit: https://opendev.org/openstack/tempest/src/branch/master/tools/generate-tempest-plugins-list.py | 21:53 |
| sean-k-mooney | ya so i guess its trying to make sure that changes to tempest dont break plugins | 21:54 |
| artom | Risky as hell though | 21:55 |
| artom | Though they can blacklist projects too | 21:55 |
| artom | So maybe not to much | 21:55 |
| artom | *so much | 21:55 |
| artom | I'll email the ML | 21:55 |
| sean-k-mooney | ya. well i think that reinforces the idea we proably should run that job on the mailing list but on the plus side they should be warned if they would breake whitebox atleast to some degree | 21:56 |
| artom | sean-k-mooney, not sure how running that job would work, it depends on scripts in the tempest rpo | 22:01 |
| artom | *repo | 22:01 |
| *** Sundar has quit IRC | 22:01 | |
| artom | Anyways, we can continue this later, I need to make supper | 22:01 |
| sean-k-mooney | ya im also done for the day. but runing that job would use the script form the tempest repo | 22:02 |
| sean-k-mooney | it would tell use if we were making a change that woudl cause it to fail before we merge it | 22:02 |
| artom | OK, so should work then | 22:02 |
| artom | But yeah, tomorrow | 22:03 |
| artom | o/ | 22:03 |
| sean-k-mooney | o/ | 22:03 |
| *** spatel has quit IRC | 22:33 | |
| *** tkajinam has joined #openstack-nova | 22:57 | |
| *** gibi has quit IRC | 23:03 | |
| *** slaweq has quit IRC | 23:06 | |
| *** brinzhang has quit IRC | 23:16 | |
| openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | 23:42 |
| openstack | bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929 | 23:42 |
| *** brinzhang has joined #openstack-nova | 23:55 | |
| *** tetsuro has joined #openstack-nova | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!