Wednesday, 2019-10-02

*** brinzhang has joined #openstack-nova00:04
*** sapd1_x has joined #openstack-nova00:07
*** brinzhang_ has quit IRC00:07
*** gyee has quit IRC00:08
*** igordc has quit IRC00:15
*** tetsuro has joined #openstack-nova00:18
*** tetsuro has quit IRC00:20
*** hongbin has joined #openstack-nova00:30
*** brinzhang_ has joined #openstack-nova00:31
*** brinzhang has quit IRC00:34
*** gbarros has quit IRC00:45
*** hongbin has quit IRC00:50
*** ricolin has joined #openstack-nova00:59
*** brinzhang has joined #openstack-nova01:03
*** brinzhang has quit IRC01:05
*** brinzhang has joined #openstack-nova01:05
*** brinzhang has joined #openstack-nova01:06
*** brinzhang_ has quit IRC01:06
*** brinzhang_ has joined #openstack-nova01:23
*** brinzhang has quit IRC01:27
*** ricolin has quit IRC01:34
*** ricolin has joined #openstack-nova01:35
*** markvoelker has quit IRC01:35
openstackgerritJing Zhang proposed openstack/nova master: Force small pages accounting for CPU pinned VMs so that memory can be reserved on NUMA 0 for host processes.  https://review.opendev.org/68607901:36
*** spsurya has joined #openstack-nova01:47
*** irclogbot_2 has quit IRC02:09
*** irclogbot_0 has joined #openstack-nova02:12
*** SonPham has joined #openstack-nova02:17
*** SonPham has quit IRC02:18
*** sapd1_x has quit IRC02:19
*** SonPham has joined #openstack-nova02:20
*** SonPham has joined #openstack-nova02:23
SonPhamHi, I want to commit my little project02:24
SonPhamIt is a small module to snapshot VM and keep running process02:24
SonPhamwhat workflow i need to do?02:25
*** alex_xu has joined #openstack-nova02:28
*** tetsuro has joined #openstack-nova02:31
*** tetsuro has quit IRC02:33
*** brinzhang has joined #openstack-nova02:44
*** brinzhang_ has quit IRC02:47
*** markvoelker has joined #openstack-nova03:10
*** BjoernT has quit IRC03:16
*** Jeffrey4l has quit IRC03:38
*** Jeffrey4l has joined #openstack-nova03:40
*** brinzhang_ has joined #openstack-nova03:46
*** brinzhang has quit IRC03:50
*** sapd1_x has joined #openstack-nova04:20
*** henriqueof has quit IRC04:33
*** henriqueof1 has joined #openstack-nova04:33
*** mmethot_ has joined #openstack-nova04:36
*** jangutter_ has joined #openstack-nova04:36
*** ricolin_ has joined #openstack-nova04:36
*** takashin has joined #openstack-nova04:37
*** kaisers has quit IRC04:37
*** cgoncalves has quit IRC04:37
*** kaisers has joined #openstack-nova04:38
*** jangutter has quit IRC04:38
*** bbobrov has quit IRC04:38
*** bbobrov has joined #openstack-nova04:40
*** ricolin has quit IRC04:40
*** sean-k-mooney has quit IRC04:40
*** mmethot has quit IRC04:40
*** eandersson has quit IRC04:40
SonPhamHi, I want to commit my little project04:40
*** dasp has quit IRC04:40
*** sean-k-mooney has joined #openstack-nova04:40
*** jamesdenton has quit IRC04:41
*** jamesdenton has joined #openstack-nova04:44
*** cgoncalves has joined #openstack-nova04:46
*** pcaruana has joined #openstack-nova04:52
*** Luzi has joined #openstack-nova05:21
*** sapd1_x has quit IRC05:34
*** dpawlik has joined #openstack-nova05:36
*** icarusfactor has joined #openstack-nova05:39
*** factor has quit IRC05:39
*** mkrai has joined #openstack-nova05:57
*** rcernin_ has joined #openstack-nova05:59
*** rcernin has quit IRC06:01
*** dpawlik has quit IRC06:06
*** ileixe_ has joined #openstack-nova06:11
*** dpawlik has joined #openstack-nova06:11
*** ileixe_ is now known as ileixe06:12
*** avolkov has joined #openstack-nova06:29
*** brinzhang_ has quit IRC06:32
*** brinzhang_ has joined #openstack-nova06:32
*** luksky has joined #openstack-nova06:35
*** ttsiouts has joined #openstack-nova06:39
*** dasp has joined #openstack-nova06:40
*** icarusfactor has quit IRC06:41
*** rcernin_ has quit IRC06:45
*** rcernin has joined #openstack-nova06:45
gibisean-k-mooney: neutron creates child RPs under the compute RP and reports bandwidth inventory on the child RPs only. Also the resource allocation is done by nova as part of the instance (and migration) allocation. So the zun and the neturon case is pretty different06:47
SonPhamHi, I want to commit my little project06:48
*** xek has joined #openstack-nova06:55
*** tesseract has joined #openstack-nova06:57
*** mkrai has quit IRC06:57
*** damien_r has joined #openstack-nova07:02
gibiSonPham: hi! most of the core developers are active after 12:00 UTC. The contribution process is documented here https://docs.openstack.org/nova/latest/contributor/index.html07:02
*** brinzhang has joined #openstack-nova07:03
*** jaosorior has joined #openstack-nova07:03
*** rcernin has quit IRC07:04
*** brinzhang_ has quit IRC07:05
*** maciejjozefczyk has joined #openstack-nova07:06
gibiSonPham: regarding your feature. How does it different from the existing snapshot feature of OpenStack?07:09
*** slaweq has joined #openstack-nova07:10
SonPhamgibi i'm using libvirt for snapshot07:10
SonPhamit can keep running process07:11
SonPhambut volume image type must be change from RAW to QCOW207:11
*** awalende has joined #openstack-nova07:13
gibiSonPham: I think nova can also do live snapshot07:15
gibiSonPham: the relevant codepath starts around here https://github.com/openstack/nova/blob/bf37bec80baa527ac013dfaa7480ef2761ed2cb9/nova/virt/libvirt/driver.py#L226707:17
openstackgerritOpenStack Proposal Bot proposed openstack/nova stable/train: Imported Translations from Zanata  https://review.opendev.org/68553407:19
*** ttsiouts has quit IRC07:26
*** ttsiouts has joined #openstack-nova07:26
SonPhamgibi i think openstakc snapshot is an image of disk. not keep process07:30
*** ttsiouts has quit IRC07:31
gibiSonPham: yes, it creates a copy of the disk while the libvirt domain is still running07:31
gibiSonPham: you what I mean is that the existing snapshot code does not require to shutdown the domain07:33
gibis/you//07:33
gibiSonPham: do you mean that in your case the snapshot itself stores the running state of the whole domain?07:34
SonPhamgibi my project create snapshot when domain running.  but when you revert snapshot. all of process still keep lock like you snapshot07:35
SonPhamgibi yep . stores the running state of the whole domain07:37
gibiSonPham: I see07:37
gibiSonPham: I suggest to try to propose a feature specification07:37
gibiSonPham: the nova spec repo is here https://opendev.org/openstack/nova-specs07:37
*** ralonsoh has joined #openstack-nova07:38
*** slaweq has quit IRC07:38
gibiSonPham: you basically need to fill the spec template https://opendev.org/openstack/nova-specs/src/branch/master/specs/ussuri-template.rst07:38
gibiSonPham: and open a blueprint pointing to the spec https://blueprints.launchpad.net/nova07:39
*** tetsuro has joined #openstack-nova07:41
*** tetsuro has quit IRC07:46
*** ivve has joined #openstack-nova07:56
*** ttsiouts has joined #openstack-nova07:56
*** slaweq has joined #openstack-nova07:57
*** rpittau|afk is now known as rpittau07:57
*** takashin has left #openstack-nova08:00
brinzhanggibi: I saw the code as you, and there is an configuration https://docs.openstack.org/nova/stein/configuration/config.html#workarounds.disable_libvirt_livesnapshot08:03
*** tkajinam has quit IRC08:04
brinzhanggibi: it seems that there is not has the API provide for user to do the live-snapshot,I think it's not support good enough(https://docs.openstack.org/nova/stein/configuration/config.html#workarounds.disable_libvirt_livesnapshot)08:05
brinzhanggibi: I didn't go deep into the logic of this piece :P08:06
*** mkrai has joined #openstack-nova08:07
*** markvoelker has quit IRC08:09
*** ttsiouts has quit IRC08:12
*** ttsiouts has joined #openstack-nova08:12
*** brinzhang_ has joined #openstack-nova08:16
*** brinzhang_ has joined #openstack-nova08:17
*** ttsiouts has quit IRC08:17
*** brinzhang_ has quit IRC08:17
*** brinzhang_ has joined #openstack-nova08:17
*** ttsiouts has joined #openstack-nova08:18
*** brinzhang_ has quit IRC08:18
*** tssurya_ has joined #openstack-nova08:19
*** brinzhang has quit IRC08:20
*** ileixe has quit IRC08:24
*** lpetrut has joined #openstack-nova08:30
*** dpawlik has quit IRC08:31
*** ociuhandu has joined #openstack-nova08:32
*** ociuhandu has quit IRC08:32
*** ociuhandu has joined #openstack-nova08:33
*** ociuhandu has quit IRC08:34
*** ociuhandu has joined #openstack-nova08:35
*** jaosorior has quit IRC08:37
*** jawad_axd has joined #openstack-nova08:38
*** derekh has joined #openstack-nova08:38
*** ricolin_ is now known as ricolin08:41
*** ileixe has joined #openstack-nova08:46
*** panda has quit IRC08:48
*** dpawlik has joined #openstack-nova08:49
*** panda has joined #openstack-nova08:49
*** tetsuro has joined #openstack-nova09:03
*** tetsuro has quit IRC09:08
*** mkrai_ has joined #openstack-nova09:12
openstackgerritBalazs Gibizer proposed openstack/nova master: Do not print default dicts during heal_allocations  https://review.opendev.org/68600109:17
openstackgerritBalazs Gibizer proposed openstack/nova master: Test heal port allocations in nova-next  https://review.opendev.org/66987909:18
gibistephenfin: fixed your comments in ^^09:18
*** tssurya_ is now known as tssurya09:19
gibistephenfin: unfortunately we dont have jq in the environment09:20
*** ab-a has quit IRC09:30
*** ab-a has joined #openstack-nova09:31
*** tetsuro has joined #openstack-nova09:39
*** tetsuro has quit IRC09:45
*** kaisers has quit IRC09:52
*** jaosorior has joined #openstack-nova09:52
*** kaisers has joined #openstack-nova09:53
*** ociuhandu has quit IRC10:02
*** ociuhandu has joined #openstack-nova10:03
*** sapd1_x has joined #openstack-nova10:05
*** ociuhandu has quit IRC10:08
*** markvoelker has joined #openstack-nova10:10
*** tetsuro has joined #openstack-nova10:13
*** sapd1_x has quit IRC10:14
*** markvoelker has quit IRC10:15
openstackgerritBalazs Gibizer proposed openstack/nova master: Add a way to spy on function calls in test  https://review.opendev.org/68594910:17
*** tetsuro has quit IRC10:18
openstackgerritBalazs Gibizer proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68595010:18
*** mkrai_ has quit IRC10:20
*** mkrai has quit IRC10:20
*** rcernin has joined #openstack-nova10:24
*** ttsiouts has quit IRC10:32
*** ivve has quit IRC10:32
*** ttsiouts has joined #openstack-nova10:33
*** ttsiouts has quit IRC10:37
*** slaweq has quit IRC10:38
*** zbr|ruck is now known as zbr|lunch10:39
*** slaweq has joined #openstack-nova10:42
*** tetsuro has joined #openstack-nova10:48
*** SonPham has quit IRC10:49
*** slaweq has quit IRC10:51
*** tetsuro has quit IRC10:53
*** SonPham has joined #openstack-nova10:57
*** slaweq has joined #openstack-nova10:57
*** ociuhandu has joined #openstack-nova10:58
*** jaosorior has quit IRC11:02
*** dpawlik has quit IRC11:03
*** sapd1_x has joined #openstack-nova11:03
*** openstackstatus has quit IRC11:09
*** rnoriega- has quit IRC11:09
*** rchurch has quit IRC11:10
*** rchurch has joined #openstack-nova11:13
*** rnoriega_ has joined #openstack-nova11:13
*** ttsiouts has joined #openstack-nova11:14
*** dpawlik has joined #openstack-nova11:14
*** SonPham has quit IRC11:15
*** jaosorior has joined #openstack-nova11:28
*** rpittau is now known as rpittau|afk11:38
*** sapd1_x has quit IRC11:44
*** artom has joined #openstack-nova11:47
*** ivve has joined #openstack-nova11:56
*** gbarros has joined #openstack-nova12:16
*** derekh has quit IRC12:27
*** gbarros has quit IRC12:32
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Use nova-net explicitly in functional tests  https://review.opendev.org/68432612:38
*** nweinber has joined #openstack-nova12:47
*** tbachman has quit IRC12:49
*** derekh has joined #openstack-nova12:59
*** gbarros has joined #openstack-nova13:00
*** ociuhandu has quit IRC13:06
*** ociuhandu has joined #openstack-nova13:07
*** mriedem has joined #openstack-nova13:08
*** jaosorior has quit IRC13:08
*** ociuhandu has quit IRC13:11
*** ivve has quit IRC13:12
*** luksky has quit IRC13:14
*** dpawlik has quit IRC13:20
*** jawad_axd has quit IRC13:27
*** jawad_axd has joined #openstack-nova13:28
*** jawad_ax_ has joined #openstack-nova13:30
*** ociuhandu has joined #openstack-nova13:31
*** jawad_axd has quit IRC13:33
*** jawad_ax_ has quit IRC13:34
*** Luzi has quit IRC13:40
openstackgerritEric Fried proposed openstack/nova master: Improve spy CM's data passing  https://review.opendev.org/68616713:41
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: lxc: make use of filter python3 compatible  https://review.opendev.org/67650013:41
*** luksky has joined #openstack-nova13:42
*** tbachman has joined #openstack-nova13:45
*** zbr|lunch is now known as zbr|ruck13:48
efriedgibi: I'm having way too much fun with spy13:49
*** henriqueof1 has quit IRC13:51
*** zul has joined #openstack-nova13:51
*** mkrai_ has joined #openstack-nova13:52
*** mkrai has joined #openstack-nova13:53
*** markvoelker has joined #openstack-nova13:53
*** jaosorior has joined #openstack-nova13:54
gibiefried: I know.I feel the same thing :)13:55
openstackgerritEric Fried proposed openstack/nova master: Improve spy CM's data passing  https://review.opendev.org/68616713:55
gibiefried: I try to answered your concerns about accessing _spy in the body of _spy()13:56
dansmithefried: gibi: maybe I'm missing something, but why isn't this as easy as: https://pastebin.com/b0g9ZFHS ?13:56
dansmiththat mock-wraps the original thing with just one line but leaves you to examine the regular mock attributes13:56
mriedemgibi: replied in https://review.opendev.org/#/c/686047/13:56
efrieddansmith: it allows you to examine the args, but it never runs the original foo()13:57
gibidansmith: is some tests we need to run the original function13:57
dansmithefried: yes it does13:57
dansmithefried: run it13:57
efriedoic13:57
mriedemnote that mock.patch(wraps=...) does that as well13:57
dansmithgibi: that runs the original13:57
dansmithah, didn't know about wraps13:58
*** markvoelker has quit IRC13:58
gibidansmith: let me try that with a bound method13:58
mriedemwraps is kind of weird to use13:58
mriedemnot documented very well13:58
efriedWhat spy does is allow you to keep track of things per invocation of the wrapped method, which wraps= doesn't13:58
*** awalende has quit IRC13:59
efriedalso selectively change the behavior13:59
efriedlike "run normally the first time, raise the second time"13:59
dansmithI don't see how it's different from side_effect13:59
gibidansmith: side_effect is not good for calling the original if the original is a bound method http://paste.openstack.org/show/780754/14:04
dansmithgibi: that's because you're mocking the class not the object14:05
dansmithI think  you're just using your spy in a different order14:05
dansmithif you mock.patch.object(a) I think you'll be fine14:05
gibidansmith: but in the functional test when we mock we tend not to have easy access to the instantiated objects just the classes14:06
dansmithin that one scenario you mean, but not necessarily as a general rule14:06
*** panda has quit IRC14:06
dansmithefried's use of spy in the patch above that does not seem to be very clean to me, especially for a supposed functional test14:07
gibidansmith: my goal is to want to get rid of (or contain) the scope magic and the original function handling in cases like https://github.com/openstack/nova/blob/bf37bec80baa527ac013dfaa7480ef2761ed2cb9/nova/tests/functional/regressions/test_bug_1843090.py#L10014:07
dansmithbut anyway, not saying it's not useful, it just seems like in a lot of cases, you could get away with base mock stuff instead of having to learn a new convention14:07
efriedoh, yeah, in the vast majority of cases14:08
*** panda has joined #openstack-nova14:08
gibispy is not a replacement for mock. There are cases where I think mock is hard to use to achive the same thing14:08
gibilike the funct test I linket above14:08
dansmithgibi: understand. using side_effect to provide a sequence of run,fail is more conventional than what either the old or new code in that patch is doing14:08
dansmithassuming you can hook at the right spot to mock the object itself14:09
efrieddansmith: I don't think you can use side_effect to run,fail14:09
dansmithefried: of course you can14:09
dansmithefried: provide it a list and it will iterate on each call14:09
efriedbecause when you pass an iterable to side_effect, it's return values, not callables.14:09
dansmithnope14:10
dansmithreturn_value behaves like that, side_effect does not14:10
*** mriedem has quit IRC14:10
efriedside_effect=one_thing calls one_thing14:10
efriedside_effect=[one, two] *returns* one followed by two14:10
gibidansmith: btw, mocking f through the instance doesn't work either http://paste.openstack.org/show/780755/14:10
*** mriedem has joined #openstack-nova14:10
dansmithefried: "If side_effect is an iterable then each call to the mock will return the next value from the iterable."14:10
efried^14:11
*** mkrai has quit IRC14:11
*** mkrai_ has quit IRC14:11
dansmithefried: https://pastebin.com/bLpt6Skz14:11
dansmithgibi: it's all about calling convention, it's tricky and magical, but you're not doing anything different other than the ordering14:12
efrieddansmith: yup, that didn't print "Foo ran 1"14:12
efriedtry to get it to do that the first time, and raise the second14:13
gibidansmith: tell me how can I call the original with side_effect if the original is a bound method because I failed to find a way14:13
efriedgibi: you can't14:13
dansmithefried: it printed 1, my local env has a different foo14:13
dansmithefried: wait, what are you arguing? that it didn't run foo? I didn't ask it to14:14
efriedI know. And you can't.14:14
efriedthat's what spy helps with14:14
gibiexaclty what efried is asking is what I'm after. having the original function is called while spying on the call14:14
dansmithah, I see what you mean.. a lambda in the list will solve that14:15
gibidansmith: no, the lambda would be returned, not called14:15
efried^14:15
dansmithgibi: having the original called is what side_effect=callable will do, but the list, fine fine, I see14:15
gibidansmith: but I cannot do that^^ if the original is a bound method as mock does not pass proper arglist to the callable14:16
dansmithyou guys seem to be describing a world where spy will make it easier to not write terrible tests like what efried is adding (and what he replaced), yet he's using spy to (if called once, then explode)14:16
efriedwhat did I write?14:16
dansmithefried: this: https://review.opendev.org/#/c/685950/3/nova/tests/functional/regressions/test_bug_1843090.py14:17
*** mkrai has joined #openstack-nova14:17
*** mkrai_ has joined #openstack-nova14:17
efriedgibi gets credit for those14:17
* gibi takes credit for the original of that14:17
efriedI think14:17
dansmithoh sorry, I thought you were commenting on your own14:17
dansmithI should have provided a url instead of a name :)14:18
gibidansmith: I want to make the test in test_bug_1843090.py better with spy. If there is other way to make it better then I'm open to suggestion14:18
*** jaosorior has quit IRC14:18
dansmithgibi: okay, but you're not in that change, right? you're still counting calls and failing once it's been called14:19
dansmithand doing gross stuff like mangling the original object to tell if you've been called twice14:19
dansmithso where is your call,fail bit you said was easy with spy?14:19
gibidansmith: we need the logic like first call fails to trigger a re-schedule but the second succeds so that the re-schedule succeeds.14:20
artomActually yeah14:20
dansmithfine, where is your fail,call ? :)14:20
gibidansmith: is there a nicer way to code that?14:20
artomWhat we're doing in https://review.opendev.org/#/c/685950/3/nova/tests/functional/regressions/test_bug_1843090.py is exactly what we said we shouldn't do in https://review.opendev.org/#/c/685950/2/nova/tests/functional/compute/test_live_migration.py@2314:20
dansmithgibi: well, for one thing you should keep track of your mock calls outside of the runtime objects you're handling14:21
efrieddansmith: Checking the actual mock's call_count is hard because you have to define the method before you create the mock before you define the method, but you have to reference the mock from within the method.14:21
dansmithgibi: point being I'm not sure what spy is making nicer about that test because it seems like the same ickiness, but with a different calling convention14:21
gibidansmith: it hides the scope magic efried is just described14:22
gibidansmith: it automates the call to the original function14:22
dansmithefried: everywhere else we keep a list above the scope and chuck things into there inside the handler14:22
dansmithgibi: so it saves the one line of saving a reference to the original method?14:22
gibidansmith: and it hides the outer scope list14:22
dansmithbut replaces it with the terrible hack of modifying the runtime objects instead of keeping that fully within the test!14:23
efriedand when there are multiple calls to the same method nested inside whatever's happening inside the context manager, it gives you a way to introspect them individually14:23
gibidansmith: and it remembers that we have to use new= instead of side_effect if the mocked function is bound method14:23
efriedwith mock.patch:14:23
efried  do_thing()14:23
efrieddo_thing calls foo() N times with different args or whatever14:23
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for migrate part of bug 1781286  https://review.opendev.org/68601714:23
openstackbug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,In progress] https://launchpad.net/bugs/1781286 - Assigned to Matt Riedemann (mriedem)14:23
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580714:23
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during build reschedule  https://review.opendev.org/68604714:23
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during migrate reschedule  https://review.opendev.org/68605014:23
openstackgerritMatt Riedemann proposed openstack/nova master: Update cells v2 up-call caveats doc  https://review.opendev.org/68605314:23
efrieddansmith: I also suggested a way we don't have to do the14:24
efried spy.claim_calls_on_node = getattr(spy, 'claim_calls_on_node', [])14:24
efriedthing, which puts creation of the tracker container back in the hands of the caller.14:24
efriedsame #LOC, just a little more scrutable14:25
dansmithefried: yes, initializing that "context" elsewhere would be a lot better14:25
dansmithanyway, it seems to do a lot for you and gibi, that's all that matters. it's something else I have to learn to comprehend the test, but...14:25
efriedgibi: I can work that ^ up if you like, at least for demonstration purposes.14:26
efriedmaybe it'll read more intuitively to dansmith14:26
*** BjoernT has joined #openstack-nova14:26
*** markvoelker has joined #openstack-nova14:26
dansmithno, not more intuitively, it'll just make the handler less gross14:26
*** openstackstatus has joined #openstack-nova14:28
*** ChanServ sets mode: +v openstackstatus14:28
mriedemi found out yesterday that we patch mock.patch globally using something in oslotest for fixing the autospec'ing stuff - what claudiub worked on a long time ago. i saw those changes merge but didn't realize we were patching the mock library.14:28
mriedemi would not be surprised when we go to a new version and that blows up in our faces14:28
openstackgerritMerged openstack/nova stable/train: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606614:29
*** awalende has joined #openstack-nova14:30
*** rcernin has quit IRC14:30
gibiefried, dansmith: fine. I'm out of steam. I think I get that we cannot remove the full uglyness of these tests. I think we can move and contain some the uglyness. But I also get that it makes the test less explicit. so meh14:31
dansmithgibi: I appreciate your efforts to improve :)14:32
mriedemgibi: if it makes you feel better, i was able to use mock.patch context manager in that migrate reschedule functional test :)14:32
gibimriedem: I haven't tried mock.patch(wraps=) recently. so it might work14:32
mriedemusing wraps= is awkward in my experience14:32
mriedemside_effect is more straightforward, but maybe that's just b/c i have more experience with it14:33
gibimriedem: and patching the mock library feels baaaad14:33
mriedemwe'll find out about ^ but this is what we call https://github.com/openstack/oslotest/blob/master/oslotest/mock_fixture.py#L18714:33
gibidansmith: I guess I will go and fined another tech debt and work on that a bit :)14:34
*** awalende has quit IRC14:34
gibis/fined/find/14:34
gibimriedem: that patching might even be relevant to my problem with side_effect and bound methods as it touches 'self'14:36
mriedemtoo much touching of self can lead to problems14:36
mriedemi've heard14:36
mriedem...14:36
gibi...14:36
dansmithmriedem: comment here: https://review.opendev.org/#/c/686017/314:38
*** kaisers has quit IRC14:39
mriedemi can probably keep a handle to the mock and assert it's called, but if you remove the mock the test fails so i'm not sure it's worthwhile14:39
mriedemactually as written i should confirm that statement,14:41
mriedembecause it probably fails anyway since the stub always raises right not, not just on the first host14:41
mriedem*now14:41
mriedemdansmith: replied inline with options14:42
dansmithmriedem: that's why I'm asking.. just seems like since we can't see the actual failure in the fault, we should make sure it's hitting what we think it is14:42
dansmithmriedem: I'd rather the explicit assertCalled personally, but it was just a suggestion anyway14:44
*** mkrai_ has quit IRC14:47
*** mkrai has quit IRC14:47
*** mkrai__ has joined #openstack-nova14:47
*** luksky has quit IRC14:48
mriedemok i can tinker with it, gonna be a bit since i'm in a meeting in 10 minutes14:50
*** mkrai__ has quit IRC14:51
*** mlavalle has joined #openstack-nova14:52
dansmithack, I dropped a few other comments on that series,14:52
dansmithbut overall..thanks for closing that loop.. it always seemed like a big deal so I never wanted to even start, but you made it look easy14:52
*** henriqueof has joined #openstack-nova14:55
* mriedem blushes14:57
mnasersean-k-mooney: cpu/libvirt-y related question, is it possible that not all flags are passed down to the vm even with host-passthrough ?14:59
*** JamesBenson has joined #openstack-nova14:59
*** jangutter_ has quit IRC15:00
sean-k-mooneyi belive that yes there are 1 or 2 instruction that are not passsed but in general no15:00
mnaserok i have way more than 1 or 2 that arent passed through in this case..15:01
sean-k-mooneythe ones that are not passed releated to things like system management mode/model specic registers15:02
sean-k-mooneymnaser: which ones?15:02
mnaserhttps://www.irccloud.com/pastebin/MAa6LBVo/15:03
mnasersean-k-mooney: ^15:03
mnaserwait sorry, thats what is exposed15:04
mnaserwhat _doesnt_ get exposed is15:04
sean-k-mooneythey are the one not passed through?15:04
sean-k-mooneyok that makes more sense15:04
mnaserhttps://www.irccloud.com/pastebin/6qVAPHMx/15:04
mnaserthose ^15:04
sean-k-mooneyi was like how is it working at all with out a floating point unit15:04
mnaserloll15:04
sean-k-mooneyi need to jump on an internal call for an hour mind if we come back to this after15:05
sean-k-mooneybut first glance several of them are not verutaliable or are cpu bug flags15:06
mnaseryeah it looked like a lot of "management" things15:06
mnaserlike seeing stuff like hw_pstate15:06
sean-k-mooneyya the svm ones are amd nested virt flags which need to be enabled in the kvm module sperately15:07
sean-k-mooneyht is hyper treading and is only expose dif you configre the guest to have 2 thrads15:08
sean-k-mooneye.g. hw:cpu_threads=215:08
sean-k-mooney"cat" "cdp" and "rdt" freatures cant be viertualsed15:09
openstackgerritBalazs Gibizer proposed openstack/nova master: Refine comments about move_allocations  https://review.opendev.org/68618215:09
mnaserokay so it looks like things are okay15:09
sean-k-mooneyyep15:09
sean-k-mooneyill look at the list again later but first glance i dont see anygin alarming15:10
kashyapmnaser: Ideally all flags from host will be passed-through as-is15:10
kashyap(I haven't read the whole scrollback, though)15:10
mnaserkashyap: that was my theory but i figured there might some blacklist of "things i will not pass because a vm shouldnt have them"15:11
mnaserhttps://www.irccloud.com/pastebin/6qVAPHMx/15:11
mnaser^ the list that didnt show up15:11
kashyapmnaser: What's the host processor?15:13
mnaserkashyap: AMD EPYC 740215:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Remove explicit 'USE_NEUTRON = True'  https://review.opendev.org/68593715:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Remove explicit 'USE_NEUTRON = True'  https://review.opendev.org/68593715:14
kashyapmnaser: Maybe silly question, did you cold-reboot the guest?15:16
kashyap(A bit slow, as on a meeting)15:16
kashyaps/on/I'm on/15:16
mnaserkashyap: yeah, and the 'proof' of that is the model name in the VM went from "AMD EPYC Processor (with IBPB)" to "AMD EPYC 7402 24-Core Processor"15:17
*** macz has joined #openstack-nova15:17
kashyapmnaser: Aside: your guest doesn't have 'virt-ssbd', 'amd-ssbd', -- that means your microcode/QEMU/libvirt versions are older.15:21
mnaserkashyap: this is latest rdo so qemu 2.12.0, libvirt 4.5.015:22
mnaserfor microcodes, amd stuff is all new to me :)15:22
*** macz has quit IRC15:22
kashyap(Nod.)  There's a QEMU web page that tells you what features you need based on your CPU15:24
mnaseryeah, well, i'll see and dig into it15:24
*** markvoelker has quit IRC15:25
kashyapmnaser: Isn't that brand new CPU?15:25
mnaseryes, like released on the 7th of august :p15:26
mnaserso part of me is also wondering "oh i wonder if we have something odd here"15:26
kashyap[https://github.com/qemu/qemu/blob/master/docs/qemu-cpu-models.texi]  Sorry for the 'texi' file15:26
kashyapmnaser: Yeah, darn, was checking on WikiChip15:26
kashyapmnaser: Okay, then here are the reasons why your guest don't see those features (also confirmed w/ a libvirt dev):15:26
mnaserthey're pretty badass :)15:26
kashyapThe reasons you don't see some host CPU features in your guest _despite_ 'host-passthrough':15:27
kashyap(1) The said CPU flag must be supported by QEMU/KVM and must be "virtualizable"15:27
kashyap(2) Some CPU flags (rather, features) are not exposed by default because, they're not migratable.15:28
kashyap... the said features from (2) must be explicitly turned on _even with_ 'host-passthrough'.15:29
openstackgerritEric Fried proposed openstack/nova master: Add a way to spy on function calls in test  https://review.opendev.org/68618415:29
openstackgerritEric Fried proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68618515:29
kashyapmnaser: Hope the above makes sense.15:29
efriedgibi, dansmith: There's my crack at it --^  (done in a separate series)15:29
gibiefried: ack15:30
efriedI have another idea that, if it works, will be an uglier helper but more intuitive calling syntax.15:30
* efried tries...15:30
*** gbarros has quit IRC15:30
*** gyee has joined #openstack-nova15:30
sean-k-mooneylyarwood: could you take a look at this reno backport https://review.opendev.org/#/c/662095/ for the rabbitmq eventlet behavior15:36
*** gbarros has joined #openstack-nova15:36
sean-k-mooneyoh its not a backprot but ya we proably want to land that before the next stable/stein release whenever that is15:37
* lyarwood walks away from RHBZ and takes a look15:39
*** ttsiouts has quit IRC15:39
* sean-k-mooney walking away for RHBZ is often the right choice in my experince :)15:39
*** ttsiouts has joined #openstack-nova15:40
lyarwoodsean-k-mooney: LGTM, I missed that Mel had updated this recently thanks.15:42
sean-k-mooneyso did i i thought i had actully merged a week or two ago but went looking for it after our downstream call15:42
*** ttsiouts has quit IRC15:44
*** JamesBenson has quit IRC15:48
*** mdbooth has quit IRC15:50
mnaserkashyap: oh i see -- i will read around a bit more15:50
kashyapmnaser: We (I?) should document this somewhere.  As many people will come to wonder.15:51
mnaserkashyap: yeah, i'm still wondering if this is a "im running really new hardware" issue or "its normal and i dont get it hat well" heh15:51
*** mdbooth has joined #openstack-nova15:51
kashyap(Maybe I'll send a patch to the upstream libvirt docs itself; and then to Nova.)15:52
*** lpetrut has quit IRC15:52
kashyapmnaser: This is a new hardware with some CPU features that are not yet supported by KVM/QEMU.15:53
mnaserkashyap: i really wanted to run el8 but rdo and friends are not there yet unfortunately15:53
*** BjoernT has quit IRC15:53
kashyapmnaser: What's the issue?  Stein not yet tested on CentOS8?15:54
mnaserkashyap: yeah there's no openstack packages for el8 afaik15:54
mnaserthere's f28 but not el815:54
kashyapmnaser: Yeah, I think the issue was CentOS-8 was released like last week.  (I _guess_ it was delayed partly due to the "CentOS Stream", the parallel distro, announcement)15:55
mnaseryes, i think that's the theory that i heard too :)15:55
kashyapmnaser: Also, from my chat w/ Alan Pevec (from RDO) the other day, EL-8 packages are in the works.15:56
*** damien_r has quit IRC15:56
kashyap(But "in the works" is boring to hear, I know.)15:56
*** BjoernT has joined #openstack-nova15:56
*** ociuhandu has quit IRC15:58
*** ociuhandu has joined #openstack-nova15:59
*** ociuhandu has quit IRC16:03
openstackgerritMerged openstack/nova master: Do not print default dicts during heal_allocations  https://review.opendev.org/68600116:04
stephenfinlyarwood: Care to look at https://review.opendev.org/#/c/685980/ so I can proceed with the other backports? https://review.opendev.org/#/c/685980/16:05
lyarwoodstephenfin: sure looking16:09
*** pcaruana has quit IRC16:10
*** pcaruana has joined #openstack-nova16:11
openstackgerritmelanie witt proposed openstack/nova stable/stein: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606316:20
openstackgerritEric Fried proposed openstack/nova master: WIP: Another swing at `spy`  https://review.opendev.org/68620716:21
openstackgerritEric Fried proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68620816:21
efriedgibi, dansmith: I think I have something that will make the caller happy. I have to run for a bit, so I haven't done up the actual tests yet, but... ^16:21
openstackgerritmelanie witt proposed openstack/nova stable/rocky: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606716:24
mriedemi'd like to get a stable/stein release out soon but i'd like it to have https://review.opendev.org/#/c/686063/ in it as well since it's a regression, but i'm not sure how that works in relation to train, i.e. can we release that in stein once it's in train rc2? i think that's probably fine16:26
mriedemwe fix stuff in master all the time, backport it and release in stable before it's released in master i guess16:27
*** tesseract has quit IRC16:29
openstackgerritStephen Finucane proposed openstack/nova master: docs: Remove a whole load of unused images  https://review.opendev.org/68621116:29
*** jawad_axd has joined #openstack-nova16:30
melwittmriedem: did you mean can we release it in stein before we release train rc2?16:32
lyarwoodI'm okay with doing that given train hasn't GA'd yet16:33
*** jawad_axd has quit IRC16:35
mriedemmelwitt: yes, we obviously released the regression on stable before train16:36
mriedemso i think it's ok16:36
openstackgerritStephen Finucane proposed openstack/nova master: docs: Remove a whole load of unused images, most remainder  https://review.opendev.org/68621116:36
mriedemi think i'm just overthinking the train stuff16:37
gibiefried: ack, looks promising16:37
melwittok16:37
*** derekh has quit IRC16:43
*** jawad_axd has joined #openstack-nova16:44
*** ociuhandu has joined #openstack-nova16:48
openstackgerritStephen Finucane proposed openstack/nova master: Add AggregateMultiTenancyIsolation filter tests  https://review.opendev.org/46400216:51
*** ociuhandu has quit IRC16:52
openstackgerritStephen Finucane proposed openstack/nova master: Update the file for IPv4-only or IPv6-only network  https://review.opendev.org/46589116:54
*** tssurya has quit IRC16:56
openstackgerritStephen Finucane proposed openstack/nova master: Add warning message for cache_utils connection failure  https://review.opendev.org/50906017:02
openstackgerritStephen Finucane proposed openstack/nova master: Add warning message for cache_utils connection failure  https://review.opendev.org/50906017:02
stephenfinefried: This looks related to what we were talking about yesterday https://review.opendev.org/#/c/518119/17:03
stephenfinOr not. I just see references to the same thing17:03
stephenfinI'm looking through super old bugs17:03
stephenfin*patches17:03
mriedemstephenfin: i think we already have a test for this now https://review.opendev.org/#/c/464002/17:05
openstackgerritStephen Finucane proposed openstack/nova master: Don't create instance_extra entry for deleted instance  https://review.opendev.org/41277117:06
stephenfinmriedem: Good spot. I guess I can rebase/rework that17:09
stephenfinTomorrow though17:09
mriedemwell, if it's duplicate we don't need it17:10
* stephenfin -> 🏃17:10
stephenfinNot sure if there's stuff missing or not17:10
stephenfinI need to diff17:10
*** jawad_axd has quit IRC17:10
stephenfinor maybe what we have is good enough. Eh17:10
mriedemup to you, o/17:11
*** factor has joined #openstack-nova17:14
*** maciejjozefczyk has quit IRC17:22
*** BjoernT has quit IRC17:22
openstackgerritMerged openstack/python-novaclient stable/train: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598017:23
*** TxGirlGeek has joined #openstack-nova17:36
*** luksky has joined #openstack-nova17:43
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for migrate part of bug 1781286  https://review.opendev.org/68601717:44
openstackbug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,In progress] https://launchpad.net/bugs/1781286 - Assigned to Matt Riedemann (mriedem)17:44
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580717:44
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during build reschedule  https://review.opendev.org/68604717:44
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during migrate reschedule  https://review.opendev.org/68605017:44
openstackgerritMatt Riedemann proposed openstack/nova master: Update cells v2 up-call caveats doc  https://review.opendev.org/68605317:44
mriedemgibi: i had to remove the mock.patch context manager usage in the migrate functional test in that series ^ because it wasn't working properly17:44
*** pcaruana has quit IRC17:45
openstackgerritMatt Riedemann proposed openstack/nova stable/train: Handle get_host_availability_zone error during reschedule  https://review.opendev.org/68622617:48
*** jmlowe has quit IRC17:50
openstackgerritMerged openstack/nova master: Handle get_host_availability_zone error during reschedule  https://review.opendev.org/68599717:54
openstackgerritMerged openstack/nova stable/stein: Add reno about nova-api eventlet monkey-patching and rabbitmq  https://review.opendev.org/66209517:54
*** awalende has joined #openstack-nova17:57
*** eandersson has joined #openstack-nova18:00
*** awalende has quit IRC18:02
*** igordc has joined #openstack-nova18:03
*** jmlowe has joined #openstack-nova18:10
openstackgerritDan Smith proposed openstack/nova-specs master: Add image-precache-support spec  https://review.opendev.org/68623218:12
*** tbachman has quit IRC18:13
openstackgerritMerged openstack/nova master: Fix incorrect usages of fake moref in VMware tests  https://review.opendev.org/68502018:17
*** TxGirlGeek has quit IRC18:29
*** jmlowe has quit IRC18:33
*** jdillaman has quit IRC18:35
openstackgerritDan Smith proposed openstack/nova-specs master: Add image-precache-support spec  https://review.opendev.org/68623218:40
*** TxGirlGeek has joined #openstack-nova18:44
*** BjoernT has joined #openstack-nova18:44
*** BjoernT has quit IRC18:49
*** BjoernT has joined #openstack-nova18:52
melwittmriedem or efried: do you know if this is a known gate bug? 409 from placement "consumer generation conflict - expected null but got 1" https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_dc8/684118/1/check/nova-multi-cell/dc8f056/testr_results.html.gz18:52
*** TxGirlGeek has quit IRC18:53
*** tbachman has joined #openstack-nova18:54
mriedemdagnabbit dansmith i was nearly done with PS1 on your spec18:54
dansmithmriedem: sorry, it was just a couple formatting things18:55
mriedemok comments inline on PS118:56
dansmiththanks18:56
mriedemartom asked some of the same18:56
*** TxGirlGeek has joined #openstack-nova18:57
mriedemmelwitt: i want to say not necessarily known, but i know i've fixed a race like that in scheduling before18:58
*** tbachman has quit IRC18:59
*** dave-mccowan has joined #openstack-nova18:59
melwittack. this one is new to me18:59
dansmith....artom reviewed something of mine?19:00
mriedemmelwitt: this is the gate issue that's like that which i fixed before https://review.opendev.org/#/c/623596/19:00
artomdansmith, payback time19:00
mriedemi thought artom reserved his reviews specifically just for -1ing my stuff19:00
melwittmriedem: thanks19:00
*** tbachman has joined #openstack-nova19:03
*** bbowen_ has quit IRC19:05
*** spsurya has quit IRC19:09
mriedemmelwitt: given the name of the test, test_delete_server_while_in_building_state, i'm not surprised that there might be a race somehow19:09
dansmithartom: mriedem thanks for all that, and in record turnaround time :D19:10
mriedemhttp://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22consumer%20generation%20conflict%20-%20expected%20null%20but%20got%201%5C%22&from=7d19:10
mriedemneither change that hit it look related to causing the issue though19:11
mriedemOct 01 22:17:03.897751 ubuntu-bionic-rax-iad-0012085716 devstack@n-api.service[17786]: INFO nova.api.openstack.wsgi [None req-c9e65a63-ea1f-4144-aace-c148ba386c8e tempest-DeleteServersTestJSON-641512281 tempest-DeleteServersTestJSON-641512281] HTTP exception thrown: Failed to delete allocations for consumer e7339a26-880b-4c27-972c-265d66eb5d82. Error: {"errors": [{"status": 409, "title": "Conflict", "detail": "There was a con19:12
mriedemt when trying to complete your request.\n\n consumer generation conflict - expected null but got 1  ", "code": "placement.concurrent_update", "request_id": "req-374479cc-0a6d-47b2-afbd-17c6e2f5413d"}]}19:12
mriedemso in this case the server probably got through scheduling and allocated resources on a provider after the delete request started but before we tried to delete the allocations19:13
mriedemin the local delete path in the api since the server doesn't have a host set yet19:13
mriedemmelwitt: might as well report a bug for tracking19:13
mriedemmelwitt: looks like this changed in stein to return the 409 https://review.opendev.org/#/c/591597/19:17
*** tbachman has quit IRC19:17
mriedembefore that we'd log but not fail the delete request19:17
*** tbachman has joined #openstack-nova19:18
mriedemwhen i wrote this in rocky anyway https://review.opendev.org/#/q/If507e23f0b7e5fa417041c3870d77786498f741d19:18
mriedemif delete_allocation_for_instance were smarter and returned a generation conflict exception, the api code could retry the call19:20
mriedemwe could look for this in the error from placement: "code": "placement.concurrent_update"19:21
mriedemwe do that elsewhere to detect races19:21
mriedemsee SchedulerReportClient.claim_resources19:21
*** ccamacho has joined #openstack-nova19:24
openstackgerritMerged openstack/nova master: conf: Remove deprecated 'project_id_regex' opt  https://review.opendev.org/50105719:26
openstackgerritMerged openstack/nova master: Replace 'fake' with a real project ID  https://review.opendev.org/68564519:26
*** ccamacho has quit IRC19:28
openstackgerritDan Smith proposed openstack/nova-specs master: Add image-precache-support spec  https://review.opendev.org/68623219:31
efriedmelwitt: did you get your answer on that 409?19:32
efriedstephenfin: yes, you're right, that appears to be hitting some of the same stuff I was looking at, will dig, thanks for the pointer.19:33
*** bbowen has joined #openstack-nova19:33
*** BjoernT has quit IRC19:37
*** jmlowe has joined #openstack-nova19:40
mriedemefried: i think so - see above19:42
mriedemefried: just need a bug and a retry decorator on delete_allocations_for_instances i think19:42
openstackgerritDan Smith proposed openstack/nova-specs master: Add image-precache-support spec  https://review.opendev.org/68623219:43
efriedmriedem: Yeah, that sounds familiar, I want to say I had that somewhere on my to-do list ages ago but haven't gotten back to it.19:43
openstackgerritMatt Riedemann proposed openstack/nova stable/stein: Handle get_host_availability_zone error during reschedule  https://review.opendev.org/68626419:44
*** BjoernT has joined #openstack-nova19:47
melwittmriedem: ok, I'll file a bug. thanks19:51
mriedemdansmith: i'm not sure that "sufficiently-endowed" should ever be in a spec19:55
dansmithmriedem: not with you around, that's for sure19:55
mriedem:P19:55
artomdansmith, "sufficiently-endowed", seriously?19:55
artommriedem, dammit19:55
mriedemnow don't go removing it yet, i'm commenting19:55
dansmithyou two being the only two reviewers, I had to bait you good19:56
artomCareful how you spell bait19:56
*** dpawlik has joined #openstack-nova19:56
* dansmith shakes his head19:57
dansmithyou guys are too easy19:57
artomIt's more fun that way.19:57
dansmithendowed with permission is a totally legit phrasing19:58
sean-k-mooneydansmith: you have a lot of references in that spec. i added it to my list of things to revew19:59
dansmithsean-k-mooney: I listed a bunch of them so that reviewers would be afraid to not approve it19:59
sean-k-mooneyit looks relitivly short for a spec but im to tired to review stuff a tthe momemnt so ill go through it tomorrow.20:00
sean-k-mooneyhehe i see you are going for the over load them with example of why this is need so you cant question it approch20:00
sean-k-mooneyit fairly effective20:01
*** dpawlik has quit IRC20:01
artomdansmith, I don't suppose setting up glance -> computes multicast is part of your MVP spec ;)20:02
artomBut it'd be a way to get both the ease of fire-and-forget RPC cast while not killing the glance server20:03
artomNot that it's, you know, realistic in any way20:03
dansmithartom: the right solution, IMHO, is bittorrent20:03
dansmithwhich was proposed and implemented in RAX at one point20:03
dansmithbasically, glance becomes a torrent seeder and only has to dish out the image once20:04
artomI genuinely thought you were kidding at first20:04
dansmithor however many workers you have, that's how many seeds you have20:04
artomBut yeah, that's actually smart20:04
dansmithimagine how good that would be for an edge site where you have four computes.. right now they all pull the image over the WAN... but that makes no sense, they should share once one of them has it20:04
dansmithright20:04
*** dave-mccowan has quit IRC20:04
dansmithand if one got it yesterday and still has it, the peer compute can pull it from it instead of over the way20:05
dansmith*wan20:05
dansmithalso imagine how in the future iteration where we need to report which computes have that image cached, if we could just ask the torrent magnet which computes are now seeds... :D20:06
artom*stunned Pikachu face*20:07
openstackgerritmelanie witt proposed openstack/nova master: Restore console proxy deployment info to cells v2 layout doc  https://review.opendev.org/68627120:08
sean-k-mooneywe talkabout using bittorrent like 2 years ago20:08
dansmithwe talked about it in 201220:08
melwittefried, mriedem: added ^ to https://etherpad.openstack.org/p/nova-train-release-todo as a possible thing for RC220:08
dansmithbut probably two years ago as well20:08
sean-k-mooneywas it that long20:08
* sean-k-mooney feels old20:09
sean-k-mooneyanyway im glad its now a thing20:09
sean-k-mooneybecause ya for edge it toptally makes sense20:09
*** dave-mccowan has joined #openstack-nova20:10
sean-k-mooneyits resilent, spreads the load across multiple peers and works over low bandwitd links20:10
dansmithI'm not really proposing that in the short term, but it's always been something I wanted us to do20:10
sean-k-mooneywell part of why i like it is its also not us e.g. openstack comping up with our own solution for a proablem that has and existing soltion already20:11
dansmithyup20:11
mriedemmore comments20:14
artomdansmith, brainstorming here - but does oslo.concurrency give us a distributed semaphore?20:14
*** remi_ness has joined #openstack-nova20:14
dansmithartom: not without some external service20:15
artomI'm thinking - cast to the computes, but have the grab the semaphore when downloading20:15
bnemecSounds like a use case for tooz.20:15
mriedemi was just going to make an etcd joke20:15
bnemecoslo.concurrency only works within a single machine.20:16
artomThat way the concurrency becomes configurable (just set the semaphore's... err, cardinality?) and you don't ddos the glance server20:16
dansmithartom: yeah, it's just not really necessary when we can threadpool it from the conductor20:16
artomdansmith, for an RPC call though, right?20:16
dansmithartom: yup20:17
artomYeah, I may be overthinking this20:17
mriedemmelwitt: ack i'm cool with that, docs only and regression in train20:17
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Noop CantStartEngineError in targets_cell if API DB not configured  https://review.opendev.org/68627620:20
*** markvoelker has joined #openstack-nova20:21
*** tbachman has quit IRC20:23
openstackgerritMerged openstack/nova master: Use os-brick locking for volume attach and detach  https://review.opendev.org/61419020:26
dansmithmriedem: yawna read my response to the parallelism thing and if you're cool with it I can squash those todos in my next rev: https://review.opendev.org/#/c/686232/4/specs/ussuri/approved/image-precache-support.rst@12320:26
melwittmriedem: coolio20:26
*** munimeha1 has joined #openstack-nova20:27
mriedemdansmith: replied20:28
dansmithmriedem: thanks20:29
openstackgerritmelanie witt proposed openstack/nova master: Restore console proxy deployment info to cells v2 layout doc  https://review.opendev.org/68627120:30
*** gbarros has quit IRC20:32
openstackgerritDan Smith proposed openstack/nova-specs master: Add image-precache-support spec  https://review.opendev.org/68623220:35
*** nweinber has quit IRC20:36
*** tbachman has joined #openstack-nova20:36
efriedmelwitt, mriedem: +2, good catch20:36
efriedaspiers: how's that SEV fixup looking? Would be nice to produce RC2 tomorrow (and not need RC3). Your fix is the last one I know of that's not well on the way to merging.20:38
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Handle get_host_availability_zone error during reschedule  https://review.opendev.org/68629220:42
mriedemefried: deadline for RCs is oct 10 right?20:44
efriedyes20:44
mriedemi mean, we want time for rc3 for sev issue #12 but sure20:44
efriedsmy point20:44
efriedOh, I misread, you're actually picking on SEV. I don't think we've had especially more issues for SEV than for other blueprints. Have there been more than one for any?20:45
efriedSeems like one for SEV, one for VPMEM, one for console tokens, one (?) for cpu-resources...20:46
efriedwhich is fine, exactly the reason FF happens before the release.20:46
sean-k-mooneyefried: i dont think we had any for cpu-resouce but we had/have one for numa migration20:51
sean-k-mooneyartom: has your fix for the RT cacheing issue landed?20:51
mriedemyes20:51
*** tbachman has quit IRC20:52
sean-k-mooneyok cool20:52
mriedemconsole proxy was a docs thing20:52
mriedemmy point is sev landed late with a ton of churn over 2 releases (speaking as an outside observer to that series), so i wouldn't be surprised if there aren't more hairy issues20:52
mriedemseems like every issue in that series is very hard to resolve20:52
mriedemi also won't be surprised when there are issues with PCPU - we haven't seen issues because we're not using it anywhere yet20:53
mriedembut in 18 months...watch out!20:53
sean-k-mooneywell there kind of is one but it was an issue we defeind as out of scope20:54
sean-k-mooneyit related to https://review.opendev.org/#/c/686079/20:54
*** remi_ness has quit IRC20:54
sean-k-mooneyyou can only have flaoting instnace and pinned instance on the same host and not have mempage issue if you enable the numa aware memory tracking using hw:mem_page_size=<any valid value>20:55
sean-k-mooneybut we know it would not address that20:56
*** dpawlik has joined #openstack-nova20:56
efriedmriedem: the console token thing we just merged? That wasn't a docs issue.20:58
efriedit may not have been related to an actual blueprint either though, not sure.20:59
sean-k-mooneythe console issue was passing the path argument in case it was not required right?21:00
dansmithI think he means the cells console proxy doc thing21:00
efriedno21:00
efriedhttps://review.opendev.org/#/c/686066/ Reduce scope of 'path' query parameter to noVNC consoles21:00
dansmiththis: https://review.opendev.org/#/c/686271/21:00
efriedno21:01
efried-----^21:01
dansmithefried: I mean he==mriedem, not you21:01
efriedright21:01
efriedand confirmed it wasn't a blueprint, it was a bug21:01
efriedanywho21:01
sean-k-mooneyim going to call it a night o/21:02
*** TxGirlGeek has quit IRC21:02
mriedemefried: oh i wasn't thinking of the serial console thing21:05
mriedemefried: it was a bug fix for a novnc 1.1.0 regression or something but dug a bit too deep21:06
*** lpetrut has joined #openstack-nova21:06
efriedy21:06
mriedemand actually wouldn't have been caught had it not been backported and released on stable21:06
mriedemspeaking of, dansmith can you get https://review.opendev.org/#/c/686063/ on stein so we can do a stein release tomorrow?21:07
mriedemi think that's the last thing i was waiting on21:07
*** maciejjozefczyk has joined #openstack-nova21:07
dansmithokay21:07
*** dpawlik has quit IRC21:07
*** xek has quit IRC21:08
*** markvoelker has quit IRC21:11
*** maciejjozefczyk has quit IRC21:12
*** markvoelker has joined #openstack-nova21:12
*** lpetrut has quit IRC21:15
*** factor has quit IRC21:17
*** dpawlik has joined #openstack-nova21:19
*** dpawlik has quit IRC21:24
openstackgerritEric Fried proposed openstack/nova master: Allow versioned discovery unauthenticated  https://review.opendev.org/68518121:25
openstackgerritEric Fried proposed openstack/nova master: Modify nova routing table to process null requests  https://review.opendev.org/51811921:27
*** BjoernT has quit IRC21:30
*** JamesBenson has joined #openstack-nova21:31
*** JamesBenson has quit IRC21:32
*** JamesBenson has joined #openstack-nova21:32
*** BjoernT has joined #openstack-nova21:35
*** JamesBenson has quit IRC21:37
*** markvoelker has quit IRC21:41
*** markvoelker has joined #openstack-nova21:41
*** tbachman has joined #openstack-nova21:45
*** markvoelker has quit IRC21:45
*** markvoelker has joined #openstack-nova21:45
ozzzoIs there an "openstack" command corresponding to every "nova" and "nova-manage" command?21:46
ozzzoFor example, can I do "nova-manage service-disable <HV UUID>" via openstack?21:47
ozzzoor "nova service-list" ?21:48
ozzzoThe obvious thing to try would be "openstack service list" but that does something different21:49
*** BjoernT has quit IRC21:50
*** takashin has joined #openstack-nova21:51
*** BjoernT has joined #openstack-nova21:54
mriedemyes,21:56
mriedemhttps://docs.openstack.org/python-openstackclient/latest/cli/command-objects/compute-service.html#compute-service-set21:56
mriedemnova-manage and nova/openstack CLIs are not the same21:56
mriedemthe former doesn't use apis, but hits the db directly when the api isn't available, and doesn't require a token, the latter requires a token and uses the api21:57
mriedemyou're also looking for this https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/compute-service.html#compute-service-list21:57
ozzzovery nice, thank you!21:58
*** TxGirlGeek has joined #openstack-nova22:01
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix unit of hw_rng:rate_period  https://review.opendev.org/68305122:01
openstackgerritTakashi NATSUME proposed openstack/nova master: Update keypairs in saving an instance object  https://review.opendev.org/68304322:02
openstackgerritTakashi NATSUME proposed openstack/nova master: Move Destination object tests to their own test class  https://review.opendev.org/68301722:02
*** TxGirlGeek has quit IRC22:12
*** markvoelker has quit IRC22:17
*** mlavalle has quit IRC22:18
ozzzoIs there an "openstack" command for "nova-manage cell_v2 discover-hosts" ?22:18
mriedemno22:19
mriedemthere is a config option to enable it in a periodic task if you don't want to hit the command line22:20
ozzzook ty22:20
mriedemhttps://docs.openstack.org/nova/latest/configuration/config.html#scheduler.discover_hosts_in_cells_interval22:20
*** TxGirlGeek has joined #openstack-nova22:23
*** BjoernT has quit IRC22:24
melwittmriedem: at first glance I wondered if this is related to the recent eventlet tpool proxy change, but it's reported against ocata https://bugs.launchpad.net/nova/+bug/184575922:27
openstackLaunchpad bug 1845759 in OpenStack Compute (nova) "Openstack nova-compute service gives cant start new thread with oslo messaging" [Undecided,New]22:27
mriedemi saw it in triage but didn't have thoughts on it22:28
melwittme neither. I've never seen anything like it before22:29
*** mriedem has quit IRC22:31
ozzzoI was thinking that "openstack compute service delete" would behave similar to "openstack resource provider delete" but the service did not come back when I typed "nova-manage cell_v2 discover_hosts"22:33
ozzzohow can I re-create the hypervisor service that I deleted?22:34
*** slaweq has quit IRC22:35
artomsean-k-mooney, https://review.opendev.org/#/c/685387/? Yeah22:38
melwittozzzo: I think just create it again https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/service.html#service-create the service command is different than the resource provider command, the resource provider command modifies only the placement service database and the service command modifies both the nova service database and the placement service database (by way of the placement API)22:39
ozzzook I'll try that, ty!22:44
melwitt'openstack compute service delete' will delete the records of the service in the nova database and then it will do an equivalent REST API call as 'openstack resource provider delete' to the placement service22:45
*** rcernin has joined #openstack-nova22:48
ozzzothat didn't seem to work; I typed "openstack service create --name us01odc-dev1-hv001 --enable nova-compute" but I still only have 2 hypervisor services under "openstack compute service list"22:50
ozzzoIt looks like I created a new service called us01odc-dev1-hv001 under "openstack service list"22:51
ozzzoit looks like "service" and "compute service" are 2 different things22:51
melwittyeah... I'm looking at the doc and the <type> looks like it should be one of compute, identity, volume but I'm not 100% sure22:52
ozzzoon the "compute service" man page I only see delete, list and set, there is no create22:52
openstackgerritEric Fried proposed openstack/nova master: Add a way to spy on function calls in test  https://review.opendev.org/68620722:54
openstackgerritEric Fried proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68620822:54
efriedgibi, dansmith: There ^22:54
*** tkajinam has joined #openstack-nova22:54
efried(recommend looking at the test deltas first)22:55
ozzzoI stop/started nova-compute on the hypervisor; that appears to have re-created the service22:56
ozzzothanks for your advice melwitt!22:56
ozzzoI'm trying to figure out how to remove, repair and re-add broken hypervisors22:57
melwittozzzo: oh, yeah ok, all you have to do it start the service, I guess that makes sense -_- heh. I guess just delete that extra service from earlier so it won't confuse you later22:57
melwittwhatever that was, the one that didn't help22:57
ozzzorighton22:58
ozzzoI guess service means 2 things in OS; there's a "nova" service that has multiple service providers, managed by "openstack service" but there's also a "nova-compute" service that runs on each of those service providers, managed by "openstack compute service"22:59
melwittyeah, it looks like the unified client (openstackclient) lumps all the services together for the 'service list' but when you do a 'service create' you have to specify the <type> which is either 'compute' (nova), 'identity' (keystone), 'volume' (cinder) etc and they are compartmentalized like that23:02
melwittand the <type> lets the client know which REST API to call23:04
ozzzook, I'm starting to get it23:04
melwittbut it sounds like there's no validation on the <type> so it allows anything and tries to do a lookup in the service catalog and doesn't raise an error? that seems odd23:05
melwittI'm just speculating, I'd have to look in the code to know for sure23:05
openstackgerritMerged openstack/nova stable/train: Imported Translations from Zanata  https://review.opendev.org/68553423:09
openstackgerritMerged openstack/nova master: Restore console proxy deployment info to cells v2 layout doc  https://review.opendev.org/68627123:09
*** JamesBenson has joined #openstack-nova23:20
*** luksky has quit IRC23:22
*** JamesBenson has quit IRC23:24
*** TxGirlGeek has quit IRC23:35
*** panda is now known as panda|off23:39
openstackgerritEric Fried proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68620823:40
*** gbarros has joined #openstack-nova23:41
*** gbarros has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!