| *** tetsuro has joined #openstack-nova | 00:20 | |
| *** whoami-rajat has quit IRC | 00:22 | |
| *** lbragstad has joined #openstack-nova | 00:42 | |
| *** d34dh0r53 has quit IRC | 01:09 | |
| *** yedongcan has joined #openstack-nova | 01:26 | |
| *** ileixe has joined #openstack-nova | 01:55 | |
| *** ircuser-1 has joined #openstack-nova | 02:03 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Add host and hypervisor_hostname flag to create server https://review.opendev.org/645520 | 02:21 |
|---|---|---|
| *** lbragstad has quit IRC | 02:46 | |
| *** d34dh0r53 has joined #openstack-nova | 02:46 | |
| openstackgerrit | Tushar Patil proposed openstack/nova-specs master: Allow compute nodes to use DISK_GB from shared storage RP https://review.opendev.org/650188 | 02:54 |
| *** yedongcan has quit IRC | 03:14 | |
| *** yedongcan has joined #openstack-nova | 03:14 | |
| *** psachin has joined #openstack-nova | 03:22 | |
| *** ileixe has quit IRC | 03:30 | |
| *** ileixe has joined #openstack-nova | 03:34 | |
| *** udesale has joined #openstack-nova | 03:56 | |
| *** yonglihe has quit IRC | 04:04 | |
| *** takashin has joined #openstack-nova | 04:18 | |
| *** ileixe has quit IRC | 04:37 | |
| *** ricolin has joined #openstack-nova | 04:38 | |
| *** ileixe has joined #openstack-nova | 04:42 | |
| *** tonyb has joined #openstack-nova | 04:46 | |
| *** ratailor has joined #openstack-nova | 05:07 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Add host and hypervisor_hostname flag to create server https://review.opendev.org/645520 | 05:18 |
| openstackgerrit | Merged openstack/nova stable/pike: Update instance.availability_zone during live migration https://review.opendev.org/647630 | 05:25 |
| openstackgerrit | Merged openstack/nova stable/pike: Add functional recreate test for bug 1819963 https://review.opendev.org/648421 | 05:25 |
| openstack | bug 1819963 in OpenStack Compute (nova) pike "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem) | 05:25 |
| openstackgerrit | Merged openstack/nova stable/pike: Update instance.availability_zone on revertResize https://review.opendev.org/648422 | 05:25 |
| openstackgerrit | Merged openstack/nova stable/pike: Add missing libvirt exception during device detach https://review.opendev.org/651642 | 05:25 |
| openstackgerrit | Merged openstack/nova stable/queens: Fix incomplete instance data returned after build failure https://review.opendev.org/647913 | 05:25 |
| *** sridharg has joined #openstack-nova | 05:34 | |
| *** cfriesen has quit IRC | 05:35 | |
| *** tonyb has quit IRC | 06:13 | |
| *** tonyb has joined #openstack-nova | 06:14 | |
| *** ccamacho has joined #openstack-nova | 06:58 | |
| *** alex_xu has joined #openstack-nova | 07:14 | |
| *** ralonsoh has joined #openstack-nova | 07:16 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8) https://review.opendev.org/575311 | 07:17 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8) https://review.opendev.org/575311 | 07:17 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9) https://review.opendev.org/575581 | 07:18 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10) https://review.opendev.org/576017 | 07:19 |
| *** pcaruana has joined #openstack-nova | 07:19 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11) https://review.opendev.org/576018 | 07:19 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12) https://review.opendev.org/576019 | 07:19 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.opendev.org/576020 | 07:20 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.opendev.org/576027 | 07:20 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031 | 07:21 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299 | 07:21 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344 | 07:21 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673 | 07:21 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | 07:22 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | 07:22 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | 07:22 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | 07:22 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.opendev.org/566153 | 07:23 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add a live migration regression test https://review.opendev.org/641200 | 07:23 |
| *** huachang has joined #openstack-nova | 07:28 | |
| openstackgerrit | zhongshengping proposed openstack/nova master: Replace git.openstack.org URLs with opendev.org URLs https://review.opendev.org/654398 | 07:29 |
| *** slaweq has joined #openstack-nova | 07:30 | |
| *** dikonoor has quit IRC | 07:31 | |
| boxiang | hi does anyone know why my nova-next fails http://logs.openstack.org/20/645520/11/check/nova-next/afbb160/? only nova-next of zuul fails. thanks :) | 07:42 |
| boxiang | is it necessary for me to do 'recheck'? | 07:42 |
| openstackgerrit | Boxiang Zhu proposed openstack/python-novaclient master: Add host and hypervisor_hostname to create servers https://review.opendev.org/647671 | 07:58 |
| *** yedongcan has quit IRC | 08:22 | |
| *** yedongcan has joined #openstack-nova | 08:22 | |
| *** tkajinam has quit IRC | 08:24 | |
| *** yikun has joined #openstack-nova | 08:41 | |
| *** pcaruana has quit IRC | 08:45 | |
| *** rcernin has joined #openstack-nova | 09:03 | |
| *** slaweq has quit IRC | 09:17 | |
| *** slaweq has joined #openstack-nova | 09:35 | |
| *** slaweq has quit IRC | 09:40 | |
| *** yan0s has joined #openstack-nova | 09:41 | |
| *** hoonetorg has quit IRC | 10:00 | |
| *** hoonetorg has joined #openstack-nova | 10:13 | |
| *** tetsuro has quit IRC | 10:20 | |
| *** huachang has quit IRC | 10:20 | |
| *** boxiang has quit IRC | 10:25 | |
| *** boxiang has joined #openstack-nova | 10:26 | |
| *** boxiang has quit IRC | 10:31 | |
| *** boxiang has joined #openstack-nova | 10:32 | |
| *** yedongcan has left #openstack-nova | 10:37 | |
| *** ttsiouts has joined #openstack-nova | 10:40 | |
| *** pcaruana has joined #openstack-nova | 10:58 | |
| *** ttsiouts has quit IRC | 11:04 | |
| *** ttsiouts has joined #openstack-nova | 11:05 | |
| *** avolkov has joined #openstack-nova | 11:05 | |
| *** ttsiouts has quit IRC | 11:10 | |
| *** tbachman has joined #openstack-nova | 11:29 | |
| *** udesale has quit IRC | 11:34 | |
| *** slaweq has joined #openstack-nova | 11:43 | |
| openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Updates for OpenDev transition https://review.opendev.org/654429 | 11:45 |
| *** tbachman has quit IRC | 11:47 | |
| *** lpetrut has joined #openstack-nova | 11:47 | |
| *** slaweq has quit IRC | 11:47 | |
| *** tbachman has joined #openstack-nova | 11:48 | |
| *** tbachman has quit IRC | 11:56 | |
| *** tbachman has joined #openstack-nova | 12:16 | |
| *** shilpasd has joined #openstack-nova | 12:19 | |
| *** _erlon_ has joined #openstack-nova | 12:41 | |
| *** yan0s has quit IRC | 12:48 | |
| *** mriedem has joined #openstack-nova | 12:50 | |
| *** yan0s has joined #openstack-nova | 12:56 | |
| *** jmlowe has quit IRC | 13:00 | |
| *** lbragstad has joined #openstack-nova | 13:09 | |
| *** ratailor has quit IRC | 13:16 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test for regression bug 1825537 https://review.opendev.org/654066 | 13:26 |
| openstack | bug 1825537 in OpenStack Compute (nova) "finish_resize failures incorrectly revert allocations" [Medium,In progress] https://launchpad.net/bugs/1825537 - Assigned to Matt Riedemann (mriedem) | 13:26 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop source node allocations if finish_resize fails https://review.opendev.org/654067 | 13:26 |
| openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Drop py35 tests https://review.opendev.org/643873 | 13:29 |
| *** psachin has quit IRC | 13:39 | |
| mriedem | bauzas: can you review this functional recreate test for a stein regression bug? https://review.opendev.org/#/c/653268/ | 13:42 |
| *** jmlowe has joined #openstack-nova | 13:44 | |
| mriedem | jaypipes: thanks for looking at https://bugs.launchpad.net/nova/+bug/1825435 - replied to your oslo.config question | 13:45 |
| openstack | Launchpad bug 1825435 in OpenStack Compute (nova) "TestRPC unit tests intermittently fail with "'>' not supported between instances of 'NoneType' and 'datetime.datetime'" - maybe due to "Fatal Python error: Cannot recover from stack overflow."" [High,Confirmed] | 13:45 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Fix {min|max}_version in ironic Adapter setup https://review.opendev.org/654457 | 13:45 |
| efried | eandersson: ^ | 13:45 |
| efried | mriedem: Just grinding into action here - any progress on that bug? | 13:46 |
| mriedem | efried: no | 13:46 |
| efried | k | 13:46 |
| lpetrut | hey, I was wondering which criteria were used when defining the list of cpu traits: https://github.com/openstack/os-traits/blob/master/os_traits/hw/cpu/x86.py | 13:46 |
| lpetrut | I mean, why not use all the cpu features defined here? https://github.com/libvirt/libvirt/blob/master/src/cpu_map/x86_features.xml | 13:46 |
| mriedem | alex_xu: ^ | 13:46 |
| jaypipes | mriedem: ack. still, the collections code does seem suspiciously related, given that stacktrace. | 13:47 |
| mriedem | jaypipes: yeah i'm trying to find the python3.6 package change log for bionic | 13:48 |
| mriedem | b/c i started seeing seg faults in pip3.6 last week also | 13:48 |
| jaypipes | lpetrut: I added those x86 traits. | 13:48 |
| jaypipes | lpetrut: I didn't use libvirt's XML files | 13:49 |
| *** eharney has joined #openstack-nova | 13:49 | |
| mriedem | hmm http://changelogs.ubuntu.com/changelogs/pool/main/p/python3.6/python3.6_3.6.7-1~18.04/changelog isn't very helpful | 13:49 |
| *** awaugama has joined #openstack-nova | 13:49 | |
| *** tbachman has quit IRC | 13:50 | |
| lpetrut | got it, I was wondering how you've picked them, whether there was already some list at the libvirt driver level. also, would it make sense to extend this list? fetching all the features from the libvirt xml may be the easiest option | 13:50 |
| jaypipes | lpetrut: there are more hypervisors than libvirt. :) | 13:51 |
| jaypipes | lpetrut: and the content of the libvirt XML file changes over time of course. | 13:52 |
| jaypipes | lpetrut: we've added things to os-traits as needed. if there's a particular instruction set extension you want/need, don't hesitate to throw up a patch adding it. :) | 13:52 |
| lpetrut | jaypapipes: indeed, in fact I was looking into implementing 'update_provider_tree' for the hyper-v driver :) but those should probably be standardized at the openstack level | 13:53 |
| jaypipes | ++ | 13:53 |
| jaypipes | lpetrut: to answer your question, I don't remember there being any rhyme or reason to the ones I added. probably I just scraped things from unit tests or boxes I had lying around ;) | 13:53 |
| lpetrut | got it, so basically each feature/trait was defined as needed (as opposed to having an extensive list) | 13:54 |
| mriedem | lpetrut: do you guys plan on upstreaming feature parity stuff like this at some point? https://opendev.org/x/compute-hyperv/commit/d4f1fa457a0fcbd1ac84c816157175c090958d6f | 13:55 |
| mriedem | https://opendev.org/x/compute-hyperv/commit/f615b509a2022025f2465341922a2e11427f8822 | 13:55 |
| mriedem | https://opendev.org/x/compute-hyperv/commit/8326d2c43c3e464f0c2e3b58bf57b4ffb9a344e7 | 13:55 |
| lpetrut | those patches look straight forward, should be easy to review/merge | 13:56 |
| lpetrut | well, maybe not the last one :) | 13:56 |
| mriedem | they are at least a specless blueprint just since they are feature additions, but it seems all the feature work on the driver has been out of tree for the last couple of years | 13:56 |
| jaypipes | lpetrut: yes, exactly. | 13:57 |
| mriedem | lpetrut: anyway, just a thought, those first 2 look simple. i'm not looking to generate work, but i'd like to see the in-tree driver close some of these feature gaps that are already in the out of tree driver. | 13:58 |
| lpetrut | mriedem: yep, we can start with a few patches to close the gap a bit. thanks for bringing this up | 13:59 |
| mriedem | bauzas: this is another simple functional recreate test for a stein regression: https://review.opendev.org/#/c/653098/ | 14:01 |
| *** huachang has joined #openstack-nova | 14:02 | |
| mriedem | jaypipes: trying to think of ways to maybe debug that oslo.config getattr thing - we could maybe have a DNM patch to oslo.config that counts the number of times we've recursed into that getattr loop and log a traceback to see what's triggering it, then push a nova change that depends-on that oslo.config debug patch? | 14:05 |
| *** takashin has left #openstack-nova | 14:07 | |
| ganso | mriedem: hey Matt! sorry for interrupting the current convo, but just a heads up of https://review.opendev.org/#/c/652153 ... I tested in my env and it did not work. It could be perhaps because I run stable packages instead of directly from source. Did it work in your env? | 14:09 |
| mriedem | ganso: i saw your comment but i haven't yet tried to recreate in a queens devstack | 14:11 |
| mriedem | do you see errors in the source node compute logs? | 14:11 |
| mriedem | did you try rocky? | 14:11 |
| ganso | mriedem: no I did not see any errors. It just did not remove the allocations | 14:11 |
| ganso | mriedem: no I haven't tried rocky | 14:11 |
| mriedem | hmm, i'm not sure what the problem would be since it's calling the same method | 14:12 |
| mriedem | alternatively i could write a functional test to recreate it | 14:13 |
| mriedem | if you can debug it would obviously speed things up | 14:14 |
| jaypipes | mriedem: yeah, the issue is we don't really know where the original call site is for the getattr that is causing the stack exhaustion. if we did know what is the thing that is the base object or thing that is being called there, we could swap out that for either a mock or an actual collections.Mapping etc and see if the issue is in the ovo code, the oslo.cfg code, or something else. | 14:25 |
| *** luksky has joined #openstack-nova | 14:26 | |
| jaypipes | mriedem: I'm wondering if there's a way to limit the Python stack recursion size to a smallish number so that the stacktrace would show the entire traceback instead of just the last hundred or so repeated recursive calls to getattr. | 14:26 |
| jaypipes | https://docs.python.org/3/library/sys.html#sys.setrecursionlimit | 14:26 |
| jaypipes | might be worth throwing that in a test, mriedem? ^ | 14:27 |
| jaypipes | just to get the whole traceback. | 14:27 |
| jaypipes | which might give us a better starting point. | 14:27 |
| *** openstackgerrit has quit IRC | 14:28 | |
| *** huachang has quit IRC | 14:31 | |
| mriedem | jaypipes: that's what i'm working on - dump the traceback to the logs before we get too far into the recursion | 14:32 |
| jaypipes | mriedem: coo. | 14:34 |
| mriedem | i'll push a nova change which depends-on it and see if we can recreate | 14:34 |
| ganso | mriedem: thanks! | 14:35 |
| *** openstackgerrit has joined #openstack-nova | 14:38 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug config opts infinite recursion https://review.opendev.org/654468 | 14:38 |
| *** mlavalle has joined #openstack-nova | 14:46 | |
| *** tbachman has joined #openstack-nova | 14:47 | |
| *** huachang has joined #openstack-nova | 14:48 | |
| *** hongbin has joined #openstack-nova | 14:55 | |
| *** hemna has joined #openstack-nova | 14:56 | |
| *** belmoreira has joined #openstack-nova | 14:59 | |
| *** lpetrut has quit IRC | 15:00 | |
| *** dklyle has joined #openstack-nova | 15:03 | |
| *** hemna has quit IRC | 15:08 | |
| *** hemna has joined #openstack-nova | 15:08 | |
| *** gyee has joined #openstack-nova | 15:11 | |
| openstackgerrit | Merged openstack/nova stable/pike: Fix incomplete instance data returned after build failure https://review.opendev.org/647916 | 15:13 |
| efried | mriedem: Where are the logs you inserted? | 15:15 |
| *** imacdonn has joined #openstack-nova | 15:17 | |
| efried | I don't see 'em in the job output or subunit results | 15:18 |
| efried | maybe logger config is set higher for oslo? | 15:22 |
| mriedem | efried: it won't dump the logs unless we hit a failure i don't think... | 15:24 |
| mriedem | maybe i need to use prints | 15:24 |
| mriedem | because what fails (TestRPC) isn't where the stack overflow happens i don't think | 15:24 |
| *** yan0s has quit IRC | 15:25 | |
| efried | Use prints... and hope the stdout fixture isn't set up | 15:26 |
| mriedem | i don't see any default log level set for oslo.config either https://github.com/openstack/oslo.log/blob/22e8a347c8ba914cefa26df42dde632937259a79/oslo_log/_options.py#L19 | 15:26 |
| efried | so what's the default default if there's no default? | 15:26 |
| mriedem | in unit tests it'd be INFO | 15:27 |
| efried | okay | 15:27 |
| mriedem | which is why we have the OS_DEBUG env var | 15:27 |
| *** tbachman has quit IRC | 15:27 | |
| *** awaugama has quit IRC | 15:27 | |
| *** openstackgerrit has quit IRC | 15:58 | |
| *** huachang has quit IRC | 15:59 | |
| *** avolkov has quit IRC | 16:05 | |
| *** wwriverrat has joined #openstack-nova | 16:05 | |
| *** ivve has joined #openstack-nova | 16:11 | |
| eandersson | Looks good efried | 16:20 |
| eandersson | As for your devstack comment, that was never meant to break | 16:20 |
| eandersson | as you are passing on endpoint = None | 16:20 |
| eandersson | https://review.opendev.org/#/c/653839/ | 16:20 |
| eandersson | This was the actual test | 16:21 |
| efried | eandersson: But that test only reports that our endpoint lookup failed | 16:22 |
| eandersson | Exactly | 16:22 |
| efried | eandersson: ...which is still a valid code path | 16:22 |
| eandersson | Sure | 16:22 |
| efried | like, if you specify a bogus region name or whatever. | 16:22 |
| eandersson | But regions don't actually work | 16:22 |
| efried | right | 16:22 |
| eandersson | Not sure what your point is | 16:22 |
| efried | see latest comment in the devstack patch. | 16:22 |
| *** tosky has joined #openstack-nova | 16:22 | |
| eandersson | Well that patch was not to prove that multi region was broken | 16:22 |
| efried | eandersson: My point is: I'm happy to have the behavior fixed, but it would be nice to have a regression test in place. | 16:22 |
| eandersson | That patch was to prove that get_endpoint was broken if region is supplied | 16:23 |
| eandersson | Of course if you introduced multi-region it would have been way worse | 16:23 |
| efried | okay, and that proof is the presence of that log message... | 16:23 |
| eandersson | Since that would never have worked | 16:23 |
| efried | so now if we reinstate that patch and rebase it onto the fix, the log message should be absent. | 16:23 |
| eandersson | Exactly | 16:24 |
| efried | In fact, we could just raise an exception and blow up the world if the endpoint is None there. | 16:24 |
| eandersson | I think we should. | 16:24 |
| efried | cool, you or me? :) | 16:24 |
| eandersson | I can follow up after your patch so people can git blame me :p | 16:24 |
| efried | So hold on a sec | 16:25 |
| efried | I'm not completely convinced we should be doing this for real | 16:25 |
| efried | just to prove the fix | 16:25 |
| eandersson | We can also take path nr 2 | 16:25 |
| eandersson | just pass on the region_name to the ironic client | 16:25 |
| efried | If ironicclient has a way to recover, and we disable/disallow it, that could piss off some operators :) | 16:25 |
| efried | So yeah, passing the region_name to the ironicclient's recovery dealy would be better I guess. | 16:26 |
| eandersson | The problem is really that we are letting the ironic client handle a failure scenario, but not actually giving it the context it needs | 16:26 |
| efried | Though once again we're throwing good code after bad. | 16:26 |
| efried | okay, to pass the region_name through, we only need changes on the nova side, right? ironicclient is already set up to handle it if it's in kwargs? | 16:26 |
| eandersson | https://github.com/openstack/python-ironicclient/blob/master/ironicclient/client.py#L121 | 16:27 |
| eandersson | Looks like it yea | 16:27 |
| efried | Cool. So two things: | 16:28 |
| efried | 1) Test patch that raises an exception if the ksa endpoint lookup fails, based on top of the fix https://review.opendev.org/654457 <== expect it *not* to blow up | 16:28 |
| efried | 2) New patch to pass region_name through to irocicclient setup | 16:28 |
| eandersson | Perfect! | 16:28 |
| efried | I wonder if 2) should actually take two completely separate code paths. | 16:28 |
| efried | One if ksa endpoint lookup succeeds, that passes in no additional kwargs | 16:28 |
| eandersson | Yes - that would be fine | 16:28 |
| eandersson | And safer | 16:28 |
| efried | and a separate one if ksa lookup fails, that passes in basically all the ksa opts :( | 16:29 |
| efried | If you have the time to propose those, I'll happily review. I'm in the midst of summit chart prep hell. | 16:29 |
| eandersson | Sure - I'll try. I am also slammed this week though. | 16:30 |
| efried | looks like interface is already passed through. service_type *should* be unnecessary, but <shrug> guess it wouldn't hurt to pass it through. | 16:31 |
| efried | and those are the only three used at the moment. | 16:31 |
| eandersson | Yea - was thinking the same. | 16:31 |
| eandersson | Unrelated to this discussion. Is it just for me that I can't open links on review.opendev.org in new tabs anymore? | 16:32 |
| eandersson | (right click, Open link in new tab) | 16:34 |
| efried | uh | 16:36 |
| * efried tries | 16:36 | |
| efried | eandersson: wfm | 16:36 |
| efried | what happens when you try it? | 16:36 |
| eandersson | Nothing | 16:36 |
| eandersson | Hmm works on Firefox | 16:36 |
| eandersson | I guess my browser is just acting up. | 16:37 |
| efried | I'm Chrome 73.0.3683.75 on Bionic fwiw | 16:37 |
| efried | kashyap: I'm reading https://review.opendev.org/#/c/645814/4/specs/train/approved/cpu-selection-with-hypervisor-consideration.rst and not understanding it, at I think a more fundamental level than other reviewers. | 16:38 |
| efried | I suspect I would be able to infer more if you addressed mriedem's comments | 16:39 |
| efried | Do you have a sec to give me a brief rundown on what actual operations or behaviors are affected by these *HypervisorCPU() calls? | 16:40 |
| *** openstackgerrit has joined #openstack-nova | 16:41 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Fix {min|max}_version in ironic Adapter setup https://review.opendev.org/654457 | 16:41 |
| *** sridharg has quit IRC | 16:48 | |
| *** lpetrut has joined #openstack-nova | 16:49 | |
| *** eharney has quit IRC | 17:00 | |
| *** tbachman has joined #openstack-nova | 17:20 | |
| *** lpetrut has quit IRC | 17:32 | |
| openstackgerrit | Nikolay Fedotov proposed openstack/nova master: Fix disappearing ironic hypervisors after rebuilding hashring https://review.opendev.org/654584 | 17:47 |
| sean-k-mooney | efried: kashyap spec is mainly techdebt removal through the use of newer libvirt apis that would allow nova to validate that the configuration specified in the nova.conf is valid | 17:54 |
| efried | sean-k-mooney: Resulting in... earlier failures in spawn()? Refusal to start nova-compute? | 17:55 |
| sean-k-mooney | i was suggesting failing to start the agent if the cpu flags were not compatible with the host/model specifed | 17:55 |
| *** zbr is now known as zbr|pto | 17:56 | |
| sean-k-mooney | but the spec does not really make it clear | 17:56 |
| *** eharney has joined #openstack-nova | 17:57 | |
| sean-k-mooney | it will in theroy also make the traits we report more acurate as it would take into account if kvm or qemu/which version of qemu is used too. | 17:57 |
| sean-k-mooney | with all that said i agree with mriedem commets. the spec does not really spell out how useing the new apis deliver a new feature or capablity to operators/users form a nova perspecitve | 18:00 |
| efried | thanks sean-k-mooney. I'm putting together charts for the project update at the summit and need to understand to *some* extent what each blueprint is about. | 18:03 |
| sean-k-mooney | efried: from kasyaps top comment on the rewview 'Nova will now be able to answer: "Is this combination of IvyBridge + 'pcid' & 'pdpe1gb' supported by KVM, QEMU and libvirt on the host?"' | 18:04 |
| efried | Right, "be able to answer" doesn't help me understand what will actually *happen* | 18:05 |
| efried | so, detect whether A+B+C is copacetic and... what? | 18:05 |
| sean-k-mooney | so really kashyap is trying to turn the a possible runtime failure when you boot the first vm into hopefully a startup failure of the agent with a detailed error message | 18:06 |
| efried | when you say "agent", you mean n-cpu? | 18:06 |
| sean-k-mooney | ya at least that what i hope is the plan | 18:06 |
| efried | ...by blowing up the virt driver init, I guess | 18:06 |
| sean-k-mooney | yep | 18:06 |
| efried | Cool. Definitely needs to be spelled out in the spec. | 18:06 |
| efried | That is *not* an implementation detail :) | 18:07 |
| sean-k-mooney | ya kasap has a second spec that he inherited that is related | 18:07 |
| sean-k-mooney | i think the agent stoping might be in that one | 18:07 |
| sean-k-mooney | https://review.opendev.org/#/c/642030/ | 18:08 |
| sean-k-mooney | ya line 71 https://review.opendev.org/#/c/642030/6/specs/train/approved/cpu-model-selection.rst@71 | 18:09 |
| *** ricolin has quit IRC | 18:10 | |
| *** rnoriega has quit IRC | 18:11 | |
| *** rnoriega has joined #openstack-nova | 18:12 | |
| openstackgerrit | sean mooney proposed openstack/nova master: [WIP] Libvirt: add nfv job https://review.opendev.org/652197 | 18:14 |
| efried | stephenfin: mriedem: I don't see a bp for nova-console removal. Who was going to be spearheading that? | 18:15 |
| sean-k-mooney | efried: fyi its a bankholiday here in ireland so stephenfin proably wont see that until tomorrow | 18:16 |
| efried | sean-k-mooney: ack, thx | 18:16 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add placement request filter for disabled computes https://review.opendev.org/654596 | 18:16 |
| mriedem | efried: sean-k-mooney: dansmith: belmoreira: ^ here is a PoC for the disabled compute pre-filtering stuff we talked about in berlin | 18:16 |
| mriedem | melwitt: ^ | 18:16 |
| efried | mdbooth_: nudge, dud you see http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005366.html ? | 18:16 |
| mriedem | efried: i don't think there is a blueprint or a driver | 18:16 |
| efried | ack | 18:17 |
| sean-k-mooney | oh cool | 18:17 |
| mriedem | efried: i wouldn't probably move forward with nova-console removal without a definitive ack from the citrix team | 18:17 |
| dansmith | isn't jaypipes going to lose his mind about that trait? | 18:17 |
| mriedem | why? | 18:17 |
| sean-k-mooney | metadata in placement | 18:18 |
| mriedem | that's what traits are | 18:18 |
| dansmith | because it's not a capability | 18:18 |
| efried | mriedem: That's a pretty big "bug fix" | 18:19 |
| sean-k-mooney | well if a compute service is disable its not capable of booting a node | 18:19 |
| sean-k-mooney | *vm | 18:19 |
| sean-k-mooney | the other way to do it if traits were not used would be a aggregate | 18:20 |
| sean-k-mooney | just add the root rps of all disabled nodes to an aggreate and frobid it | 18:20 |
| mriedem | efried: see the todo | 18:20 |
| sean-k-mooney | but a trait works too | 18:20 |
| mriedem | so, this is a poc hacked together for discussion at the ptg | 18:20 |
| mriedem | i would write a spec | 18:21 |
| mriedem | but wanted to post this first after working on it one morning | 18:21 |
| efried | dig | 18:21 |
| sean-k-mooney | an active service aggrate + member of is likely less of a sticking point in that an aggrate is just a bunch of resouce providers | 18:21 |
| *** cfriesen has joined #openstack-nova | 18:29 | |
| mriedem | efried: i thought i had something in the ptg etherpad for this bug but can't find it now - what would you prefer? nova general etherpad or the xp one with placement? | 18:32 |
| efried | mriedem: "this bug" - disabled CPU prefiltering? | 18:33 |
| mriedem | yes | 18:33 |
| efried | Doesn't require any placement changes (I don't count os-traits really) so nova etherpad. | 18:33 |
| *** ivve has quit IRC | 18:33 | |
| efried | but please ask yourself: does it need PTG time? | 18:33 |
| mriedem | alright | 18:33 |
| efried | I haven't read your patch yet | 18:34 |
| mriedem | given the questions i just got after posting this... | 18:34 |
| efried | sure, questions that would be answered in a spec; I guess I'm asking whether discussion in the spec wolud suffice. | 18:34 |
| efried | would | 18:34 |
| sean-k-mooney | am it might help to do some of it in real time too | 18:35 |
| mriedem | yeah idk, i guess i was thinking i'd rather crank it out in 15 minutes rather than waffle on a spec for 2 months | 18:35 |
| sean-k-mooney | but i think the idea of adressing this in placment in some way is a good one | 18:35 |
| efried | sean-k-mooney: Do you mean you think there's potential placement API changes needed? | 18:36 |
| mriedem | sean-k-mooney: it has to be if cern (and huawei public cloud) want to grab more allocation candidates from placement during scheduling, cern is currently hacking around this | 18:36 |
| sean-k-mooney | efried: no this should not need any api changes in placment | 18:36 |
| *** ttsiouts has joined #openstack-nova | 18:37 | |
| sean-k-mooney | but we need to either use a trait or aggrage of some kind to track it in placement so placment can use its existing apis to filter out disabled hosts | 18:37 |
| sean-k-mooney | mriedem: right if you use limit its going to be a huge pain | 18:37 |
| mriedem | cern limits their allocation candidate results to like 20 and as a result were only getting back disabled computes in some cases | 18:38 |
| sean-k-mooney | i left a quick comment https://review.opendev.org/#/c/654596/1/nova/scheduler/request_filter.py@104 by the way ill review it properly tomorow | 18:38 |
| * jaypipes flies in | 18:38 | |
| efried | Yeah, so the concept seems pretty simple to me, but if you think we need PTG time... | 18:38 |
| mriedem | efried: i just assume that like all things placement related there will be a holy war | 18:39 |
| efried | I really hope we're getting close to having an established process for these GET /a_c-based filters | 18:39 |
| efried | to the point where soon we'll be able to have specless bps for them. "Add trait called X. Set trait X on compute RP when Y. Add filter for X a) in flavor, b) in image, c) when condition Z is present. Done." | 18:40 |
| sean-k-mooney | that would make my "port num instance filter to placemtn" personal backlog item simpler :) | 18:41 |
| efried | melwitt, lbragstad: I seem to recall this session https://www.openstack.org/summit/denver-2019/summit-schedule/events/23712/migrating-nova-apis-to-keystone-scope-types being merged with another - link please? | 18:42 |
| efried | d'oh, never mind. | 18:43 |
| jaypipes | mriedem: https://review.opendev.org/#/c/623558/ addresses the *reason* why CERN limits to 20. That's a more appropriate solution IMHO than making placement into another servicegroup API where we will need to sync active status between the nova cell DB tables and placement. | 18:45 |
| jaypipes | mriedem: it's not a holy war. it's just fundamentally you'll get into a situation where there are two sources of truth for whether a provider is active or not. | 18:46 |
| jaypipes | I'd prefer to avoid that if possilbe. | 18:47 |
| mriedem | we already have 2 sources of truth for aggregates yeah? | 18:47 |
| mriedem | i thought one of the goals of placement was to do as much filtering in sql as possible to avoid the slower post-filtering in python we have | 18:47 |
| mriedem | so this would also align with that | 18:48 |
| mriedem | i realize the downside of this is nova and placement are out of sync on the status of the thing - we could potentially heal that in our compute update_available_resource periodic | 18:48 |
| efried | Maybe eventually placement can be the sole source of truth | 18:48 |
| jaypipes | erm, not really if it doesn't usually filter anything at all... | 18:48 |
| mriedem | i imagine in a large cloud the ComputeFilter gets used a lot | 18:49 |
| mriedem | during maintenance and such | 18:49 |
| jaypipes | mriedem: we could run the servicegroup API ahead of time and pass a parameter like ?uuids=!in:<DISABLED_SET> | 18:50 |
| jaypipes | which actually *would* be an efficient filter. | 18:50 |
| jaypipes | but I digress. | 18:50 |
| sean-k-mooney | that could be a long list of uuids | 18:51 |
| sean-k-mooney | which is why an aggregate would make more sense | 18:51 |
| jaypipes | sean-k-mooney: rarely. | 18:51 |
| jaypipes | sean-k-mooney: you're going to keep an aggregate membership sync'd with yet another periodic task? :( | 18:51 |
| sean-k-mooney | no on servcei startup have the compute node remove itself form the aggrrate if its currently listed | 18:52 |
| mriedem | sean-k-mooney: i don't think you'd want an aggregate for all non-disabled computes if you have 14K computes | 18:52 |
| sean-k-mooney | then we can add it if we miss a heartbeat or someone use force down | 18:52 |
| jaypipes | mriedem: I think he's saying the opposite. an agg for disabled providers. | 18:53 |
| sean-k-mooney | mriedem: sure but you can do it the other way around too as you said in the review | 18:53 |
| mriedem | sean-k-mooney: if we did aggregates for this yes that's what i'd do, an aggregate for only disabled computes | 18:53 |
| jaypipes | oh look, a case for forbidden aggregates :) | 18:53 |
| mriedem | and yeah that's what we'd use ^ | 18:54 |
| sean-k-mooney | jaypipes: orgiginly in the code is said an aggrate for enabled because i fogot negitive member of was merged | 18:54 |
| jaypipes | sean-k-mooney: it's the forbidden fruit. | 18:54 |
| sean-k-mooney | from a db perspecitve this really would not be that expensice either way | 18:55 |
| sean-k-mooney | the negitive case would be better but it will be similar | 18:55 |
| jaypipes | so, I suppose, in order of preference, I would personally go with: a) finish up https://review.opendev.org/#/c/623558/ and the other cell-specific-service-listing thing mentioned in there, b) use an aggregate for disabled computes and member_of=!<uuid> placement request filter | 18:55 |
| sean-k-mooney | the aggrates table is just too uuid fields both of which are indexed | 18:55 |
| jaypipes | actually, it's not really an either/or. can/should do https://review.opendev.org/#/c/623558/ as well as a placement request filter for !member_of a disabled aggregate. | 18:56 |
| sean-k-mooney | yes they are not mutally exclucive and i think both are useful | 18:57 |
| jaypipes | mriedem, sean-k-mooney: maybe adding a CONF.disabled_aggregate_uuid option that would be used to construct a placement request filter that used a member_of=!<uuid> query parameter to allocation candidates? | 18:58 |
| jaypipes | or something like that.. | 18:58 |
| sean-k-mooney | jaypipes: well it would be a prefilter so it would be enabled via config | 18:59 |
| sean-k-mooney | or are you also suggesting a config option for the compute node too | 18:59 |
| mriedem | if we use an aggregate we have to use the same uuid everywhere, | 18:59 |
| mriedem | either via config or hard-coded in api/compute/scheduler | 18:59 |
| sean-k-mooney | ya that is simple however | 18:59 |
| sean-k-mooney | just use a uuid5 that we caluate for a constat or hardcode it | 19:00 |
| *** tbachman has quit IRC | 19:06 | |
| *** slaweq has joined #openstack-nova | 19:08 | |
| *** nicolasbock has joined #openstack-nova | 19:08 | |
| efried | oo, a special aggregate UUID. It's almost like... metadata. | 19:14 |
| sean-k-mooney | that is sotred in nova | 19:15 |
| sean-k-mooney | not placement | 19:15 |
| sean-k-mooney | clinets of placement are free to use aggreate to group rps whatever way the like | 19:15 |
| *** tbachman has joined #openstack-nova | 19:25 | |
| *** amodi has quit IRC | 19:27 | |
| sean-k-mooney | by the way before i go i noticed some strange errors in the nfv-ci job | 19:27 |
| sean-k-mooney | libvirtError: Cannot access storage file '/opt/stack/data/nova/instances/b4afcea9-12d0-4f79-953c-d09dad3e5515/disk' (as uid:107, gid:107): Permission denied | 19:27 |
| sean-k-mooney | i also got DiskNotFound: No disk at /opt/stack/data/nova/instances/f5f5ce5f-a616-4e5e-aa35-d3cc469f869a/disk | 19:27 |
| *** amodi has joined #openstack-nova | 19:28 | |
| sean-k-mooney | i have set the concurance to 1 and its reruning currently as i think that is part of the issue but i have no iday way those disk errors are showing up | 19:28 |
| sean-k-mooney | if people want to take a look the logs are here http://logs.openstack.org/97/652197/11/check/tempest-nfv-multinode/e1fa793/compute/logs/screen-n-cpu.txt.gz?level=TRACE#_Apr_20_22_14_03_503486 | 19:30 |
| mriedem | efried: except metadata that is unparseable by the human eye :) | 19:32 |
| sean-k-mooney | other vms worked fine so im wondering if this is related to load/memory pressure. | 19:32 |
| *** dasp has joined #openstack-nova | 19:33 | |
| *** lbragstad has quit IRC | 19:40 | |
| mriedem | d15ab1ed-0000-0000-0000-00000000000 :) | 19:41 |
| efried | sean-k-mooney: speaking of CI... | 19:43 |
| sean-k-mooney | what did i break now :) | 19:44 |
| efried | I got an extremely vague question from within the blue walls today about what kinds of tests the PCI job ought to be running. | 19:44 |
| efried | I hadn't a clue, but thought you might. | 19:44 |
| sean-k-mooney | ideally pci passthough via alaise in the flavor + neutron sriov interface, then it would be nice to alos test cold migration/resize and possibly live migration in the neutron case | 19:46 |
| efried | someone said something like the PCI job was just setting up devstack and not running any actual tests. I'm not sure how accurate that could be, or if they were looking at the wrong thing, or if at some point we disabled all the actual tests while we were working on the infra, or what. | 19:46 |
| sean-k-mooney | i would just run the standared tempest test with tweeked flavors and different vnic_types | 19:46 |
| *** dakshina-ilangov has joined #openstack-nova | 19:46 | |
| sean-k-mooney | efried: it runs test but it boots like 1 vm or very few tests | 19:46 |
| efried | was the PCI job just hitting SR-IOV? | 19:48 |
| efried | Cause there's an SRIOV job that runs against neutron | 19:48 |
| efried | not sure if those are related | 19:48 |
| sean-k-mooney | the pci job was ment for testing QAT integration with flavor bassed passthough | 19:49 |
| sean-k-mooney | there was a seperate one for neutron sriov testing | 19:49 |
| sean-k-mooney | looking at http://52.27.155.124/pci/482200/7/ | 19:49 |
| sean-k-mooney | there are not test in the test repofrt | 19:50 |
| sean-k-mooney | but im not sure that means threare are not test | 19:50 |
| sean-k-mooney | it looks like some test are run http://52.27.155.124/pci/482200/7/console_status.log.gz | 19:51 |
| efried | I can't tell if they pass | 19:52 |
| sean-k-mooney | nor can i | 19:54 |
| sean-k-mooney | looking at the devstack output and the tempest config i cant see tehm creating any custom flavor either | 19:54 |
| efried | how did you know where to find these logs? | 19:55 |
| efried | the CI is running, just not reporting? | 19:56 |
| *** slaweq has quit IRC | 19:56 | |
| sean-k-mooney | i clicked on an old log got a 404 and went up one directory | 19:56 |
| efried | how did you know this patch had any results? Did you search for reviewedby:Intel PCI? | 19:57 |
| * efried tries to learn how to fish | 19:57 | |
| sean-k-mooney | i looked at one of my patches that i knew it had commented on in the past | 19:57 |
| sean-k-mooney | http://52.27.155.124/ you can see the rest of the logs are here for the other intel cis | 19:58 |
| *** jmlowe has quit IRC | 19:58 | |
| efried | IC | 19:58 |
| sean-k-mooney | this is a ovh log server that i used ot own/payfor via intel for the ci logs | 19:59 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug config opts infinite recursion https://review.opendev.org/654468 | 19:59 |
| sean-k-mooney | or maybe it moved | 20:00 |
| sean-k-mooney | it used to be http://intel-openstack-ci-logs.ovh | 20:00 |
| efried | I think the 52. is aws | 20:00 |
| sean-k-mooney | ya the pci ci used an aws log server. they moved to using our ovh one for a while i guess it moved back to aws at some point | 20:01 |
| efried | yeah, that whole class A is amazon's. Not sure if that means aws, but probably | 20:01 |
| sean-k-mooney | basicaly it just had to be a server that was not on the intel network | 20:01 |
| efried | right, so the outside world could see it. | 20:02 |
| sean-k-mooney | actully no os that you did not have to go through a 3 level security review to host an external facing service on the coperate network | 20:02 |
| *** belmoreira has quit IRC | 20:02 | |
| efried | yeah, that too | 20:03 |
| sean-k-mooney | if its not on the copreate network it was way eaiser aslo the pci ci was originally in china which casued issues | 20:03 |
| efried | okay, I'm going to dive over the firewall and throw some of this intel back at intel. Thanks for the help. | 20:04 |
| sean-k-mooney | cool anyway my adivice would be to do something like im doing in the nfv ci test job | 20:04 |
| sean-k-mooney | e.g. run standared tempest test but use a custom flavor or tweek teh vnic_type used in the tempest config | 20:05 |
| sean-k-mooney | that will test 90% of the edgecases | 20:05 |
| sean-k-mooney | at that point if the ci is stable it can be extended to test more neich things | 20:05 |
| mriedem | dansmith: despite my typo in https://review.opendev.org/#/c/640197/ i'd like to get that in so we can move the grenade live migration job in-tree (this is under the queens backport) - because of devstack changes in queens + grenade i have to disable the ceph stuff in the grenade case, but it's either that or just drop the job altogether from queens which i'd like to avoid | 20:07 |
| mriedem | the commit message attempts to explain the mess and reasoning for just punting and disabling the ceph case | 20:07 |
| dansmith | okay | 20:09 |
| *** jmlowe has joined #openstack-nova | 20:14 | |
| *** ralonsoh has quit IRC | 20:14 | |
| sean-k-mooney | mriedem: speaking of ceph we have a downstream request to backport two of your patches to newton wich is eol upstrema. https://review.opendev.org/#/q/topic:bug/1635008+(status:open+OR+status:merged) i know its been quite a while but are you aware of any reaon of the top of your head why that woudl be a bad idea. | 20:14 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test for regression bug 1825537 https://review.opendev.org/654066 | 20:15 |
| openstack | bug 1825537 in OpenStack Compute (nova) "finish_resize failures incorrectly revert allocations" [Medium,In progress] https://launchpad.net/bugs/1825537 - Assigned to Matt Riedemann (mriedem) | 20:15 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop source node allocations if finish_resize fails https://review.opendev.org/654067 | 20:15 |
| sean-k-mooney | it looked resonable to me at first glance but just said i woudl ask | 20:15 |
| mriedem | sean-k-mooney: yes | 20:15 |
| mriedem | there was a regression introduced | 20:16 |
| sean-k-mooney | oh im glad i asked :) | 20:16 |
| mriedem | for $5 i'll tell you the secret | 20:16 |
| mriedem | https://review.opendev.org/#/q/I6fc7108817fcd9df4a342c9dabbf14ab7911d06a | 20:17 |
| sean-k-mooney | https://bugs.launchpad.net/charm-nova-compute/+bug/1671422? | 20:18 |
| openstack | Launchpad bug 1671422 in OpenStack nova-compute charm "charms: nova/cinder/ceph rbd integration broken on Ocata" [Critical,Fix released] - Assigned to James Page (james-page) | 20:18 |
| mriedem | that might be the other one | 20:18 |
| mriedem | https://review.opendev.org/#/q/Ieba216275c07ab16414065ee47e66915e9e9477d | 20:18 |
| sean-k-mooney | ok so it did not handel olde connection info. so if we are to backprot it we need to backport the fixes for the regression too | 20:19 |
| mriedem | sure | 20:19 |
| mriedem | or just tell the bank to upgrade | 20:19 |
| sean-k-mooney | i mean if i could i would but... | 20:20 |
| sean-k-mooney | that said depending on how invasive the other fixes are that might be answer | 20:20 |
| sean-k-mooney | huh the last one was only fixed recently in the grand scheme fo things | 20:23 |
| *** lpetrut has joined #openstack-nova | 20:29 | |
| *** lpetrut has quit IRC | 20:33 | |
| efried | sean-k-mooney: I can see whitelist/alias stuff in the nova.confZ | 20:41 |
| efried | pci_passthrough_whitelist = [{"vendor_id":"8086","product_id":"1520"}] | 20:42 |
| efried | pci_alias = {"name":"pci_network_card","vendor_id":"8086","product_id":"1520"} | 20:42 |
| efried | Presumably that does no good without a flavor requesting something at that alias? | 20:42 |
| efried | well, I mean, I guess it at least proves that the parts of the pci subsystem that parse those and generate the list of legal devices don't explode. | 20:42 |
| efried | Do you have any idea where a guy could find the script that's running the show here? | 20:43 |
| *** pcaruana has quit IRC | 20:46 | |
| sean-k-mooney | correct | 20:47 |
| sean-k-mooney | unless the flavor request the alaise it does not do anything | 20:47 |
| sean-k-mooney | for the device to be useable for neutron the physnet would have to be set in the whitelist | 20:47 |
| sean-k-mooney | they could be using a local.conf like i am. my guess would be it burried in a jenkins job somewhere | 20:48 |
| efried | that's what I figured | 20:48 |
| efried | I also really, really don't want to get too deeply involved here. | 20:48 |
| sean-k-mooney | ha to late :P | 20:48 |
| efried | I recognize it's a Learning Opportunity | 20:49 |
| sean-k-mooney | well its being converted to zull v3 right | 20:49 |
| efried | but if I go start debugging the CI, my life is over. | 20:49 |
| efried | yes, afaik | 20:49 |
| sean-k-mooney | so i woudl ignore how its currently running | 20:49 |
| sean-k-mooney | and plan for how it will run | 20:50 |
| efried | I sort of get the impression that there was someone competent doing all the work here, and then whoever that is left the company. | 20:50 |
| efried | And those left behind are scrambling to figure out wtf is going on. | 20:50 |
| sean-k-mooney | well yes but that happened liek 2 years ago :P | 20:51 |
| efried | oh | 20:51 |
| sean-k-mooney | actully the pci ci was always different | 20:51 |
| *** seyeongkim has quit IRC | 20:51 | |
| efried | well, I know it would take *me* at least two years to figure out how this stuff works... | 20:51 |
| sean-k-mooney | but the nfv ci was run by me an waldek but he left to joing redhat 2 years ago | 20:52 |
| sean-k-mooney | the pci ci was run by a team of 1 person out of the PRC site but they had other respociblityes | 20:52 |
| efried | different topic: "completing numa affinity polices for neutron sriov interfaces." <== on the PTG agenda. What's the status of this? Bullet says you were going to write a spec or something? | 20:53 |
| *** icey has quit IRC | 20:53 | |
| sean-k-mooney | ya basically what i want to get out of that topic it the direct | 20:53 |
| *** fyx has quit IRC | 20:53 | |
| sean-k-mooney | i could just repopose the old spec | 20:53 |
| sean-k-mooney | which say use a flavor/image property to specify the policy | 20:53 |
| *** yikun has quit IRC | 20:53 | |
| sean-k-mooney | or i could pass the policy via a neutron port | 20:54 |
| efried | is there any in-person design work needed? | 20:54 |
| sean-k-mooney | as this is numa affinty for neutron interfaces | 20:54 |
| *** jungleboyj has quit IRC | 20:54 | |
| sean-k-mooney | well it will need a spec in either case the quest is nova or neutron | 20:54 |
| sean-k-mooney | i also want to expand it to all neutron ports not just sriov to pick up the numa awrere vswitch work stephenfin did | 20:55 |
| efried | Not understanding the subject, it sounds like you need to a) put things into a port def (neutron), and then b) process them into affinity thingies (nova) | 20:55 |
| sean-k-mooney | yes but the question is where is the policy for the numa afinity expressed | 20:56 |
| sean-k-mooney | per neutron port? or in the flavor/image | 20:56 |
| efried | Even if it's on the port, would the neutron side require any actual code change, or is it just stuffing whatever into the already-garbage-in-garbage-out binding profile dict? | 20:56 |
| *** seyeongkim has joined #openstack-nova | 20:56 | |
| sean-k-mooney | binding profile is admin only | 20:57 |
| sean-k-mooney | so that is one of the questions use binding profile or add it as a new qos policy | 20:57 |
| efried | Is this something to discuss in the nova/neutron xproj? | 20:57 |
| *** icey has joined #openstack-nova | 20:57 | |
| *** jungleboyj has joined #openstack-nova | 20:57 | |
| sean-k-mooney | well that is why i addeed it there | 20:58 |
| efried | oh look | 20:58 |
| *** seyeongkim has quit IRC | 20:58 | |
| efried | it's already in there | 20:58 |
| sean-k-mooney | is it also in the main nova one | 20:58 |
| sean-k-mooney | if so kill the nova one | 20:58 |
| efried | so can I nix it from the main nova one? | 20:58 |
| sean-k-mooney | ya | 20:58 |
| efried | cool | 20:58 |
| sean-k-mooney | sorry that was added before the cross project etherpad i forgot to kill it when i move it over | 20:59 |
| *** gmann has quit IRC | 20:59 | |
| efried | yeah, that's cool, I'm just desperately trying to figure out how to schedule everything, and my first look is always going to be, "can we get rid of this??" | 21:00 |
| sean-k-mooney | im debating if we should drop one of the other topic i have in the xproj doc | 21:00 |
| sean-k-mooney | i have moved https://review.opendev.org/#/c/645173/ down in the xprojc etherpad by one as i think its something we shoudl discuss but im not sure ill have time to work on it. that said it might still be good to start the neutron work | 21:03 |
| *** zbr|pto has quit IRC | 21:03 | |
| *** icey has quit IRC | 21:04 | |
| *** jungleboyj has quit IRC | 21:04 | |
| *** mnaser has quit IRC | 21:04 | |
| *** gmann has joined #openstack-nova | 21:07 | |
| *** fyx has joined #openstack-nova | 21:07 | |
| *** jungleboyj has joined #openstack-nova | 21:07 | |
| *** mnaser has joined #openstack-nova | 21:07 | |
| *** icey has joined #openstack-nova | 21:07 | |
| *** seyeongkim has joined #openstack-nova | 21:08 | |
| *** yikun has joined #openstack-nova | 21:09 | |
| efried | jaypipes: Would you please put https://review.opendev.org/#/c/641994/ (AMD SEV reproposal) on your radar for review? There has been a nontrivial change of direction: namely making SEV contexts a quantifiable resource and therefore a resource class. | 21:13 |
| *** Sundar has joined #openstack-nova | 21:16 | |
| openstackgerrit | Merged openstack/nova-specs master: Tools & docs for backlog & abandoned spec process https://review.opendev.org/648800 | 21:17 |
| *** ttsiouts_ has joined #openstack-nova | 21:24 | |
| *** ttsiouts has quit IRC | 21:25 | |
| *** boxiang has quit IRC | 21:27 | |
| *** boxiang has joined #openstack-nova | 21:27 | |
| jaypipes | efried: yessir. | 21:44 |
| efried | thankyousir | 21:44 |
| jaypipes | npsir | 21:45 |
| *** ttsiouts has joined #openstack-nova | 21:50 | |
| *** ttsiouts_ has quit IRC | 21:52 | |
| *** rchurch_ has joined #openstack-nova | 21:57 | |
| *** bauzas_ has joined #openstack-nova | 21:58 | |
| *** manjeets_ has joined #openstack-nova | 21:58 | |
| *** bauzas has quit IRC | 21:58 | |
| *** tonyb has quit IRC | 21:58 | |
| *** johanssone has quit IRC | 21:58 | |
| *** bauzas_ is now known as bauzas | 21:58 | |
| *** panda has quit IRC | 21:58 | |
| *** sambetts_ has quit IRC | 21:58 | |
| *** odyssey4me has quit IRC | 21:58 | |
| *** rchurch has quit IRC | 21:59 | |
| *** Zara has quit IRC | 21:59 | |
| *** edleafe has quit IRC | 21:59 | |
| *** manjeets has quit IRC | 21:59 | |
| *** dansmith has quit IRC | 21:59 | |
| *** irclogbot_3 has quit IRC | 21:59 | |
| *** imacdonn has quit IRC | 21:59 | |
| *** panda has joined #openstack-nova | 22:00 | |
| *** odyssey4me has joined #openstack-nova | 22:00 | |
| *** seyeongkim has quit IRC | 22:01 | |
| *** dansmith has joined #openstack-nova | 22:01 | |
| *** imacdonn has joined #openstack-nova | 22:01 | |
| *** sambetts_ has joined #openstack-nova | 22:01 | |
| *** johanssone has joined #openstack-nova | 22:02 | |
| *** irclogbot_3 has joined #openstack-nova | 22:02 | |
| *** seyeongkim has joined #openstack-nova | 22:03 | |
| *** boxiang has quit IRC | 22:03 | |
| *** cfriesen has quit IRC | 22:03 | |
| *** boxiang has joined #openstack-nova | 22:04 | |
| efried | mriedem: https://review.opendev.org/#/c/645316/ is related to https://bugs.launchpad.net/nova/+bug/1817927 yah? | 22:06 |
| openstack | Launchpad bug 1817927 in OpenStack Compute (nova) "device tagging support is not checked during move operations" [Undecided,New] | 22:06 |
| *** jungleboyj has quit IRC | 22:07 | |
| *** gmann has quit IRC | 22:07 | |
| *** amodi has quit IRC | 22:07 | |
| *** nicolasbock has quit IRC | 22:07 | |
| *** jungleboyj has joined #openstack-nova | 22:07 | |
| *** nicolasbock has joined #openstack-nova | 22:07 | |
| *** mdbooth has joined #openstack-nova | 22:09 | |
| *** NewBruce0 has joined #openstack-nova | 22:10 | |
| *** tjgresha_nope has joined #openstack-nova | 22:10 | |
| *** irclogbot_3 has quit IRC | 22:11 | |
| *** amotoki_ has joined #openstack-nova | 22:11 | |
| *** sambetts has joined #openstack-nova | 22:12 | |
| *** gmann has joined #openstack-nova | 22:12 | |
| *** kinrui has joined #openstack-nova | 22:13 | |
| *** mlavalle has quit IRC | 22:14 | |
| *** NewBruce has quit IRC | 22:16 | |
| *** amotoki has quit IRC | 22:16 | |
| *** sambetts_ has quit IRC | 22:16 | |
| *** alex_xu has quit IRC | 22:16 | |
| *** wwriverrat has quit IRC | 22:16 | |
| *** yankcrime has quit IRC | 22:16 | |
| *** mdbooth_ has quit IRC | 22:16 | |
| *** kashyap has quit IRC | 22:16 | |
| *** dansmith has quit IRC | 22:16 | |
| *** jenglisch has quit IRC | 22:16 | |
| *** fungi has quit IRC | 22:16 | |
| *** tjgresha has quit IRC | 22:16 | |
| *** dansmith has joined #openstack-nova | 22:17 | |
| *** kinrui is now known as fungi | 22:19 | |
| *** irclogbot_0 has joined #openstack-nova | 22:19 | |
| *** irclogbot_0 has quit IRC | 22:19 | |
| *** edleafe has joined #openstack-nova | 22:19 | |
| *** irclogbot_3 has joined #openstack-nova | 22:20 | |
| *** ttsiouts_ has joined #openstack-nova | 22:22 | |
| *** ttsiouts has quit IRC | 22:23 | |
| *** jenglisch_ has joined #openstack-nova | 22:30 | |
| *** ttsiouts has joined #openstack-nova | 22:36 | |
| *** ttsiouts_ has quit IRC | 22:38 | |
| *** rcernin has quit IRC | 22:45 | |
| *** rcernin has joined #openstack-nova | 22:45 | |
| *** luksky has quit IRC | 22:50 | |
| *** tkajinam has joined #openstack-nova | 22:57 | |
| openstackgerrit | Michael Still proposed openstack/nova master: Remove fake_libvirt_utils from connection tests. https://review.opendev.org/642557 | 22:58 |
| openstackgerrit | Michael Still proposed openstack/nova master: Remove fake_libvirt_utils from snapshot tests. https://review.opendev.org/642558 | 22:58 |
| openstackgerrit | Michael Still proposed openstack/nova master: Remove fake_libvirt_utils from virt driver tests. https://review.opendev.org/643894 | 22:58 |
| openstackgerrit | Michael Still proposed openstack/nova master: Remove fake_libvirt_utils from libvirt imagebackend tests. https://review.opendev.org/643895 | 22:58 |
| openstackgerrit | Michael Still proposed openstack/nova master: Remove remaining vestiges of fake_libvirt_utils from unit tests. https://review.opendev.org/643896 | 22:58 |
| openstackgerrit | Michael Still proposed openstack/nova master: Remove fake_libvirt_utils users in functional testing. https://review.opendev.org/644793 | 22:58 |
| *** threestrands has joined #openstack-nova | 23:04 | |
| *** cfriesen has joined #openstack-nova | 23:07 | |
| *** tosky has quit IRC | 23:09 | |
| *** ttsiouts has quit IRC | 23:11 | |
| *** ttsiouts has joined #openstack-nova | 23:12 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/os-traits master: Dropping the py35 testing https://review.opendev.org/654651 | 23:13 |
| *** alex_xu has joined #openstack-nova | 23:15 | |
| *** rcernin has quit IRC | 23:15 | |
| *** rcernin has joined #openstack-nova | 23:16 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/os-resource-classes master: Dropping the py35 testing https://review.opendev.org/654653 | 23:16 |
| *** ttsiouts has quit IRC | 23:17 | |
| openstackgerrit | Merged openstack/nova stable/queens: Fix legacy-grenade-dsvm-neutron-multinode-live-migration https://review.opendev.org/640197 | 23:18 |
| mriedem | efried: yeah somewhat | 23:24 |
| mriedem | tonyb[m]: can you get this queens backport? https://review.opendev.org/#/c/640198/2 | 23:24 |
| *** mriedem has quit IRC | 23:24 | |
| *** mriedem has joined #openstack-nova | 23:25 | |
| mriedem | efried: unfortunately because i changed nova/test.py in https://review.opendev.org/#/c/654468/ it runs a full tempest run and all so i'm sitting in the check queue for 3.5 hours waiting to see if that makes any difference in the output | 23:27 |
| mriedem | maybe i can speed that up | 23:31 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug config opts infinite recursion https://review.opendev.org/654468 | 23:31 |
| sean-k-mooney | mriedem: so.... if i just run the compute and networking senario test and dont run all the api test in the nfv job do you think that is enough? | 23:32 |
| sean-k-mooney | run with concurancy 1 is... slow | 23:32 |
| *** hongbin has quit IRC | 23:33 | |
| mriedem | sean-k-mooney: those don't test nfv things do they? or are you making the ports sriov ports or something? | 23:33 |
| sean-k-mooney | im using flavors that enable hugepage pinning realtime eumlator thread polices a cpu topolgies | 23:34 |
| sean-k-mooney | i can also add custom tests but i wanted a baseline of tests we know work as a base and add to it if needed. | 23:34 |
| sean-k-mooney | but ya its still rung the tests but the job has been running for a little over 5 hours and 20 minutes. if i just run the senario test that will drop back to about an hour | 23:36 |
| sean-k-mooney | its multinode so the senario test will also chack migrations/resize. | 23:37 |
| sean-k-mooney | i was expecting concurancy 1 to take maybe 3 hours with current tests set but 5+ is stupidly slow. ill let this finish and go back to the smaller test set. we can always add more after if there are specific test that make sense | 23:39 |
| sean-k-mooney | anyway goodnight o/ | 23:40 |
| mriedem | yes just start with scenario tests in --serial or --concurrency=2 if you can | 23:43 |
| sean-k-mooney | concurrency=2 was ocationally resulting in no valid host errors | 23:44 |
| sean-k-mooney | pinning disable over subsription so its easy to end up running out for free cores | 23:45 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!