Tuesday, 2019-07-16

« Monday, 2019-07-15 Index Wednesday, 2019-07-17 »
*** artom has joined #openstack-nova00:08
openstackgerritMerged openstack/nova master: doc: Add links to novaclient contributor guide  https://review.opendev.org/66902800:27
*** tbachman has quit IRC00:30
*** brinzhang_ has joined #openstack-nova00:34
*** brinzhang has quit IRC00:37
*** jangutter has quit IRC00:42
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Make the '--config-drive' be a boolean option  https://review.opendev.org/65368300:46
*** TxGirlGeek has joined #openstack-nova00:48
*** tbachman has joined #openstack-nova00:52
*** mlavalle has quit IRC00:53
*** tbachman_ has joined #openstack-nova00:56
*** tbachman has quit IRC00:57
*** tbachman_ is now known as tbachman00:57
*** igordc has quit IRC01:01
*** tbachman has quit IRC01:01
*** bbowen has quit IRC01:02
*** bbowen has joined #openstack-nova01:02
*** gyee has quit IRC01:14
*** imacdonn has quit IRC01:15
*** imacdonn has joined #openstack-nova01:16
*** yedongcan has joined #openstack-nova01:18
*** spatel has joined #openstack-nova01:27
*** hongda has joined #openstack-nova01:59
openstackgerritMing Jin proposed openstack/nova master: Nova handle empty vcpu_pin_set scenario  https://review.opendev.org/67093502:23
*** tkajinam has quit IRC02:23
*** tkajinam has joined #openstack-nova02:24
openstackgerritYongli He proposed openstack/python-novaclient master: Microversion 2.75 - show server topology  https://review.opendev.org/67079002:25
*** hemna has quit IRC02:30
*** hongda has quit IRC02:38
*** whoami-rajat has joined #openstack-nova02:42
*** takashin has joined #openstack-nova02:57
*** hongbin has joined #openstack-nova03:00
openstackgerritMing Jin proposed openstack/nova master: Nova handle empty vcpu_pin_set scenario  https://review.opendev.org/67093503:05
*** logan- has quit IRC03:11
*** logan- has joined #openstack-nova03:14
*** factor has joined #openstack-nova03:20
*** psachin has joined #openstack-nova03:26
*** hongbin has quit IRC03:42
*** TxGirlGeek has quit IRC03:59
*** damien_r has joined #openstack-nova04:08
*** udesale has joined #openstack-nova04:09
*** damien_r has quit IRC04:13
*** damien_r has joined #openstack-nova04:35
*** damien_r has quit IRC04:38
*** damien_r has joined #openstack-nova04:41
*** mdbooth_ has joined #openstack-nova04:43
*** mdbooth has quit IRC04:46
openstackgerritYongli He proposed openstack/python-novaclient master: Microversion 2.75 - show server topology  https://review.opendev.org/67079004:46
*** igordc has joined #openstack-nova05:07
*** pcaruana has joined #openstack-nova05:08
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix a unit test for pre_live_migration method  https://review.opendev.org/67094805:09
*** ricolin_ has joined #openstack-nova05:10
*** spatel has quit IRC05:28
*** udesale has quit IRC05:39
*** udesale has joined #openstack-nova05:40
*** udesale has quit IRC05:42
*** udesale has joined #openstack-nova05:57
*** ratailor has joined #openstack-nova06:06
*** jmlowe has quit IRC06:07
*** jmlowe has joined #openstack-nova06:08
*** belmoreira has joined #openstack-nova06:19
openstackgerritAlex Xu proposed openstack/nova master: Add the virt driver interface for claim and unclaim the devices  https://review.opendev.org/67078206:27
openstackgerritAlex Xu proposed openstack/nova master: Move the allocation retrieving early  https://review.opendev.org/67078306:27
openstackgerritAlex Xu proposed openstack/nova master: Claim and unclaim deivces when spawn and destory the instance  https://review.opendev.org/67078406:27
openstackgerritAlex Xu proposed openstack/nova master: Claim/unclaim the deivces when rebuilding the instance  https://review.opendev.org/67078506:27
openstackgerritAlex Xu proposed openstack/nova master: Claim/unclaim the devices when resizing instance  https://review.opendev.org/67078606:27
openstackgerritAlex Xu proposed openstack/nova master: Populate the existing mediated devices in the libvirt device manager  https://review.opendev.org/67078706:27
*** damien_r has quit IRC06:31
openstackgerritYongli He proposed openstack/nova master: clean up orphan instances  https://review.opendev.org/62776506:38
*** dpawlik has joined #openstack-nova06:40
*** damien_r has joined #openstack-nova06:53
*** rcernin has quit IRC07:00
*** maciejjozefczyk has joined #openstack-nova07:02
openstackgerritmelanie witt proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware  https://review.opendev.org/50748607:06
*** rpittau|afk is now known as rpittau07:08
*** xek has joined #openstack-nova07:09
*** Alon_KS has quit IRC07:15
*** igordc has quit IRC07:16
*** tssurya has joined #openstack-nova07:20
*** spatel has joined #openstack-nova07:24
*** spatel has quit IRC07:29
*** belmoreira has quit IRC07:30
*** jangutter has joined #openstack-nova07:32
*** jangutter has quit IRC07:33
*** jangutter has joined #openstack-nova07:33
*** ricolin_ is now known as ricolin07:33
*** ratailor_ has joined #openstack-nova07:36
*** ratailor has quit IRC07:39
*** ratailor__ has joined #openstack-nova07:39
*** helenafm has joined #openstack-nova07:40
*** ratailor_ has quit IRC07:41
*** slaweq has joined #openstack-nova07:43
*** priteau has joined #openstack-nova07:43
*** udesale has quit IRC07:46
*** udesale has joined #openstack-nova07:46
*** igordc has joined #openstack-nova07:47
openstackgerritLee Yarwood proposed openstack/nova stable/stein: Add 'path' query parameter to console access url  https://review.opendev.org/67097207:51
*** cdent has joined #openstack-nova07:56
*** ttsiouts has joined #openstack-nova08:03
*** belmoreira has joined #openstack-nova08:05
*** igordc has quit IRC08:06
*** ralonsoh has joined #openstack-nova08:13
*** jangutter has quit IRC08:15
*** jangutter_ has joined #openstack-nova08:15
*** jangutter_ has quit IRC08:20
*** jangutter has joined #openstack-nova08:24
*** tkajinam has quit IRC08:25
tssuryatakashin: are you around ?08:28
takashintssurya: yes.08:29
tssuryayou review comment here: https://review.opendev.org/#/c/666792/2/novaclient/tests/unit/v2/test_shell.py@427708:29
tssuryacan I ask why 74 was added there in the first place ?08:29
tssuryacould you respond to the question when you have time ? thanks in advance08:30
tssuryayour*08:30
takashintssurya: okay08:30
*** mdbooth has joined #openstack-nova08:36
*** mdbooth_ has quit IRC08:37
tssurya@cores: reviews for https://review.opendev.org/#/c/645611/ would be appreciated, its in the runways. Thanks in advance08:38
*** derekh has joined #openstack-nova08:44
*** belmoreira has quit IRC08:48
*** panda has quit IRC08:54
*** belmoreira has joined #openstack-nova08:54
*** panda has joined #openstack-nova08:57
*** tetsuro has joined #openstack-nova09:00
openstackgerritSurya Seetharaman proposed openstack/python-novaclient master: API microversion 2.75: Add 'power-update' external event  https://review.opendev.org/66679209:13
*** ratailor_ has joined #openstack-nova09:14
*** ratailor__ has quit IRC09:17
*** ricolin has quit IRC09:20
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: doc: Clarify versioned wrapped method  https://review.opendev.org/67098909:25
*** ratailor__ has joined #openstack-nova09:30
*** takashin has left #openstack-nova09:30
*** ratailor_ has quit IRC09:34
slaweqsean-k-mooney: hi, can You help me with one bug?09:37
slaweqsean-k-mooney: https://bugs.launchpad.net/neutron/+bug/183664209:37
openstackLaunchpad bug 1836642 in OpenStack Compute (nova) "Metadata responses are very slow sometimes" [Undecided,New]09:37
*** boxiang has joined #openstack-nova09:41
*** belmoreira has quit IRC09:44
*** priteau has quit IRC10:10
openstackgerritYumengBao proposed openstack/nova master: add cyborg tempest job  https://review.opendev.org/67099910:15
*** tetsuro has quit IRC10:16
*** jojoda has joined #openstack-nova10:18
*** ttsiouts has quit IRC10:18
*** ttsiouts has joined #openstack-nova10:19
*** ttsiouts has quit IRC10:23
*** udesale has quit IRC10:29
*** derekh has quit IRC10:29
*** derekh has joined #openstack-nova10:38
openstackgerritMerged openstack/python-novaclient master: doc: Clarify versioned wrapped method  https://review.opendev.org/67098910:47
*** jojoda has quit IRC10:48
sean-k-mooneyslaweq: its posible that that is just a result of the limited resouces we have in the gate10:48
slaweqsean-k-mooney: yes, it's possible but it is causing many failures various jobs :/10:49
*** bbowen has quit IRC10:49
gibitssurya: left comments in https://review.opendev.org/#/c/645611/10:51
sean-k-mooneythe first two reuest are long in http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-n-api-meta.txt.gz#_Jul_11_23_43_16_70497910:51
sean-k-mooneythat one is 16 second and the previous one was 1.410:52
sean-k-mooneybut the request after that are 2ms10:52
openstackgerritMerged openstack/nova master: Add Python 3 Train unit tests  https://review.opendev.org/66943710:55
sean-k-mooneyslaweq: it look like that was the first request on that worker thread and it had to initalise the db connection andproably some other setup stuff10:58
*** jojoda has joined #openstack-nova10:58
*** brinzhang has joined #openstack-nova10:59
sean-k-mooneyslaweq: we can see the request is recived here http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-n-api-meta.txt.gz#_Jul_11_23_43_00_52307610:59
slaweqsean-k-mooney: yes, and response is send here: http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-n-api-meta.txt.gz#_Jul_11_23_43_16_70497911:00
slaweqit took 16 seconds to send it back11:00
slaweqafter 10 seconds this "cloud-init" like script in cirros is failing and instance don't have SSH key configured thus test fails :/11:01
*** davidsha has joined #openstack-nova11:01
sean-k-mooneyyes but if you look at the pids it was setting up the db connection on pid 9232 which is the pid that serviced the request11:01
slaweqsean-k-mooney: and what is strange for me, is that only this one request was so slow11:02
sean-k-mooneyso im guessing it was delayed becasue the api worker hadn to do some intal setup11:02
slaweqwhy all other requests around this are much faster?11:02
*** brinzhang_ has quit IRC11:03
sean-k-mooneybecasue they dont have to inisllise the wsgi app and set up the db connections11:03
slaweqsean-k-mooney: but don't You think it is bug if nova needs sometimes 16 seconds to send public-key?11:03
sean-k-mooneythey can reuse all the work that has been done11:03
sean-k-mooneyi dont think it actully too 16 second i think it was waiting for the database to return with the reponce11:04
slaweqfrom user (VM) PoV it was 16 seconds11:04
sean-k-mooneysure11:04
sean-k-mooneybut as i said it look like this is because its the frist api request recived by that worker11:05
sean-k-mooney16 seconds is not really that un resonable given the load that the gate vms are under. we can see if there is a way to speed it up but i dont think this would be an issue in production11:06
openstackgerritLee Yarwood proposed openstack/nova master: Get rid of args to RBDDriver.__init__()  https://review.opendev.org/66856411:09
slaweqsean-k-mooney: the problem is that this is an issue in the gate :/11:09
openstackgerritBalazs Gibizer proposed openstack/nova master: Move consts from neutronv2/api to constants module  https://review.opendev.org/66894511:10
sean-k-mooneyslaweq: yep which we may not be able to fix in nova. im goint to look at the other logs and then ill look at the code11:10
sean-k-mooneybut its like saying the ovs neutorn agenet does not wire up new port imediatly after teh agent is restart11:10
slaweqsean-k-mooney: thx11:10
openstackgerritBalazs Gibizer proposed openstack/nova master: Translatable output strings in heal allocation  https://review.opendev.org/66892511:11
sean-k-mooneywe know it take a whlie for it to recompute everything because it has a lot of work to do on start up. maybe we tweak something or have devstack ping the metadta api11:11
sean-k-mooneye.g. before tempest runs to make sure this setup is done before a test hits it11:12
slaweqsean-k-mooney: or other "gate only workaround" could be to patch cirros image and retry those calls if it fails first time11:12
*** ratailor__ has quit IRC11:13
cdent"gate only workaround"--11:14
sean-k-mooneylooking at the logs more closely the reponces that are 2ms are all cached11:19
sean-k-mooneyuncached responces for new intanstances are between 300-1000ms generally with the 16000 ms and similar long resonces also doing addtional work11:21
*** ttsiouts has joined #openstack-nova11:22
sean-k-mooneyby addtional work i mean it has to aquire a lock and call nova.context.get_or_set_cached_cell_and_set_connections before it then generates the metadata11:23
sean-k-mooneyor in the case of the first request also set up the db connection11:23
sean-k-mooneyok so when its not cached it has to call this11:25
sean-k-mooneyhttps://github.com/openstack/nova/blob/af4cd78e8537d8f0933e30598066cedac5844866/nova/context.py#L355-L37511:25
*** luksky11 has joined #openstack-nova11:26
openstackgerritBalazs Gibizer proposed openstack/nova master: Use neutron contants in cmd/manage.py  https://review.opendev.org/66894611:26
openstackgerritBalazs Gibizer proposed openstack/nova master: Add 'resource_request' to neutronv2/constants  https://review.opendev.org/66894711:26
sean-k-mooneywhich set up both the db connection and rabbitmq connection if its not already cached11:26
*** tesseract has joined #openstack-nova11:30
sean-k-mooneyok so that is invoked when teh context target_cell context manager is used to lookup the instace by its uuid here https://github.com/openstack/nova/blob/master/nova/api/metadata/base.py#L68011:35
slaweqsean-k-mooney: thx for looking into this11:35
openstackgerritBalazs Gibizer proposed openstack/nova master: Use the safe get_binding_profile  https://review.opendev.org/66981711:37
gibiefried: if you have time, could you look back to https://review.opendev.org/#/c/666857 ?11:49
*** yedongcan has left #openstack-nova11:55
sean-k-mooneyslaweq: im not 100% certin but the metadata api request that took 16 seconds may have been waiting on neutron for 12 second to respond with the security groups for the instance http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-q-svc.txt.gz#_Jul_11_23_43_16_66075111:55
*** lpetrut has joined #openstack-nova11:57
*** bbowen has joined #openstack-nova12:00
sean-k-mooneyslaweq: actully the device id of the port matches the instance id of the metadata request so yes its was waiting for neuton to return the security group info12:01
sean-k-mooneywhen we are building the metata data we call neutron to get the security group info here https://github.com/openstack/nova/blob/master/nova/api/metadata/base.py#L14512:02
sean-k-mooneywhich does this https://github.com/openstack/nova/blob/46a3bcd80b41e99ec4923c7cf3d0f8dd8505e97c/nova/network/security_group/neutron_driver.py#L374-L40912:02
sean-k-mooneywe list the ports associated with the server which we can see in the neutorn log here http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-q-svc.txt.gz#_Jul_11_23_43_04_46350512:02
sean-k-mooneythen we retrive the securtiy group info which takes 12 seconds http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-q-svc.txt.gz#_Jul_11_23_43_16_66075112:03
sean-k-mooneyit starts here http://logs.openstack.org/09/666409/7/check/tempest-full/08f4c53/controller/logs/screen-q-svc.txt.gz#_Jul_11_23_43_04_57587912:04
sean-k-mooneyslaweq: what does  "Synchronizing usage tracker for tenant" mean in the context of security groups?12:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Defaults missing group_policy to 'none'  https://review.opendev.org/65779612:07
*** boxiang has quit IRC12:17
slaweqsean-k-mooney: let me look at this now12:26
*** lpetrut has quit IRC12:28
kashyapaspiers: Hi, if you have a SUSE system near by, can you please check if any of the OVMF/EDK2 packages provide this file: /usr/share/OVMF/OVMF_CODE.fd12:31
* kashyap guesses there is no such file (even as a symlink)as the above in SUSE, except: /usr/share/qemu/ovmf-x86_64-opensuse-code.bin12:31
*** lpetrut has joined #openstack-nova12:32
*** Luzi has joined #openstack-nova12:32
slaweqsean-k-mooney: it looks that this "Synchronizing usage tracker for tenant.." is done here: https://github.com/openstack/neutron/blob/master/neutron/quota/resource.py#L245 and it counts number of security groups in db and update quota usage in db for this tenant12:34
slaweqsean-k-mooney: I will investigate this more, thx a lot for help12:34
*** ricolin has joined #openstack-nova12:35
sean-k-mooneyim updating the bug and tyring to see if its the same cause for other cases12:35
slaweqsean-k-mooney: thx12:35
sean-k-mooneyi am not sure its the same issue in all cases but i did see a another 12 second wait to get netwrok info in another case12:35
sean-k-mooneyim currently confirming its coming form a metadta build request12:36
artomlyarwood, heya, could you hit https://review.opendev.org/#/q/topic:bug/1832028+(status:open+OR+status:merged) ?12:37
artomAnd we'd need to find another stable core, with mriedem having a much earned PTO12:37
*** mchlumsky has joined #openstack-nova12:48
lyarwoodartom: just hit the stable/stein one, quick open question around testing12:52
*** udesale has joined #openstack-nova12:52
*** helenafm has quit IRC12:52
lyarwoodartom: I'm not sure if we need another DMN testing change in Neutron on stable/stein for this or if everything in Nova stable/stein is already enough12:52
*** tesseract has quit IRC12:53
*** helenafm has joined #openstack-nova12:53
openstackgerritMatthew Booth proposed openstack/nova master: Failing unit test for bug 1836212  https://review.opendev.org/67102312:54
openstackbug 1836212 in OpenStack Compute (nova) "libvirt: Failure to recover from failed detach" [Undecided,New] https://launchpad.net/bugs/183621212:54
efriedgibi: on it12:55
efriedAnyone know where cfriesen has been lately?12:55
*** tesseract has joined #openstack-nova12:55
*** lpetrut has quit IRC12:57
*** mchlumsky has quit IRC12:58
*** mchlumsky has joined #openstack-nova12:59
efriedgibi: are you still here?13:12
kashyapefried: I was wondering, too.13:17
kashyapProbably buried in downstream stuff13:17
efriedkashyap: Do you know anything about the libvirt patches to make vTPM more secure?13:17
kashyapefried: I do, peripherially.  And funny you ask!13:18
kashyapJust today I was browsing that vTPM thread on upstream libvirt list13:18
* kashyap brings the URL13:18
*** ricolin has quit IRC13:18
kashyapThe patch series is at v5, and almost ready to be merged13:19
efriedLibVirt RFE: https://bugzilla.redhat.com/show_bug.cgi?id=172803013:19
efriedRH has already written a patch https://www.redhat.com/archives/libvir-list/2019-July/msg00584.html13:19
openstackbugzilla.redhat.com bug 1728030 in libvirt "RFE: Add encryption of TPM emulator state" [Unspecified,New] - Assigned to libvirt-maint13:19
kashyapefried: https://www.redhat.com/archives/libvir-list/2019-July/msg00854.html13:19
kashyap"PATCH v5 00/20] Add support for vTPM state encryption13:19
kashyap"13:19
efriedcool, I don't know how to read any of that, so "almost ready to be merged" is a good bit of info.13:19
kashyapIt has gone through a good few rounds of feedback from long-time reviewers upstream13:20
efriedwhen that happens, will there need to be any special affordance in the nova design to enable it?13:20
efriedlike, do we have to (conditionally based on available libvirt version) do something different to the xml?13:20
kashyapefried: Probably the config classes and XML exposure, and the relevant calls in libvirt/driver.py?13:20
efriedI guess at the very least we would have to expose a trait saying that "more secure vTPM is available here"13:20
kashyapRight, on the version conditionals13:21
efriedBut someone needs to be working on the code in the first place. It hasn't been touched since the PTG.13:21
* kashyap is, meanwhile, shaving unending yaks for the Secure Boot stuff. Now that the Fedora patch is merged (https://src.fedoraproject.org/rpms/edk2/c/674b3c8a27a8), now getting the Debian and Ubuntu folks to follow suit.13:22
kashyapefried: Yeah, if the author has gone AWOL, then probably "punt it to backlog"?13:22
*** eharney has quit IRC13:22
kashyapWhen they come back, it can be revived.13:22
efriedno can do13:22
kashyapSorry, what do you mean?13:22
efriedmore likely "find new owner asap"13:22
kashyapAh13:22
efrieddownstream pressure13:23
kashyapAh-ha, nod.13:23
kashyapIt is a useful, self-contained solution -- for those who want it.13:23
*** tbachman has joined #openstack-nova13:24
*** tbachman_ has joined #openstack-nova13:26
*** hemna has joined #openstack-nova13:27
kashyapefried: You do know that the core vTPM stuff is merged in upstream libvirt, yeah?13:27
kashyap(The series linked above is for the vTPM state being ecrypted)13:27
efriedkashyap: Yeah13:27
efriedlike, the base design (what we have merged) could be completed any time somebody wanted to pick it up and run with it.13:27
kashyapYeah, just double-confirming.  Just checked with DanPB, he said he hasn't looked at the v5 libvirt series yet13:27
artomlyarwood, yeah, good point, lemme look into that13:28
*** brinzhang has quit IRC13:28
*** tbachman has quit IRC13:28
*** tbachman_ is now known as tbachman13:28
*** shilpasd has quit IRC13:31
sean-k-mooneykashyap: has it been released in a libvirt version13:33
*** liuyulong has joined #openstack-nova13:33
kashyapsean-k-mooney: The vTPM state being encrypted?13:33
kashyapNo, it is still in review upstream.13:33
kashyap(As noted above)13:33
sean-k-mooneyok then we cant have a feature that depends on that untill its released. not just merged13:33
sean-k-mooneyor in this case still under review13:34
*** helenafm has quit IRC13:34
sean-k-mooneybut unencrypted vTPM can still proceed13:34
*** ttsiouts has quit IRC13:34
*** ganso has quit IRC13:34
kashyapRight, I know the rule.  But as long as it is in an upstream libvirt release, it should damn well be fine to have a feature depend on a brand new upstream release13:34
sean-k-mooneyit shoudl hopefully be minor to extend unencypted vTPM to encyrpeted once the feature is released13:35
*** ttsiouts has joined #openstack-nova13:35
sean-k-mooneyyep13:35
sean-k-mooneyjust need to be in a livirt releases tar13:35
sean-k-mooneyit doesnt need to be in a distro13:35
kashyapRight.  That's good.  In the past there was an absolute bonkers discussion that libvirt release should "bake" in a distro for a month13:36
kashyap... and receive some "testing" and some such utter nonsense.13:36
kashyapGlad we are doing away with that crap.13:36
*** helenafm has joined #openstack-nova13:36
sean-k-mooneywell the reason for "master is not enough" is like use other porject somtimes merge stuff and revert it before a release13:37
sean-k-mooneyso it has to be released but it does not need to be packaged13:37
kashyapsean-k-mooney: Right, I'm with you on that -- so long as it's in a released tarball, we should be (and are) good.13:37
*** trident has quit IRC13:38
*** ttsiouts has quit IRC13:38
*** ttsiouts has joined #openstack-nova13:38
*** trident has joined #openstack-nova13:39
tssuryathanks gibi :)13:41
*** ganso has joined #openstack-nova13:43
*** Luzi has quit IRC13:43
artomlyarwood, wrote some words on https://review.opendev.org/#/c/670645/213:44
efriedartom: btw, I did *not* get around to opening that bug yesterday.13:44
efriedin case you had a yen to do it13:44
* artom reloads context13:44
* artom does the needful13:44
artomlyarwood, if you agree with said words, I shall do the needful for that as well13:45
* artom pokes sean-k-mooney to look at said words as well13:45
* artom feels like sean-k-mooney's bus-factor is a bit too high13:46
lyarwoodartom: I am if we can run A against stable/stein, I'm not sure that we can13:46
* sean-k-mooney sniggers "does the needful" sounds like a euphemism for going to the bathroom13:47
lyarwoodartom: A being OVS hybrid same host13:47
sean-k-mooneyartom: that is the backport13:47
artomlyarwood, any reason we can't add it to stein's experimental job?13:47
artomsean-k-mooney, yeah, lyarwood is concerned about testing13:47
artom(rightfully so)13:47
lyarwoodartom: yeah I just don't know if we just inherit that all from master or if we need to change stuff13:48
sean-k-mooneyyou should be able to backport the DNM test13:48
lyarwoodI'll try experimental first and go from there13:48
* artom does all the things13:48
sean-k-mooneyand merge in the neutorn-iptabels job too if needed13:48
*** TxGirlGeek has joined #openstack-nova13:48
artomAnd since https://review.opendev.org/#/c/670593/1 popped up when looking through my reviews trying to find the one where efried'd bug manifested itself, it'd be cool if someone could look at that as well, but there's no "natural" person for that area (simple change, but involved understanding SRIOV and device tagging)13:49
*** tbachman has quit IRC13:49
sean-k-mooneylyarwood: we do not have teh iptables job in the experimental pipline on stable/*13:49
lyarwoodsean-k-mooney: kk so we need a DNM change right?13:50
lyarwoodsean-k-mooney: sorry I wasn't sure which part you were talking about above :D13:50
sean-k-mooneyi have https://review.opendev.org/#/c/664442/ but i need to add the neutron iptables jobs and backport it13:51
*** luksky11 has quit IRC13:51
artomsean-k-mooney, you have to split the 2 changes as well13:51
artomBecause we need to test multi-host both without and with hybrid plug13:52
sean-k-mooneyi can create a version that will drop the unneeded jobs and just have teh two we need and propsoe the backpor if ye want13:52
*** psachin has quit IRC13:52
artomsean-k-mooney, I'd really appreciated if you handled covering all the test cases, yeah :)13:52
sean-k-mooneyso we need 3 or 4 test jobs13:53
artomsean-k-mooney, 3, for cases A, B, D13:54
artomA = neutron-tempest-iptables_hybrid13:54
*** mlavalle has joined #openstack-nova13:54
artomD = first half of https://review.opendev.org/#/c/664442/ (run revert tests in nova-live-migration)13:55
sean-k-mooneyb and d is the neutorn multinode job with and without iptables?13:55
artomB = second half of https://review.opendev.org/#/c/664442/ (iptables in nova-live-migration/nova-multinode)13:55
sean-k-mooneyok i can go do that now13:55
sean-k-mooney*nova-multinode which was nova-live-migration13:56
mdboothFolks, I'm undecided on https://review.opendev.org/#/c/671023/ . It looks to me like a customer actually hit this race in practise while running a heat stack. However, in general I suspect it's fairly uncommon. My preference is generally to fix this stuff, but this kind of cleanup also tends to go ignored. Is a fix for this likely to be of interest?13:58
* mdbooth doesn't want to add to his pile of abandoned code.14:00
openstackgerritMerged openstack/nova master: Remove Rocky-era min compute trusted certs compat check  https://review.opendev.org/66953914:00
sean-k-mooneyassuming your runit test actully repoduced the race then it should be fairly staitforwad to show your patch is correct14:01
sean-k-mooneyso as long as the fix is not too invasive i dont see why this would be contovertial14:01
sean-k-mooneybut i dont know14:01
sean-k-mooneyits a race that is proably not hard to debug so if we dont fix it now it will proably bite us later14:03
*** TxGirlGeek has quit IRC14:03
*** TxGirlGeek has joined #openstack-nova14:03
mdboothsean-k-mooney: It will require a refactor of a method in libvirt.guest. Nobody likes refactors.14:04
sean-k-mooneymdbooth: can  you do it without changing the api signiture14:04
mdboothOf course, it's internal to the libvirt driver.14:04
sean-k-mooneyby api signiture i ment the signiture of the funtion14:05
mdboothYes, probably, although the function signature does also need to be fixed.14:05
sean-k-mooneyas you said the guest stuff shoudl be internal to the libvirt driver so your e not modifying the virt direver api so it should be fine to backport14:06
artomefried, hah, see where else that error popped up: http://logs.openstack.org/09/666409/8/check/tempest-full-py3/38bf84e/job-output.txt#_2019-07-14_17_19_10_67755514:07
mdboothsean-k-mooney: That's fine. I'm more interested to know if anybody cares. If nobody cares I'm not going to fix it.14:07
yongliheefried:  per the run queue, there is 2.75 ahead my patch, seems i better  moving to 2.76 in advance?14:07
efriedartom: okay, so we should sort this out.14:07
mdboothsean-k-mooney: It's bad for my mental health ;)14:08
sean-k-mooneymdbooth: well at least we have the bug and a repoducer unit test14:08
efriedyonglihe: You mean in the runway queue?14:08
sean-k-mooneyso if you dont fix it someone else can at a later date14:08
artomefried, there aren't that many hits, but yeah, our theory from last night is pretty much confirmed14:08
*** TxGirlGeek has quit IRC14:08
yongliheefried, yeah.14:08
yonglihe'power-update' might take the 2.7514:09
efriedyonglihe: Tough call. Just because it's ahead of you in the runway queue doesn't mean it'll get reviewed first. If you go to 2.76, you'll also need to stack yours on top of the 2.75 one, which is pretty weird since there's no actual dependency.14:09
efriedprobably just best to ride it out.14:09
yongliheok, let's see what happens then.14:09
artomefried, in my experience it was basically a race to see which one lands first, and then the other person is stuck rebasing14:10
efriedartom: test_delete_server_while_in_building_state sounds like a test designed to expose this bug.14:10
*** TxGirlGeek has joined #openstack-nova14:10
mdboothartom: Is that holding up your revert resize change, btw?14:10
artomefried, for serious14:10
artommdbooth, what is?14:10
mdboothartom: I don't know, that's what I'm asking.14:10
sean-k-mooneymdbooth: the revert resize change has merged14:10
yonglihebtw, the corresponding novaclent patch realse released: https://review.opendev.org/#/c/670790/14:10
mdboothsean-k-mooney artom: \o/14:10
artommdbooth, no, this is unrelated, something that efried noticed last night14:10
*** TxGirlGeek has quit IRC14:10
sean-k-mooneymdbooth: on master. im working on creating a DNM patch to test stable14:11
artomsean-k-mooney, patch_ES_ :)14:11
sean-k-mooneyartom has the backport alreay up14:11
sean-k-mooneyartom: im going to do 1 patch with 3 jobs14:11
mdbooth[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate is not yet valid14:12
sean-k-mooneyits a dnm patch so i have no issue with copy pasting the job definiton and just not seting iptabels in one of them14:12
mdboothI bet I'm the only person to see that regularly :)14:12
sean-k-mooneymdbooth: are you doing multi node but have not syncted your clock on each of the hosts14:13
artomefried, https://bugs.launchpad.net/nova/+bug/1836754 Should I also propose something for elastic-recheck?14:14
openstackLaunchpad bug 1836754 in OpenStack Compute (nova) "Conflict when deleting allocations for an instance that hasn't finished building" [Undecided,New]14:14
*** tbachman has joined #openstack-nova14:14
mdboothsean-k-mooney: Nah, I'm using Hyper-V, which stops the clock while my laptop is suspended. It was still early last week.14:14
artomsean-k-mooney, ack, that works too14:14
mdboothThe guest tools are *supposed* to auto sync the time from the HV, but I've never made that work.14:14
sean-k-mooneymdbooth: right but do you not have ntpd installed to fix that14:14
mdboothYeah, I have to fix it with chronyd manually, which is a hack. It has to be manual because it won't auto-fix a jump that large.14:15
sean-k-mooneyah i see. i have only ever used hyperv on a desktop or server so never had that issue14:16
sean-k-mooneythat reminds me i need to order an ssd so i dont need to keep dual booting my workstation14:17
* sean-k-mooney i think its technical a workstation pc but i guess it could be a tower server... im sure there is a diffence to may wallet but other then that i think they are the same?14:19
mdboothsean-k-mooney: For an SSD? That seems like a weird distinction, but I'm out of touch with such things.14:20
sean-k-mooneyno i was wondering if my dell t7610 is technically a server or a workstation14:21
sean-k-mooneythe ssd boot drive died so im using the one out of my windows gaming pc as a stop gap but that means i have to dual boot it currently14:22
sean-k-mooneyim planning to get a seccond ssd and install linux then pass the windows ssd into a vm14:22
artom... do I need something special to propose patches to elastic-recheck? Gerrit's refusing me14:23
efriedartom: elastic-recheck would be nice, though I'm not sure who approves those when mriedem isn't around.14:23
sean-k-mooneyartom: no you shouldnt14:23
sean-k-mooneyartom: whats the error?14:23
*** eharney has joined #openstack-nova14:23
artomoh, no one had updated .gitreview to point to opendev/ instead of openstack-infra14:24
sean-k-mooneyah ya that would break it14:24
sean-k-mooneyi think the redirects dont work proably in all cases14:24
artomOh, I think it's because I cloned from github and no one's re-setup the mirroring14:29
sean-k-mooneyi dont think the infra projects are going to be mirror going forward14:30
*** icarusfactor has joined #openstack-nova14:32
*** luksky11 has joined #openstack-nova14:33
efriedstephenfin: you around to give https://review.opendev.org/#/c/657796/ a bump?14:33
stephenfinLooking14:34
*** factor has quit IRC14:35
stephenfinand done. That was nice and easy14:36
*** dpawlik has quit IRC14:36
gibiefried, stephenfin: thanks14:36
efriedgibi: I only regret I can't send "heal port allocations" to the gate.14:37
efriedhalf a dozen +As stacked up behind that one.14:37
gibiefried: no worries. we will wait for Matt to to +A that after he is back14:38
gibimeanwhile I can finally refocus on the server move operations + bandwidth series14:38
*** jc_ has joined #openstack-nova14:56
*** jc_ has left #openstack-nova14:57
openstackgerritShilpa Devharakar proposed openstack/nova master: Nova object changes for forbidden aggregates request filter  https://review.opendev.org/67107215:01
openstackgerritShilpa Devharakar proposed openstack/nova master: DB API changes to get non-matching aggregates from metadata  https://review.opendev.org/67107415:02
openstackgerritShilpa Devharakar proposed openstack/nova master: Add a new request filter to forbidden aggregates  https://review.opendev.org/67107515:03
openstackgerritShilpa Devharakar proposed openstack/nova master: Enable request filter forbidden_aggregates  https://review.opendev.org/66795215:03
*** shilpasd has joined #openstack-nova15:05
openstackgerritsean mooney proposed openstack/nova stable/stein: [DNM] testing bug/1813789 revert resize events  https://review.opendev.org/67107915:22
artomThat was fast15:22
sean-k-mooneyit took longer then i planned but i think ^ is right15:23
sean-k-mooneyi guess not15:25
sean-k-mooneyoh i think im missing a new line15:26
sean-k-mooneyand a space15:27
openstackgerritsean mooney proposed openstack/nova stable/stein: [DNM] testing bug/1813789 revert resize events  https://review.opendev.org/67107915:27
sean-k-mooneyok zull accpeted that so it should report back eventually15:34
*** gyee has joined #openstack-nova15:39
*** cdent has quit IRC15:45
*** eharney has quit IRC15:46
*** eharney has joined #openstack-nova15:46
*** igordc has joined #openstack-nova15:47
*** ttsiouts has quit IRC15:51
*** ttsiouts has joined #openstack-nova15:52
*** helenafm has quit IRC15:52
*** luksky11 has quit IRC15:55
*** tssurya has quit IRC15:55
*** ttsiouts has quit IRC15:57
*** ricolin has joined #openstack-nova16:09
*** hemna has quit IRC16:22
*** damien_r has quit IRC16:23
*** hamzy has quit IRC16:25
*** mmethot has quit IRC16:25
*** tesseract has quit IRC16:33
efriedshilpasd: still around?16:34
openstackgerritBoris Bobrov proposed openstack/nova master: Provide HW_CPU_X86_AMD_SEV trait when SEV is supported  https://review.opendev.org/63868016:34
openstackgerritBoris Bobrov proposed openstack/nova master: Add extra spec parameter and image property for memory encryption  https://review.opendev.org/66442016:34
openstackgerritBoris Bobrov proposed openstack/nova master: Extract SEV-specific bits on host detection  https://review.opendev.org/63633416:34
openstackgerritBoris Bobrov proposed openstack/nova master: Add <launchSecurity> and <driver iommu='on' /> to config.py  https://review.opendev.org/63631816:34
openstackgerritBoris Bobrov proposed openstack/nova master: Apply SEV-specific guest config when SEV is required  https://review.opendev.org/64456516:34
openstackgerritBoris Bobrov proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption  https://review.opendev.org/66661616:34
efriedshilpasd: I'm going to edit the commit message of https://review.opendev.org/#/c/671075 real quick so I can be +2 on the whole series.16:35
openstackgerritEric Fried proposed openstack/nova master: Add a new request filter to forbidden aggregates  https://review.opendev.org/67107516:37
openstackgerritEric Fried proposed openstack/nova master: Enable request filter forbidden_aggregates  https://review.opendev.org/66795216:37
efrieddansmith: I'm +2 all the way up on the forbidden aggs series ^16:38
*** rpittau is now known as rpittau|afk16:38
efriedthe biggish final patch is mostly doc16:40
*** TxGirlGeek has joined #openstack-nova16:40
*** atahardjebbar has joined #openstack-nova16:40
openstackgerritRaubvogel proposed openstack/nova master:    Author: Mauricio Tavares <raubvogel@gmail.com>  https://review.opendev.org/67110616:44
*** hemna has joined #openstack-nova16:44
openstackgerritRaubvogel proposed openstack/nova master:    Author: Mauricio Tavares <raubvogel@gmail.com>  https://review.opendev.org/67110616:45
*** mmethot has joined #openstack-nova16:47
*** amodi has quit IRC16:48
*** hemna has quit IRC16:49
*** sapd1_x has joined #openstack-nova16:50
*** hamzy has joined #openstack-nova16:55
*** derekh has quit IRC17:01
openstackgerritBoris Bobrov proposed openstack/nova master: Apply SEV-specific guest config when SEV is required  https://review.opendev.org/64456517:03
openstackgerritBoris Bobrov proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption  https://review.opendev.org/66661617:03
*** sapd1_x has quit IRC17:04
*** ganso has quit IRC17:06
*** ganso has joined #openstack-nova17:08
*** udesale has quit IRC17:09
*** davidsha has quit IRC17:11
*** atahardjebbar has quit IRC17:13
*** hemna has joined #openstack-nova17:14
*** TxGirlGeek has quit IRC17:17
*** igordc has quit IRC17:17
*** hemna has quit IRC17:18
efriedsean-k-mooney: you around?17:20
sean-k-mooneykind of17:20
efriedI try and I try, but I just can't follow this stuff. Would you please review https://review.opendev.org/#/c/670588/ for me? I'll proxy your +1.17:20
sean-k-mooneyim wait on food to be delivered in 20 mins or so17:21
efriedI think it was your suggestion anyway from the subsequent patch17:21
sean-k-mooneyoh right. in the non ovs case we added a noop pluging to os vif to handel generic vhost-user plugging17:21
sean-k-mooneyand for the ovs case its already handeled by os-vif17:22
efriedyeah, see, I can read those words, but I've never been able to follow the code around that area17:23
sean-k-mooneyso those functions should never get called. but ill trace through the code and confim17:23
efriedthanks17:23
sean-k-mooneywell it used to do an indrect function call by appending the vif type to a prefix and using gettr to look up the function17:24
sean-k-mooneyso it was not exactly the simplest code17:24
sean-k-mooneythat actully predates me working on openstack but i never really understood why we did it that way17:25
efriedyeah, the getattr removal is in the subsequent patch, and I was able to follow that. But it's also dependent on this being right.17:25
sean-k-mooneythis code path is for netwroking-vpp or snabb switch17:26
sean-k-mooneyits very very rarly used but we still dont want to break people that use it17:26
*** hemna has joined #openstack-nova17:34
openstackgerritMerged openstack/nova master: Defaults missing group_policy to 'none'  https://review.opendev.org/65779617:37
*** hemna has quit IRC17:38
*** ricolin has quit IRC17:40
*** luksky11 has joined #openstack-nova17:42
*** igordc has joined #openstack-nova17:44
*** TxGirlGeek has joined #openstack-nova17:53
sean-k-mooneyefried: done. i have put a walk through of how the dead code would have been called in the past but it is never used after https://review.opendev.org/#/q/Ifab3006454708ab290b93f02d82b794c334c3946 was merged last cycle17:53
efriedthanks sean-k-mooney17:53
*** hemna has joined #openstack-nova17:54
sean-k-mooneyno worries dinner just arrived so ill be away for a while. i might be back in an hour or two17:54
*** hemna has quit IRC17:59
*** spatel has joined #openstack-nova18:01
spatelsean-k-mooney: yt?18:01
spatelvirsh list command showing  instance-<uuid>  is there anyway it can show hostname of instance18:01
*** eharney has quit IRC18:02
spatelIn google people saying "virsh domhostname <domain>" but what is domain going to be?18:03
spatelalso i don't think kvm support that18:03
*** eharney has joined #openstack-nova18:06
*** dpawlik has joined #openstack-nova18:10
*** efried has quit IRC18:12
donnydWas there still interest in the NUMA for fortnebula?18:20
donnydLOL... interest in NUMA topologies18:20
*** efried has joined #openstack-nova18:21
donnydI have things to a more stable state with decent performance, so now I can move forward18:21
openstackgerritMauricio Tavares proposed openstack/nova master:    Increases the level of detail provided by the debug messages in pci_passthrough_filter.py.  https://review.opendev.org/67110618:25
efriedartom, sean-k-mooney: ---^  ?18:30
*** hemna has joined #openstack-nova18:37
artomdonnyd, yes please, I would like 1 the NUMA ;)18:39
donnydLOL +218:40
artomFor serious though, yeah, flavors with >= 2 NUMA nodes would be awesome18:40
donnydyou get 2 numas artom18:40
donnydI am not deep in the zuul or nodepool code, so how does the test work. Can you ask for a specific flavor?18:41
donnydI need a way to stay compatible with the other jobs18:41
artomefried, I don't even know where to start with that. I guess look at the logic before drowning them with the formatting and whitespace nits?18:41
artomdonnyd, that's not my strong suite, I'm afraid18:42
efriedartom: I was pointing at donnyd, not at Mauricio's patch18:42
artomefried, oh, hah18:43
efriedmy ---^ was an attempt to go around the corner18:43
artomdonnyd, sean-k-mooney would most likely know, though I hate to pile even more things on this plate18:44
artomOr ask in -infra? Is that good advice?18:44
donnyd2:44 PM <clarkb> there isnt18:46
donnyd2:44 PM <clarkb> we can do provider specific flavors but generally avoid that to handle cloud outages18:46
donnydartom: I will have to learn how to setup NUMA aware flavors, (unless someone knows off the top of their head) and it looks like we have a path forward19:04
*** bbowen has quit IRC19:05
artomdonnyd, it's kinda tedious, but not super hard: https://docs.openstack.org/nova/latest/user/flavors.html19:05
artomScroll down to CPU topology19:06
artomSorry, NUMA topology19:06
donnydOk, I will take a look19:06
artomActually both19:07
artomhw:cpu_ controls the guest topology, hw:numa_ controls how it maps onto host hardware19:07
*** eharney has quit IRC19:08
openstackgerritMerged openstack/nova master: Use Adapter global_request_id kwarg  https://review.opendev.org/67090719:11
*** maciejjozefczyk has quit IRC19:46
*** prometheanfire has joined #openstack-nova19:57
prometheanfireis one of the use cases for nova to run 1-3 compute nodes in 3-4 DCs across the world?  (via host aggregates and AZs)19:58
openstackgerritEric Fried proposed openstack/nova master: Nova object changes for forbidden aggregates request filter  https://review.opendev.org/67107219:59
openstackgerritEric Fried proposed openstack/nova master: DB API changes to get non-matching aggregates from metadata  https://review.opendev.org/67107419:59
openstackgerritEric Fried proposed openstack/nova master: Add a new request filter to forbidden aggregates  https://review.opendev.org/67107519:59
openstackgerritEric Fried proposed openstack/nova master: Enable request filter forbidden_aggregates  https://review.opendev.org/66795219:59
efriedprometheanfire: What's this in reference to?20:00
prometheanfiresometihng I may be doing soon20:00
prometheanfirecross continental rabbit could be fun20:01
prometheanfirejust not sure if it's an intended (supported) use case for nova20:01
openstackgerritMerged openstack/nova master: vif: Remove 'plug_vhostuser', 'unplug_vhostuser'  https://review.opendev.org/67058820:03
efriedohh, okay. So you're asking if you can make a setup like that?20:06
efriedprometheanfire: in a single cell?20:06
*** eharney has joined #openstack-nova20:06
efriedCause I would imagine you can surely do a multi-cell that way20:06
efrieddonnyd: do you know about this?20:06
prometheanfireya, could do it multi-cell I suppose, single cell may be nice management wise though20:07
efriedprometheanfire: isn't that how zuul is set up?20:07
prometheanfirethe main question I have is if I'll need to run non-compute stuff at each location or not (I'd rather it just be nova-compute and neutron agent type stuff)20:07
donnydwell if you look at any of the big clouds out there,  it would appear they try to stay away from this type of thing20:08
efriedrunning a nodepool across a dozen different locations20:08
donnydControl plane / internal stuff usually stays on the same l2//3 networks20:08
efriedYou might have to run a cell conductor at each site. I would think your db performance would be crap if you didn't.20:08
donnydBut I am not entirely sure what the ask is20:09
prometheanfirehow to best build a cloud that spans multiple DCs, but it still small beans (a couple hosts per location)20:09
donnydWhat is the plan for N/S E/W traffic... that has been my big hurdle in the past...20:10
artomprometheanfire, ooooo, are you being edgy? It's all the rage these days ;20:10
artom;)20:10
donnydwanted to host my control plane in someone elses cloud, and then just have hypervisors and storage here20:10
prometheanfireartom: kinda, but not quite that scale20:11
*** bbowen has joined #openstack-nova20:11
prometheanfiredonnyd: as far as traffic, we'll have site to sites, non-dedicated.  the E/W should be minimal traffic wise (between DCs at least)20:12
donnydI don't see any reason why it cannot be done if you already have S2S connections to secure traffic.. Gonna spend quite a lot of time having fun with rabbit...20:15
prometheanfireyep, I dread rabbit20:16
donnydand def be using DVR20:17
prometheanfireI'm not sure we'd use more than linuxbridge and maybe gre (or just vlan)20:18
prometheanfirekiss20:18
donnydThe way its supposed to work is the same l2 between l3 agent and the vm20:20
prometheanfireyep20:20
prometheanfirethat's possible20:20
donnydyou would want to use DVR because the L3 agent will be over a wan connection, and all your broadcast traffic would also be doing the same20:21
prometheanfireya, that'd be rough20:22
prometheanfireat least this isn't 4 years ago, dvr is better now I hear20:22
prometheanfirehttps://wiki.openstack.org/wiki/Neutron/DVR still mentions the road to juno :D20:23
prometheanfirefirst result when googling for 'neutron dvr'20:23
donnydI am not surprised20:25
donnydjuno was only a couple releases back20:26
prometheanfirefeels like20:28
prometheanfire9 releases?20:28
*** hamzy has quit IRC20:30
*** ralonsoh has quit IRC20:35
*** dasp has quit IRC20:45
*** dasp has joined #openstack-nova20:47
openstackgerritMerged openstack/nova master: vif: Stop using getattr for VIF lookups  https://review.opendev.org/66663020:47
*** pcaruana has quit IRC21:01
*** dpawlik has quit IRC21:15
*** gokhani has quit IRC21:23
*** dpawlik has joined #openstack-nova21:27
*** amodi has joined #openstack-nova21:27
*** dpawlik has quit IRC21:32
*** luksky11 has quit IRC21:36
openstackgerritmelanie witt proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware  https://review.opendev.org/50748621:47
*** spatel has quit IRC21:50
*** xek has quit IRC22:02
*** betherly has joined #openstack-nova22:26
Nick_Awe have snapshots set to qcow2 in nova.conf. I noticed they take 2x disk usage until uploaded to glance. Is it trying to convert qcow2 > qcow2 or something?22:26
*** betherly has quit IRC22:31
*** whoami-rajat has quit IRC22:31
*** rcernin has joined #openstack-nova22:36
*** tkajinam has joined #openstack-nova22:53
*** slaweq has quit IRC22:55
*** betherly has joined #openstack-nova22:56
*** betherly has quit IRC23:01
*** hemna has quit IRC23:24
*** hamzy has joined #openstack-nova23:27
*** betherly has joined #openstack-nova23:29
*** mlavalle has quit IRC23:29
*** betherly has quit IRC23:34
*** brinzhang has joined #openstack-nova23:49
« Monday, 2019-07-15 Index Wednesday, 2019-07-17 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!