| *** CeeMac has quit IRC | 00:10 | |
| *** tosky has quit IRC | 00:18 | |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: ksa auth conf and client for Cyborg access https://review.opendev.org/631242 | 00:32 |
|---|---|---|
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Add Cyborg device profile groups to request spec. https://review.opendev.org/631243 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Create and bind Cyborg ARQs. https://review.opendev.org/631244 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Pass accelerator requests to each virt driver from compute manager. https://review.opendev.org/698581 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Compose accelerator PCI devices into domain XML in libvirt driver. https://review.opendev.org/631245 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Enable hard/soft reboot with accelerators. https://review.opendev.org/697940 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Enable start/stop of instances with accelerators. https://review.opendev.org/699553 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Enable and use COMPUTE_ACCELERATORS trait. https://review.opendev.org/699554 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Bump compute rpcapi version and reduce Cyborg calls. https://review.opendev.org/704227 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Block unsupported instance operations with accelerators. https://review.opendev.org/674726 | 00:32 |
| openstackgerrit | Sundar Nadathur proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999 | 00:32 |
| *** tetsuro has joined #openstack-nova | 00:36 | |
| *** vishalmanchanda has joined #openstack-nova | 00:43 | |
| *** mlavalle has quit IRC | 00:46 | |
| openstackgerrit | Merged openstack/nova stable/stein: Fix os-keypairs pagination links https://review.opendev.org/711896 | 00:53 |
| *** tbachman_ has joined #openstack-nova | 00:56 | |
| *** tbachman has quit IRC | 00:56 | |
| *** tbachman_ is now known as tbachman | 00:56 | |
| *** tetsuro_ has joined #openstack-nova | 01:01 | |
| *** tetsuro has quit IRC | 01:05 | |
| *** gyee has quit IRC | 01:05 | |
| *** CeeMac has joined #openstack-nova | 01:05 | |
| *** zhanglong has joined #openstack-nova | 01:10 | |
| *** tbachman_ has joined #openstack-nova | 01:13 | |
| *** tbachman has quit IRC | 01:14 | |
| *** tbachman_ is now known as tbachman | 01:14 | |
| brinzhang | gmann:hi | 01:25 |
| brinzhang | gmann: I combined the os-instance-actions policy patch, and that has previous issue in cmd/test_policy https://review.opendev.org/#/c/706470/ | 01:27 |
| brinzhang | gmann: can you have a fast check ? | 01:27 |
| *** spatel has joined #openstack-nova | 01:28 | |
| *** dave-mccowan has joined #openstack-nova | 01:32 | |
| *** spatel has quit IRC | 01:32 | |
| *** mkrai has joined #openstack-nova | 02:37 | |
| *** ociuhandu has joined #openstack-nova | 02:43 | |
| *** psachin has joined #openstack-nova | 02:43 | |
| *** ociuhandu has quit IRC | 02:48 | |
| *** spatel has joined #openstack-nova | 03:02 | |
| *** spatel has quit IRC | 03:13 | |
| *** CeeMac has quit IRC | 03:15 | |
| *** mkrai has quit IRC | 03:17 | |
| *** mkrai has joined #openstack-nova | 03:18 | |
| *** psachin has quit IRC | 03:30 | |
| *** dave-mccowan has quit IRC | 03:42 | |
| *** tetsuro_ has quit IRC | 03:54 | |
| *** zhanglong has quit IRC | 03:55 | |
| *** tetsuro has joined #openstack-nova | 04:00 | |
| openstackgerrit | Merged openstack/nova master: Catch exception when use invalid architecture of image https://review.opendev.org/711363 | 04:09 |
| *** udesale has joined #openstack-nova | 04:40 | |
| openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.83: Add volume-patch CLI https://review.opendev.org/712651 | 04:52 |
| *** mkrai has quit IRC | 05:02 | |
| *** bnemec has quit IRC | 05:19 | |
| *** bnemec has joined #openstack-nova | 05:28 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add new default roles in os-instance-actions policies https://review.opendev.org/706470 | 05:33 |
| *** evrardjp has quit IRC | 05:35 | |
| *** evrardjp has joined #openstack-nova | 05:36 | |
| *** links has joined #openstack-nova | 05:36 | |
| *** mkrai has joined #openstack-nova | 05:53 | |
| *** dpawlik has quit IRC | 06:37 | |
| *** dpawlik has joined #openstack-nova | 06:55 | |
| *** dpawlik has quit IRC | 07:10 | |
| *** dpawlik has joined #openstack-nova | 07:16 | |
| *** rpittau|afk is now known as rpittau | 07:24 | |
| *** factor has joined #openstack-nova | 07:50 | |
| *** slaweq has joined #openstack-nova | 07:56 | |
| *** damien_r has joined #openstack-nova | 08:00 | |
| *** ociuhandu has joined #openstack-nova | 08:02 | |
| *** ociuhandu has quit IRC | 08:07 | |
| *** maciejjozefczyk has joined #openstack-nova | 08:08 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add SYSTEM_READER role to servers actions API https://review.opendev.org/706179 | 08:11 |
| *** tesseract has joined #openstack-nova | 08:12 | |
| *** amoralej|off is now known as amoralej | 08:17 | |
| *** ociuhandu has joined #openstack-nova | 08:20 | |
| *** ociuhandu has quit IRC | 08:24 | |
| *** rcernin has quit IRC | 08:27 | |
| *** tkajinam has quit IRC | 08:31 | |
| *** ociuhandu has joined #openstack-nova | 08:40 | |
| *** tosky has joined #openstack-nova | 08:46 | |
| *** ociuhandu has quit IRC | 08:49 | |
| *** ociuhandu has joined #openstack-nova | 08:50 | |
| *** xek has joined #openstack-nova | 08:50 | |
| kashyap | melwitt: stephenfin: Just caught up with that "lift the SCSI unit restriction" libivirt issue: I'm pretty damn sure it's fixed upstream _and_ for CentOS / RHEL | 08:58 |
| *** ralonsoh has joined #openstack-nova | 08:59 | |
| kashyap | Okay, I see that stephenfin has pointed out the RHBZ | 08:59 |
| kashyap | melwitt: stephenfin: It looks like Ubunutu Virt folks need to backport that patch series | 09:00 |
| *** ociuhandu has quit IRC | 09:02 | |
| *** links has quit IRC | 09:04 | |
| *** ociuhandu has joined #openstack-nova | 09:14 | |
| *** ociuhandu has quit IRC | 09:15 | |
| *** ociuhandu has joined #openstack-nova | 09:16 | |
| *** ociuhandu has quit IRC | 09:25 | |
| *** ociuhandu has joined #openstack-nova | 09:26 | |
| *** derekh has joined #openstack-nova | 09:33 | |
| *** martinkennelly has joined #openstack-nova | 09:35 | |
| *** jangutter has joined #openstack-nova | 09:45 | |
| *** lpetrut has joined #openstack-nova | 09:53 | |
| *** ociuhandu has quit IRC | 09:53 | |
| *** jangutter has quit IRC | 09:57 | |
| *** jangutter has joined #openstack-nova | 09:58 | |
| *** tbachman has quit IRC | 10:04 | |
| *** lpetrut has quit IRC | 10:05 | |
| *** lpetrut has joined #openstack-nova | 10:09 | |
| *** ociuhandu has joined #openstack-nova | 10:10 | |
| *** lpetrut has quit IRC | 10:10 | |
| *** lpetrut has joined #openstack-nova | 10:12 | |
| *** ociuhandu has quit IRC | 10:12 | |
| *** ociuhandu has joined #openstack-nova | 10:12 | |
| *** tetsuro has quit IRC | 10:19 | |
| *** ociuhandu has quit IRC | 10:22 | |
| *** ociuhandu has joined #openstack-nova | 10:24 | |
| *** ociuhandu has quit IRC | 10:28 | |
| *** mkrai has quit IRC | 10:36 | |
| *** mkrai has joined #openstack-nova | 10:37 | |
| *** mkrai has quit IRC | 10:44 | |
| *** ociuhandu has joined #openstack-nova | 11:00 | |
| *** mvkr has quit IRC | 11:08 | |
| *** udesale_ has joined #openstack-nova | 11:09 | |
| *** mvkr has joined #openstack-nova | 11:10 | |
| *** udesale has quit IRC | 11:12 | |
| *** nicolasbock has joined #openstack-nova | 11:25 | |
| *** rpittau is now known as rpittau|bbl | 11:28 | |
| *** jangutter has quit IRC | 11:29 | |
| *** hoonetorg has quit IRC | 11:47 | |
| *** ociuhandu has quit IRC | 11:48 | |
| *** hoonetorg has joined #openstack-nova | 12:00 | |
| *** nicolasbock has quit IRC | 12:07 | |
| *** nicolasbock has joined #openstack-nova | 12:07 | |
| *** ociuhandu has joined #openstack-nova | 12:09 | |
| *** mkrai has joined #openstack-nova | 12:12 | |
| *** udesale_ is now known as udesale | 12:16 | |
| *** jangutter has joined #openstack-nova | 12:23 | |
| * gibi is on and off during the day due to downstream stuff | 12:24 | |
| *** grandchild has joined #openstack-nova | 12:36 | |
| *** grandchild has quit IRC | 12:44 | |
| mnaser | has anyne seen this in nova's ci? | 12:48 |
| mnaser | libvirt.libvirtError: Requested operation is not valid: format of backing image '/var/lib/nova/instances/_base/791111176e5cb97db82b0a71a670431f65838a05' of image '/var/lib/nova/instances/ff811290-bbb0-4d30-b1e7-ee9aec4913ee/disk' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting) | 12:48 |
| mnaser | OSA's jobs are failing with that and i wonder if its because the libvirt version with bionic is different... maybe | 12:48 |
| mnaser | i fonud this - https://bugzilla.redhat.com/show_bug.cgi?id=1798148 | 12:50 |
| openstack | bugzilla.redhat.com bug 1798148 in libvirt "Regression: Requested operation is not valid: format of backing image ... was not specified in the image metadata" [Unspecified,On_qa] - Assigned to pkrempa | 12:50 |
| mnaser | ok i see https://bugs.launchpad.net/nova/+bug/1864020 | 12:50 |
| openstack | Launchpad bug 1864020 in OpenStack Compute (nova) "libvirt.libvirtError: Requested operation is not valid: format of backing image %s of image %s was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting)" [Undecided,In progress] - Assigned to Lee Yarwood (lyarwood) | 12:50 |
| *** vishalmanchanda has quit IRC | 13:03 | |
| lyarwood | mnaser: yeah that should be resolved now | 13:03 |
| *** rpittau|bbl is now known as rpittau | 13:04 | |
| mnaser | lyarwood: cool, we are bumping our versions which should help with this in OSA. thanks for looking into it! | 13:05 |
| lyarwood | mnaser: np, which jobs hit this in OSA btw? You need a very recent version of libvirt to hit this. | 13:06 |
| * lyarwood only found this with a new fedora virt-preview job | 13:07 | |
| *** jangutter has quit IRC | 13:09 | |
| sean-k-mooney | gibi: efried_gone https://review.opendev.org/#/c/676522/44/nova/compute/resource_tracker.py@1738 i think that adresses the duplicate resouce provider definitons in different yaml files | 13:10 |
| sean-k-mooney | also is efried_gone gone permently now or will erric be back before he finishes his current role | 13:10 |
| gibi | sean-k-mooney: i will look at the provider series next week. thanks for taking it over | 13:17 |
| sean-k-mooney | gibi: no rush bar m3, i still need to adress the final patch and then look at the testing | 13:17 |
| sean-k-mooney | i think i covered the main funcitonal changes already however | 13:18 |
| gibi | sean-k-mooney: ack | 13:18 |
| gibi | m3 is in 4 weeks | 13:19 |
| sean-k-mooney | yep so no rush | 13:19 |
| gibi | sh*t i have to fix up the qos series too til | 13:19 |
| gibi | anyhow I expect a rush of reviews at m3 as usual | 13:20 |
| efried_gone | sean-k-mooney: I'm gone gone. Spending maybe a few minutes a day paying attention to OpenStack stuff. | 13:22 |
| sean-k-mooney | cool good to know. ill miss having you around | 13:22 |
| *** lbragstad__ has quit IRC | 13:29 | |
| *** lbragstad has joined #openstack-nova | 13:31 | |
| *** mriedem has joined #openstack-nova | 13:36 | |
| *** nweinber has joined #openstack-nova | 13:36 | |
| *** rcernin has joined #openstack-nova | 13:41 | |
| *** jangutter has joined #openstack-nova | 13:41 | |
| *** mlavalle has joined #openstack-nova | 13:44 | |
| kashyap | efried_gone: Was good knowing you; see you on the Other Side(tm). | 13:48 |
| kashyap | And thanks for all the outstanding (as in, excellent, not "remaining") work! ;-) | 13:48 |
| kashyap | mnaser: Hey, yes... | 13:49 |
| kashyap | mnaser: That issue known due to a libvirt regression, as lyarwood got a patch merged to solve it in Nova -- which is the right thing to do _anyway) | 13:49 |
| kashyap | mnaser: See this one: | 13:50 |
| stephenfin | lyarwood, smcginnis: Think you folks could hit this backport from elod today? It fixes issues with failing tests seen in other backports :) https://review.opendev.org/#/c/712751/ | 13:50 |
| kashyap | mnaser: https://review.opendev.org/#/c/708745/ ("libvirt: Provide the backing file format when creating qcow2 disks") | 13:50 |
| kashyap | mnaser: It's merged in master, Train backport in-progress. | 13:50 |
| stephenfin | (like this one https://review.opendev.org/#/c/711670/) | 13:50 |
| kashyap | Oh, lyarwood already answered it; silly me. I should read the scrollback in full before spamming the channel. | 13:51 |
| *** amoralej is now known as amoralej|lunch | 13:55 | |
| *** mkrai has quit IRC | 13:56 | |
| *** lucidguy has quit IRC | 13:57 | |
| *** ociuhandu has quit IRC | 13:59 | |
| *** factor has quit IRC | 14:00 | |
| *** factor has joined #openstack-nova | 14:00 | |
| lyarwood | kashyap: looking | 14:02 |
| lyarwood | kashyap: ah this one | 14:03 |
| brinzhang | stephenfin: https://review.opendev.org/#/c/712651/ could please check this patch of novaclient? I add a PATCH ``volume-patch`` CLI, debug it, and not found where is wrong | 14:03 |
| kashyap | lyarwood: Yeah, sorry for the noise. | 14:03 |
| *** factor has quit IRC | 14:03 | |
| *** factor has joined #openstack-nova | 14:04 | |
| lyarwood | stephenfin: ack I'll take a swing later today | 14:05 |
| *** ociuhandu has joined #openstack-nova | 14:07 | |
| *** ociuhandu has quit IRC | 14:07 | |
| *** ociuhandu has joined #openstack-nova | 14:07 | |
| *** TxGirlGeek has joined #openstack-nova | 14:09 | |
| *** bnemec is now known as beekneemech | 14:10 | |
| stephenfin | brinzhang: You need to add a 'patch_servers_1234_os_volume_attachments_Work' method to 'FakeSessionClient' in 'novaclient/tests/unit/v2/fakes.py' | 14:10 |
| stephenfin | with a response mocking what you'd see from nova-api | 14:11 |
| brinzhang | stephenfin: cools, thanks, yep, I will try, I missing that response in 'FakeSessionClient' | 14:13 |
| *** factor has quit IRC | 14:14 | |
| *** factor has joined #openstack-nova | 14:14 | |
| *** factor has quit IRC | 14:15 | |
| sean-k-mooney | stephenfin: i cant cross link to source code form nova docs right | 14:17 |
| brinzhang | stephenfin: thanks, I will complete its unit tests, and then update ^^ | 14:17 |
| stephenfin | sean-k-mooney: Only if the code is autodoc'd somewhere, and I think only the notifier stuff falls in that bracket | 14:17 |
| stephenfin | I _think_ you should still be able to use e.g. :py:method:`nova.foo.bar` but it won't resolve to anything | 14:18 |
| sean-k-mooney | stephenfin: ok in that case i think ill have to link to opendev/github | 14:18 |
| sean-k-mooney | stephenfin: context is https://review.opendev.org/#/c/693460/18/doc/source/admin/managing-resource-providers.rst,unified@66 | 14:18 |
| sean-k-mooney | just trying to figure out how i would go ablout liking to the schem file for the provider.yaml validation | 14:19 |
| sean-k-mooney | i think an external link is the only way to do it so ill figure out what the opendev path would be | 14:19 |
| stephenfin | you could just '.. include' it | 14:19 |
| sean-k-mooney | oh i didnt know you could do that am ill try | 14:20 |
| *** ociuhandu has quit IRC | 14:23 | |
| brinzhang | johnthetubaguy: I resolved the os-instance-actions policy issue, and granular the GET API, pls review again. https://review.opendev.org/#/c/706470/ | 14:24 |
| *** ociuhandu has joined #openstack-nova | 14:29 | |
| *** TxGirlGeek has quit IRC | 14:31 | |
| *** grandchild has joined #openstack-nova | 14:33 | |
| *** Liang__ has quit IRC | 14:43 | |
| *** ociuhandu has quit IRC | 14:49 | |
| dansmith | brinzhang: assume you saw this right? https://review.opendev.org/#/c/712697/ | 14:55 |
| *** amoralej|lunch is now known as amoralej | 14:55 | |
| brinzhang | dansmith: yeah, I have seen it, but I am not very understand why | 14:55 |
| brinzhang | dansmith: so I am not update my patch :( | 14:56 |
| dansmith | brinzhang: understand why what? | 14:56 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Use virDomainBlockCopy to swap volumes with Libvirt >= 5.10.0 https://review.opendev.org/696834 | 14:58 |
| dansmith | brinzhang: your patch needs to be rebased on top of that, but otherwise there isn't much you need to do I think | 14:58 |
| brinzhang | dansmith: no, I am still confusing of that change ... | 14:58 |
| brinzhang | I don't seem to grasp the main reason for doing this. | 14:59 |
| *** mlavalle has quit IRC | 14:59 | |
| brinzhang | I think that's why I am confusing | 14:59 |
| *** mlavalle has joined #openstack-nova | 15:00 | |
| dansmith | brinzhang: the main reason for doing the "format_message()" instead of just "str(e)" ? | 15:00 |
| sean-k-mooney | brinzhang: to not leak sensitive infomation to end users when there are errors | 15:00 |
| brinzhang | <sean-k-mooney> yeah, I know, but can you give me a example? | 15:01 |
| dansmith | yes, | 15:01 |
| brinzhang | I think I need a sample to understand this well | 15:01 |
| dansmith | we had a ceph exception which was something like "Failed to connect to 12.34.56.78" | 15:01 |
| kashyap | lyarwood: I just noticed I didn't hit send on my new comment; done. (See the regression mentioned there.) | 15:01 |
| brinzhang | s/example/sample | 15:01 |
| dansmith | which gets exposed out of the API and users get to see sensitive details like the internal ceph ip address | 15:01 |
| dansmith | brinzhang: I think there was even a case where a credential got leaked.. mriedem might remember the bug(s) to point to | 15:02 |
| brinzhang | dansmith: now we can get the exception only, and cannot get "Failed to connect to 12.34.56.78", right? | 15:03 |
| dansmith | brinzhang: yeah so if the exception is not known (i.e. not inherited from NovaException) we just get the exception *name*, so FailedToConnectToCeph or ConnectionFailure or something like that | 15:04 |
| brinzhang | In other words, we cannot get the details message for the sensitive | 15:04 |
| dansmith | brinzhang: we can *log* it in the compute log for the admin, but we don't want to expose it to the user | 15:04 |
| dansmith | right | 15:04 |
| brinzhang | dansmith: ok, let me taste your changes again | 15:05 |
| dansmith | lol | 15:05 |
| dansmith | brinzhang: um, global pandemic going on right now.. probably best not to lick anything | 15:06 |
| lyarwood | kashyap: ack, we aren't going to hit that FWIW | 15:06 |
| lyarwood | kashyap: we don't use the shallow copy flag | 15:06 |
| brinzhang | dansmith: haha | 15:06 |
| dansmith | brinzhang: :) | 15:06 |
| lyarwood | kashyap: but thanks for raising that | 15:06 |
| brinzhang | dansmith: Thanks to a guy named Guo WeiPeng, I have been in quarantine for another 15 days at home | 15:07 |
| kashyap | lyarwood: Yeah, was reading the details; just wanted to think through if there are any other places we need to bear in mind. | 15:07 |
| dansmith | brinzhang: well, the definitely don't lick *my* patch :) | 15:07 |
| brinzhang | dansmith: so I have too much time to talk with you | 15:07 |
| dansmith | brinzhang: heh, well, I work from home all the time, so I'm pretty much on quarantine normally :) | 15:08 |
| brinzhang | dansmith: you are happiness, it's my dream :) | 15:08 |
| kashyap | Wauw, Dan _is_ Happiness; that's something. | 15:09 |
| dansmith | heh | 15:09 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Store instance action event exc_val fault details https://review.opendev.org/694428 | 15:13 |
| dansmith | brinzhang: we might want to make sure we have some functional tests that raise both nova and non-nova exceptions in such a way that we can examine them from the API | 15:14 |
| brinzhang | dansmith: I saw you add non-nova test exception in https://review.opendev.org/#/c/712697/1/nova/tests/unit/objects/test_objects.py@1003 | 15:17 |
| brinzhang | You mean, I should add some nova and non-nova exception for os-instance-actions API? | 15:17 |
| mriedem | dansmith: brinzhang: i can look up the cve, but we were exposing credentials to the rbd backing a compute host via instance faults | 15:17 |
| dansmith | brinzhang: yeah, but in a functional test I think | 15:17 |
| mriedem | that's why i referred to the nova.compute.utils code that handles faults | 15:17 |
| dansmith | mriedem: ack yeah | 15:18 |
| mriedem | you could build on the functional test i wrote for ^ | 15:18 |
| mriedem | dansmith: brinzhang: https://review.opendev.org/#/c/674821/ | 15:19 |
| *** ociuhandu has joined #openstack-nova | 15:19 | |
| *** eharney has quit IRC | 15:19 | |
| melwitt | kashyap: fyi I added you to this review about adding aarch64 cpu model | 15:20 |
| melwitt | https://review.opendev.org/709494 | 15:20 |
| kashyap | melwitt: Hiya | 15:20 |
| kashyap | melwitt: Will look; thanks for the heads-up | 15:20 |
| *** ociuhandu has quit IRC | 15:20 | |
| *** ociuhandu has joined #openstack-nova | 15:20 | |
| melwitt | cool thanks | 15:21 |
| brinzhang | mriedem: dansmith: Looks like I need to add a functional test file for instance_actions, such as nova/tests/functional/test_instance_action.py | 15:22 |
| mriedem | *shrug* there are lots of existing functional tests that make assertions using instance actions, | 15:22 |
| mriedem | i'm not sure there is a module dedicated to instance actions outside of the api samples | 15:22 |
| dansmith | yeah, just another case in one of those, or just add to test_server_faults, IMHO.. it's mostly the same thing | 15:23 |
| *** gyee has joined #openstack-nova | 15:23 | |
| mriedem | umm https://github.com/openstack/nova/blob/master/nova/tests/functional/test_instance_actions.py | 15:23 |
| mriedem | or just use the existing module :) | 15:23 |
| dansmith | heh yeah | 15:24 |
| brinzhang | mriedem: yeah, I saw another api sample tests in nova\tests\functional\api_sample_tests\test_instance_actions.py | 15:24 |
| dansmith | *gasp* | 15:24 |
| dansmith | backslashes! | 15:24 |
| mriedem | brinzhang: api samples are generally not really for this type of testing | 15:25 |
| mriedem | they are more about happy path positive test scenarios with minimal fixture | 15:25 |
| brinzhang | mriedem: I know, I will use your paste linke | 15:25 |
| brinzhang | s/linke/link | 15:25 |
| brinzhang | dansmith: I copied from windows, so it's backslashes! | 15:26 |
| dansmith | brinzhang: hence the gasp! :) | 15:26 |
| brinzhang | dansmith: The github sometime I cannot open it fastly, sometime 400 for me, I donot know why .. | 15:27 |
| *** ociuhandu_ has joined #openstack-nova | 15:28 | |
| brinzhang | dansmith: You are a humorous technology *bull*. I think I missed a lot of interesting things :) | 15:28 |
| dansmith | brinzhang: I'm just joking around, don't take me seriously :) | 15:29 |
| mriedem | brinzhang: the great firewall :) | 15:29 |
| mriedem | i'm assuming you're tunneled in through a windows VM for development and being able to be on IRC | 15:29 |
| mriedem | like all of my old huawei coworkers in china | 15:30 |
| brinzhang | dansmith: I like this style, It makes me free. | 15:30 |
| * mriedem goes back to what he's actually supposed to be working on | 15:31 | |
| brinzhang | mriedem: yes, the great firewall, we bought shadowscokets, but it cannot work now .. | 15:31 |
| kashyap | LOL, interesting choice of words to describe dansmith: "humorous technology *bull*" :D | 15:31 |
| kashyap | He _is_, though. But still, the vividness of the metaphor. | 15:31 |
| *** ociuhandu has quit IRC | 15:32 | |
| *** ccamacho has quit IRC | 15:32 | |
| *** ociuhandu_ has quit IRC | 15:33 | |
| brinzhang | kashyap: Forgive my native language is chinese | 15:33 |
| brinzhang | kashyap: :) | 15:33 |
| kashyap | brinzhang: No-no, I was just saying in jest (joking). Your English is fine | 15:33 |
| *** _mlavalle_1 has joined #openstack-nova | 15:33 | |
| *** grandchild has quit IRC | 15:34 | |
| brinzhang | kashyap: thanks, a little.. | 15:34 |
| *** mlavalle has quit IRC | 15:36 | |
| brinzhang | mriedem: I used windows OS and installed IRC(HexChat) in windows, but my VM boot from our company's server(NODE), it's an OpenStack cloud deployed by kolla, just connect by xshell :) | 15:38 |
| kashyap | melwitt: I didn't notice it earlier, but confusingly the method is returning 'mode', while the actual content is 'model': https://review.opendev.org/#/c/709494/3/nova/virt/libvirt/utils.py | 15:40 |
| kashyap | melwitt: I'll add some words in the change; didn't notice it before. | 15:41 |
| melwitt | kashyap: yeah, me neither. though that's not the fault of that patch, right? | 15:41 |
| melwitt | or are you saying the patch is returning the wrong thing | 15:42 |
| kashyap | melwitt: You're right - not the fault of the patch; it's existing | 15:42 |
| kashyap | melwitt: We can change that later, but the core idea (on the basis to set up a CI) is good | 15:42 |
| kashyap | melwitt: I just asked the upstream QEMU AArch64 folks about the model chosen in the patch ('cortex-a57'). I'll report back on the patch. | 15:43 |
| sean-k-mooney | brinzhang: i used a windows laptop basically as a thin client with cygwin and ssh to linux server most of my time working upstream | 15:43 |
| sean-k-mooney | brinzhang: i only started runing linux on my main worstation/laptop when i left intel as we were required to use windows on the it provisioned laptop | 15:44 |
| melwitt | kashyap: ok. yeah, I wasn't sure about the order and placement of the aarch64 checks. and maybe could use some code comments | 15:44 |
| sean-k-mooney | also i like kolla. its easy to debug and uses | 15:44 |
| kashyap | melwitt: Yes, defintely +1 on the code comments. | 15:45 |
| brinzhang | sean-k-mooney: we are same, I think you like to use linux | 15:47 |
| sean-k-mooney | i do but i dont dislike windoes either. | 15:48 |
| *** lpetrut has quit IRC | 15:48 | |
| brinzhang | sean-k-mooney: yeah, what suits you is the best :) | 15:51 |
| brinzhang | it's time too later for me, I will go, thanks damsmith, mriedem, kashyap, sean-k-mooney (good morning) ^^ | 15:52 |
| *** _mlavalle_1 has quit IRC | 16:03 | |
| *** mlavalle has joined #openstack-nova | 16:03 | |
| kashyap | melwitt: Ah, only noticed your comments _after_ I've hit send on mine. Along with your questions, I have added a few more. | 16:04 |
| melwitt | kashyap: cool, better to have more comments to show if there's any agreement or if I'm only asking dumb questions :P | 16:06 |
| kashyap | melwitt: No, just reading your questions; you make perfectly valid points there. | 16:06 |
| kashyap | You are not a mind reader to know the intention; so asking for code comments is the only reasonable thing :D | 16:06 |
| melwitt | this stuff is greek to me. I'd rather have some explanations there for those that venture to the code in the future, looking to refactor or whatever | 16:08 |
| kashyap | melwitt: Okay, got some more input from QEMU maintainer (Peter Maydell) - he has special interest in AArc64 - I'll add it in the change. He recommends a bunch of things. | 16:08 |
| melwitt | awesome! | 16:08 |
| kashyap | melwitt: I know the mechanics of how QEMU handles things; but the innards of AArch64 and the usage is Greek for me too. (I find Greek aesthetically pleasing, though. I have a couple of books with Greek on left, the English translation on right. :D) | 16:09 |
| melwitt | :) | 16:09 |
| *** damien_r has quit IRC | 16:12 | |
| *** lpetrut has joined #openstack-nova | 16:14 | |
| melwitt | kashyap: is this libvirt kvm aarch64 (which is mentioned on feature support matrix)? or is it libvirt qemu aarch64 (not on feature support matrix yet)? https://docs.openstack.org/nova/train/user/support-matrix.html | 16:19 |
| kashyap | melwitt: Yeah, there was two things: | 16:19 |
| kashyap | s/was/are/ | 16:19 |
| kashyap | (1) TCG (the emulatd bits), or what is also referred to as "QEMU"-only; and (2) KVM (with hardware accelerated) in context of AArch64. I guess we want 'care' about both | 16:21 |
| kashyap | Luckily, Peter from QEMU informs that for _both TCG/KVM, we can just use one model: 'max' | 16:22 |
| kashyap | (Comments on why upcoming...) | 16:22 |
| melwitt | kashyap: ok, so the proposed patches aren't for one in particular only | 16:22 |
| kashyap | melwitt: Although for CI, perhaps the dev just cares about QEMU (TCG). Just like how Nova x86 CI runs on TCG guests (because no nested, to get KVM). | 16:23 |
| kashyap | Not sure if I'm helping or confusing :D | 16:23 |
| melwitt | uh ... helping a little :) but it's not you, it's me | 16:23 |
| melwitt | I'm just trying to write something on the lp bug for the patch above the one we've been looking at | 16:23 |
| sean-k-mooney | kashyap: what is the max model usef for? | 16:24 |
| melwitt | the bug is unable to attach volume to instance with config drive on arm64 and according to the feature support matrix the support is "unknown" so I was thinking to set the bug Low based on that https://docs.openstack.org/nova/latest/user/support-matrix.html#operation_attach_volume_driver_libvirt_kvm_aarch64 | 16:24 |
| kashyap | melwitt: I see. I'll collect thoughts and write it in the change, and we can take it from there. | 16:25 |
| kashyap | sean-k-mooney: The 'max' model will apparently give you the moving-target of "all the stuff we [QEMU] can currently emulate". | 16:25 |
| kashyap | melwitt: Yeah, 'low' for now is fine. | 16:26 |
| sean-k-mooney | kashyap: so totally non portable | 16:26 |
| sean-k-mooney | kashyap: where is max used | 16:27 |
| sean-k-mooney | e.g.is it a cpu model or a scisi contoler or what | 16:27 |
| kashyap | sean-k-mooney: Well, the recommendation from the AArch64 experts is to use 'max' for _both_ TCG and KVM: | 16:27 |
| kashyap | ... "unless you really specifically want an always-the-same-thing even in newer QEMU versions fixed target". | 16:27 |
| sean-k-mooney | im trying to figure out the context | 16:27 |
| kashyap | Please read the scrollback with Mel. | 16:27 |
| kashyap | It's in context of https://review.opendev.org/#/c/709494/ | 16:28 |
| sean-k-mooney | ya i was trying to and could not fiutre it out | 16:28 |
| sean-k-mooney | so this has changed form the old advise of use the VIRT cpu model | 16:28 |
| melwitt | kashyap: yeah so I think what should likely happen is that once they have the CI set up and running we will update the feature support matrix with all of the things that are working in the CI? | 16:28 |
| kashyap | sean-k-mooney: No, no you're mixing up CPU model and machine type for AArch64 | 16:31 |
| kashyap | sean-k-mooney: 'virt' is still the recommended machine type for AArch64. | 16:31 |
| kashyap | melwitt: Yeap. | 16:31 |
| sean-k-mooney | ah yes i am | 16:31 |
| melwitt | cool | 16:31 |
| *** openstackgerrit has quit IRC | 16:31 | |
| sean-k-mooney | this still will cause issue for live migration if we use max | 16:31 |
| sean-k-mooney | so i dont know if that is a good default | 16:32 |
| kashyap | melwitt: I _think_ first want to care about TCG, because for KVM, you'd need AArch64 hardware in the CI | 16:32 |
| kashyap | sean-k-mooney: I've talked to the AArch64 maintainer, I'm posting the recommendations in the patch. | 16:32 |
| sean-k-mooney | kashyap: we have AArch64 hardware in ci | 16:32 |
| melwitt | kashyap: ack | 16:32 |
| sean-k-mooney | kashyap: thats the whole point lenario donated some | 16:32 |
| kashyap | sean-k-mooney: Good, then. We've got the recommendations for that, too. | 16:32 |
| melwitt | kashyap: so do you think TCG would be a separate column in the matrix? | 16:32 |
| kashyap | Linaro, I take it. Yeah | 16:32 |
| kashyap | melwitt: Yeah, I'd say so. | 16:33 |
| sean-k-mooney | ah yes | 16:33 |
| melwitt | ok. thanks for explaining all this | 16:33 |
| kashyap | (We need to clearly distinguish both cases.) | 16:33 |
| kashyap | No worries, I need to refresh this every few months mysel f:D | 16:33 |
| melwitt | :) | 16:33 |
| sean-k-mooney | kashyap: we proably should be using the cpu_mode config option to define this | 16:33 |
| *** nweinber has quit IRC | 16:34 | |
| sean-k-mooney | e.g. tie it in to host-passthough and host-model some how | 16:34 |
| stephenfin | sean-k-mooney: how strongly do you feel about https://review.opendev.org/#/c/468203 ? | 16:35 |
| stephenfin | specifically jaypipes arguments there | 16:35 |
| kashyap | sean-k-mooney: One thing at a time :-) | 16:35 |
| sean-k-mooney | im also not sure how i feel about htis being a bug, it fells more like a specless blueprint but im not going to really object too stronly to it beign a bug | 16:36 |
| stephenfin | I ask because I'm trying to decide how to say "these cores should be dedicated" in a mixed instance | 16:37 |
| sean-k-mooney | kashyap: well for now they could jsut use cpu_mode=custom and cpu_model=max or cpu_model=min right | 16:37 |
| stephenfin | Currently I'm going with 'hw:cpu_dedicated_mask', which is a CPU list | 16:37 |
| sean-k-mooney | stephenfin: am ill take a look now | 16:37 |
| sean-k-mooney | stephenfin: ya i would be fine with that | 16:37 |
| sean-k-mooney | stephenfin: whats the other option | 16:37 |
| stephenfin | there are a few | 16:38 |
| stephenfin | in this scenario that I'm following, you'll be able to use hw:cpu_dedicated_mask *or* hw:cpu_realtime_mask | 16:38 |
| sean-k-mooney | stephenfin: no you would use both | 16:38 |
| stephenfin | I see no reason to say these cores are shared, these are dedicated but non-realtime, and these are dedicated and realtime | 16:39 |
| sean-k-mooney | well optinally | 16:39 |
| sean-k-mooney | e.g. not an exclucive or | 16:39 |
| sean-k-mooney | i dont see a reason to block it | 16:39 |
| stephenfin | why? What real-world user is going to use all three types of core in an instance? | 16:39 |
| stephenfin | Because it's less complicated | 16:39 |
| sean-k-mooney | its more complicated | 16:39 |
| sean-k-mooney | the validation logic to prevent all 3 is extra logic we dont need if we allow it | 16:40 |
| stephenfin | A|B is easier grok than A.issubset(B) | 16:40 |
| stephenfin | and the it makes my XML generation easier | 16:40 |
| stephenfin | s/the // | 16:41 |
| sean-k-mooney | stephenfin: you are over loading hw:cpu_realtime_mask | 16:41 |
| sean-k-mooney | its behavior would change based on the hw:cpu_policy | 16:41 |
| sean-k-mooney | so that gets harder to reason about | 16:41 |
| stephenfin | nope, it stays the same: these are cores that are real-time | 16:41 |
| sean-k-mooney | if we allow both it does not | 16:41 |
| stephenfin | what changes is what happens to the other cores | 16:41 |
| stephenfin | and that's purely based on hw:cpu_policy | 16:41 |
| sean-k-mooney | no it chacnge form tehse are realtime to these are realtime and dedicated and the rest flaot | 16:42 |
| kashyap | sean-k-mooney: Not entirely; there's also a quirk of making sure to specify the interrupt controller (as the default is less featureful) -- `-machine gic-version=max` | 16:42 |
| kashyap | melwitt: For later, added my notes in the change. | 16:42 |
| stephenfin | they're always realtime and dedicated | 16:42 |
| stephenfin | you can't have realtime floating cores | 16:42 |
| melwitt | kashyap: thanks | 16:42 |
| sean-k-mooney | well libvirt allows you to but that is a seperate thign | 16:43 |
| stephenfin | ...in nova | 16:43 |
| kashyap | Yep | 16:43 |
| sean-k-mooney | stephenfin: my perference would be to allow both to be set | 16:43 |
| stephenfin | I could allow it, but I don't want to force it | 16:43 |
| sean-k-mooney | and always require hw:dedicated_cpu_mask for mixed | 16:43 |
| stephenfin | and because I don't want to force it, I'd rather say there's only one way to do this | 16:43 |
| sean-k-mooney | which one do you not want to force | 16:44 |
| *** nweinber has joined #openstack-nova | 16:44 | |
| sean-k-mooney | the realtime mask or the dedicated mask | 16:44 |
| stephenfin | having to set both hw:cpu_dedicated_mask and hw:cpu_realtime_mask | 16:44 |
| sean-k-mooney | right so i would make the cpu_realtime_mask optional | 16:44 |
| stephenfin | if you want a real-time instance with the non-realtime cores floating | 16:44 |
| sean-k-mooney | and always require the cpu_dedicated_mask | 16:44 |
| stephenfin | but then you have a difference of behavior elsewhere | 16:45 |
| sean-k-mooney | and make it so that if you dont set cpu_realtime_mask but do set hw:cpu_realtime=true then we use the hw:dedciated_cpu_mask | 16:45 |
| stephenfin | now you have to use 'hw:cpu_realtime_mask' when using dedicated | 16:45 |
| stephenfin | but use 'hw:cpu_dedicated_mask' when using mixed | 16:45 |
| stephenfin | anyway, we'll invariably debate this in the patch so back to my original question | 16:46 |
| sean-k-mooney | i think its a much simpler rule to say if you want realtime always use realtime_mask and if you want mixed always use the dedicated_mask | 16:46 |
| sean-k-mooney | if you want both use both and require the realtime mask must be a subset of the dedicated mask | 16:47 |
| stephenfin | if we're doing this, do we want to fix that annoying thing where hw:cpu_realtime_mask has to be preceded by carat? | 16:47 |
| stephenfin | because I don't want to force that 'hw:cpu_dedicated_mask' | 16:47 |
| stephenfin | and it would be nice for them to behave similarly | 16:47 |
| sean-k-mooney | stephenfin: ya i would not mind doing that | 16:48 |
| sean-k-mooney | stephenfin: we said the dedicated mask should follow the rules for the config opntion | 16:48 |
| sean-k-mooney | not the rules for the realtime mask | 16:48 |
| stephenfin | the only reason I see to not do that is jaypipes wanted us to kill 'hw:cpu_realtime_mask' in favor of 'hw:cpu_realtime_set' | 16:48 |
| stephenfin | but tbh, I don't think it's worth the effort | 16:48 |
| stephenfin | we haven't deprecated flavor extra specs before. I don't even want to get into that | 16:48 |
| sean-k-mooney | i would be ok with that i guess but yat that ^ | 16:49 |
| stephenfin | ditto for image metadata props, for that matter | 16:49 |
| stephenfin | okay, sweet | 16:49 |
| stephenfin | I can revive those patches so | 16:49 |
| stephenfin | whoo, rebase fun! | 16:49 |
| sean-k-mooney | stephenfin: cool. i prefer each option to do one thing and one thing only. but do what you think is best | 16:50 |
| sean-k-mooney | its a preference not a blocker for me | 16:50 |
| sean-k-mooney | and sice we cant do cross extra spec validation in your validation propsoeal i also prefer that form the avlidation point of view | 16:51 |
| sean-k-mooney | let me know when you want me to review and or play around with it | 16:51 |
| sean-k-mooney | im going to drop soon just an fyi | 16:51 |
| stephenfin | will do | 16:51 |
| stephenfin | ah yeah, it'll be next week anyway | 16:51 |
| *** rpittau is now known as rpittau|afk | 16:58 | |
| *** maciejjozefczyk has quit IRC | 17:03 | |
| *** tesseract has quit IRC | 17:07 | |
| *** openstackgerrit has joined #openstack-nova | 17:16 | |
| openstackgerrit | Elod Illes proposed openstack/nova stable/rocky: Enhance service restart in functional env https://review.opendev.org/713033 | 17:16 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713035 | 17:19 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: WIP nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713036 | 17:21 |
| *** maciejjozefczyk has joined #openstack-nova | 17:21 | |
| *** lbragstad has quit IRC | 17:23 | |
| *** lbragstad has joined #openstack-nova | 17:26 | |
| openstackgerrit | Merged openstack/nova stable/rocky: Remove global state from the FakeDriver https://review.opendev.org/712751 | 17:29 |
| melwitt | lyarwood: yay, thanks for looking at that. is the same thing happening with the intermittent failures on master? | 17:29 |
| melwitt | *intermittent nova-live-migration failures | 17:29 |
| lyarwood | melwitt: I don't recall seeing this on master but let me have a quick look in logstash | 17:30 |
| *** dpawlik has quit IRC | 17:30 | |
| melwitt | lyarwood: ok, was just wondering if you knew off the top of your head. I've seen occasional failures of nova-live-migration on master and just curious if it's the same thing. I hope so, cause that would mean your fix would fix that too | 17:31 |
| *** iurygregory has quit IRC | 17:32 | |
| *** evrardjp has quit IRC | 17:35 | |
| *** evrardjp has joined #openstack-nova | 17:36 | |
| lyarwood | melwitt: yeah, I can't see anything on master but my elasticsearch foo is awful | 17:37 |
| melwitt | ok np | 17:37 |
| lyarwood | melwitt: FWIW your https://review.opendev.org/#/c/712226/ change failed because of http://status.openstack.org/elastic-recheck/#1813789 | 17:39 |
| lyarwood | melwitt: I've not had time to look into that but I have been seeing that across master | 17:39 |
| melwitt | lyarwood: ah thanks. I've been looking at http://status.openstack.org/elastic-recheck/#1844929 again yesterday and today | 17:40 |
| melwitt | so far, no dice | 17:41 |
| lyarwood | melwitt: kk, these all appear rather tricky | 17:42 |
| melwitt | yeah :( | 17:43 |
| *** hamzy has joined #openstack-nova | 17:45 | |
| *** udesale_ has joined #openstack-nova | 17:52 | |
| *** eharney has joined #openstack-nova | 17:52 | |
| *** udesale has quit IRC | 17:54 | |
| *** jangutter_ has joined #openstack-nova | 17:54 | |
| *** jangutter has quit IRC | 17:58 | |
| *** derekh has quit IRC | 18:01 | |
| *** lseki has joined #openstack-nova | 18:11 | |
| *** maciejjozefczyk_ has joined #openstack-nova | 18:16 | |
| *** maciejjozefczyk has quit IRC | 18:16 | |
| melwitt | zzzeek: could you pls sanity check me on this -- this logging is done after getting a response back from the database server right? it's not a client side logging before making the query https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/engine/result.py#L1579 | 18:28 |
| zzzeek | melwitt: row logging is after we've executed the statement and we've received rows back from the DBAPI cursor, that line logs the row itself | 18:28 |
| zzzeek | there's no "row" that we would have before invoking a statement | 18:29 |
| melwitt | thanks zzzeek++ | 18:29 |
| melwitt | haha yeah. makes sense | 18:29 |
| *** TxGirlGeek has joined #openstack-nova | 18:31 | |
| melwitt | zzzeek: I'm investigating a gate bug (and we enabled connection_debug=100) and noticing that in the failure cases, we get no rows logged for our query for compute nodes. the gate environment on the nodes where it fails is known to be running on nodes with restricted disk iops. do you have any idea what could be happening or how we could tune to handle this environment better? | 18:32 |
| melwitt | like, in what scenarios could we make a query to the db server and then get no response within 60 seconds? | 18:33 |
| zzzeek | melwitt: well eventlet can do that, if you're using eventlet in some case | 18:33 |
| zzzeek | melwitt: another is, the query has a huge cartesian product and is taking too long to ORDER BY | 18:33 |
| melwitt | and if there's any rule of thumb way to deal with that, assuming that the disks cannot be made faster or allow more iops | 18:33 |
| melwitt | we are using eventlet to manage the 60 second client side timeout waiting for the result | 18:34 |
| zzzeek | the eventlet issue is that if some other greenlets are hogging the CPU then the greenlet in question might not be able to get a result back, but this has never been observed at the scale of 60 seconds | 18:35 |
| zzzeek | if the "restricted disk iops" is making MySQL chug to a halt, that would be a thing | 18:35 |
| zzzeek | is that both read and write ops? | 18:36 |
| *** ralonsoh has quit IRC | 18:36 | |
| *** lbragstad_ has joined #openstack-nova | 18:37 | |
| melwitt | hm, not sure. I'm referencing this ML thread http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010505.html | 18:37 |
| *** udesale_ has quit IRC | 18:39 | |
| zzzeek | so...the issue is, MySQL is actually being observed to be overworked in this case and we want to tune it for reduced disk IO ? | 18:39 |
| *** lbragstad has quit IRC | 18:39 | |
| melwitt | I was wondering that as a possibility, yes | 18:40 |
| melwitt | I'm playing around with some innodb_ settings in devstack to experiment, but mostly don't know what to try | 18:40 |
| melwitt | I wondered if tuning for that might enable it to work under the reduced disk IO condition | 18:41 |
| melwitt | zzzeek: I have to run for a bit, but if you think of any good tunables I could try in my.conf, pls give me a shout | 18:53 |
| zzzeek | sure...there are some write flags but read is tricky | 18:53 |
| melwitt | *my.cnf | 18:54 |
| *** tbachman has joined #openstack-nova | 19:04 | |
| *** lpetrut has quit IRC | 19:05 | |
| *** amoralej is now known as amoralej|off | 19:06 | |
| *** ociuhandu has joined #openstack-nova | 19:32 | |
| *** brinzhang_ has joined #openstack-nova | 19:38 | |
| *** jangutter_ has quit IRC | 19:39 | |
| *** brinzhang has quit IRC | 19:42 | |
| *** mgariepy has quit IRC | 19:53 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: WIP nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713036 | 20:13 |
| *** ociuhandu has quit IRC | 20:23 | |
| sean-k-mooney | melwitt: how big is the db | 20:27 |
| sean-k-mooney | it would be a bit of a hack but it might be posible to mount its data on tempfs or otherwise use some form of filesystem or block level caching to mask the io limitations | 20:28 |
| *** tbachman has quit IRC | 20:33 | |
| melwitt | sean-k-mooney: how can I tell how big is the db? I have been using this page to get an idea of things to try https://dev.mysql.com/doc/refman/5.6/en/optimizing-innodb-diskio.html | 20:34 |
| sean-k-mooney | i was just thining of useing du | 20:34 |
| melwitt | oh, lemme see, there's df output on the job runs | 20:36 |
| *** tbachman has joined #openstack-nova | 20:36 | |
| *** TxGirlGeek has quit IRC | 20:36 | |
| melwitt | https://zuul.opendev.org/t/openstack/build/8c91fd21815148d9894ac2bf60893a9e/log/logs/df.txt | 20:37 |
| sean-k-mooney | ya that unfortunetly is not going to show us | 20:37 |
| sean-k-mooney | ill check my local devstack | 20:37 |
| sean-k-mooney | so im seeing about 180mb | 20:38 |
| sean-k-mooney | in /var/lib/mysql | 20:38 |
| sean-k-mooney | so we could just try mounting that on tempfs | 20:38 |
| sean-k-mooney | i think we would have enough ram to do that | 20:39 |
| melwitt | oh, thanks | 20:39 |
| sean-k-mooney | runing a db on tempfs is normaly really dumb but its a ci so ... | 20:40 |
| sean-k-mooney | its not like we care if the data goes away after a reboot | 20:40 |
| melwitt | heh | 20:40 |
| *** nweinber has quit IRC | 20:43 | |
| *** ociuhandu has joined #openstack-nova | 20:55 | |
| *** maciejjozefczyk_ has quit IRC | 20:59 | |
| *** slaweq has quit IRC | 21:04 | |
| *** ociuhandu has quit IRC | 21:06 | |
| *** ociuhandu has joined #openstack-nova | 21:07 | |
| *** ociuhandu has quit IRC | 21:16 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713035 | 21:40 |
| lyarwood | melwitt / sean-k-mooney ; ^ if you're around, thoughts on that would be appreciated. Appears to fix the issues we've been seeing on stable/pike. | 21:41 |
| melwitt | cool, will look. thanks for digging into that | 21:42 |
| lyarwood | thanks :) | 21:47 |
| * lyarwood heads offline for the weekend \o | 21:47 | |
| *** tbachman has quit IRC | 22:01 | |
| *** ociuhandu has joined #openstack-nova | 22:07 | |
| *** ociuhandu has quit IRC | 22:20 | |
| sean-k-mooney | lyarwood: oh is that why the migration job was failing. it was not waiting for ceph after it reconfigred and restarted n-cpu | 22:23 |
| *** mlavalle has quit IRC | 22:31 | |
| *** mriedem has left #openstack-nova | 22:35 | |
| *** rcernin has quit IRC | 22:58 | |
| *** rcernin has joined #openstack-nova | 22:59 | |
| *** sean-k-mooney has quit IRC | 23:02 | |
| *** gyee has quit IRC | 23:22 | |
| *** bbowen_ has quit IRC | 23:29 | |
| *** martinkennelly has quit IRC | 23:31 | |
| *** xek has quit IRC | 23:38 | |
| *** bbowen has joined #openstack-nova | 23:39 | |
| *** sean-k-mooney has joined #openstack-nova | 23:47 | |
| *** lbragstad_ has quit IRC | 23:49 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!