Wednesday, 2024-11-13

*** fungi is now known as Guest924901:33
*** kinrui is now known as fungi01:41
opendevreviewArtem Goncharov proposed openstack/nova master: Fix add/remove SecurityGroup action json schemas  https://review.opendev.org/c/openstack/nova/+/93482308:24
opendevreviewMax proposed openstack/nova master: fix: leftover BDM after instance delete  https://review.opendev.org/c/openstack/nova/+/93498415:28
dansmithsean-k-mooney: can you look at the osvif failure here? https://review.opendev.org/c/openstack/requirements/+/93417615:44
dansmithcc frickler ^15:44
tkajinamdansmith, we need https://review.opendev.org/c/openstack/os-vif/+/932436 and a new release of os-vif.15:53
tkajinambut that change is blocked due to broken openstack-tox-functional-ovs-with-sudo15:53
dansmithah I didn't see a comment on that requirements patch so I didn't know15:54
dansmiththat requirements thing is holding up another nova fix, among other things15:54
s3rj1ksean-k-mooney: Hi, was able to repro that live-migration issue with broken DNS, please see details here https://bugs.launchpad.net/nova/+bug/2086867/comments/316:10
opendevreviewMerged openstack/nova-specs master: Update and Re-propose "Allow Manila shares to be directly attached to an instance when using libvirt" for Epoxy  https://review.opendev.org/c/openstack/nova-specs/+/93047116:12
sean-k-mooneys3rj1k: ok that not a nova bug :)16:15
sean-k-mooneydansmith: i cna take a look but the the last time i looked at it a few months ago it work for me locally and i coudl not repoduce the failure16:16
sean-k-mooneydansmith: so ill see if it repodcues localy now16:16
sean-k-mooneys3rj1k: oh this was in relation to using the ip adress16:16
sean-k-mooneys3rj1k: so have you defiend the two nova config option i noted to use the ip?16:17
sean-k-mooneyoh this is failing on config drive16:18
sean-k-mooneythis i shiti som old fallback cod ethat is not needed anymore i think16:19
sean-k-mooneywe used to have a workaround for old libvirt version that we have not supproted for many many years16:21
sean-k-mooneywhere we copy the config drive in nova instead of leaving libvirt do it16:21
sean-k-mooneyya this https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L11456-L1146816:22
sean-k-mooneys3rj1k: so that cod ethat we can delete now, there is a patch somewhere to fix that16:22
sean-k-mooneyas a workaround you could use vfat config drives but we shoudl just delete that code.16:23
sean-k-mooneyhttps://review.opendev.org/c/openstack/nova/+/90912216:24
sean-k-mooneys3rj1k: ^ thats the fix for your issue16:24
opendevreviewDoug Szumski proposed openstack/nova master: Revert "[libvirt] Live migration fails when config_drive_format=iso9660"  https://review.opendev.org/c/openstack/nova/+/90912216:24
s3rj1ksean-k-mooney: yea both options on both computes with corresponding IPs16:24
s3rj1khmm, https://review.opendev.org/c/openstack/nova/+/909122 will remove support for config-drive?16:26
sean-k-mooneyno16:26
sean-k-mooneylibvirt and qemu can automatically copy the iso file as part fo the live migration16:27
sean-k-mooneybut libvirt had a bug 8-9 years ago16:27
s3rj1kok, let me try that, I'll report back16:27
sean-k-mooneyand that was a tempory workaround until it could be fixed and we never reverted it16:27
sean-k-mooneyi think this stopped being a thing we needed to care about when we dropped ubuntu 12.04 or 14.04 support16:28
sean-k-mooneys3rj1k: ah someone else already reported a similar issue to yours https://bugs.launchpad.net/nova/+bug/193986916:31
s3rj1ksimilar yes, fails on "Host key verification failed"16:37
pas-ha[m]hi folks, today we faced what seems another fallout of those qcow-related CVE fixes. After update to latest Antelope code, some of our previously working (windows) images no longer boot (and volume can not be created). Upon closer inspection, those happened to be qcow v1 images uploaded to glance with disk_format=qcow2. This is clearly not right, yet it was working before and I wonder who else might get bitten..16:44
dansmithpas-ha[m]: it was working before because glance does absolutely no inspection of images16:46
pas-ha[m]from the comment in oslo.utils image format inspector, I get that this was intentional, yet I do not see that in release notes or smth.16:47
pas-ha[m]yeah, I know that šŸ™‚ 16:47
dansmithalso I suspect your images are not qcow1, but qcow2v116:47
dansmithpas-ha[m]: https://review.opendev.org/c/openstack/nova/+/92893716:48
pas-ha[m]Just wanted to highlight that may be it worth mentioning that in release notes, even retro-actively16:48
dansmithI wasn't planning to merge that, I mostly did it just so it was available16:48
dansmithpas-ha[m]: ack, it wasn't mentioned in the release notes because most of us had assumed people had long since moved off of unsupported formats like that.. not sure we'd go back and put in a retroactive reno like that or not16:49
pas-ha[m]this is tricky semantic šŸ™‚ qemu-img calls it 'qcow' (not qcow2), calling 'file' shows 'QEMU QCOW Image (v1)'.16:53
pas-ha[m]in my understanding it is really qcow == qcow v1. then came qcow v2 == qcow2 which is very different internally, and then came qcow v3 which is an extension of v2 and is backward compatible, so it is still called qcow2 file format.16:53
pas-ha[m]when I tried to repro such an image, this is what you get with 'qemu-img convert -O qcow'16:54
pas-ha[m]and there was QED which is even older than QCOW16:55
dansmithpas-ha[m]: I'm quite sure you don't have any *actual* qcowv1 files16:57
dansmithif it really is, I think we can safely say we don't intend to support your use of those16:58
dansmithI know some *cough* distributions of qemu disable write support for the actual v1 format16:58
pas-ha[m]as I said, this is what 'file' and 'qemu-img info' showed once I fetched files locally. I personally think someone fat fingered -O qcow2 šŸ™‚ still, came as a surprise once it broke and stopped working16:59
dansmithyeah, if qemu-img is showing "file format: qcow" then you're using the ancient version that uses totally different code in qemu (as opposed to 2 and 2v3)17:03
dansmithand we have never intended to support that, so I wouldn't even say we broke it.. I'd say that fits into another category of issues where you can fool nova into booting images on libvirt we intend to disallow by lying in glance17:04
dansmithso this is that sort of thing, you claimed it was a v2, it's not, but we sent it to the compute node anyway and didn't notice that it's actually something other than what we expected17:04
dansmiththe "unintended" inspector stuff was around breaking qcow2 (vs qcow2v3) *not* about breaking the old-old format we didn't intend to support anyway17:05
s3rj1ksean-k-mooney: with https://review.opendev.org/c/openstack/nova/+/909122 the actual stack trace error is gone but I still see migration failure, "Live Migration failure: Cannot recv data: ssh: Could not resolve hostname"17:20
sean-k-mooneys3rj1k: ok so the revert is not enough to fix https://bugs.launchpad.net/nova/+bug/1939869 then its just part of it17:23
sean-k-mooneys3rj1k: you tested that with both options set to the ip yes adn the revert applied to the dest node 17:24
sean-k-mooney(it shoudl be on both but i think it actully runs on the dest node)17:24
s3rj1kboth options set yea, also checked with and without config-drive after applying patch17:26
s3rj1krevert is on both computes17:26
sean-k-mooneyok you got the same issue without config drive 17:26
sean-k-mooneywhat are you using for the live migration url17:27
sean-k-mooneyare you using the default17:27
s3rj1kyea, I think before revert without config drive migration was working17:27
sean-k-mooneyhttps://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_uri17:27
s3rj1kdefault migration url17:27
sean-k-mooneyok so plain text over tcp qemu+tcp://%s/system17:28
sean-k-mooneyunless you have tls enabled17:28
s3rj1kok no so default then, I have live_migration_uri = qemu+ssh://stack@%s/system17:28
sean-k-mooneyah you using ssh. its what most do when not using tls17:29
sean-k-mooneyso %s is replaced with the hostname17:29
sean-k-mooneywell17:29
sean-k-mooneythe docs say17:29
sean-k-mooney Any included ā€œ%sā€ is replaced with the migration target hostname, or live_migration_inbound_addr if set.17:29
sean-k-mooneyso that shoudl use the ip17:30
s3rj1klive_migration_inbound_addr is set to IP17:30
sean-k-mooneydo you have a trace fo the error youc an share with the revert applied17:30
sean-k-mooneys3rj1k: basically im wondering if there is another scp somewhere 17:30
s3rj1kactually no trace where in logs, let me check again17:30
s3rj1kactually no trace where in logs, let me check again17:30
s3rj1kyea, see it, I'll update launchpad in a moment17:32
sean-k-mooneyack. my gues is we just missed using the config option somewhere and hardcoded the hypervisor_hostname, the other possibleity is it is using the ip now and that is not in the ssh know hosts17:32
sean-k-mooneyoh its might be that change this https://review.opendev.org/c/openstack/nova/+/906053/4/nova/virt/libvirt/driver.py17:34
sean-k-mooneyyou not tryign to migrate an isntance that uses an image that was deleted in glance are you17:34
sean-k-mooneyat the moment we are passign instance.host as the fallback host but that is likely not correct in this case17:35
s3rj1kupdated https://bugs.launchpad.net/nova/+bug/2086867/comments/417:36
sean-k-mooneyoh its comming form migrateToURI317:36
s3rj1kusing cirros for debug17:37
sean-k-mooneythat does not make sense based on what you said was set on the config17:37
sean-k-mooneyya the guest iamge should not factor in here other then if its deleted in glance17:37
s3rj1kit's definitely in config, let me restart services just in case and retry17:38
sean-k-mooneywe might just need to set up  this evn and debug it17:38
sean-k-mooneythis is basiclly just a normal multi node devstack but instead of regestierign the host in dns, mdns or /etchost you have set the ips17:39
sean-k-mooneywhich on master shoudl work17:39
sean-k-mooneyyou coudl also set [DEFAULT]host=<ip> i guess17:39
sean-k-mooneyyou should not have too17:40
sean-k-mooneybut im pretty sure that would fix this and break neutorn unless you did the same there....17:40
sean-k-mooneyim debating how easy it would be to repoduce this in the ci17:41
sean-k-mooneyvs the hour it woudl take me to set this up by hand. that is proably quicker17:41
sean-k-mooneybut you know lazy17:41
s3rj1kok, so cold-migrate works fine, live-migrate is not17:47
s3rj1kI am pretty sure that this is one-shot repro, so CI is overkill17:48
s3rj1kidea is to have a fallback on IPs when DNS has issues, so directly using would work but UX is bad in this case17:49
sean-k-mooneys3rj1k: so yes an no17:52
sean-k-mooneynova does not genreally have ips to fall back too17:52
sean-k-mooneyand in this case its failing in libvirt not nova17:52
sean-k-mooneyso nova required that you either have dns for each of the compute nodes ro that you provie hostname resolution some ohter way like /etc/hosts17:53
sean-k-mooneynow if you put the ips int eh config that is intended to work17:53
sean-k-mooneyso there is a bug somewhere17:53
sean-k-mooneybut we require the opertor to make the hostnames resouceabel or provide the ips17:53
s3rj1kso, should we start with merging revert and fixing edge after merge or we should fix-all-the-things :) ?17:57
s3rj1k* edge case17:57
sean-k-mooneys3rj1k: we proably should proceed with the revert if that does not regess anything then fix what is left in a follow up patch18:29
sean-k-mooneys3rj1k: what release are you trying to adress this in by the way18:30
s3rj1ksean-k-mooney: hopefully upcoming 2025.1 Epoxy18:33
sean-k-mooneyoh ok you dont need this in older releases as well18:33
sean-k-mooneyseperate form the revert im wondering if there was a bug in https://github.com/openstack/nova/commit/6bca37e904e9e56b250b123abde1901e951c8c9a18:33
s3rj1kwell, probably some older releases could be potentially needed, but this is another story that I hope I could avoid :)18:34
sean-k-mooneyno worries18:34
sean-k-mooneywhat im condiering is if we consider it a regresion intoduced by the "libvirt-migrate-with-hostname-instead-of-ip" which was not ment to change the default behaicvor then it would be a bug and backportable18:36
s3rj1kmigration_inbound_addr is for none-live migration? if so, none-live migration worked for me when DNS is blocked18:36
sean-k-mooneyotherwise this is kind of a mini featur and only fixable on master18:36
sean-k-mooneys3rj1k: it is but its also for some shared commdns that are used during live migration18:37
sean-k-mooneys3rj1k: im wondering if this https://github.com/openstack/nova/commit/6bca37e904e9e56b250b123abde1901e951c8c9a#diff-486ddc37ce8e7c105a51566dd2cffbda034f28fae7eaad2e74abf470ab296352R4514-R452318:37
sean-k-mooneyimpacted live migrtaion in a way we did not plan it too18:37
sean-k-mooneyalthough you have set both to an ip18:38
sean-k-mooneyso proably something else18:38
sean-k-mooneyi need to set up a multi node devstack for somethign else18:38
s3rj1ksean-k-mooney: I can retested after revert gets merged, so no problem with that18:38
s3rj1khmm, "if "%s" in addr:"18:38
sean-k-mooneyill see if i can repoduce this when i do18:39
sean-k-mooneyoh18:39
sean-k-mooneyya that is the bug18:39
sean-k-mooneythat code is assumeing that %s is always going to repalce with the hostname18:39
sean-k-mooneybut if you set the livemigration config var to the ip we should not be using get_hostname there18:40
sean-k-mooneywell hum 18:40
sean-k-mooneyi think that depned on where get_host_ip_addr is being called form at the time18:40
s3rj1kfix would be moving line 4520 below if I guess18:40
s3rj1kwith some check if conf is set18:41
sean-k-mooneysomething along those lines maybe18:42
sean-k-mooneycould you try regerting get_host_ip_addr to just         return CONF.my_ip18:42
sean-k-mooneyand see if that fixes it for you locally18:42
sean-k-mooneyit will break the other fature so that not the correct fix but it will confirm if that is the souce of the regression18:43
s3rj1kyea, give me few minutes18:43
sean-k-mooneyalthough you do not have "%s" in  CONF.libvirt.migration_inbound_addr do you?18:45
sean-k-mooneyyou do in the migration uri but that is just an ip adress reight18:45
sean-k-mooney*right18:45
s3rj1kI have live_migration_uri = qemu+ssh://stack@%s/system and live_migration_inbound_addr, migration_inbound_addr set to IPs, nothing more related to migration in libvirt section18:54
s3rj1kso I've tried to return both CONF.my_ip and CONF.libvirt.migration_inbound_addr, migration still tries to resolve by dns18:55
sean-k-mooneys3rj1k: ok we are going to need ot look at exaclty what we are passing to the migrate too uri libvirt call19:35
s3rj1ksean-k-mooney: do you have some specific debug changes in mind?19:36
sean-k-mooneybaiscally debugging this call https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10854-L1085919:40
sean-k-mooneywe can do that by putting some debug logs or prints in https://github.com/openstack/nova/blob/master/nova/virt/libvirt/guest.py#L59119:41
sean-k-mooneys3rj1k: in your config we would expect both destination and migrate_uri to have the ip 19:42
sean-k-mooneybut i suspect one or both of them have the hostname19:42
sean-k-mooneys3rj1k: the excption form libvirt is comming form this funciton call https://github.com/openstack/nova/blob/master/nova/virt/libvirt/guest.py#L64719:42
s3rj1khmm, destination has hostname as expected I guess, but migrate_uri is None for some reason19:51
s3rj1ksmth wrong in here https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10781-L10786 ?19:54
s3rj1k"non tunneled migration"19:55
sean-k-mooneywe do not want to use tunneled migration so i think that is fine20:09
sean-k-mooneyso the quetion is where did destinaiton get set ot the host name20:09
s3rj1kmigrate_uri is None, this is expected?20:10
sean-k-mooneyi dont recall of the top of my head but we do not adivse using tunneld migration20:10
sean-k-mooneyso that being none is proably ok 20:10
sean-k-mooneydist i think is coming form (self._live_migration_uri(dest)20:11
sean-k-mooneyok that just ipv6 escapign the dest that past in20:11
sean-k-mooneyso thats passed in to live_migrate in the driver and that invoked over rpc via the compute manager20:14
s3rj1khere https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10733 it is already a hostname20:16
s3rj1k(dest var)20:17
sean-k-mooneyyep20:17
sean-k-mooneyso in the conduction its a destation object i think it being converted to the host name in the compute manager but im trying to find that20:17
sean-k-mooneyor not https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L903720:19
sean-k-mooneythat looks like its alrady going to be the host name on the comptue manager side of the rpc20:19
sean-k-mooneys3rj1k: so https://github.com/openstack/nova/blob/master/nova/conductor/tasks/live_migrate.py#L509-L562 is returning the host form the select_destinations call to the schduler20:21
sean-k-mooneyselection.service_host20:22
sean-k-mooneyis what   self.destination has here https://github.com/openstack/nova/blob/master/nova/conductor/tasks/live_migrate.py#L100 20:22
sean-k-mooneywhich we pass here https://github.com/openstack/nova/blob/master/nova/conductor/tasks/live_migrate.py#L15720:23
s3rj1kI assume that we want this hotfixed near live_migration_inbound_addr handling?20:24
sean-k-mooneyso the porblem is the schduelr does not realy knwo about the ips of the comptue nodes so it just sleect the dstionation compute service to hanel the request20:26
sean-k-mooneywhich means that we need to transport the information form the dest comptue node back to the source eiver via the database or rpc20:27
sean-k-mooneyto then pass it to libvirt on the souce node20:27
sean-k-mooneyin other words the source compute node need the value of live_migration_inbound_addr form the destioantion host when it calls libvirt20:28
s3rj1kahh, that makes sense, and makes fix more complicated :(20:28
sean-k-mooneywe shoudl proably do that by adding it to the libvirtLiveMigrationData object https://github.com/openstack/nova/blob/master/nova/objects/migrate_data.py#L22920:30
sean-k-mooneyin pre live migrate20:30
sean-k-mooneyso we need to add a new field "destiantion_address"20:31
s3rj1ktarget_connect_addr seems familiar20:31
s3rj1khttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10781-L1078620:31
sean-k-mooneyit deso but im not sure if that is related to the storage network20:32
sean-k-mooneyhum20:32
sean-k-mooneycan you pritn some of that locally and see what branch we are taking20:33
s3rj1kyea, few min20:33
sean-k-mooneyif im readign that proprly then if target_connect_addr was set by the dest node to the ip then it would use it20:33
sean-k-mooneyassuming tunneled migraiton is off which it is by default20:34
sean-k-mooneyhum it should be uncondtionally set here https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L11503-L1150520:35
s3rj1ktarget_connect_addr in migrate_data but value is None and (migration_flags & libvirt.VIR_MIGRATE_TUNNELLED == 0)20:39
sean-k-mooneythats odd because its clearly bing set unconditionally on the dest so either we are loosign the value or there is a bug in how we are setting it form the config20:40
s3rj1ksean-k-mooney: ok, so just to verify, live_migration_inbound_addr is under libvirt in /etc/nova/nova.conf for devstack right?20:47
sean-k-mooneygood question i think so20:48
sean-k-mooneyyep https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_inbound_addr20:49
s3rj1kbecause I've just added log.debug in https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L11503-L11505 on target node and I see that value is None20:49
sean-k-mooneywhen the comptue start up in debug mode it dumps the config to the log20:50
sean-k-mooneyis it set to the correct value there?20:50
opendevreviewribaudr proposed openstack/nova master: Amend ShareMappingStatus due to asynchronous call  https://review.opendev.org/c/openstack/nova/+/90886420:50
opendevreviewribaudr proposed openstack/nova master: Amend DB model add a unique constraint.  https://review.opendev.org/c/openstack/nova/+/91251820:50
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (manila abstraction)  https://review.opendev.org/c/openstack/nova/+/83119420:50
opendevreviewribaudr proposed openstack/nova master: Use client token when talking to manila  https://review.opendev.org/c/openstack/nova/+/92527720:50
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (drivers and compute manager part)  https://review.opendev.org/c/openstack/nova/+/83309020:50
opendevreviewribaudr proposed openstack/nova master: Mounting the shares as part of the initialization process  https://review.opendev.org/c/openstack/nova/+/88007520:50
opendevreviewribaudr proposed openstack/nova master: Deletion of associated share mappings on instance deletion  https://review.opendev.org/c/openstack/nova/+/88147220:50
opendevreviewribaudr proposed openstack/nova master: Add metadata for shares  https://review.opendev.org/c/openstack/nova/+/85050020:50
opendevreviewribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part)  https://review.opendev.org/c/openstack/nova/+/85482320:50
opendevreviewribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute manager part)  https://review.opendev.org/c/openstack/nova/+/85482420:50
opendevreviewribaudr proposed openstack/nova master: Add share_info parameter to resume method for each driver (driver part)  https://review.opendev.org/c/openstack/nova/+/86028420:50
opendevreviewribaudr proposed openstack/nova master: Support resuming an instance with shares (compute manager part)  https://review.opendev.org/c/openstack/nova/+/86028520:50
opendevreviewribaudr proposed openstack/nova master: Add helper methods to rescue/unrescue shares  https://review.opendev.org/c/openstack/nova/+/86028620:50
opendevreviewribaudr proposed openstack/nova master: Support rescuing an instance with shares  https://review.opendev.org/c/openstack/nova/+/86028720:51
opendevreviewribaudr proposed openstack/nova master: Allow to mount manila share using Cephfs protocol  https://review.opendev.org/c/openstack/nova/+/88386220:51
opendevreviewribaudr proposed openstack/nova master: Check shares support (compute manager)  https://review.opendev.org/c/openstack/nova/+/88575120:51
opendevreviewribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (API)  https://review.opendev.org/c/openstack/nova/+/83683020:51
opendevreviewribaudr proposed openstack/nova master: Add helper methods to attach/detach shares  https://review.opendev.org/c/openstack/nova/+/88575320:51
opendevreviewribaudr proposed openstack/nova master: Add instance.share_attach notification  https://review.opendev.org/c/openstack/nova/+/85050120:51
opendevreviewribaudr proposed openstack/nova master: Add instance.share_detach notification  https://review.opendev.org/c/openstack/nova/+/85102820:51
opendevreviewribaudr proposed openstack/nova master: Add shares to InstancePayload  https://review.opendev.org/c/openstack/nova/+/85102920:51
opendevreviewribaudr proposed openstack/nova master: Add instance.share_attach_error notification  https://review.opendev.org/c/openstack/nova/+/86028220:51
opendevreviewribaudr proposed openstack/nova master: Add instance.share_detach_error notification  https://review.opendev.org/c/openstack/nova/+/86028320:51
opendevreviewribaudr proposed openstack/nova master: Reports instance events to the DB regarding attaching and detaching a share  https://review.opendev.org/c/openstack/nova/+/92708820:51
opendevreviewribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working.  https://review.opendev.org/c/openstack/nova/+/85208620:51
opendevreviewribaudr proposed openstack/nova master: Add virt/libvirt error test cases  https://review.opendev.org/c/openstack/nova/+/85208720:51
opendevreviewribaudr proposed openstack/nova master: Manila shares admin guide documentation  https://review.opendev.org/c/openstack/nova/+/87164220:51
sean-k-mooneys3rj1k: its defiend as a HostDomainOpt type shich shoudl accpaate any ip, hostanem or fqdn20:51
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/conf/libvirt.py#L239-L25420:51
sean-k-mooneyhttps://github.com/openstack/oslo.config/blob/82552116f827030ce1e493506fe83ef71656e447/oslo_config/cfg.py#L118920:52
s3rj1ksean-k-mooney: yea, I see value in logs DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = 10.42.71.101 {{(pid=205504)...20:54
sean-k-mooneywait where are you definign the values ?20:55
sean-k-mooneyyour putting them on the compute nodes nova.conf right20:55
sean-k-mooneyfor devstack this is called /etc/nova/nova-cpu.conf20:55
s3rj1kwas putting it in nova.conf :(20:56
s3rj1klemme check again20:56
s3rj1k::facepalm20:58
s3rj1kwell, in works20:59
sean-k-mooneywith the revert rightt20:59
s3rj1ksean-k-mooney: sorry for that, yea, with revert20:59
sean-k-mooneyso it woudl b einteresting if it fails if you drop the revert20:59
sean-k-mooneycause if so then that is the probelm and the fix20:59
s3rj1khmm, lemme check21:00
sean-k-mooneyno rush can you update the bug wiht your finding either way21:01
sean-k-mooneythe revert is good to do anyway but i think im going to finish for today21:03
s3rj1ksean-k-mooney: yep, fails without revert21:06
s3rj1ki'll update bugreport21:06
s3rj1kdone https://bugs.launchpad.net/nova/+bug/2086867/comments/521:10
s3rj1kthanks Sean!21:11
sean-k-mooneycool then lets proceed with the revert we can follow up with others to review tomrorow21:12
s3rj1kI'll also will revalidate that after merge on clean DevStack, just in case :)21:13
sean-k-mooneywe may want to consider twakign ci to be able to test this but i think that is best left to a followup21:13
*** tkajinam is now known as Guest933822:33

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!