Monday, 2020-12-14

« Sunday, 2020-12-13 Index Tuesday, 2020-12-15 »
*** zzzeek has quit IRC00:08
*** jangutter has quit IRC00:16
*** zzzeek has joined #openstack-nova00:16
*** jangutter has joined #openstack-nova00:17
openstackgerritArtom Lifshitz proposed openstack/nova master: WIP: db: Add `socket` to pci_devices  https://review.opendev.org/c/openstack/nova/+/76686200:19
openstackgerritArtom Lifshitz proposed openstack/nova master: WIP: objects: Add `socket` to NUMACell and PciDevice  https://review.opendev.org/c/openstack/nova/+/76679500:19
openstackgerritArtom Lifshitz proposed openstack/nova master: WIP: Start populating `socket_id` in host NUMA topology  https://review.opendev.org/c/openstack/nova/+/76681600:19
*** prometheanfire has joined #openstack-nova00:20
*** zzzeek has quit IRC00:33
*** zzzeek has joined #openstack-nova00:35
*** tosky has quit IRC00:36
*** zzzeek has quit IRC00:45
*** zzzeek has joined #openstack-nova00:49
*** eharney has quit IRC01:00
*** Shatadru has joined #openstack-nova01:02
*** Shatadru has quit IRC01:05
*** eharney has joined #openstack-nova01:18
*** lifeless has quit IRC01:42
*** lifeless has joined #openstack-nova01:44
*** zzzeek has quit IRC01:46
*** zzzeek has joined #openstack-nova01:47
*** hamalq has joined #openstack-nova02:31
*** ociuhandu has joined #openstack-nova02:31
*** zzzeek has quit IRC02:33
*** hamalq has quit IRC02:35
*** zzzeek has joined #openstack-nova02:35
*** ociuhandu has quit IRC02:39
*** jangutter_ has joined #openstack-nova02:40
*** jangutter_ has quit IRC02:40
*** jangutter has quit IRC02:41
*** jangutter has joined #openstack-nova02:41
openstackgerritMerged openstack/nova-specs master: Smart-nic Support Specification  https://review.opendev.org/c/openstack/nova-specs/+/74278502:52
*** rcernin has quit IRC02:53
*** dave-mccowan has joined #openstack-nova02:56
*** rcernin has joined #openstack-nova03:05
*** sapd1 has joined #openstack-nova03:21
*** psachin has joined #openstack-nova03:32
*** rcernin has quit IRC03:33
*** rcernin has joined #openstack-nova03:40
*** rcernin has quit IRC03:41
*** rcernin has joined #openstack-nova03:42
*** zzzeek has quit IRC03:59
*** zzzeek has joined #openstack-nova04:01
*** ratailor has joined #openstack-nova04:13
*** ociuhandu has joined #openstack-nova04:17
*** ociuhandu has quit IRC04:21
*** vishalmanchanda has joined #openstack-nova04:23
*** bnemec has joined #openstack-nova04:41
*** hamalq has joined #openstack-nova04:42
*** efried has quit IRC04:46
*** hamalq has quit IRC04:46
*** efried has joined #openstack-nova04:47
*** bnemec has quit IRC04:50
*** efried has quit IRC05:01
openstackgerritHemanth N proposed openstack/nova stable/ussuri: functional: Clean up PCI tests  https://review.opendev.org/c/openstack/nova/+/76687905:02
*** hemanth_n has joined #openstack-nova05:05
*** bnemec has joined #openstack-nova05:07
*** efried has joined #openstack-nova05:18
*** bnemec has quit IRC05:20
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-nova05:33
*** sapd1 has quit IRC05:35
*** efried has quit IRC05:49
*** efried has joined #openstack-nova05:58
*** efried has quit IRC06:01
*** efried has joined #openstack-nova06:01
*** songwenping__ has quit IRC06:03
*** ociuhandu has joined #openstack-nova06:05
*** ociuhandu has quit IRC06:09
*** sapd1 has joined #openstack-nova06:19
*** sapd1 has quit IRC06:23
*** hamalq has joined #openstack-nova06:43
*** hamalq has quit IRC06:47
*** zzzeek has quit IRC06:56
*** k_mouza has joined #openstack-nova06:56
*** zzzeek has joined #openstack-nova06:56
*** k_mouza has quit IRC07:01
*** ociuhandu has joined #openstack-nova07:01
*** rcernin has quit IRC07:01
*** ociuhandu has quit IRC07:06
*** ralonsoh has joined #openstack-nova07:19
*** rcernin has joined #openstack-nova07:27
*** rcernin has quit IRC07:33
*** sapd1 has joined #openstack-nova07:34
*** efried has quit IRC07:34
*** rcernin has joined #openstack-nova07:45
*** mkrai has joined #openstack-nova07:46
*** hamalq has joined #openstack-nova07:48
*** zzzeek has quit IRC07:50
*** rcernin has quit IRC07:50
*** slaweq has joined #openstack-nova07:51
*** ralonsoh has quit IRC07:51
*** zzzeek has joined #openstack-nova07:52
*** hamalq has quit IRC07:53
*** ralonsoh has joined #openstack-nova07:53
*** ociuhandu has joined #openstack-nova07:59
gibigood morning08:01
*** ociuhandu has quit IRC08:03
*** zzzeek has quit IRC08:06
*** rcernin has joined #openstack-nova08:06
*** zzzeek has joined #openstack-nova08:09
*** ratailor_ has joined #openstack-nova08:15
*** ratailor has quit IRC08:15
*** masterpe has quit IRC08:16
*** tesseract has joined #openstack-nova08:16
*** masterpe has joined #openstack-nova08:16
*** mkrai has quit IRC08:16
*** hemanth_n has quit IRC08:16
*** psachin has quit IRC08:16
*** jangutter has quit IRC08:16
*** elod has quit IRC08:16
*** zer0c00l has quit IRC08:16
*** tbarron has quit IRC08:16
*** dansmith has quit IRC08:16
*** valleedelisle has quit IRC08:16
*** stephenfin has quit IRC08:16
*** johanssone has quit IRC08:19
*** andrewbonney has joined #openstack-nova08:20
*** zzzeek has quit IRC08:20
*** mkrai has joined #openstack-nova08:22
*** hemanth_n has joined #openstack-nova08:22
*** psachin has joined #openstack-nova08:22
*** jangutter has joined #openstack-nova08:22
*** elod has joined #openstack-nova08:22
*** zer0c00l has joined #openstack-nova08:22
*** tbarron has joined #openstack-nova08:22
*** dansmith has joined #openstack-nova08:22
*** valleedelisle has joined #openstack-nova08:22
*** stephenfin has joined #openstack-nova08:22
*** zzzeek has joined #openstack-nova08:22
*** rcernin has quit IRC08:23
*** johanssone has joined #openstack-nova08:23
*** sapd1 has quit IRC08:26
*** rpittau|afk is now known as rpittau08:27
*** sapd1 has joined #openstack-nova08:28
*** ociuhandu has joined #openstack-nova08:30
*** hamalq has joined #openstack-nova08:30
*** ociuhandu has quit IRC08:34
*** rcernin has joined #openstack-nova08:35
*** k_mouza has joined #openstack-nova08:36
*** zzzeek has quit IRC08:37
*** tosky has joined #openstack-nova08:38
*** k_mouza has quit IRC08:41
openstackgerritBalazs Gibizer proposed openstack/nova master: Reproduce bug 1907522 in functional test  https://review.opendev.org/c/openstack/nova/+/76679108:41
openstackbug 1907522 in OpenStack Compute (nova) "test_*_with_qos_min_bw_allocation fails in the nova-multi-cell job with: nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host <host>" [High,Triaged] https://launchpad.net/bugs/1907522 - Assigned to Balazs Gibizer (balazs-gibizer)08:41
*** zzzeek has joined #openstack-nova08:41
openstackgerritMerged openstack/nova stable/victoria: Reproduce bug 1896463 in func env  https://review.opendev.org/c/openstack/nova/+/76315208:42
openstackbug 1896463 in OpenStack Compute (nova) victoria "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 - Assigned to Balazs Gibizer (balazs-gibizer)08:42
*** rcernin has quit IRC08:49
*** jangutter_ has joined #openstack-nova08:55
*** psachin has quit IRC08:57
*** ociuhandu has joined #openstack-nova08:57
*** ociuhandu has quit IRC08:57
*** jangutter has quit IRC08:58
*** ociuhandu has joined #openstack-nova08:58
*** rcernin has joined #openstack-nova08:59
*** ociuhandu has quit IRC09:03
*** dtantsur|afk is now known as dtantsur09:07
*** rcernin has quit IRC09:11
*** derekh has joined #openstack-nova09:17
*** macz_ has joined #openstack-nova09:20
*** ociuhandu has joined #openstack-nova09:22
*** macz_ has quit IRC09:24
*** zzzeek has quit IRC09:28
*** zzzeek has joined #openstack-nova09:31
*** hamalq has quit IRC09:34
*** martinkennelly has joined #openstack-nova09:35
*** sapd1 has quit IRC09:38
*** zzzeek has quit IRC09:51
*** zzzeek has joined #openstack-nova09:53
lyarwoodgibi: morning, just sent a reply to fungi on the ML without CC'ing you in reply. He raised a number of valid points about our tox envs and requirements files, wanted to ask if you wanted them tracked in LP or somewhere else09:56
*** rcernin has joined #openstack-nova09:59
*** zzzeek has quit IRC10:01
*** zzzeek has joined #openstack-nova10:03
gibilyarwood: hi! ack, replying10:05
*** rcernin has quit IRC10:12
openstackgerritLee Yarwood proposed openstack/nova stable/ussuri: [stable-only] lower-contraints: Bump various packages  https://review.opendev.org/c/openstack/nova/+/76673810:13
*** mkrai has quit IRC10:17
gibilyarwood: I'm not considering these as serious issues, more like improvements, but bugs are OK to track them10:19
*** mkrai has joined #openstack-nova10:23
lyarwoodright that's why I asked10:25
lyarwoodcool10:25
lyarwoodgibi: https://etherpad.opendev.org/p/nova-stable-pip-fallout btw I'm trying to capture things here re the broken stable branches etc10:31
lyarwoodelod: ^ also10:31
gibithanks lyarwood10:32
*** rcernin has joined #openstack-nova10:35
openstackgerritLee Yarwood proposed openstack/nova stable/train: [stable-only] Cap bandit to 1.6.2 and raise hacking and flake8 in LC  https://review.opendev.org/c/openstack/nova/+/76617110:37
lyarwoodhmm does Depends-On work across branches with grenade?10:39
* lyarwood gives it a go10:40
openstackgerritLee Yarwood proposed openstack/nova stable/train: [stable-only] Cap bandit to 1.6.2 and raise hacking and flake8 in LC  https://review.opendev.org/c/openstack/nova/+/76617110:40
openstackgerritBalazs Gibizer proposed openstack/nova master: Reproduce bug 1907522 in functional test  https://review.opendev.org/c/openstack/nova/+/76679110:41
openstackbug 1907522 in OpenStack Compute (nova) "test_*_with_qos_min_bw_allocation fails in the nova-multi-cell job with: nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host <host>" [High,Triaged] https://launchpad.net/bugs/1907522 - Assigned to Balazs Gibizer (balazs-gibizer)10:41
*** rcernin has quit IRC10:41
openstackgerritBalazs Gibizer proposed openstack/nova master: Fallback to same-cell resize with qos ports  https://review.opendev.org/c/openstack/nova/+/76692510:42
openstackgerritBalazs Gibizer proposed openstack/nova master: DNM: test how multi-cell job behaves now  https://review.opendev.org/c/openstack/nova/+/76692610:44
*** ociuhandu_ has joined #openstack-nova11:01
*** mkrai has quit IRC11:01
*** ociuhandu has quit IRC11:03
*** mgoddard has quit IRC11:08
elodlyarwood: fyi, bandit 1.6.3 is yanked, so hopefully that error is now history :) >>> https://pypi.org/project/bandit/#history11:09
elodlyarwood: sorry, my bad. it seems 1.7.0 is still universal :'(11:11
lyarwood /o\11:11
*** mgoddard has joined #openstack-nova11:13
elodquite frustrating :/11:13
elodthe new bug report: https://github.com/PyCQA/bandit/issues/66511:14
lyarwoodI'm going to close out my bandit changes either way as this should eventually get resolved there11:17
lyarwooddid we want to switch our LC jobs to non-voting btw while we wait for an agreement on dropping them out right or not?11:17
elodlyarwood: maybe we could even remove them (at least from old branches)11:18
lyarwoodelod: personally I'd like to wait for distros to comment more, I wanted to chase up with the RDO folks for example11:18
elodwhere it's clearly contradicts with constraints (i.e. LC fails)11:18
lyarwoodelod: switching to non-voting at least gives them a chance to respond and later fix these issues if they find it useful11:19
elodlyarwood: hmmm, yes, you are right. Let's just set them non-voting11:19
lyarwoodkk11:19
lyarwoodoh it's a template11:20
lyarwoodmy bad11:20
lyarwoodcan we mark templates as non-voting?11:20
elodwe should be able to11:21
elodworst case the job needs to be added in .zuul.yaml outside of the template11:22
elodand then set non-voting11:22
lyarwoodyup true11:22
openstackgerritLee Yarwood proposed openstack/nova master: zuul: Make lower-constraints jobs non-voting  https://review.opendev.org/c/openstack/nova/+/76693111:26
elodlyarwood: btw, if now the lower-constraints do work on master and victoria then maybe it's enough to set non-voting from ussuri. what do you think?11:27
*** sean-k-mooney has joined #openstack-nova11:28
elodas the main problem is where the LC's are kinda irreparable11:29
*** rcernin has joined #openstack-nova11:32
elodor time-consuming to fix and involves lots of LC changes (a real "butterfly effect" :P)11:33
lyarwoodelod: tbh I'd still like to mark them all as non-voting to signal that we don't think they are useful going forward11:33
*** zzzeek has quit IRC11:35
*** zzzeek has joined #openstack-nova11:36
elodlyarwood: you are probably right. however it feels now almost useful - if the constraints are really kept with the new pip resolver. But I'm okay with setting it non-voting on master.11:37
*** ftarasenko has joined #openstack-nova11:55
*** jangutter_ has quit IRC11:58
*** jangutter has joined #openstack-nova11:58
ftarasenkohi! Team, can someone help my colleague with review of tiny commit? https://review.opendev.org/c/openstack/nova/+/76492712:28
*** avolkov has joined #openstack-nova12:29
*** ociuhandu_ has quit IRC12:33
*** ociuhandu has joined #openstack-nova12:34
sean-k-mooneyftarasenko: im not sure that makes sense12:38
sean-k-mooneythe constuction of the loop is such that we sleep for 1 seconed on every iteration12:39
sean-k-mooneyi understand the logic but just addint that while keeping the current looping seams odd12:40
sean-k-mooneywe shoudl proably drop the loop var and instead just do a while loop instead12:41
sean-k-mooneyftarasenko: as written the current look will at most be off by the time it takes for the body to execute12:43
sean-k-mooneyso i agree it can be longer then you expect but not by a large interval12:43
ftarasenkoThank you, your explanation seems good to me. Forwarded to my colleague. He says that he can rewrite it to while loop, if someone will review)12:46
ftarasenkoIs it ok to ask for review here? I'm new to nova project.12:47
sean-k-mooneyyes its ok12:47
sean-k-mooneyftarasenko: we encorage people to ask question about developement here espcially if they are new12:49
sean-k-mooneyftarasenko: by the way they should ingore the lower constraints failure.12:55
sean-k-mooneyftarasenko: we have an ongoing issue with that and the new pip depency resolver12:55
lyarwoodgibi: https://zuul.opendev.org/t/openstack/build/02576a8cc541451bb8add7f1423d14bf - is that a known issue?12:57
lyarwoodopenstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://199.204.45.203/compute/v2.1/os-aggregates/openstack.tests.functional.cloud.test_aggregate.TestAggregate.test_aggregates-1, Invalid input received: id must be an integer'12:57
lyarwoodah sorry I thought that was part of test_qos_minimum_bandwidth_rule_lifecycle12:57
sean-k-mooneywell it must be yes12:58
ftarasenkosean-k-mooney: tnx!12:58
lyarwoodit isn't ignore me12:58
sean-k-mooneylyarwood: its not part of the serise but the error is correct https://docs.openstack.org/api-ref/compute/?expanded=show-aggregate-details-detail13:00
sean-k-mooneylyarwood: there is an osc bug in a few cases where it passes the a name to the /os-aggregates/{aggregate_id} instead of checking if its an id or not13:02
sean-k-mooneyit then calls the list endpoint and filters in python13:02
sean-k-mooneyat least that is what it does in the server show case13:02
*** ygk_12345 has joined #openstack-nova13:03
sean-k-mooneywe had a downstream bz for the server show case but i suspect its a incorrect pattern that exits in many cases that needs to be corrected. osc is just not doing input validation to determin which api to call13:03
sean-k-mooneyi assume tempest has the same bug13:04
*** k_mouza has joined #openstack-nova13:08
*** k_mouza has quit IRC13:10
hemanth_nsean-k-mooney, lyardwood: As discussed on Friday, i started backporting Iabe37dbdc244b019cb68099739b2c1c81b94a30d, and it requires I999fc4aaf14cca02e4ed730afb844a043b59530c for the tests to be successful. So, i am going ahead with backport of I999fc4aaf14cca02e4ed730afb844a043b59530c (after some tests on my local).. hope this is ok13:10
openstackgerritMamduh proposed openstack/os-vif stable/ussuri: Refactor code of linux_net to more cleaner and increase performace  https://review.opendev.org/c/openstack/os-vif/+/76541913:11
ygk_12345hi all13:11
ygk_12345I am facing this error in nova-api log in the Train nova setup.   http://paste.openstack.org/raw/801004/13:11
ygk_12345is this a known bug ?13:11
sean-k-mooneyhemanth_n: that touches more files then i would like but i think its ok13:12
sean-k-mooneyhemanth_n:looking at the change its minimal13:12
hemanth_nsean-k-mooney: yep more backports, ack i will test in local before i submit a patch13:13
*** ociuhandu has quit IRC13:13
sean-k-mooneyygk_12345: that looks like you have not run the required db migration our your backporint some with a db change which is not allowed13:14
ygk_12345sean-k-mooney what would be the exact command ?13:14
openstackgerritMamduh proposed openstack/os-vif stable/train: Refactor code of linux_net to more cleaner and increase performace  https://review.opendev.org/c/openstack/os-vif/+/76591213:15
sean-k-mooneyygk_12345: hum it was added in pike https://github.com/openstack/nova/commit/3674a4268d177230375fa1b581dbdf6f62755cee13:15
sean-k-mooneyygk_12345: "nova-manage db sync"13:16
sean-k-mooneybut given how long ago it was added its unlikely you have not run any db migration since pike13:16
ygk_12345sean-k-mooney is this the one "su -s /bin/sh -c "nova-manage db sync" nova" ?13:16
sean-k-mooneyygk_12345: more or less yes although it proably not the issue13:18
ygk_12345sean-k-mooney now I got this after the command http://paste.openstack.org/raw/801005/13:18
*** mgariepy has joined #openstack-nova13:18
openstackgerritMamduh proposed openstack/os-vif stable/stein: Refactor code of linux_net to more cleaner and increase performace  https://review.opendev.org/c/openstack/os-vif/+/76591413:18
sean-k-mooneyso this is train13:19
sean-k-mooneywas the cloud initall deploy as train or upgraded form a previous release13:20
ygk_12345sean-k-mooney yes13:20
ygk_12345intially as train only13:20
sean-k-mooneyso that failing here https://github.com/openstack/nova/blob/0e7cd9d1a95a30455e3c91916ece590454235e0e/nova/db/sqlalchemy/migrate_repo/versions/316_add_disk_ratio_for_compute_nodes.py13:22
ygk_12345sean-k-mooney yes exactly13:22
sean-k-mooneyso if the migration is failing there which is mitaka then ist possbale that the rest fo the migration have not been run either13:23
sean-k-mooneythe service uuid was added in a later migration13:23
sean-k-mooneylyarwood: gibi  do you know if we ahve a way to print the crrent db version via nova manage13:24
ygk_12345sean-k-mooney so what would be the workaround now ?13:24
sean-k-mooneystephenfin: maybe you know ^13:24
sean-k-mooneyygk_12345: well we need to figure out what version your db actull is13:25
sean-k-mooneyit sound like your deployment didnt actully work properly13:25
ygk_12345sean-k-mooney i am following the train minimal nova installation guide13:25
ygk_12345sean-k-mooney oh ok13:25
ygk_12345sean-k-mooney i used this on ubuntu18  add-apt-repository cloud-archive:train13:26
sean-k-mooneythe install gudies are not really tested at least not by the project teams13:27
lyarwoodsean-k-mooney: yeah isn't there a version subcommand or something?13:27
* lyarwood looks13:27
lyarwoodhttps://docs.openstack.org/nova/latest/cli/nova-manage.html#nova-database - nova-manage db version13:27
ygk_12345sean-k-mooney its version 31513:28
ygk_12345lyarwood sean-k-mooney its  db version 31513:28
sean-k-mooneyygk_12345: ya so that pre mitaka13:28
*** redrobot has quit IRC13:29
openstackgerritMamduh proposed openstack/os-vif stable/rocky: Refactor code of linux_net to more cleaner and increase performace  https://review.opendev.org/c/openstack/os-vif/+/76596213:29
ygk_12345sean-k-mooney so what would be the workaroud now ?13:29
sean-k-mooneyygk_12345: so at that point you should not have the disk_allocation_ratio column in that table13:29
sean-k-mooneyi think what has happened here is that since you started the services before the db was synced13:30
sean-k-mooneythe conductor or one of the other service created teh db column13:30
ygk_12345sean-k-mooney shall I remove the db and nova pkgs and try again ?13:31
sean-k-mooneyyou could do that yes if you have not workloads on it13:31
*** hamalq has joined #openstack-nova13:31
ygk_12345sean-k-mooney just a test platform. no workloads13:31
sean-k-mooneyyou dont really need to uninstall the service just stop them13:31
sean-k-mooneythen drop the db and recreate it13:32
sean-k-mooneymaking sure that the db is fully migrated13:32
*** bnemec has joined #openstack-nova13:32
ygk_12345sean-k-mooney ok. sounds fine. will do that now13:33
sean-k-mooneyyou were following https://docs.openstack.org/nova/train/install/controller-install-ubuntu.html#install-and-configure-components13:34
ygk_12345sean-k-mooney exactly. you spotted it :)13:35
sean-k-mooneythat has you restart the services at the end https://docs.openstack.org/nova/train/install/controller-install-ubuntu.html#finalize-installation implying they are running before you populate the db13:35
sean-k-mooneyi think that is wrong13:35
sean-k-mooneyreally it should have you install nova-manage only13:35
sean-k-mooneythen populate the db13:35
sean-k-mooneythen install the contoler services after13:36
*** hamalq has quit IRC13:36
sean-k-mooneyi think that is what caused your issue13:36
openstackgerritMamduh proposed openstack/os-vif stable/queens: Refactor code of linux_net to more cleaner and increase performace  https://review.opendev.org/c/openstack/os-vif/+/76594113:37
ygk_12345sean-k-mooney so i have deleted the dbs and created again13:37
ygk_12345sean-k-mooney i have created the dbs again now. Whats the next step ?13:38
sean-k-mooneyygk_12345: are all the nova services stopoed13:38
sean-k-mooney*stopped13:38
ygk_12345sean-k-mooney not on the compute node though. will it affect ?13:39
sean-k-mooneypossibly13:40
sean-k-mooneyalthough no it should not13:40
sean-k-mooneythe compute node only access the db via the conductor13:40
ygk_12345sean-k-mooney now shall I populate the dbs ?13:41
ygk_12345sean-k-mooney also the services are shown to be started in the end after the db stuff in that doc. so whats wrong with that ?13:42
sean-k-mooneyyep thats the next step13:42
sean-k-mooneystarting them at the end is ok13:42
sean-k-mooneybut they should not be started before the db is ready13:43
sean-k-mooneysteps 3-6 should be done before any of the serviecs are started13:43
ygk_12345@se13:44
ygk_12345sean-k-mooney oh ok got it. it went well this time. lets see further13:44
sean-k-mooneybefore you start the services check the db version13:45
ygk_12345oh ok13:46
sean-k-mooneyi think it shoudl be 40213:46
sean-k-mooneyhttps://github.com/openstack/nova/tree/stable/train/nova/db/sqlalchemy/migrate_repo/versions based on the last migration13:46
*** ociuhandu has joined #openstack-nova13:47
sean-k-mooneyif it gets to 402 then your good to go with starting the services13:47
ygk_12345yes its 402. awesome13:48
sean-k-mooneyfeel free to submit a patch to update the install guide with this and any other issue you find13:49
sean-k-mooneyas i said in this case it should have had you install just nova-manage then set up the db then install the other nova packages for the api conducort and scheduler13:50
sean-k-mooneyygk_12345: when you install the packages the post init script starts the service automatically which can cause issues as you found13:51
ygk_12345sean-k-mooney ok sure thanks for your time as well13:51
sean-k-mooneyno worries13:51
*** dave-mccowan has quit IRC13:55
*** rcernin has quit IRC13:58
*** _erlon_ has joined #openstack-nova14:01
*** ratailor_ has quit IRC14:02
*** mgariepy has quit IRC14:06
*** mgariepy has joined #openstack-nova14:06
*** ociuhandu has quit IRC14:08
*** ociuhandu has joined #openstack-nova14:08
gibilyarwood: sorry I was away, but I see you found the reasons14:28
*** efried has joined #openstack-nova14:28
*** mkrai has joined #openstack-nova14:37
*** nweinber has joined #openstack-nova14:39
*** mgariepy has quit IRC14:39
*** artom has joined #openstack-nova14:45
*** ygk_12345 has quit IRC14:50
*** mgariepy has joined #openstack-nova14:56
sean-k-mooneygibi: i suspect the conductor proably created the column not really sure what actully did it but yes14:59
sean-k-mooneygibi: i have seen that with devstack often enought to know that running the service before the db is ready end is weired issue like that14:59
gibisean-k-mooney: oh I referred to the aggregate question from lyarwood15:00
sean-k-mooneyoh ok15:00
sean-k-mooneysorry15:01
*** efried has quit IRC15:05
*** efried has joined #openstack-nova15:06
openstackgerritLance Bragstad proposed openstack/placement master: Pass context objects to oslo.policy directly  https://review.opendev.org/c/openstack/placement/+/76678515:12
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for resource providers  https://review.opendev.org/c/openstack/placement/+/76024015:12
*** bnemec has quit IRC15:25
*** martinkennelly has quit IRC15:25
*** bnemec has joined #openstack-nova15:26
*** bnemec has quit IRC15:27
*** hamalq has joined #openstack-nova15:32
*** macz_ has joined #openstack-nova15:37
*** macz_ has quit IRC15:37
*** hamalq has quit IRC15:37
*** macz_ has joined #openstack-nova15:38
openstackgerritJan Gutter proposed openstack/os-vif stable/train: DNM: testing lower-constraints  https://review.opendev.org/c/openstack/os-vif/+/76697515:41
*** hemanth_n has quit IRC15:43
*** mlavalle has joined #openstack-nova15:43
*** dklyle has joined #openstack-nova15:44
openstackgerritLee Yarwood proposed openstack/nova master: Add regression test for bug #1908075  https://review.opendev.org/c/openstack/nova/+/76697615:45
openstackbug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Undecided,New] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood)15:45
*** mkrai has quit IRC15:54
*** amodi has joined #openstack-nova15:56
*** mlavalle has quit IRC15:59
gibilyarwood: do you happen to know if all the issues we fixed in https://review.opendev.org/c/openstack/nova/+/765749 for stable/victoria needs to be backported to stable/ussuri? if yes I can do the cherry-pick16:01
*** mlavalle has joined #openstack-nova16:01
lyarwoodgibi: there are other issues to resolve on the older branches, I've been waiting to see if people wanted to drop the LC jobs16:03
gibilyarwood: ack, then I16:03
lyarwoodgibi: and for the bandit issue to be resolved16:03
gibithen I'm not pushing the backport now, the qos bug is targeted to ussuri16:04
melwittgibi: wondering why is this considered closed? this one is about the lack of cell targeting for lookup of the service record, not really about the lack of multi-cell support for min bw requests? https://bugs.launchpad.net/nova/+bug/190751116:09
openstackLaunchpad bug 1907511 in OpenStack Compute (nova) ussuri "nova-multi-cell fails on stable branches with: Failed to compute_task_migrate_server: 'NoneType' object has no attribute 'version'" [Undecided,New]16:09
melwitt*closed for victoria16:09
*** Guest5440 has joined #openstack-nova16:09
gibimelwitt: my thinking is to backport his fix to victoria instead https://review.opendev.org/c/openstack/nova/+/76692516:10
gibimelwitt:16:10
gibimelwitt: I considered https://bugs.launchpad.net/nova/+bug/1907511 as a gate blocker bug which we resolved with turning off the test16:11
openstackLaunchpad bug 1907511 in OpenStack Compute (nova) ussuri "nova-multi-cell fails on stable branches with: Failed to compute_task_migrate_server: 'NoneType' object has no attribute 'version'" [Undecided,New]16:11
melwittoh, I didn't see that16:11
gibibut I can reopen it and close it with https://review.opendev.org/c/openstack/nova/+/766925 if that make more sense16:11
gibiI just wanted to limit the number of bugs to track16:11
gibiI would keep https://bugs.launchpad.net/nova/+bug/1907522 open until https://review.opendev.org/c/openstack/nova/+/766925 is merged16:12
openstackLaunchpad bug 1907522 in OpenStack Compute (nova) "test_*_with_qos_min_bw_allocation fails in the nova-multi-cell job with: nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host <host>" [High,In progress] - Assigned to Balazs Gibizer (balazs-gibizer)16:12
melwittyeah, I see now. without context of the other approach in the bug, it didn't make sense to me16:12
gibimelwitt: I can add more context to the bug I closed16:12
*** Guest5440 is now known as redrobot16:12
melwittI am ok, just explaining why I asked. but it might help to add that info for reading back in history16:13
melwittI will abandon my patch16:13
*** ociuhandu has quit IRC16:13
gibisure16:13
gibiI will add it16:14
*** ociuhandu has joined #openstack-nova16:14
melwittcool thanks16:14
gibidone16:16
*** ociuhandu has quit IRC16:18
*** martinkennelly has joined #openstack-nova16:25
prometheanfiremelwitt: how goes the mock updates? :P16:26
melwittprometheanfire: will start on it maybe today, last week our gate was failing 100% so was focused on things related to that16:27
prometheanfireyep16:27
prometheanfireoffhand, is it possible to switch a flavor from public to private via the API?  I'm thinking no16:28
*** bnemec has joined #openstack-nova16:31
*** bnemec has quit IRC16:32
melwittprometheanfire: looks like no, can't switch flavor from public to private via the API16:32
gmannprometheanfire: melwitt: yeah, only modification allowed in flavor is description or add/remove tenant access.16:33
gmanngibi: melwitt lyarwood looks like placement l-c is also broken. https://review.opendev.org/c/openstack/placement/+/76439416:34
gmannany fix up otherwise i can work on that16:34
lyarwoodgmann: nothing from me, hack away :)16:35
gmannk16:35
gibigmann: same,16:35
melwittme either, feel free16:35
melwittthanks gmann16:35
gmannsure16:35
*** lbragstad_ has joined #openstack-nova16:38
*** ircuser-1 has joined #openstack-nova16:39
*** bnemec has joined #openstack-nova16:41
*** lbragstad has quit IRC16:41
openstackgerritGhanshyam proposed openstack/placement master: Fix l-c job for pyflakes version conflict  https://review.opendev.org/c/openstack/placement/+/76699416:47
*** ociuhandu has joined #openstack-nova16:48
*** ociuhandu has quit IRC17:00
*** ygk_12345 has joined #openstack-nova17:00
*** nweinber has quit IRC17:00
*** ociuhandu has joined #openstack-nova17:02
*** nweinber has joined #openstack-nova17:05
*** ygk_12345 has left #openstack-nova17:09
*** lbragstad__ has joined #openstack-nova17:13
*** lbragstad_ has quit IRC17:16
*** mgariepy has quit IRC17:19
*** hamalq has joined #openstack-nova17:30
*** hamalq_ has joined #openstack-nova17:35
*** hamalq has quit IRC17:38
gibistephenfin: regarding https://review.opendev.org/c/openstack/nova/+/765300 do you happen to know how to retry a unit test with all the setup and teardown executed properly?17:39
gibijust plugging a retry decorator to the test function will not execute teardown, cleanup and the setup again17:40
sean-k-mooneygibi: stephenfin  is on pto today but im not sure we have an example of that in our code bases17:40
sean-k-mooneyyou could do it manually i guess17:41
sean-k-mooneycall tear down and setup on failure excpet for the last execution17:41
sean-k-mooneyyou can acess them via the self argument passed to the inner funciton17:42
*** jangutter has quit IRC17:43
*** jangutter has joined #openstack-nova17:43
gibisean-k-mooney: my grep failed me to find that example. Yeah I can try calling all these manually I'm just affraid I will miss something that the unit test framework does17:44
sean-k-mooneyi know some other project like pip have a @flaky decorator17:45
*** johanssone has quit IRC17:45
sean-k-mooneythey call pypi in some of there unit tests17:45
sean-k-mooneyso the have to retry them ocationally17:45
*** johanssone has joined #openstack-nova17:47
gibiflaky is for nose a pytest :/17:48
*** ociuhandu has quit IRC17:51
gibiok I don't think I can make it work without digging too deep into the internals of the untit test framework. So I17:51
gibiSo I stop here17:52
gibihonestly I don't know what we can do with https://bugs.launchpad.net/nova/+bug/182325117:52
openstackLaunchpad bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed]17:52
*** jangutter has quit IRC17:52
*** jangutter has joined #openstack-nova17:53
sean-k-mooneygibi: what is it ment to be testing17:53
sean-k-mooneygibi: e.g. shoud we just remove the test and be done with it17:53
gibidb migrations as far as I understand17:53
sean-k-mooneybut what does walk version mean17:54
gibiversion by version do the db migrations17:54
sean-k-mooneyah ok17:54
sean-k-mooneydo we need to do that for mysql?17:54
sean-k-mooneyis sqlight enough17:54
gibiwe do it for sqlite mysql and postgres17:54
sean-k-mooneyyep because fo the opertunistic testing code17:55
sean-k-mooneybut im wondering if we should exclude it form that17:55
gibiI guess we added all these because we thought there are differences between the db engines regarding schema upgrade17:55
gibibut I don't have historical context to say more17:55
sean-k-mooneyshould that not be caught but the indivual upgrade check17:55
sean-k-mooneyhttps://github.com/openstack/nova/blob/7dcc4cfea7da2ef1163e0a65618cb784f5159c6d/nova/tests/unit/db/test_migrations.py#L253-L25417:56
sean-k-mooneyis the one that fails right17:56
*** rpittau is now known as rpittau|afk17:56
gibiall the walk_versions can fail17:57
gibiat least I saw failures both form the unit and the functional one17:57
sean-k-mooneyah ok so its not jsut for mysql17:57
gibiyes, the api_db is tested in the func env the cell db tested in the unit env17:57
gibinow I'm not sure I see other than mysql failures17:58
sean-k-mooneyright but is it only with mysql backend that if fails or with any db backend17:58
gibithe e-r query filters for mysql18:00
gibibut I think I did not see non-mysql failures18:00
sean-k-mooneyya i only recall mysql but that could just be my falut human memory18:01
*** ociuhandu has joined #openstack-nova18:01
sean-k-mooneywhich was why i was ondering would running it with sqlithg only be an option18:01
gibithe question is, do we lose test coverage?18:02
gibiI guess I stop here today. see all tomorrow o/18:04
*** ociuhandu has quit IRC18:05
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/tests/unit/db/test_migrations.py#L1062-L112818:05
sean-k-mooneyarguably yes but if it unrelayable its value is limited18:06
sean-k-mooneymaybe sqlacamy has a retry mechanium we can use18:06
sean-k-mooneybut ya enjoy your evning18:07
sean-k-mooneygibi: this might be of interest https://review.opendev.org/c/openstack/manila/+/291397/18:11
*** derekh has quit IRC18:13
sean-k-mooneygibi: it looks like we are getting a timeout exception18:16
sean-k-mooneygibi: so adding the different timeout for this like manilla does mught be the path forward18:16
*** eharney has quit IRC18:19
*** mgariepy has joined #openstack-nova18:20
*** tesseract has quit IRC18:21
*** lbragstad__ has quit IRC18:24
*** jangutter_ has joined #openstack-nova18:25
*** jangutter has quit IRC18:28
*** eharney has joined #openstack-nova18:31
*** dtantsur is now known as dtantsur|afk18:32
openstackgerritGhanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict  https://review.opendev.org/c/openstack/placement/+/76699418:39
*** jangutter_ has quit IRC18:40
*** jangutter has joined #openstack-nova18:41
*** jangutter has quit IRC18:41
*** eharney has quit IRC18:41
*** eharney has joined #openstack-nova18:41
*** jangutter has joined #openstack-nova18:42
*** lbragstad__ has joined #openstack-nova18:42
*** openstackgerrit has quit IRC18:50
*** openstackgerrit has joined #openstack-nova18:52
openstackgerritGhanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict  https://review.opendev.org/c/openstack/placement/+/76699418:52
*** gmann is now known as gmann_lunch18:53
*** gyee has joined #openstack-nova18:58
*** jangutter_ has joined #openstack-nova19:19
*** jangutter has quit IRC19:22
*** hoonetorg has quit IRC19:41
*** tbachman_ has joined #openstack-nova19:54
*** hoonetorg has joined #openstack-nova19:54
*** tbachman has quit IRC19:56
*** tbachman_ is now known as tbachman19:56
*** yonglihe has quit IRC20:03
*** viks____ has quit IRC20:03
*** arne_wiebalck has quit IRC20:03
*** yonglihe has joined #openstack-nova20:03
*** arne_wiebalck has joined #openstack-nova20:03
openstackgerritLance Bragstad proposed openstack/placement master: Bump oslo.log version to 4.3.0  https://review.opendev.org/c/openstack/placement/+/76022920:05
openstackgerritLance Bragstad proposed openstack/placement master: Pass context objects to oslo.policy directly  https://review.opendev.org/c/openstack/placement/+/76678520:05
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for resource providers  https://review.opendev.org/c/openstack/placement/+/76024020:05
*** gmann_lunch is now known as gmann20:08
*** jangutter has joined #openstack-nova20:11
*** vishalmanchanda has quit IRC20:12
*** jangutter_ has quit IRC20:14
*** andrewbonney has quit IRC20:17
*** raildo has quit IRC20:18
*** lbragstad__ has quit IRC20:19
*** raildo has joined #openstack-nova20:21
*** lbragstad has joined #openstack-nova20:22
JamesBensonsean-k-mooney:  FWIW, I deployed ubuntu 18 with kernel: 5.4.0-58-lowlatency, worked just fine. :-)20:33
sean-k-mooneythats good to hear. it appeared to be working when i did it but as i said i had fault ram so did not get to test it that way for long20:37
openstackgerritmelanie witt proposed openstack/nova master: Initialize global data separately and run_once in WSGI app init  https://review.opendev.org/c/openstack/nova/+/73362720:39
*** martinkennelly has quit IRC20:43
JamesBensonyeah, just did it, thought I would follow up with you. :-) Have a good one.20:44
*** martinkennelly has joined #openstack-nova20:45
JamesBensonHaven't done any performance testing between the kernels though, so don't know if there is any benefit.  My refstacks took the same amount of time though.20:46
sean-k-mooneyin thory if anything it should be slower but the performace should be more consitent espcially with regard to io latency20:47
JamesBensonthe lowlatency should be slower?20:48
sean-k-mooneyslower is too blunt20:48
sean-k-mooneyin theory the throughput of concurrent task may go down but the execution fo a singel item should complete faster20:49
sean-k-mooneythere is a trade of in general between latency and througput20:49
sean-k-mooneybout shoudl get more consistent performace too20:50
JamesBensonah, I see. Interesting20:50
sean-k-mooneyif you are not hevialy over subsribing the host i think it would generally be an improvment20:51
JamesBensonThanks for the info, always more to learn and further dig into things :-)20:52
sean-k-mooneyyep i was couious about that too and its still on my todo list to dig into in more detail20:52
*** ociuhandu has joined #openstack-nova21:00
*** ralonsoh has quit IRC21:04
*** ociuhandu has quit IRC21:05
*** jangutter has quit IRC21:11
openstackgerritsean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif  https://review.opendev.org/c/openstack/nova/+/60243221:22
sean-k-mooneymelwitt: ^ i have not had time to manually test that but i belvie that should resolve the grenade issues.21:23
openstackgerritGhanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict  https://review.opendev.org/c/openstack/placement/+/76699421:26
*** raildo has quit IRC21:29
*** rcernin has joined #openstack-nova21:30
melwittsean-k-mooney: ack thanks for the heads up21:31
sean-k-mooneyi left some comments in gerrit basically to explain the code i added to do the fallback. we will see what grenade thinks when it runs21:32
sean-k-mooneymelwitt: i shoudl proably an addtional unit test or two also so ill proably respin it tomorrow but if there is anything else you think i shoudl add let me know. im going to call it a day there o/21:34
melwittcool sean-k-mooney, will do. seeya later o/21:35
*** jmlowe has quit IRC21:43
*** jmlowe has joined #openstack-nova21:45
jawad_axdHi! Can someone push me on this one, with newly added compute . I can see in 'openstack compute service list' but not in 'openstack hypervisor list'.  This is nova-compute log http://paste.openstack.org/show/801032/21:47
openstackgerritGhanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict  https://review.opendev.org/c/openstack/placement/+/76699421:50
*** ralonsoh has joined #openstack-nova21:50
openstackgerritGhanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict  https://review.opendev.org/c/openstack/placement/+/76699422:03
*** avolkov has quit IRC22:09
openstackgerritGhanshyam proposed openstack/nova stable/rocky: DNM: testing gate  https://review.opendev.org/c/openstack/nova/+/76702722:39
*** nweinber has quit IRC22:46
*** spatel has joined #openstack-nova23:05
*** spatel has quit IRC23:09
*** martinkennelly has quit IRC23:17
*** k_mouza has joined #openstack-nova23:32
*** k_mouza has quit IRC23:34
*** JamesBenson has quit IRC23:34
*** k_mouza has joined #openstack-nova23:37
*** k_mouza has quit IRC23:45
« Sunday, 2020-12-13 Index Tuesday, 2020-12-15 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!