Thursday, 2021-07-22

opendevreviewmelanie witt proposed openstack/placement master: Add consumer_types migration, database and object changes  https://review.opendev.org/c/openstack/placement/+/66917001:34
opendevreviewmelanie witt proposed openstack/placement master: Microversion 1.37: API support for consumer types  https://review.opendev.org/c/openstack/placement/+/67944101:34
opendevreviewmelanie witt proposed openstack/placement master: Switch ConsumerType to use an AttributeCache  https://review.opendev.org/c/openstack/placement/+/67948601:34
melwittgibi: finally I have an update for consumer types https://review.opendev.org/c/openstack/placement/+/679441 sorry for the long delay01:40
opendevreviewMerged openstack/nova-specs master: Move placement specs to placement repo  https://review.opendev.org/c/openstack/nova-specs/+/80077507:15
gibimelwitt: OK, I've added it to my queue07:20
gibimelwitt: I have the RP re-parenting patch ready too https://review.opendev.org/c/openstack/placement/+/78402007:21
gibiif you have time07:22
*** rpittau|afk is now known as rpittau07:49
*** bhagyashris_ is now known as bhagyashris08:05
opendevreviewLee Yarwood proposed openstack/nova master: libvirt: Handle silent failures to extend volume within os-brick  https://review.opendev.org/c/openstack/nova/+/80171408:10
stephenfinlyarwood: Could you review a few outstanding docs changes for me today? https://review.opendev.org/c/openstack/nova/+/782410/109:12
lyarwoodstephenfin: sure thing09:13
stephenfinThanks. I addressed your comments in that one specifically too, gibi ^09:13
gibistephenfin: ack09:13
stephenfinbauzas: In case you didn't see it, I addressed your issues on https://review.opendev.org/c/openstack/nova/+/798145 with a follow-up https://review.opendev.org/c/openstack/nova/+/80152309:19
bauzasstephenfin: ok, will look09:19
bauzasand thanks09:19
opendevreviewSylvain Bauza proposed openstack/nova master: DNM (yet) : Expose the mdev class  https://review.opendev.org/c/openstack/nova/+/80174309:25
lyarwoodhttps://github.com/openstack/nova/blob/3545356ae3a719442833cb8c3c911408d4bd3c15/nova/conductor/manager.py#L166-L184 - This is likely a stupid idea but why aren't we logging anything here as breadcrumbs for RPC timeouts etc?09:26
lyarwoodHave an example downstream of a bdm.save() timing out on the compute and I've got no way of seeing what n-cond was doing AFAICT09:26
lyarwoodI guess any breadcrumbs need to land in the actual object methods being exercised09:28
sean-k-mooneyya i dont know really09:29
sean-k-mooneyi guess one question would be where09:29
sean-k-mooneyas in in what log would they end up09:30
sean-k-mooneythe api/conductor wont know where it actuly got too09:30
lyarwoodit should end up in n-cond right?09:30
lyarwoodalmost need a TRACE log level below DEBUG for stuff like this09:31
gibistephenfin: what should I do to enable ':oslo.config:option:' directive in the api-guide doc build? I've added 'oslo_config.sphinxext' extension to the config but it fails with https://paste.opendev.org/show/807644/09:34
*** bhagyashris_ is now known as bhagyashris09:35
lyarwoodnvm another n-cond log is full of `Too many heartbeats missed` errors from oslo.msg09:40
lyarwoodreally wish gerrit had a `move the unified diff to the middle of your giant ass screen` option10:10
stephenfingibi: You'd need to render the config options somewhere like we do here https://raw.githubusercontent.com/openstack/nova/master/doc/source/configuration/config.rst10:12
stephenfin(the '.. show-options::' directive)10:13
stephenfinwhich we probably don't want to do. We could build something that relies on intersphinx to link back to the main docs but that doesn't exist yet10:13
opendevreviewStephen Finucane proposed openstack/nova master: neutron: Remove references to dead 'qos-queue' extension  https://review.opendev.org/c/openstack/nova/+/75444810:43
stephenfingibi: turns out that extension was used by the VMWare NSX neutron plugin, but they dropped it about two years ago. Have updated the commit message to state as much ^10:44
sean-k-mooneystephenfin: nice find11:35
sean-k-mooneystephenfin: im kind of surpised we have any third party exstions reference in nova11:36
sean-k-mooneyat least on the common code. i would have expect those constance to live in teh vmware dirver11:37
sean-k-mooneydo we know if vnic-index is used still11:37
opendevreviewStephen Finucane proposed openstack/nova master: db: Final cleanups  https://review.opendev.org/c/openstack/nova/+/80048411:44
stephenfinsean-k-mooney: vnic-index?11:45
stephenfinoh, wait, yeah11:45
stephenfinhmm, probably not actually11:45
gibistephenfin: re sphinx: OK, then I stop fighting with sphinx :)11:45
stephenfinsean-k-mooney: actually no, it's still there https://opendev.org/x/vmware-nsx/src/branch/master/vmware_nsx/extensions/vnicindex.py11:46
stephenfinwhether it actually works is a different thing of course11:46
gibistephenfin: I'm +2 on the QoS extension removal11:47
stephenfinhurrah11:48
stephenfinthanks gibi++11:48
gibistephenfin: I'm fixing your comments in the pps series I hope I can push it today11:48
gibimoving around the func test was a nice exercise :)11:49
* gibi loves sweedish summer time when there is no internal meetings and no interrupts11:49
stephenfinyeah, it felt like the right thing to do11:50
gibiyepp, it is much better now. and pycharm is glad not to reparse the whole test_servers.py when I edit something in the func tests11:51
stephenfinfair to say that 5kloc+ files are a thing best avoided where possible. 5kloc+ *test* files are 100% worth avoiding :)11:51
stephenfin \o/11:51
gibiI hope not just me but jetbrains using the nova codebase to performance test pycharm :D 11:52
stephenfinthere a few projects that could benefit from using nova as a test bed11:53
gibi:D11:53
stephenfingitea for one11:53
gibiwe should promote that as a goal of nova :D being big and complicated python codebase for "you" to test your python tooling on11:54
stephenfindon't know if you clicked through the links on the nsx-mh removal patch but it basically crashed my browser11:54
stephenfinGitHub (smartly) refuses to render large files by default11:54
stephenfinlol11:54
gibimine is not crashed (yet)11:55
opendevreviewStephen Finucane proposed openstack/nova master: tox: Encode specific Python versions  https://review.opendev.org/c/openstack/nova/+/79705412:12
sean-k-mooneyanyone ohter then stephenfin  have time to reivew an os-vif patch https://review.opendev.org/c/openstack/os-vif/+/79805512:39
*** elodilles is now known as elodilles_afk12:48
gibisean-k-mooney: done12:54
gibiincoming...12:56
sean-k-mooneythanks im finsihing up some testing downstream this week ill be working on some other os-vif patches next week and ill start with a followup for the nits12:56
sean-k-mooneypps series?12:56
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] create pps resource on OVS agent RP  https://review.opendev.org/c/openstack/nova/+/78720512:56
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] move port creation to the NeutronFixture  https://review.opendev.org/c/openstack/nova/+/78720612:56
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] neutron fixture for extended resource request  https://review.opendev.org/c/openstack/nova/+/79430612:56
opendevreviewBalazs Gibizer proposed openstack/nova master: Detect extended_resource_request neutron API extension  https://review.opendev.org/c/openstack/nova/+/79361812:56
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] move port resource request tests  https://review.opendev.org/c/openstack/nova/+/80181512:56
gibisean-k-mooney: yepp, pps :D12:58
opendevreviewBalazs Gibizer proposed openstack/nova master: Reject server create with extended resource req  https://review.opendev.org/c/openstack/nova/+/79361913:00
opendevreviewBalazs Gibizer proposed openstack/nova master: Reject server operations with extended resource req  https://review.opendev.org/c/openstack/nova/+/79362013:00
opendevreviewBalazs Gibizer proposed openstack/nova master: Add same_subtree field to RequestLevelParams  https://review.opendev.org/c/openstack/nova/+/79150313:04
opendevreviewBalazs Gibizer proposed openstack/nova master: Bump min placement microversion to 1.36  https://review.opendev.org/c/openstack/nova/+/79150413:05
opendevreviewBalazs Gibizer proposed openstack/nova master: Support same_subtree in allocation_canadidate query  https://review.opendev.org/c/openstack/nova/+/79150513:05
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] refactor assertPortMatchesAllocation  https://review.opendev.org/c/openstack/nova/+/79245813:05
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] refactor asserts in qos tests  https://review.opendev.org/c/openstack/nova/+/79893013:09
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] ports with both bw and pps resources  https://review.opendev.org/c/openstack/nova/+/79239413:10
opendevreviewBalazs Gibizer proposed openstack/nova master: Parse extended resource request from the port data  https://review.opendev.org/c/openstack/nova/+/80008513:12
opendevreviewBalazs Gibizer proposed openstack/nova master: Transfer RequestLevelParams from ports to scheduling  https://review.opendev.org/c/openstack/nova/+/79150613:13
opendevreviewBalazs Gibizer proposed openstack/nova master: Support move ops with extended resource request  https://review.opendev.org/c/openstack/nova/+/80008713:14
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] refactor interface attach with qos  https://review.opendev.org/c/openstack/nova/+/80008813:16
opendevreviewBalazs Gibizer proposed openstack/nova master: Support interaface attach / detach with new resource request format  https://review.opendev.org/c/openstack/nova/+/80008913:17
bauzasstephenfin: awesome ! thanks for the FUP <313:18
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] move unshelve test to the proper place  https://review.opendev.org/c/openstack/nova/+/79362113:18
gibiand I f*ck up the rebase at some point and lost a patch in the series :/13:22
sean-k-mooneyhopefully you have it localy13:26
gibigit reflog has it13:27
gibiI basically merged two subseqent commit together 13:28
gibiin the series13:28
gibiduring the rebase13:28
gibiI hate that git does not warn against commit --amend when during interactive rebase it stops at the commit both because it marked as edit and becuase it has a merge conflict13:38
belmoreiraHello everyone... I have a question regarding DBs... but also related with nova :)13:39
sean-k-mooneywell thats becasue you can do commit --amend for valid reasons13:39
sean-k-mooneywith an interactive rebase13:39
sean-k-mooneybut yes i have done that too13:39
gibisean-k-mooney: yeah, but not during a conflict resolution I guess13:39
sean-k-mooneybelmoreira: nova does have them13:39
sean-k-mooneygibi: i have used it where i was not finsihed with the full resolution13:39
belmoreirasean-k-mooney :)13:40
sean-k-mooneythen came back13:40
belmoreiraWe will need to move some nova DBs from mysql 5.7.x to 8.x. Our DB team was testing this DB migration and a character convertion is required (utf8mb3 -> utf8mb4)13:40
belmoreiraApparently utf8 in mysql 8.x uses 4 bytes instead of 3 bytes (like previously)13:40
sean-k-mooneybelmoreira: nova currently does not supeciric any charicterset or coalation type in our schema13:40
sean-k-mooneybelmoreira: utf8 in gereal uses between 1 an 4 bytes depending on the charater13:41
belmoreirasean-k-mooney right13:41
belmoreiramy concern is that they reported that (silent) changes of data types can happen, for example from TEXT to MEDIUMTEXT to allow the column to accommodate the original data using 4bytes for each character instead of 3. For similar reasons we could also potentially hit the innodb index key max size...13:42
belmoreiraIs there something that you are aware? Or this should be safe for Nova13:43
sean-k-mooneyi think nova largely should not care but im not sure. 13:44
sean-k-mooneynova likely should jsut use utf_8_bin13:44
sean-k-mooneyat some point13:44
sean-k-mooneyhttps://en.wikipedia.org/wiki/UTF-8#MySQL_utf8mb313:45
belmoreirathanks sean-k-mooney13:46
sean-k-mooneywe do use mediumetext in somplaces13:47
sean-k-mooneyin general nova does not allow you to change the column data types form what we have in the schema13:48
sean-k-mooneybtu the encoding i dont think will break anything13:48
sean-k-mooneybelmoreira: are they plannign to actully modify the types in the tables13:49
sean-k-mooneywe generally are allowing sqlalchmy to mange the filed types13:50
belmoreiraI'm afraid to just discover some of these changes in prod. Anyway I will try to test it as much as possible and I will let you know if I discover something13:50
belmoreirasean-k-mooney no they will not explicitly change the schema but they say it can happen when moving to utf8mb413:51
sean-k-mooneywell text files unlike nvarcars are not actully stored in the db correct13:51
belmoreiraI'm learning about all of this utf8 encoding :)13:51
sean-k-mooneythey are stored as sepereate files13:51
opendevreviewBalazs Gibizer proposed openstack/nova master: Transfer RequestLevelParams from ports to scheduling  https://review.opendev.org/c/openstack/nova/+/79150613:51
opendevreviewBalazs Gibizer proposed openstack/nova master: Support boot with extended resource request  https://review.opendev.org/c/openstack/nova/+/80008613:51
opendevreviewBalazs Gibizer proposed openstack/nova master: Support move ops with extended resource request  https://review.opendev.org/c/openstack/nova/+/80008713:52
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] refactor interface attach with qos  https://review.opendev.org/c/openstack/nova/+/80008813:52
sean-k-mooneywe use string https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/models.py#L256 when we wnat fixed lenght nvarchar columns13:52
belmoreiraI think the best is to do the conversion of a test DB and I will go manually through the schema. Maybe at the end there is nothing about this13:53
sean-k-mooneyya13:54
sean-k-mooneybut ya just confirm that text filds are stored off table https://www.navicat.com/en/company/aboutus/blog/1308-choosing-between-varchar-and-text-in-mysql13:55
sean-k-mooneybelmoreira: where the issue might happen13:55
sean-k-mooneywould be for user-data13:56
sean-k-mooneytext field cans sture up to 64k fo data13:56
belmoreirasean-k-mooney thanks a lot. I will let you know if something comes up13:57
sean-k-mooneyuser-data is defiened to be medium text already https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/models.py#L28813:57
opendevreviewBalazs Gibizer proposed openstack/nova master: Support interaface attach / detach with new resource request format  https://review.opendev.org/c/openstack/nova/+/80008913:57
sean-k-mooneyso it should not need to be upgraded but ya let us know13:57
opendevreviewBalazs Gibizer proposed openstack/nova master: [func test] move unshelve test to the proper place  https://review.opendev.org/c/openstack/nova/+/79362113:58
sean-k-mooneyMediumText can store 16 MB but we limit the user-data to 64k i belive in the api so it should fit thats the only filed that is likely to get close to 64k i think so hopefully it will just work out13:58
gibistephenfin:I've fixed your comments in the pps series14:04
gibiI think we have a gate block due to https://zuul.opendev.org/t/openstack/builds?job_name=openstacksdk-functional-devstack&project=openstack/nova14:07
gibicoudl be due to https://review.opendev.org/c/openstack/cinder/+/79229914:08
gibimoving to cinder ...14:09
lyarwoodYeah you should be able to drop the v2 tests from osc now on master14:14
lyarwoodthe sdk even sorry14:15
gibilyarwood: hm, good point, these are v2 tests14:16
gibithen I'm going to delete them ...14:16
*** elodilles_afk is now known as elodilles14:34
gibilyarwood: fyi, here is the removal of those tests https://review.opendev.org/c/openstack/openstacksdk/+/80186414:38
sean-k-mooneygibi: how did the v2 removal impact the nova gate by the way15:00
sean-k-mooneyhttps://review.opendev.org/c/openstack/openstacksdk/+/801864 should not affefct us right15:01
melwittgibi: thanks! and ack, will check the re-parenting15:03
gibisean-k-mooney: we gate on openstacksdk-functional-devstack job that started failing as it runs against a devstack and trying to find block storage v215:03
lyarwoodsean-k-mooney: I don't think it did15:04
lyarwoodsean-k-mooney: all of our cinderv2 stuff goes through cinderv3 now anyway15:04
sean-k-mooneygibi: oh it is15:06
lyarwoodgibi: re osc, sorry I'm not trying to be difficult but I think we can just exclude these tests for now 15:14
gibilyarwood: sure no worries. I #openstack-sdks gtema said he will inventigate a better way to fix this so I defer to him 15:17
lyarwoodgibi: ah sorry I'm not in that channel, sounds good :)15:17
gibiyeah it is in good hands now :)15:17
gibi1215:26
gibiups15:26
gibimelwitt: re consumer_types: so you created one transaction that containst both the ensure_consumer and the Allocation.replace_all calls 15:26
gibimelwitt: with a @db_api.placement_context_manager.writer decorator on the top. 15:27
melwittgibi: yes, put both calls into a helper function decorated with the database context manager (this initiates the session/transaction)15:27
melwittit took me a long time to arrive at that because previously I was looking at the wrong code path for a different "set allocations" path that was already a single transaction /o\15:28
gibimelwitt: and later on when the ensure_consumer or the Allocation.replace_all calls the db_api.placement_context_manager.writer again then it won't start a separate transaction as the new, bigger one, is already started. Is it how the nested db decorator works?15:28
melwittgibi: right, it will check whether there is a session already created and if not it creates a new one (which will be a new transaction). if it finds a session already there, it will use that and that's how the nested things get connected. the session is an attribute on the RequestContext15:32
gibimelwitt: awesome, thanks for the explanation15:33
melwittso you put the decorator on the outermost function with RequestContext as the first argument to group things into same transaction15:33
gibicool, clever solution15:35
melwittI couldn't move past the "why is it not a single transaction" because from all the code comments around setting allocations, it should have been. so I was going to keep looking until I found out why 😆15:36
opendevreviewMerged openstack/nova master: docs: Clarify host-model, host-passthrough differences  https://review.opendev.org/c/openstack/nova/+/78241015:36
gibi:)15:37
gmanngibi: lyarwood for sdk https://review.opendev.org/c/openstack/openstacksdk/+/80187215:42
melwitteasy review to correct a reno https://review.opendev.org/c/openstack/nova/+/801144 if anyone can take a look. has a +2 from stephenfin already15:42
opendevreviewSamuel proposed openstack/nova master: Add REST and compute APIs to migrate instances between projects  https://review.opendev.org/c/openstack/nova/+/80137715:43
melwittI was able to consult a SME on telemetry and they let me know telemetry doesn't call the audit log API, it uses only the notifications15:44
lyarwoodgmann: cool thanks15:44
gibimelwitt: I left 3 small comment in the consumer_type impl but over all it looks good https://review.opendev.org/c/openstack/placement/+/67944115:55
gibigmann: will check soon15:55
gibimelwitt: approved the reno update. thanks for that!15:56
melwittgibi: thanks for your patience on this one! and one of your comments looks familiar, apologies if it's one I missed in previous PS's15:56
melwittI will address them15:57
gibimelwitt: no worries. 15:58
*** rpittau is now known as rpittau|afk16:02
opendevreviewMerged openstack/nova master: Correct reno for archiving task_log database records  https://review.opendev.org/c/openstack/nova/+/80114416:19
opendevreviewStephen Finucane proposed openstack/nova master: Move 'hw:pmu', 'hw_pmu' parsing to nova.virt.hardware  https://review.opendev.org/c/openstack/nova/+/79236416:43
opendevreviewStephen Finucane proposed openstack/nova master: api: Rename 'parameter_types.hostname' -> 'fqdn'  https://review.opendev.org/c/openstack/nova/+/77854916:48
opendevreviewStephen Finucane proposed openstack/nova master: api: Add support for 'hostname' parameter  https://review.opendev.org/c/openstack/nova/+/77855016:48
opendevreviewStephen Finucane proposed openstack/nova master: objects: Don't use generic 'Field' container  https://review.opendev.org/c/openstack/nova/+/73823916:57
opendevreviewStephen Finucane proposed openstack/nova master: objects: Remove unnecessary type aliases, exceptions  https://review.opendev.org/c/openstack/nova/+/73824016:57
opendevreviewStephen Finucane proposed openstack/nova master: objects: Use imports instead of type aliases  https://review.opendev.org/c/openstack/nova/+/73801816:57
opendevreviewStephen Finucane proposed openstack/nova master: objects: Remove wrappers around ovo mixins  https://review.opendev.org/c/openstack/nova/+/73801916:57
opendevreviewStephen Finucane proposed openstack/nova master: WIP: add ovo-mypy-plugin to type hinting o.vos  https://review.opendev.org/c/openstack/nova/+/75885116:57
opendevreviewStephen Finucane proposed openstack/nova master: docs: Further tweaks to the CPU models document  https://review.opendev.org/c/openstack/nova/+/78406616:59
opendevreviewmelanie witt proposed openstack/placement master: Microversion 1.37: API support for consumer types  https://review.opendev.org/c/openstack/placement/+/67944117:36
opendevreviewmelanie witt proposed openstack/placement master: Switch ConsumerType to use an AttributeCache  https://review.opendev.org/c/openstack/placement/+/67948617:36
opendevreviewmelanie witt proposed openstack/placement master: Microversion 1.37: API support for consumer types  https://review.opendev.org/c/openstack/placement/+/67944118:21
opendevreviewmelanie witt proposed openstack/placement master: Switch ConsumerType to use an AttributeCache  https://review.opendev.org/c/openstack/placement/+/67948618:21
opendevreviewmelanie witt proposed openstack/nova master: WIP Add force kwarg to delete_allocation_for_instance  https://review.opendev.org/c/openstack/nova/+/68880221:22

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!