Wednesday, 2017-06-21

openstackgerritAnanth Narayan S proposed openstack/valence-specs master: [WIP] Add task queueing to address scalability  https://review.openstack.org/47409908:22
openstackgerritRaghavendra proposed openstack/valence master: [WIP] Task Manager and task queue updated api part to support RPC mechanism Change-Id: I5bc741575e153b16c98fe7351facec3a3b5f87f7  https://review.openstack.org/47406812:35
*** akhiljain32 has joined #openstack-valence14:21
*** ntpttr_laptop has joined #openstack-valence14:22
*** ramineni_ has joined #openstack-valence14:26
*** hubian has joined #openstack-valence14:27
hubianHello14:29
ntpttr_laptopo/14:30
hubian:)14:30
ramineni_Hi All14:30
lin_yangHello everyone14:30
ntpttr_laptoplooks like maybe ananth isn't here yet, but he sent out an email for agenda right? so he'll likely be here14:31
hubianyeah , there is an agenda14:32
lin_yangyep, I believe so, let's wait three mins14:32
mkraiHello everyone14:34
ntpttr_laptophi mkrai14:34
*** ananth_n has joined #openstack-valence14:35
ananth_nhi all14:35
hubianhere ananth comes ~14:35
ntpttr_laptophi ananth_n14:35
hubian:)14:35
lin_yanghi mkrai and ananth_n14:35
ananth_nsorry, got delayed abit14:35
ntpttr_laptoplooks like we have enough folks to get started on the agenda :)14:35
hubiannp14:35
ntpttr_laptopananth_n: do you want to chair the meeting?14:36
ananth_nyou can go ahead :)14:36
ntpttr_laptopokay, looks like the first item is a discussions on the device management spec14:37
ntpttr_laptopwe've got a pretty full agenda, so let's try not to go to deep into the rabbit hole for these14:37
ntpttr_laptopramineni_ and I have been talking back and forth on this spec about a couple things, has anyone looked at it?14:37
ananth_ni want to, but haven't had the time. i did skim over to understand what it talks aobout14:38
ananth_n*about14:38
ananth_nbut that's about it14:38
ntpttr_laptophttps://review.openstack.org/#/c/471322/14:38
hubianplease show the link , i want to see14:38
lin_yangsorry, didn't get chance to read it14:38
*** mrittika has joined #openstack-valence14:38
ananth_ni will try to make time tomorrow and post my comments14:38
ntpttr_laptopa couple things we're talking about - should we have endpoints in the API for compute/storage/network and then another one for additional devices called pci_devices or something?14:38
ntpttr_laptopor should all of those be encapsulated in one pooled API14:39
ramineni_right14:39
ananth_ni would prefer the former. but let me read the spec. if it makes a strong case for the latter, i'd be open to it14:39
ntpttr_laptopI'm sort of of the opinion that it might be helpful to have separate API endpoints for compute network and storage, and then have another API for other devices14:39
ramineni_ntpttr_laptop: my concern is it would be hard to categorize, and cause more confusion14:40
mrittikahi. redfish has pooled apis14:40
ntpttr_laptopmrittika: the redfish APIs were actually sort of confusing to me. There's the "services" API for storage services, but it looks like new NVMe storage can be found under Chassis/Drives?14:41
mkraiI think seperate APIs is always better14:41
ananth_n@ramineni could you explain14:41
lin_yangntpttr_laptop: what't the other kind of devices besides compute/network/storage, FPGA?14:41
ntpttr_laptopramineni_: why do you think it'd be hard to categorize? Which devices do you think would cause trouble?14:41
ananth_nlin_yang fpga, pcie, nvme...14:41
ntpttr_laptopananth_n: would nvme not be considered storage?14:42
ramineni_if we expose 2 endpoints, pci devices and storage , nvm should be logically under both14:42
mrittikawe should  not need APIs for devices in valence.. refer to this link http://redfish.dmtf.org/schemas/14:42
ntpttr_laptopthat's true.14:42
ntpttr_laptopmrittika: but that link contains an endpoint called pci_devices14:43
ntpttr_laptopPCIeDevice more specificalley14:43
mrittikantpttr_laptop: and what would you want to add14:44
ramineni_mrittika: you are saying having pooled devices which can attached dynamically to compose node is not possible via redfish14:44
ramineni_like on demand storage .. or gpu for additional computing power etc14:44
ntpttr_laptopmrittika: just an endpoint in valence for managing and attaching them14:44
ramineni_?14:44
mrittikaok got it. for both ntpttr_laptop and ramineni_ 's points14:45
ntpttr_laptopso I've been trying to work on updating the storage API and I think ramineni_'s point has been what's thrown me for a loop - where to put NVMe. Because it's not under the storage services endpoint in redfish14:45
mrittikaramineni_: right now dynamic addition is not there14:46
ntpttr_laptopmaybe we should actually just put it in a pooled resources endpoint of type storage14:46
mrittikabut can be proposed to redfish composition service14:46
ntpttr_laptopmrittika: it is actually, for pci devices14:46
ntpttr_laptopmrittika: there's an API in podm/redfish for attaching NVMe drives to already composed nodes14:46
ramineni_i think chester mentioned sometime it would be added in 2.1 ..14:46
ntpttr_laptopI was looking through the 2.1 specs and found it14:46
ramineni_im not very sure about redfish though14:47
mrittikaok is it in PODM or redfish?14:47
mrittikaok let's take an action item to find out14:47
ntpttr_laptopmight be podm only, but shouldn't we support it then?14:47
mrittikaananth_n: what other items did we have for agenda14:47
ntpttr_laptophttps://etherpad.openstack.org/p/Valence14:48
mrittikantpttr_laptop: lets put a bp14:48
ananth_nwe have a packed agenda14:48
ntpttr_laptopsounds good14:48
ananth_nntpttr_laptop: is chairing14:48
ntpttr_laptopyes, next is multipodm14:48
mrittikaoh cool!14:48
ntpttr_laptopI think we've gotten some good headway on that topic over email14:48
ntpttr_laptopis there much to discuss here?14:48
mrittikayes14:48
ramineni_mrittika: hope we could finalize , i would like this feature in valence,14:49
ntpttr_laptophubian: anything you'd like to? or we can go to the next item14:49
hubianYeah , just need finialize some thing14:49
ntpttr_laptopsure14:49
ananth_njust one thing, between python redfish and sushy, both would implement redfish spec which could be behind PODM specs14:49
hubianI saw the next items is about the redfish lib14:49
lin_yangfor multiple-podm, my spec for scheduler https://review.openstack.org/#/c/474053/14:49
ntpttr_laptopthanks lin_yang I'll check it out14:49
ntpttr_laptopredfish lib is later in the agenda, we'll get to that14:50
hubianOK14:50
ntpttr_laptopokay so next was going to be pooled resource management, we sort of covered that with device management14:50
ntpttr_laptopwe can continue the discussion on ramineni_ spec - please check it out when you get time14:50
hubianAnd for thta , i wll seperate my patch into two small patch , to make this whole work more solid ~14:50
ntpttr_laptopsounds good hubian :)14:51
ramineni_ntpttr_laptop: right, thanks :)14:51
ntpttr_laptopthat brings us to patches in ened of review14:51
ntpttr_laptopif anyone has any patches they'd like us to review please link them14:51
ntpttr_laptopthough there' not usually that many at the same time, and reviewers can see all patches in need of review on gerrit :)14:52
ananth_nthere are a few pending for  a while. so i added that item14:52
ntpttr_laptopsounds good ananth_n, let's all take some time to check them out14:52
ananth_nyes, please link them. +1 ntpttr_laptop14:52
lin_yangrecently we got many new specs14:52
ntpttr_laptopno need to link here, if you sort reviews based on project you'll see all patches in need of review. I'll put together a link that organizes all patches based on our projects and sorts them based on what needs reviewing and email it out, okay?14:53
ntpttr_laptopI think that will help out our review process a lot14:53
ananth_nhttps://review.openstack.org/#/q/project:openstack/valence14:54
ananth_n:)14:54
lin_yangsound good, thanks ntpttr_laptop14:54
ntpttr_laptopananth_n: here this might be better!14:54
ntpttr_laptophttps://review.openstack.org/#/dashboard/?foreach=project:^openstack/.*valence.* status:open NOT owner:self NOT label:Workflow<=-1 label:Verified>=1 NOT label:Code-Review<=-1,self NOT label:Code-Review>=1,self&title=Valence Review Inbox&Valence Specs=project:openstack/manila-specs&Bug Fixes=topic:^bug/.*&Needs Feedback (Changes older than 5 days that have not been reviewed by anyone)=NOT label:Code-Review<=2 age:5d&You are a14:54
ntpttr_laptopreviewer, but haven't voted in the current revision=reviewer:self&Needs final  2=label:Code-Review>=2 limit:50&New Contributors=reviewer:10068&Passed Jenkins, No Negative Feedback=NOT label:Code-Review>=2 NOT label:Code-Review<=-1 limit:50&Wayward Changes (Changes with no code review in the last 2days)=NOT label:Code-Review<=2 age:2d14:54
ananth_nntpttr_laptop: +1 :) thanks14:54
ntpttr_laptopoops link too long for IRC14:54
ntpttr_laptopI'll send it over email14:54
ntpttr_laptopit groups things nicely, AND includes both valence and valence-specs14:54
ntpttr_laptopokay - next on the agenda is valenceclient patches.14:55
ntpttr_laptopShuquan sent out that email saying they were breaking them down and were going to submit next week right?14:55
mrittikayes14:55
lin_yangshuquan mentioned they will submit the rest of them to gerrrit14:56
ntpttr_laptopI think we can wait to see what they've got and then take some ARs to test it out14:56
ananth_ni agree14:56
ntpttr_laptopwe can move on for now I think14:56
lin_yangthey need to somehow refactor the one that used on demo14:57
ntpttr_laptopthat makes sense14:57
ntpttr_laptophubian that brings us to sushy/python-redfish :)14:57
hubian+114:57
hubianok14:57
ananth_nwe can add client discussion to the agenda in 2 weeks time, which gives a week for review14:57
ntpttr_laptopananth_n: sounds good14:57
mrittika+1 for Inherit/reuse sushy/python-redfish in valence redfish module14:58
ananth_ni guess we can move to the next topic then14:58
ntpttr_laptopso which one would we use, sushy or python-redfish?14:58
hubianyeah , that's the key point ~14:58
lin_yangI think most of us will agree to reuse the redfish library, this is why I added this item in agenda14:58
ananth_n(sorry, some lag in my connection)14:58
lin_yangthe problem here is which one, sushy or python-redfish?14:59
hubianI need your guys give me input , since they both can do help to valence14:59
mrittikasushy14:59
ananth_ni would suggest sushy. if it has prefered by Ironic, then the chances are it will be maintained longer14:59
ntpttr_laptopI agree mrittika14:59
ananth_nand be more up-to-date14:59
ntpttr_laptopI think sushy makes more sense14:59
ntpttr_laptopand it's not as complicated as python-redfish, it keeps things simpler15:00
mkrai+1 for sushy15:00
hubianok , seems sushy wins  ~15:00
hubian:)15:00
lin_yangI don't have strong feeling right now, most likely we need to work on it to add more features15:00
ananth_nmy only concern is what i said earlier - PODM's APIs would likely be ahead of what sushy supports.15:00
mrittikalin_yang: we can talk about it15:01
ntpttr_laptopananth_n: +115:01
ananth_nthat's because it takes time for standards bodies to agree & accept15:01
ntpttr_laptopit does sort of make us dependent on another project's review and submission process15:01
hubianyeah ~ more things need to do after the descision15:01
ntpttr_laptopwhich could hinder our own development15:01
mrittikaananth_n: we want to improve sushy15:02
lin_yangif my understanding right, the node composition is out of standard redfish api15:02
mrittikawe can submit bps for sushy15:02
ntpttr_laptopmrittika: yes but we don't have power over what gets merged15:02
hubianimprove sushy ??? you mean we may need to contribute to sushy ?15:02
ntpttr_laptopso it could create blockers15:02
lin_yangwe can try to push it to sushy. If the maintainer don't like it, we need to keep this in valence redfish module15:03
mrittikamaybe we develop to our redfish driver but have a integration with sushy too, like every 2 weeks15:03
ananth_nmrittika: I agree. but when we depend on sushy, we need to wait for patches to be accepted into it, before we can use it15:03
hubianSo , i think we just improve our own useage of "redfish" feature15:03
lin_yangntpttr_laptop: agree, we might see pushback from maintainer, but we can try it at first15:03
ntpttr_laptopsure, and we can always implement our own stuff to make up for gaps if we need15:04
ananth_nwould forking sushy make sense?15:04
ntpttr_laptopananth_n: I don't think we should go that route really15:04
lin_yangananth_n: if necessary, we can keep a sushy branch15:04
hubianwalk two lines , but valence's own usage is our first focus15:04
ntpttr_laptophubian: yeah15:04
lin_yangthen push patches back to sushy15:04
hubianyeah15:05
mkraiIts always preferable to keep things on master15:05
ananth_nntpttr_laptop: if we fork, update sushy for Valence needs, and then submit those back to sushy?15:05
hubianit is better sushy could accept , but if not we also not blocked by that thing15:05
ntpttr_laptophmm, i don't know, it is usually better to just try to submit to master first15:06
ntpttr_laptopif we're going to submit it later anyways - but I can see if the maintainer is adamantly against something needing a fork15:06
ananth_nwould the maintainers be ok with a valence branch of sushy :D15:07
mrittikawe need to get the sushy layer owner involved in Valence requirements15:07
ntpttr_laptopananth_n: interesting though there15:07
ntpttr_laptopthought*15:07
ananth_nvalence team submits to valence branch and raise a pull request, which can merge after review. that way, valence branch has the features valence requires15:08
ntpttr_laptopthat would be good if that's osmething they allow15:09
ntpttr_laptopfor sure15:09
lin_yangananth_n: +1 we can discuss it with sushy maintainer15:09
mkraiBut first let's talk about the direct integration15:09
*** akhiljain32 has quit IRC15:09
ntpttr_laptopmkrai: +115:10
lin_yangmkrai: +1 the master is first target15:10
ntpttr_laptopthough I think they were hesitant to add a few things like composition15:10
ntpttr_laptopmaybe that's changed15:10
ntpttr_laptopokay next item, resource DB entries15:10
ntpttr_laptopI was going over ramineni_'s spec, and trying to rework the storage patch that I had out, and I created a DB entry for storage resources15:11
hubianSo , we finialized using shshy  !!!15:11
hubianThanks ~15:11
ntpttr_laptophubian: yeah for sure15:11
hubianI'll make in asap ~15:11
hubian:)15:11
ntpttr_laptopbut the DB entry was basically just a pointer - it contains the type of resource, the podm that it exists in, and a link to it like /redfish/v1/Services/1/Drives/1 or something15:12
ntpttr_laptopso I was thinking maybe we should just have one 'resource' db entry for things like that rather than a different one for each different kind of resource, does that make sense?15:12
ntpttr_laptopsince they're mostly going to be pointers to where to find them in the podm anyways15:13
ramineni_+1 from me for single resource DB15:13
ntpttr_laptopyeah just wanted to ask what you guys think15:13
mkraintpttr_laptop: You mean to have only one table for all resources?15:13
lin_yangntpttr_laptop: who will use this table15:14
ramineni_all pooled resources15:14
ntpttr_laptopmkrai: well, we'd have DB entries for podm and flavor and valence things15:14
ntpttr_laptopand then all resources in the podm could be in one table, or we could split htem up15:14
ntpttr_laptopbut they're basically all going to be the same, pointers to the podm object, and when a get request happens it uses the pointer in the DB to make the request to the pod manager15:15
hubianeh , i'm confusing about How can one table cover different resoures's item. Is it convient for the logic coding ?15:15
ntpttr_laptopso we're not storing a bunch of info in the db15:15
ramineni_may be spec makes things more clear15:15
ntpttr_laptophubian: we're not actually storing info about the resources in the DB beyond just the link to it15:15
ntpttr_laptophubian: because things might change, like health of the item, etc. And those changes would update our DB15:15
mkraintpttr_laptop: ramineni_ There could be some valence related information related to these resources, let's say label for example so we need to store those informations as well in valence15:16
ntpttr_laptopso we have to make calls out to the pod manager anyways15:16
ntpttr_laptopmkrai: yes15:16
ntpttr_laptopour DB is just for valence things like that, and a link to get the podm info15:16
mkraintpttr_laptop: In that case we would want to have different table for each resources15:16
mkraintpttr_laptop: Right15:16
ntpttr_laptopmkrai: why? what would differ from table to table?15:16
ntpttr_laptopwould one row specifying 'type' be enough distinction?15:17
hubianntpttr_laptop: agree with your thought of updating DB Frequently15:17
mkraiI am not sure about each resource now but I think that case will not be rare15:17
ntpttr_laptopmkrai: not sure what you mean15:18
mkraifor example, a flavor is related to node but not to network15:18
ramineni_mkrai: it stores same info for each resource15:18
ntpttr_laptopmkrai: a flavor isn't related to a node though15:18
ramineni_mkrai: here resource is device15:18
mkraiSo here we want to store flavor id with nodes detail in the node table15:18
ntpttr_laptopwe're going to keep the flavor DB, we don't need to mark the flavor id of a node that's been composed already15:18
mkrairamineni_: And what about nodes, storages ?15:18
ntpttr_laptopmkrai: all we need are pointers to those things15:19
ntpttr_laptopand keep track of what type they are in the db15:19
mkraiIf there's no specific info to any of these resources and only link, then I am ok with one table15:20
ntpttr_laptopyeah, if we can come up with useful info to separte them by in the valence DB i'm all for it15:20
mkraiOk sounds good15:21
mrittikahave we digressed from the agenda15:21
ntpttr_laptopmrittika: no we haven't, we're still on item 6 :)15:21
ntpttr_laptopsorry i mean item 515:21
mrittikaok :)15:21
ntpttr_laptopthat brings us to the last item, which is open discussion15:21
ntpttr_laptopdoes anyone have anything else they'd like to discuss?15:22
mrittikatask manager and queue manager ananth?15:22
ananth_nWIP :) i'm working on the spec15:23
hubianntpttr_laptop : eh ~ I'm sorry that could you send us an email to share more details of your thoughts for the "one table DB" ?15:23
ntpttr_laptopcool will take a look when done ananth_n15:23
ntpttr_laptophubian: yes for sure, and I'll try and write out examples15:23
hubianok , Thanks ~15:23
hubian:)15:24
ntpttr_laptopmrittika ananth_n: Oh I did have one question regarding the RPC stuff we talked about last week15:24
ntpttr_laptopyou just intended to have one controller service separate from the API that gets RPC messages right, not a bunch of separate ones for compute/storage/network?15:24
ntpttr_laptopother openstack projects tend to just have the one controller service, having more than that would get heavier than we need15:25
ananth_nthe code is also WIP, but we have something working. a PoC which is submitted15:25
ntpttr_laptopananth_n: oh a PoC where?15:26
mkraiIn the PoC there is only one RPC service like a conductor15:26
ananth_ni meant poc implementation :) code is on gerrit. Raghavendra has submitted15:26
mrittikantpttr_laptop: we can de-prioritize rpc stuff for now15:26
ntpttr_laptopcool15:26
ntpttr_laptopokay I'll take a look, I think I've actually seen that patch15:26
ntpttr_laptopanything else guys?15:27
ananth_nthe patch is failing some jenkins checks. but since it is a poc patch, we might abandon it anyways for an improved implementation15:28
ananth_nntpttr_laptop: no. i don't have anything els15:28
ananth_nelse15:28
ntpttr_laptopah yeah that might be why I didn't look at it too long, jenkins15:28
ntpttr_laptopgoing once15:28
ntpttr_laptopgoing twice15:28
ntpttr_laptopall right thanks all, good meeting :)15:28
ramineni_Thanks All, Bye15:28
mrittikabye everyone!15:28
*** ramineni_ has left #openstack-valence15:28
ntpttr_laptopbye!15:28
mrittikaexit15:28
mkraiBye everyone!15:28
lin_yangthanks all bye15:28
*** mrittika has quit IRC15:29
hubianThanks all15:29
hubianbye everyone , see you next week  ~15:29
*** hubian has quit IRC15:29
*** ananth_n has quit IRC15:30
*** ntpttr_laptop has quit IRC16:37
*** ntpttr_laptop has joined #openstack-valence17:10
*** ntpttr_laptop has quit IRC17:11
*** openstackgerrit has quit IRC20:03
*** openstackgerrit has joined #openstack-valence21:11
openstackgerritNate Potter proposed openstack/valence-specs master: Add unified resource database  https://review.openstack.org/47627221:11
openstackgerritNate Potter proposed openstack/valence-specs master: Add unified resource database  https://review.openstack.org/47627221:39

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!