openstackgerrit | Ananth Narayan S proposed openstack/valence-specs master: [WIP] Add task queueing to address scalability https://review.openstack.org/474099 | 08:22 |
---|---|---|
openstackgerrit | Raghavendra proposed openstack/valence master: [WIP] Task Manager and task queue updated api part to support RPC mechanism Change-Id: I5bc741575e153b16c98fe7351facec3a3b5f87f7 https://review.openstack.org/474068 | 12:35 |
*** akhiljain32 has joined #openstack-valence | 14:21 | |
*** ntpttr_laptop has joined #openstack-valence | 14:22 | |
*** ramineni_ has joined #openstack-valence | 14:26 | |
*** hubian has joined #openstack-valence | 14:27 | |
hubian | Hello | 14:29 |
ntpttr_laptop | o/ | 14:30 |
hubian | :) | 14:30 |
ramineni_ | Hi All | 14:30 |
lin_yang | Hello everyone | 14:30 |
ntpttr_laptop | looks like maybe ananth isn't here yet, but he sent out an email for agenda right? so he'll likely be here | 14:31 |
hubian | yeah , there is an agenda | 14:32 |
lin_yang | yep, I believe so, let's wait three mins | 14:32 |
mkrai | Hello everyone | 14:34 |
ntpttr_laptop | hi mkrai | 14:34 |
*** ananth_n has joined #openstack-valence | 14:35 | |
ananth_n | hi all | 14:35 |
hubian | here ananth comes ~ | 14:35 |
ntpttr_laptop | hi ananth_n | 14:35 |
hubian | :) | 14:35 |
lin_yang | hi mkrai and ananth_n | 14:35 |
ananth_n | sorry, got delayed abit | 14:35 |
ntpttr_laptop | looks like we have enough folks to get started on the agenda :) | 14:35 |
hubian | np | 14:35 |
ntpttr_laptop | ananth_n: do you want to chair the meeting? | 14:36 |
ananth_n | you can go ahead :) | 14:36 |
ntpttr_laptop | okay, looks like the first item is a discussions on the device management spec | 14:37 |
ntpttr_laptop | we've got a pretty full agenda, so let's try not to go to deep into the rabbit hole for these | 14:37 |
ntpttr_laptop | ramineni_ and I have been talking back and forth on this spec about a couple things, has anyone looked at it? | 14:37 |
ananth_n | i want to, but haven't had the time. i did skim over to understand what it talks aobout | 14:38 |
ananth_n | *about | 14:38 |
ananth_n | but that's about it | 14:38 |
ntpttr_laptop | https://review.openstack.org/#/c/471322/ | 14:38 |
hubian | please show the link , i want to see | 14:38 |
lin_yang | sorry, didn't get chance to read it | 14:38 |
*** mrittika has joined #openstack-valence | 14:38 | |
ananth_n | i will try to make time tomorrow and post my comments | 14:38 |
ntpttr_laptop | a couple things we're talking about - should we have endpoints in the API for compute/storage/network and then another one for additional devices called pci_devices or something? | 14:38 |
ntpttr_laptop | or should all of those be encapsulated in one pooled API | 14:39 |
ramineni_ | right | 14:39 |
ananth_n | i would prefer the former. but let me read the spec. if it makes a strong case for the latter, i'd be open to it | 14:39 |
ntpttr_laptop | I'm sort of of the opinion that it might be helpful to have separate API endpoints for compute network and storage, and then have another API for other devices | 14:39 |
ramineni_ | ntpttr_laptop: my concern is it would be hard to categorize, and cause more confusion | 14:40 |
mrittika | hi. redfish has pooled apis | 14:40 |
ntpttr_laptop | mrittika: the redfish APIs were actually sort of confusing to me. There's the "services" API for storage services, but it looks like new NVMe storage can be found under Chassis/Drives? | 14:41 |
mkrai | I think seperate APIs is always better | 14:41 |
ananth_n | @ramineni could you explain | 14:41 |
lin_yang | ntpttr_laptop: what't the other kind of devices besides compute/network/storage, FPGA? | 14:41 |
ntpttr_laptop | ramineni_: why do you think it'd be hard to categorize? Which devices do you think would cause trouble? | 14:41 |
ananth_n | lin_yang fpga, pcie, nvme... | 14:41 |
ntpttr_laptop | ananth_n: would nvme not be considered storage? | 14:42 |
ramineni_ | if we expose 2 endpoints, pci devices and storage , nvm should be logically under both | 14:42 |
mrittika | we should not need APIs for devices in valence.. refer to this link http://redfish.dmtf.org/schemas/ | 14:42 |
ntpttr_laptop | that's true. | 14:42 |
ntpttr_laptop | mrittika: but that link contains an endpoint called pci_devices | 14:43 |
ntpttr_laptop | PCIeDevice more specificalley | 14:43 |
mrittika | ntpttr_laptop: and what would you want to add | 14:44 |
ramineni_ | mrittika: you are saying having pooled devices which can attached dynamically to compose node is not possible via redfish | 14:44 |
ramineni_ | like on demand storage .. or gpu for additional computing power etc | 14:44 |
ntpttr_laptop | mrittika: just an endpoint in valence for managing and attaching them | 14:44 |
ramineni_ | ? | 14:44 |
mrittika | ok got it. for both ntpttr_laptop and ramineni_ 's points | 14:45 |
ntpttr_laptop | so I've been trying to work on updating the storage API and I think ramineni_'s point has been what's thrown me for a loop - where to put NVMe. Because it's not under the storage services endpoint in redfish | 14:45 |
mrittika | ramineni_: right now dynamic addition is not there | 14:46 |
ntpttr_laptop | maybe we should actually just put it in a pooled resources endpoint of type storage | 14:46 |
mrittika | but can be proposed to redfish composition service | 14:46 |
ntpttr_laptop | mrittika: it is actually, for pci devices | 14:46 |
ntpttr_laptop | mrittika: there's an API in podm/redfish for attaching NVMe drives to already composed nodes | 14:46 |
ramineni_ | i think chester mentioned sometime it would be added in 2.1 .. | 14:46 |
ntpttr_laptop | I was looking through the 2.1 specs and found it | 14:46 |
ramineni_ | im not very sure about redfish though | 14:47 |
mrittika | ok is it in PODM or redfish? | 14:47 |
mrittika | ok let's take an action item to find out | 14:47 |
ntpttr_laptop | might be podm only, but shouldn't we support it then? | 14:47 |
mrittika | ananth_n: what other items did we have for agenda | 14:47 |
ntpttr_laptop | https://etherpad.openstack.org/p/Valence | 14:48 |
mrittika | ntpttr_laptop: lets put a bp | 14:48 |
ananth_n | we have a packed agenda | 14:48 |
ntpttr_laptop | sounds good | 14:48 |
ananth_n | ntpttr_laptop: is chairing | 14:48 |
ntpttr_laptop | yes, next is multipodm | 14:48 |
mrittika | oh cool! | 14:48 |
ntpttr_laptop | I think we've gotten some good headway on that topic over email | 14:48 |
ntpttr_laptop | is there much to discuss here? | 14:48 |
mrittika | yes | 14:48 |
ramineni_ | mrittika: hope we could finalize , i would like this feature in valence, | 14:49 |
ntpttr_laptop | hubian: anything you'd like to? or we can go to the next item | 14:49 |
hubian | Yeah , just need finialize some thing | 14:49 |
ntpttr_laptop | sure | 14:49 |
ananth_n | just one thing, between python redfish and sushy, both would implement redfish spec which could be behind PODM specs | 14:49 |
hubian | I saw the next items is about the redfish lib | 14:49 |
lin_yang | for multiple-podm, my spec for scheduler https://review.openstack.org/#/c/474053/ | 14:49 |
ntpttr_laptop | thanks lin_yang I'll check it out | 14:49 |
ntpttr_laptop | redfish lib is later in the agenda, we'll get to that | 14:50 |
hubian | OK | 14:50 |
ntpttr_laptop | okay so next was going to be pooled resource management, we sort of covered that with device management | 14:50 |
ntpttr_laptop | we can continue the discussion on ramineni_ spec - please check it out when you get time | 14:50 |
hubian | And for thta , i wll seperate my patch into two small patch , to make this whole work more solid ~ | 14:50 |
ntpttr_laptop | sounds good hubian :) | 14:51 |
ramineni_ | ntpttr_laptop: right, thanks :) | 14:51 |
ntpttr_laptop | that brings us to patches in ened of review | 14:51 |
ntpttr_laptop | if anyone has any patches they'd like us to review please link them | 14:51 |
ntpttr_laptop | though there' not usually that many at the same time, and reviewers can see all patches in need of review on gerrit :) | 14:52 |
ananth_n | there are a few pending for a while. so i added that item | 14:52 |
ntpttr_laptop | sounds good ananth_n, let's all take some time to check them out | 14:52 |
ananth_n | yes, please link them. +1 ntpttr_laptop | 14:52 |
lin_yang | recently we got many new specs | 14:52 |
ntpttr_laptop | no need to link here, if you sort reviews based on project you'll see all patches in need of review. I'll put together a link that organizes all patches based on our projects and sorts them based on what needs reviewing and email it out, okay? | 14:53 |
ntpttr_laptop | I think that will help out our review process a lot | 14:53 |
ananth_n | https://review.openstack.org/#/q/project:openstack/valence | 14:54 |
ananth_n | :) | 14:54 |
lin_yang | sound good, thanks ntpttr_laptop | 14:54 |
ntpttr_laptop | ananth_n: here this might be better! | 14:54 |
ntpttr_laptop | https://review.openstack.org/#/dashboard/?foreach=project:^openstack/.*valence.* status:open NOT owner:self NOT label:Workflow<=-1 label:Verified>=1 NOT label:Code-Review<=-1,self NOT label:Code-Review>=1,self&title=Valence Review Inbox&Valence Specs=project:openstack/manila-specs&Bug Fixes=topic:^bug/.*&Needs Feedback (Changes older than 5 days that have not been reviewed by anyone)=NOT label:Code-Review<=2 age:5d&You are a | 14:54 |
ntpttr_laptop | reviewer, but haven't voted in the current revision=reviewer:self&Needs final 2=label:Code-Review>=2 limit:50&New Contributors=reviewer:10068&Passed Jenkins, No Negative Feedback=NOT label:Code-Review>=2 NOT label:Code-Review<=-1 limit:50&Wayward Changes (Changes with no code review in the last 2days)=NOT label:Code-Review<=2 age:2d | 14:54 |
ananth_n | ntpttr_laptop: +1 :) thanks | 14:54 |
ntpttr_laptop | oops link too long for IRC | 14:54 |
ntpttr_laptop | I'll send it over email | 14:54 |
ntpttr_laptop | it groups things nicely, AND includes both valence and valence-specs | 14:54 |
ntpttr_laptop | okay - next on the agenda is valenceclient patches. | 14:55 |
ntpttr_laptop | Shuquan sent out that email saying they were breaking them down and were going to submit next week right? | 14:55 |
mrittika | yes | 14:55 |
lin_yang | shuquan mentioned they will submit the rest of them to gerrrit | 14:56 |
ntpttr_laptop | I think we can wait to see what they've got and then take some ARs to test it out | 14:56 |
ananth_n | i agree | 14:56 |
ntpttr_laptop | we can move on for now I think | 14:56 |
lin_yang | they need to somehow refactor the one that used on demo | 14:57 |
ntpttr_laptop | that makes sense | 14:57 |
ntpttr_laptop | hubian that brings us to sushy/python-redfish :) | 14:57 |
hubian | +1 | 14:57 |
hubian | ok | 14:57 |
ananth_n | we can add client discussion to the agenda in 2 weeks time, which gives a week for review | 14:57 |
ntpttr_laptop | ananth_n: sounds good | 14:57 |
mrittika | +1 for Inherit/reuse sushy/python-redfish in valence redfish module | 14:58 |
ananth_n | i guess we can move to the next topic then | 14:58 |
ntpttr_laptop | so which one would we use, sushy or python-redfish? | 14:58 |
hubian | yeah , that's the key point ~ | 14:58 |
lin_yang | I think most of us will agree to reuse the redfish library, this is why I added this item in agenda | 14:58 |
ananth_n | (sorry, some lag in my connection) | 14:58 |
lin_yang | the problem here is which one, sushy or python-redfish? | 14:59 |
hubian | I need your guys give me input , since they both can do help to valence | 14:59 |
mrittika | sushy | 14:59 |
ananth_n | i would suggest sushy. if it has prefered by Ironic, then the chances are it will be maintained longer | 14:59 |
ntpttr_laptop | I agree mrittika | 14:59 |
ananth_n | and be more up-to-date | 14:59 |
ntpttr_laptop | I think sushy makes more sense | 14:59 |
ntpttr_laptop | and it's not as complicated as python-redfish, it keeps things simpler | 15:00 |
mkrai | +1 for sushy | 15:00 |
hubian | ok , seems sushy wins ~ | 15:00 |
hubian | :) | 15:00 |
lin_yang | I don't have strong feeling right now, most likely we need to work on it to add more features | 15:00 |
ananth_n | my only concern is what i said earlier - PODM's APIs would likely be ahead of what sushy supports. | 15:00 |
mrittika | lin_yang: we can talk about it | 15:01 |
ntpttr_laptop | ananth_n: +1 | 15:01 |
ananth_n | that's because it takes time for standards bodies to agree & accept | 15:01 |
ntpttr_laptop | it does sort of make us dependent on another project's review and submission process | 15:01 |
hubian | yeah ~ more things need to do after the descision | 15:01 |
ntpttr_laptop | which could hinder our own development | 15:01 |
mrittika | ananth_n: we want to improve sushy | 15:02 |
lin_yang | if my understanding right, the node composition is out of standard redfish api | 15:02 |
mrittika | we can submit bps for sushy | 15:02 |
ntpttr_laptop | mrittika: yes but we don't have power over what gets merged | 15:02 |
hubian | improve sushy ??? you mean we may need to contribute to sushy ? | 15:02 |
ntpttr_laptop | so it could create blockers | 15:02 |
lin_yang | we can try to push it to sushy. If the maintainer don't like it, we need to keep this in valence redfish module | 15:03 |
mrittika | maybe we develop to our redfish driver but have a integration with sushy too, like every 2 weeks | 15:03 |
ananth_n | mrittika: I agree. but when we depend on sushy, we need to wait for patches to be accepted into it, before we can use it | 15:03 |
hubian | So , i think we just improve our own useage of "redfish" feature | 15:03 |
lin_yang | ntpttr_laptop: agree, we might see pushback from maintainer, but we can try it at first | 15:03 |
ntpttr_laptop | sure, and we can always implement our own stuff to make up for gaps if we need | 15:04 |
ananth_n | would forking sushy make sense? | 15:04 |
ntpttr_laptop | ananth_n: I don't think we should go that route really | 15:04 |
lin_yang | ananth_n: if necessary, we can keep a sushy branch | 15:04 |
hubian | walk two lines , but valence's own usage is our first focus | 15:04 |
ntpttr_laptop | hubian: yeah | 15:04 |
lin_yang | then push patches back to sushy | 15:04 |
hubian | yeah | 15:05 |
mkrai | Its always preferable to keep things on master | 15:05 |
ananth_n | ntpttr_laptop: if we fork, update sushy for Valence needs, and then submit those back to sushy? | 15:05 |
hubian | it is better sushy could accept , but if not we also not blocked by that thing | 15:05 |
ntpttr_laptop | hmm, i don't know, it is usually better to just try to submit to master first | 15:06 |
ntpttr_laptop | if we're going to submit it later anyways - but I can see if the maintainer is adamantly against something needing a fork | 15:06 |
ananth_n | would the maintainers be ok with a valence branch of sushy :D | 15:07 |
mrittika | we need to get the sushy layer owner involved in Valence requirements | 15:07 |
ntpttr_laptop | ananth_n: interesting though there | 15:07 |
ntpttr_laptop | thought* | 15:07 |
ananth_n | valence team submits to valence branch and raise a pull request, which can merge after review. that way, valence branch has the features valence requires | 15:08 |
ntpttr_laptop | that would be good if that's osmething they allow | 15:09 |
ntpttr_laptop | for sure | 15:09 |
lin_yang | ananth_n: +1 we can discuss it with sushy maintainer | 15:09 |
mkrai | But first let's talk about the direct integration | 15:09 |
*** akhiljain32 has quit IRC | 15:09 | |
ntpttr_laptop | mkrai: +1 | 15:10 |
lin_yang | mkrai: +1 the master is first target | 15:10 |
ntpttr_laptop | though I think they were hesitant to add a few things like composition | 15:10 |
ntpttr_laptop | maybe that's changed | 15:10 |
ntpttr_laptop | okay next item, resource DB entries | 15:10 |
ntpttr_laptop | I was going over ramineni_'s spec, and trying to rework the storage patch that I had out, and I created a DB entry for storage resources | 15:11 |
hubian | So , we finialized using shshy !!! | 15:11 |
hubian | Thanks ~ | 15:11 |
ntpttr_laptop | hubian: yeah for sure | 15:11 |
hubian | I'll make in asap ~ | 15:11 |
hubian | :) | 15:11 |
ntpttr_laptop | but the DB entry was basically just a pointer - it contains the type of resource, the podm that it exists in, and a link to it like /redfish/v1/Services/1/Drives/1 or something | 15:12 |
ntpttr_laptop | so I was thinking maybe we should just have one 'resource' db entry for things like that rather than a different one for each different kind of resource, does that make sense? | 15:12 |
ntpttr_laptop | since they're mostly going to be pointers to where to find them in the podm anyways | 15:13 |
ramineni_ | +1 from me for single resource DB | 15:13 |
ntpttr_laptop | yeah just wanted to ask what you guys think | 15:13 |
mkrai | ntpttr_laptop: You mean to have only one table for all resources? | 15:13 |
lin_yang | ntpttr_laptop: who will use this table | 15:14 |
ramineni_ | all pooled resources | 15:14 |
ntpttr_laptop | mkrai: well, we'd have DB entries for podm and flavor and valence things | 15:14 |
ntpttr_laptop | and then all resources in the podm could be in one table, or we could split htem up | 15:14 |
ntpttr_laptop | but they're basically all going to be the same, pointers to the podm object, and when a get request happens it uses the pointer in the DB to make the request to the pod manager | 15:15 |
hubian | eh , i'm confusing about How can one table cover different resoures's item. Is it convient for the logic coding ? | 15:15 |
ntpttr_laptop | so we're not storing a bunch of info in the db | 15:15 |
ramineni_ | may be spec makes things more clear | 15:15 |
ntpttr_laptop | hubian: we're not actually storing info about the resources in the DB beyond just the link to it | 15:15 |
ntpttr_laptop | hubian: because things might change, like health of the item, etc. And those changes would update our DB | 15:15 |
mkrai | ntpttr_laptop: ramineni_ There could be some valence related information related to these resources, let's say label for example so we need to store those informations as well in valence | 15:16 |
ntpttr_laptop | so we have to make calls out to the pod manager anyways | 15:16 |
ntpttr_laptop | mkrai: yes | 15:16 |
ntpttr_laptop | our DB is just for valence things like that, and a link to get the podm info | 15:16 |
mkrai | ntpttr_laptop: In that case we would want to have different table for each resources | 15:16 |
mkrai | ntpttr_laptop: Right | 15:16 |
ntpttr_laptop | mkrai: why? what would differ from table to table? | 15:16 |
ntpttr_laptop | would one row specifying 'type' be enough distinction? | 15:17 |
hubian | ntpttr_laptop: agree with your thought of updating DB Frequently | 15:17 |
mkrai | I am not sure about each resource now but I think that case will not be rare | 15:17 |
ntpttr_laptop | mkrai: not sure what you mean | 15:18 |
mkrai | for example, a flavor is related to node but not to network | 15:18 |
ramineni_ | mkrai: it stores same info for each resource | 15:18 |
ntpttr_laptop | mkrai: a flavor isn't related to a node though | 15:18 |
ramineni_ | mkrai: here resource is device | 15:18 |
mkrai | So here we want to store flavor id with nodes detail in the node table | 15:18 |
ntpttr_laptop | we're going to keep the flavor DB, we don't need to mark the flavor id of a node that's been composed already | 15:18 |
mkrai | ramineni_: And what about nodes, storages ? | 15:18 |
ntpttr_laptop | mkrai: all we need are pointers to those things | 15:19 |
ntpttr_laptop | and keep track of what type they are in the db | 15:19 |
mkrai | If there's no specific info to any of these resources and only link, then I am ok with one table | 15:20 |
ntpttr_laptop | yeah, if we can come up with useful info to separte them by in the valence DB i'm all for it | 15:20 |
mkrai | Ok sounds good | 15:21 |
mrittika | have we digressed from the agenda | 15:21 |
ntpttr_laptop | mrittika: no we haven't, we're still on item 6 :) | 15:21 |
ntpttr_laptop | sorry i mean item 5 | 15:21 |
mrittika | ok :) | 15:21 |
ntpttr_laptop | that brings us to the last item, which is open discussion | 15:21 |
ntpttr_laptop | does anyone have anything else they'd like to discuss? | 15:22 |
mrittika | task manager and queue manager ananth? | 15:22 |
ananth_n | WIP :) i'm working on the spec | 15:23 |
hubian | ntpttr_laptop : eh ~ I'm sorry that could you send us an email to share more details of your thoughts for the "one table DB" ? | 15:23 |
ntpttr_laptop | cool will take a look when done ananth_n | 15:23 |
ntpttr_laptop | hubian: yes for sure, and I'll try and write out examples | 15:23 |
hubian | ok , Thanks ~ | 15:23 |
hubian | :) | 15:24 |
ntpttr_laptop | mrittika ananth_n: Oh I did have one question regarding the RPC stuff we talked about last week | 15:24 |
ntpttr_laptop | you just intended to have one controller service separate from the API that gets RPC messages right, not a bunch of separate ones for compute/storage/network? | 15:24 |
ntpttr_laptop | other openstack projects tend to just have the one controller service, having more than that would get heavier than we need | 15:25 |
ananth_n | the code is also WIP, but we have something working. a PoC which is submitted | 15:25 |
ntpttr_laptop | ananth_n: oh a PoC where? | 15:26 |
mkrai | In the PoC there is only one RPC service like a conductor | 15:26 |
ananth_n | i meant poc implementation :) code is on gerrit. Raghavendra has submitted | 15:26 |
mrittika | ntpttr_laptop: we can de-prioritize rpc stuff for now | 15:26 |
ntpttr_laptop | cool | 15:26 |
ntpttr_laptop | okay I'll take a look, I think I've actually seen that patch | 15:26 |
ntpttr_laptop | anything else guys? | 15:27 |
ananth_n | the patch is failing some jenkins checks. but since it is a poc patch, we might abandon it anyways for an improved implementation | 15:28 |
ananth_n | ntpttr_laptop: no. i don't have anything els | 15:28 |
ananth_n | else | 15:28 |
ntpttr_laptop | ah yeah that might be why I didn't look at it too long, jenkins | 15:28 |
ntpttr_laptop | going once | 15:28 |
ntpttr_laptop | going twice | 15:28 |
ntpttr_laptop | all right thanks all, good meeting :) | 15:28 |
ramineni_ | Thanks All, Bye | 15:28 |
mrittika | bye everyone! | 15:28 |
*** ramineni_ has left #openstack-valence | 15:28 | |
ntpttr_laptop | bye! | 15:28 |
mrittika | exit | 15:28 |
mkrai | Bye everyone! | 15:28 |
lin_yang | thanks all bye | 15:28 |
*** mrittika has quit IRC | 15:29 | |
hubian | Thanks all | 15:29 |
hubian | bye everyone , see you next week ~ | 15:29 |
*** hubian has quit IRC | 15:29 | |
*** ananth_n has quit IRC | 15:30 | |
*** ntpttr_laptop has quit IRC | 16:37 | |
*** ntpttr_laptop has joined #openstack-valence | 17:10 | |
*** ntpttr_laptop has quit IRC | 17:11 | |
*** openstackgerrit has quit IRC | 20:03 | |
*** openstackgerrit has joined #openstack-valence | 21:11 | |
openstackgerrit | Nate Potter proposed openstack/valence-specs master: Add unified resource database https://review.openstack.org/476272 | 21:11 |
openstackgerrit | Nate Potter proposed openstack/valence-specs master: Add unified resource database https://review.openstack.org/476272 | 21:39 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!