| *** yamamot__ has joined #openstack-dragonflow | 01:41 | |
| openstackgerrit | Merged openstack/dragonflow: Add manual deployment guide https://review.openstack.org/413987 | 01:47 |
|---|---|---|
| openstackgerrit | Merged openstack/dragonflow: Increase the number of retries for db-lock https://review.openstack.org/413352 | 01:47 |
| *** gongysh has joined #openstack-dragonflow | 02:05 | |
| *** gongysh has quit IRC | 02:20 | |
| openstackgerrit | hujie proposed openstack/dragonflow: check valid resource version for nb_api https://review.openstack.org/413120 | 02:38 |
| *** gongysh has joined #openstack-dragonflow | 03:41 | |
| openstackgerrit | Hong Hui Xiao proposed openstack/dragonflow: Rename tunnel_type in Chassis to tunnel_types https://review.openstack.org/415087 | 04:00 |
| *** yamamot__ has quit IRC | 05:26 | |
| *** yamamot__ has joined #openstack-dragonflow | 06:09 | |
| *** gongysh has quit IRC | 06:23 | |
| *** yamamot__ has quit IRC | 06:50 | |
| openstackgerrit | hujie proposed openstack/dragonflow: separate control plane and data plane ip address https://review.openstack.org/415116 | 06:57 |
| *** yamamot__ has joined #openstack-dragonflow | 07:00 | |
| *** yuval has joined #openstack-dragonflow | 07:18 | |
| *** gongysh has joined #openstack-dragonflow | 07:37 | |
| openstackgerrit | Omer Anson proposed openstack/dragonflow: Add legacy sNAT rule to l3_proactive_app https://review.openstack.org/413993 | 07:42 |
| openstackgerrit | Merged openstack/dragonflow: neutron-lib: use L3 constant from neutron-lib https://review.openstack.org/414970 | 07:48 |
| oanson | Yo | 08:30 |
| dimak | Hello | 08:30 |
| oanson | xiaohhui, hujie, irenab, you guys here? | 08:30 |
| irenab | hi | 08:30 |
| * xiaohhui raises his hand | 08:30 | |
| oanson | o/^~ | 08:31 |
| dimak | lihi promised to join us as well | 08:31 |
| oanson | That was supposed to be a towel, but it didn't come out right :) | 08:31 |
| oanson | yamamot__, are you joining us for the NB refactor discussion? | 08:32 |
| dimak | Maybe you should install the emoji keyboard 😂 | 08:32 |
| lihi | Hi | 08:32 |
| oanson | All right. So the spec was updated here: https://review.openstack.org/#/c/410298/ | 08:32 |
| oanson | dimak, if something can't be done in ascii, it's not worth doing | 08:32 |
| oanson | Actually, I take that back | 08:32 |
| oanson | dimak also uploaded some example code here: https://review.openstack.org/#/c/414984/2/dragonflow/db/models3.py | 08:33 |
| dimak | The actual models down below are just for presentation | 08:34 |
| dimak | All the names are suggestions :) | 08:34 |
| irenab | dimak: models can be defined each in its own module, right? | 08:35 |
| dimak | irenab, yes, we'll provide decorators to register them | 08:35 |
| oanson | irenab, I think that's a requirement | 08:35 |
| dimak | and to extend the provided classes to include the required features | 08:36 |
| irenab | I think probably we need one for based model (L2 mainly) and then there will be separate modules per services | 08:36 |
| dimak | irenab, I agree | 08:36 |
| oanson | Sounds good | 08:37 |
| oanson | Then we will also have to add the configuration based registration mechanism | 08:37 |
| dimak | We have to see if we can get jsonmodels accepted as requirement | 08:37 |
| dimak | Or do we have to find something else | 08:37 |
| oanson | dimak, you said you couldn't find any alternative that isnt' db-backed, right? | 08:38 |
| irenab | The proposal looks reasonable to me, expecially from the service dev perspective. Adding new model seems quite simple. RefWrapper /Ref is a bit tricky though | 08:38 |
| oanson | That means it's either that, or writing something ourselves. | 08:38 |
| dimak | oanson, I did not find anything of the kind | 08:38 |
| oanson | If that's the case (and we need to verify it), we may prefer to take over the project (if it's abandoned) rather than start anew | 08:38 |
| oanson | dimak, could you explain a bit more about the RefWrapper/Ref design? | 08:39 |
| dimak | Ok yes | 08:40 |
| dimak | There are 2 parts to this | 08:40 |
| dimak | Names might not be 100% great yet | 08:40 |
| dimak | Ref wrapper is quite simple. | 08:40 |
| dimak | Its a proxy to an object | 08:40 |
| dimak | if we request its ID, it returns it | 08:41 |
| dimak | other requests go to a backing object | 08:41 |
| dimak | The way you use it is by specifying the model, and the id of the object | 08:41 |
| dimak | Now ref is a model field, that wraps a "foreign key" field in the json | 08:42 |
| oanson | The names are a bit confusing. | 08:42 |
| xiaohhui | This way, we achieve some relationship between models like sql, right? | 08:43 |
| oanson | Maybe rename RefWrapper to Proxy, or ModelProxy? | 08:43 |
| dimak | xiaohhui, its not exactly for relationship purposes | 08:43 |
| dimak | xiaohhui, more for easy retrieval of pointed object | 08:44 |
| xiaohhui | In which scenario, the pointed object will be retrieved? | 08:44 |
| dimak | Some of our objects are contained (subnets, lrouter ports), and some are pointed by ID (lport->lswitch), this way we can use both the same way | 08:44 |
| yuli_s | hello | 08:45 |
| oanson | This is in a sense relationship. lports point to networks. networks contain subnets. etc. | 08:45 |
| oanson | yuli_s, hi | 08:45 |
| dimak | xiaohhui, object retrieval can be configured to be either lazy, then it will happen first time you access a field other than id on the ref | 08:45 |
| dimak | or eager, then you fetch it right away | 08:45 |
| dimak | I didn't write that code though we though of going to db store and if its not there to nb db | 08:46 |
| oanson | dimak, I don't see our discussion of cache-dirty marking in the example code. Was that removed? | 08:46 |
| dimak | the example code doesn't have a cache | 08:47 |
| xiaohhui | Is it for the case for something like get_lswitch_by_lport_id, this way we can call something like get_lswitch(lport_id) | 08:47 |
| oanson | There is a db store. It's just very basic. | 08:47 |
| dimak | all lookups are using iteration :) | 08:47 |
| dimak | Rewriting DbStore is a task by itself | 08:47 |
| xiaohhui | I meant lport.get_lswitch() | 08:48 |
| oanson | Yes. But this example should contain as many features as possible, even if they are only stubs. | 08:48 |
| oanson | The cache dirty mark is very cool. I wouldn't want to miss it | 08:48 |
| dimak | xiaohhui, we can get it by getting the lport, then lport.lswitch | 08:48 |
| dimak | is a ref to the lswitch | 08:48 |
| *** yamamot__ has quit IRC | 08:48 | |
| xiaohhui | OK, thanks for the explaination | 08:48 |
| dimak | oanson, I can upload another example with the dirty mark later on | 08:49 |
| oanson | dimak, thanks. Please do. | 08:49 |
| lihi | I also think that the names are a bit confusing, but basically that's sounds good | 08:50 |
| dimak | I'll send a patch soon to request adding jsonmodels to openstack-requirement unless anyone wants to propose another framework | 08:50 |
| lihi | + | 08:50 |
| lihi | oops sorry | 08:50 |
| oanson | dimak, +1 | 08:51 |
| dimak | Umm, what about the model registration part? | 08:52 |
| dimak | Do we want to just have a list of model modules like we do with apps? | 08:52 |
| oanson | In the spec I suggested using a (separate) register_model decorator | 08:53 |
| oanson | This way 'built' models can be registered immediately | 08:53 |
| oanson | The containing module will have to be loaded. This will have to be passed via configuration. | 08:53 |
| dimak | I meant that in case I add my own file | 08:53 |
| dimak | with my own models | 08:53 |
| dimak | someone has to import it | 08:53 |
| oanson | Yes. I'd like something like the App / DB driver mechanism - a namespace for models (e.g. dragonflow_models) and the name of the service | 08:54 |
| irenab | dimak: we can consider using stevedore, but this can be done later | 08:54 |
| dimak | Ok, we have to make sure we're loading the same models on both the controllers and the neutron nodes | 08:55 |
| xiaohhui | Can't the models just reside in the app module? | 08:55 |
| xiaohhui | some basic models in a common file | 08:55 |
| oanson | xiaohhui, the core models, yes | 08:55 |
| xiaohhui | the floatingip in dnat_app file | 08:55 |
| dimak | xiaohhui, apps are loaded in the neutron server code? | 08:55 |
| xiaohhui | apps are loaded in df-local-controller | 08:56 |
| oanson | xiaohhui, but we want to allow external vendors to implement their own services/apps. | 08:56 |
| oanson | It would be easier if a developer could write everything in their own repository, and then just include it in the config file. | 08:56 |
| xiaohhui | since we are going to load the app anyway, we can register the model when we load that app | 08:56 |
| oanson | It will also allow us to create a Dragonflow theatre - | 08:56 |
| oanson | The Dragonflow base (controller, API) | 08:56 |
| lihi | But eventually, each vendor will be required to changed the configuration file with the models? | 08:57 |
| oanson | and a theatre project per feature (l3, qos, lbaas, fwaas, etc.) | 08:57 |
| oanson | lihi, yes, but that is done in deployment | 08:57 |
| oanson | And from the looks of stevedore, it might solve that problem too (but I need to read more) | 08:57 |
| xiaohhui | So the vendor needs to config the app of their own, and also configure the db models, right? | 08:58 |
| dimak | it seems that way | 08:58 |
| oanson | To a point though. I don't want any 'magic' in loading new services - I want it to be easily trackable through the code. | 08:58 |
| oanson | xiaohhui, yes. The vendor will have to add his models, services, and apps. | 08:58 |
| xiaohhui | I think we can hide models from vendors, or users. After all, it is a bit duplicating with app. | 08:59 |
| oanson | Only in definition | 08:59 |
| xiaohhui | That is a bit too detailed for user need to know. | 08:59 |
| dimak | maybe we want to wrap all of it together, service plugin, app, and models | 09:00 |
| dimak | for example, SFC has all of the three and it doesn't make sense to include just part of them | 09:00 |
| oanson | xiaohhui, sorry, confused users with developers/vendors | 09:00 |
| xiaohhui | If there are in the configuration files, user will eventually see them. | 09:01 |
| oanson | Yes, that makes sense. Offer an 'application' registry, where the application defines its models, apps, and api adaptors | 09:01 |
| xiaohhui | service plugin is for neutron server to know what to run | 09:01 |
| xiaohhui | app is for df controller to know what to run | 09:01 |
| dimak | models can be required for both | 09:01 |
| dimak | to send messages over nb api | 09:01 |
| oanson | But if we tell the developer they can provide one name, and then read <name>.app, <name>.models, and <name>.<api adaptors> | 09:02 |
| oanson | And make it overridable, so that we don't force the structure on the vendor more than we have to | 09:02 |
| oanson | Yes, models are needed on both sides | 09:02 |
| xiaohhui | basically, I know what models I need, if I specify some service plugin and app | 09:03 |
| oanson | stevedore appears to solve that too. It has extensions pattern, which allow you to define multiple entrypoints with different names | 09:03 |
| xiaohhui | oanson, I think that is feasible, use one name for both app and models | 09:04 |
| oanson | xiaohhui, you know what models, app, and service plugin you need if you specify an overall application. e.g. if you want 'FWaaS', you are going to want all 3: models, API, and implementation (DF App) | 09:04 |
| irenab | oanson: so you suggestt tackle the app as a whole or step by step starting with NB DB refactor? | 09:06 |
| oanson | step by step | 09:06 |
| oanson | Mostly because otherwise the change would be too big, and we won't reach the end | 09:06 |
| irenab | ok, I got confused by discussion scope change :-) | 09:07 |
| dimak | I think we can do this after we figure out the NB refactor, and have a list of model files to include | 09:07 |
| oanson | Once the nb refactor is finished, I suggest a SB refactor (DF controller and applications) | 09:07 |
| xiaohhui | Agree | 09:07 |
| oanson | Once the SB refactor is done, consider maybe moving over to a more general pluggability model, e.g. stevedore | 09:07 |
| oanson | In parallel, once the NB refactor is done, I would like to push the native API forwards too | 09:08 |
| oanson | And next cycle, we're going to solve P=NP | 09:08 |
| irenab | oanson: REST on top of the models? | 09:08 |
| dimak | You meant P!=NP :) | 09:08 |
| oanson | dimak, we'll see next cycle | 09:09 |
| oanson | irenab, yes and no | 09:09 |
| oanson | irenab, I was thinking a programmable api, and a REST layer on top of it. | 09:09 |
| dimak | irenab, oanson jsonmodels allows exporting jsonschema out of the defined models | 09:09 |
| oanson | Neutron API and CLI don't have to go through REST | 09:09 |
| oanson | dimak, that makes sense. We'll probably use that | 09:09 |
| oanson | In fact, I'd prefer Neutron API not to go through REST. I think it will add an unnecessary overhead | 09:10 |
| dimak | oanson, but maybe having a single API would be simpler? | 09:10 |
| irenab | oanson: maybe you just need to clarify what native API means | 09:10 |
| oanson | irenab, yes, I think I do :) | 09:11 |
| oanson | dimak, we will have a single API, and adaptors on top of that | 09:11 |
| oanson | By native API, I mean a set of function calls, with all the necessary plumbing (e.g. AAA) to control a dragonflow deployment | 09:11 |
| xiaohhui | Sorry to interrupt, one last comment. Can we just import the db model module in service plugin and in app? We will do that import anyway, right? If so, then the db models can be registered when imported, and we don't need to make them into config fule. | 09:12 |
| oanson | A REST API on top of that can be a thin https server that calls these functions, and translates that info | 09:12 |
| xiaohhui | fule->file | 09:12 |
| dimak | xiaohhui, we will need the models in mech driver as well | 09:12 |
| irenab | oanson: +1 | 09:12 |
| dimak | and it doesn't import the apps | 09:12 |
| xiaohhui | but, mech driver will run with service plugin | 09:13 |
| oanson | xiaohhui, in any case we will need to write down the application somewhere in configuration. I think using a centralised place will make things a bit more organised | 09:13 |
| xiaohhui | in one process | 09:13 |
| oanson | Instead of importing the service plugin/app that will import the models, import the application's registration code which will provide both models and service plugin/app | 09:13 |
| oanson | In my opinion that will be clearer later. (But obviously I may be wrong) | 09:14 |
| xiaohhui | I just feel it might be confused if I am a newbie. I need to figure out what models I need when configure neutron.conf, and dragonflow.ini | 09:15 |
| irenab | Is anyone ok with the basic proposal that oanson and dimak posted in the spec and POC patches? | 09:15 |
| oanson | And this IRC chat? | 09:15 |
| oanson | xiaohhui, we will have to prepare some examples, showing how to load apps in the configuration | 09:16 |
| dimak | BTW, everyone are welcome check out the poc patch and play with it, all the tested things should work (conditions apply) | 09:16 |
| irenab | oanson: seems we are raiseng much more advanced steps, maybe worth to start from the DB refactor and graduate to the app dynamic management | 09:16 |
| xiaohhui | Sure, I just bring out my concens | 09:16 |
| oanson | xiaohhui, in any case, we can use the same configuration in the neutron.conf and dragonflow.ini : dragonflow_services = FW, LB, L3, ... | 09:16 |
| lihi | xiaohhui, that something that a newbie will be required to know, if he'll want to add new models | 09:16 |
| oanson | xiaohhui, and then by context load what we need | 09:17 |
| oanson | irenab, that's the short term plan, yes | 09:17 |
| lihi | or cahnge the defaukt configration | 09:17 |
| oanson | That's what we said | 09:17 |
| oanson | lihi, yes. But the way things are done now are not exactly newbie-friendly either | 09:18 |
| dimak | oanson, but when you add a new model you get to touch almost all parts of code so by the time you're done you're dragonflow expert | 09:18 |
| lihi | I don't think there is a way this will be really newbie-friendly | 09:19 |
| irenab | dimak: :-) | 09:19 |
| oanson | dimak, that's how things are done now. This is what we're here to solve (today) | 09:19 |
| oanson | lihi, true. But I hope by the time we're done, it will look... better :) | 09:20 |
| dimak | Anything else we want to address? | 09:20 |
| dimak | How do we proceed with the spec? | 09:21 |
| oanson | The spec has to be voted on and approved. In theory that includes the work plan within it | 09:21 |
| oanson | Once that is done, we start on the work plan. | 09:21 |
| oanson | Obviously, even after the spec is approved, nothing is set in stone. We can revisit our decisions if we realised we've made a terrible, terrible mistake | 09:22 |
| irenab | oanson: keeping scope on DB refactor, correct? | 09:22 |
| oanson | This spec, yes | 09:22 |
| oanson | Once this spec is approved, and we've made decent progress, I'll introduce a SB refactor spec | 09:22 |
| dimak | Do you want to wait until jsonmodels is approved as requirement? | 09:22 |
| oanson | It should be much easier, since most of the work was done in this spec | 09:23 |
| oanson | dimak, no | 09:23 |
| oanson | As I said, due to the circumstances, if it will be necessary, we will take control of the project | 09:23 |
| oanson | (If possible) | 09:23 |
| dimak | Maybe infra folks will suggest some other alternatives | 09:24 |
| irenab | dimak: did you try to ask on the infra channel? | 09:24 |
| dimak | irenab, nope, I will | 09:24 |
| oanson | I hope there's someone there. It's Christmas. | 09:25 |
| oanson | dimak, if that's the case, we will re-review | 09:25 |
| dimak | Yeah, I'll send the patch in the meantime | 09:25 |
| oanson | But I suspect that even if we change a backend, the change will not be great | 09:25 |
| dimak | oanson, I agree | 09:25 |
| oanson | We don't rely on any special features that are not expected from any ORM-based solution. | 09:25 |
| oanson | All right. Let's wrap up. | 09:26 |
| oanson | Please vote on the spec. I will update it again later today if necessary. | 09:26 |
| dimak | Thanks everyone | 09:26 |
| oanson | ^^^^^^^^^^^^^^^ | 09:26 |
| lihi | Thank you :) | 09:26 |
| irenab | thanks | 09:26 |
| xiaohhui | 😏 | 09:27 |
| oanson | #endmeeting (oh wait...) | 09:27 |
| irenab | oanson: sleepless nights … | 09:27 |
| *** gongysh has quit IRC | 09:30 | |
| *** zenoway has joined #openstack-dragonflow | 09:38 | |
| *** irenab_ has joined #openstack-dragonflow | 09:38 | |
| *** irenab_ has quit IRC | 09:38 | |
| *** zenoway has quit IRC | 09:40 | |
| *** zenoway has joined #openstack-dragonflow | 09:40 | |
| openstackgerrit | hujie proposed openstack/dragonflow: separate control plane and data plane ip address https://review.openstack.org/415116 | 09:54 |
| *** zenoway has quit IRC | 11:51 | |
| *** zenoway has joined #openstack-dragonflow | 11:51 | |
| openstackgerrit | hujie proposed openstack/dragonflow: separate control plane and data plane ip address https://review.openstack.org/415116 | 11:55 |
| *** zenoway has quit IRC | 11:56 | |
| openstackgerrit | Merged openstack/dragonflow: Tap-as-a-service plugin https://review.openstack.org/396307 | 12:32 |
| *** dkb has joined #openstack-dragonflow | 12:37 | |
| *** duankebo has joined #openstack-dragonflow | 12:38 | |
| *** yamamoto has joined #openstack-dragonflow | 12:46 | |
| *** yamamoto has quit IRC | 12:53 | |
| *** yamamoto has joined #openstack-dragonflow | 12:53 | |
| *** yamamoto has quit IRC | 13:00 | |
| *** dkb has quit IRC | 13:20 | |
| *** duankebo has quit IRC | 13:24 | |
| openstackgerrit | Merged openstack/dragonflow: check valid resource version for nb_api https://review.openstack.org/413120 | 13:27 |
| openstackgerrit | Merged openstack/dragonflow: Rename tunnel_type in Chassis to tunnel_types https://review.openstack.org/415087 | 13:37 |
| *** yamamoto has joined #openstack-dragonflow | 13:47 | |
| *** yuval has quit IRC | 14:29 | |
| *** yamamoto has quit IRC | 14:35 | |
| *** yamamoto has joined #openstack-dragonflow | 14:39 | |
| *** yamamoto has quit IRC | 14:44 | |
| *** lihi has quit IRC | 14:48 | |
| *** lihi has joined #openstack-dragonflow | 14:49 | |
| *** yamamoto has joined #openstack-dragonflow | 14:49 | |
| *** yuval has joined #openstack-dragonflow | 14:52 | |
| *** yuval has quit IRC | 15:06 | |
| *** openstackgerrit_ has joined #openstack-dragonflow | 15:56 | |
| *** openstackgerrit_ has quit IRC | 15:57 | |
| *** yamamoto has quit IRC | 17:37 | |
| *** yamamoto has joined #openstack-dragonflow | 18:37 | |
| *** yamamoto has quit IRC | 18:48 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!