Wednesday, 2019-02-06

*** tetsuro has joined #openstack-placement00:03
*** tetsuro has quit IRC00:16
*** tetsuro_ has joined #openstack-placement00:16
*** mriedem has quit IRC00:43
*** tetsuro_ has quit IRC01:08
*** tetsuro has joined #openstack-placement01:09
*** takashin has joined #openstack-placement01:24
*** tetsuro has quit IRC01:33
*** dims has quit IRC01:40
*** dims has joined #openstack-placement02:11
*** dims has quit IRC02:25
*** tetsuro has joined #openstack-placement02:30
*** dims has joined #openstack-placement02:33
*** tetsuro has quit IRC02:46
*** tetsuro_ has joined #openstack-placement02:46
*** tetsuro_ has quit IRC03:53
*** tetsuro has joined #openstack-placement06:30
*** takashin has quit IRC07:32
*** takashin has joined #openstack-placement07:50
*** takashin has left #openstack-placement08:00
*** belmoreira has joined #openstack-placement08:02
*** e0ne has joined #openstack-placement08:10
*** e0ne has quit IRC08:33
*** ttsiouts has joined #openstack-placement08:33
*** ttsiouts has quit IRC08:47
*** ttsiouts has joined #openstack-placement08:48
*** tetsuro has quit IRC08:48
*** ttsiouts has quit IRC08:52
*** tssurya has joined #openstack-placement08:54
*** e0ne has joined #openstack-placement09:01
*** ttsiouts has joined #openstack-placement09:09
*** cdent has joined #openstack-placement10:25
*** ttsiouts has quit IRC11:13
*** ttsiouts has joined #openstack-placement11:58
*** cdent has quit IRC12:26
*** mriedem has joined #openstack-placement13:32
*** cdent has joined #openstack-placement13:56
*** ttsiouts has quit IRC14:09
*** ttsiouts has joined #openstack-placement14:17
cdentlyarwood: thanks for the response on that thread. I have to admit I still don't fully get it, but will think upon it a while before I make any questions.14:19
*** belmoreira has quit IRC14:20
*** belmoreira has joined #openstack-placement14:21
lyarwoodcdent: np, so IOW, having both the original placement and extracted openstack/placement around into the T release would allow projects like TripleO additional time to write and more importantly validate the required upgrade logic to migrate between the two.14:29
lyarwoodcdent: If anything it just highlights how broken our approach to upgrades is in TripleO14:29
cdentbut things like stable/rocky are always around?14:30
lyarwoodcdent: right but we don't deploy mixed versions and always upgrade all components in the same step to the same release14:33
lyarwoodcdent: so we wouldn't run stable/rocky openstack-nova-placement-api with stein for example14:33
cdentah, right. okay that clarifies things a bit better. So if an actual deployment wanted to run rocky nova with stein placement they couldn't do that: the overcloud is a sort of unit?14:34
lyarwoodcdent: if you really wanted to you could edit the images being pulled in but the deployment logic itself would assume you were using stein placement14:35
lyarwoodcdent: btw, have the OSA folks actually landed support for an extracted placement service?14:36
lyarwoodI'm digging around in gerrit now but I can't find any evidence that they have14:36
lyarwoodand the same for kolla-ansible14:36
cdentas far as I can tell it is still in progress14:37
cdentI haven't been tracking it that closely in the past few weeks because of other obligations.14:37
lyarwoodkk np14:38
cdentI also didn't want to be responsible for all things. I feel like if I was supposed to be responsible for being aware of all that, the ordering of things would have been different...14:38
lyarwoodyeah understood, just trying to load context ahead of the call later14:40
cdentyup, is good to do that loading14:41
cdentlyarwood: from a tripleo standpoint, if the placement remains in nova after the end of stein, that helps. Is the fact that it is not tested an issue? If there is divergence between those codesets (there already is underneath, but not at the API level) is that a problem?14:45
* cdent walks14:53
cdentbiab14:53
*** cdent has quit IRC14:53
mnaserlyarwood: we have not landed it yet unfortunately15:30
mnaserthere's a bit of work that's left and our 'testing mechanism' really relies on most openstack projects using tempest15:31
mnaserso we dont have an easy way to validate placement working.. other than trying to find some other way to do it15:31
mnaserlyarwood: how are you validating the placement module in ic?15:31
mnasers/ic/ci/15:31
lyarwoodmnaser: indirectly in TripleO and the puppet projects through the compute related tempest tests15:44
lyarwoodmnaser: directly in RDO but only through the normal unit and func tests15:45
mnaserlyarwood: yeah i was hoping that we can just run tempest tests for placement only but apparently those dont exist15:45
mnaserand i dont really want to deploy with nova and all inside the role tests so yeah15:45
lyarwoodmnaser: yeah and never have tbh so I don't see that as a blocker15:45
mnaserin OSA, we validate roles independently and then in openstack-ansible we test it all integrated together15:45
lyarwoodmnaser: understood, do you have a topic for all of this btw?15:47
mnaserlyarwood: on gerrit? not really, we just have one kinda big role that's stalled a bit. https://review.openstack.org/#/c/618820/15:47
mnaseri'm planning to get that moving again soon.15:47
lyarwoodmnaser: ack thanks15:51
*** cdent has joined #openstack-placement15:56
lyarwoodcdent: re your question earlier about testing, the fact this isn't tested in the upstream gate isn't an issue, unit and func are tested when building the packages in RDO and we are also testing the original service in the TripleO gate until the extraction lands.15:57
cdentlyarwood: cool, thanks15:58
mnasercdent: i was asking what's the best way to run functional tests agains the api?15:59
cdentmnaser: it depends on what you mean by against the api15:59
mnaserin OSA world, we run tests against a specific service in the role, and we run everything all integerated in our integrated openstack-ansible repo15:59
cdentthe existing functional tests in placement are "against the api"15:59
mnaserso for example, i'm building a mistral role, we'll drop mistral_tempest_tests and run mistral tests only inside the role16:00
mnaserok so do they expect the api to be setup and running at a certain host/port/etc?16:00
cdentno, the api runs in the same process as the test16:00
cdentif what you're after is tests where the api is a different process, that's relatively easy to do, but fairly meaningless16:01
cdentsorry, fairly meaningless without something on the client side (like nova)16:01
mnaserit's meaningful in our case because in our role tests, we deploy a specific service only, so sometimes it's literally a matter of passing a "list all consumers and return a 200 ok" or something like that16:01
cdentdoes it have to be tempest or would you be happy with something simpler/faster?16:02
cdentsee, for example: https://git.openstack.org/cgit/openstack/placement/tree/playbooks/perfload.yaml16:02
mnasertempest would be the fastest path to success (i.e. we literally have to add a variable pointing to a tempest plugin and a whitelist of tests)16:02
cdentIt was recently agreed that we wouldn't do placement api tests in tempest (there was a thread on that mid january)16:04
mnaserthat's fine16:04
cdentwhat tempest would have is "tests of things that use placement"16:04
mnaserright, which would mean we'd have to deploy an entire openstack to test placement which will happen in our integrated repo, but in our 'role' only tests, it's a bit unnecessary16:04
cdentanother pending strategy for live tests of a placement service is: https://review.openstack.org/#/c/607508/16:04
cdentI get that that's bad, so I'm trying to understand what you want instead?16:05
cdentIn a perfect universe what would exist?16:05
*** openstackgerrit has joined #openstack-placement16:05
openstackgerritMerged openstack/placement master: Also time placeload when doing perfload  https://review.openstack.org/62802316:05
mnaser"something" i can run which can make requests against the placement service i just deployed to validate that it's functional (and in the term functional here, not run the entire test suite, but, maybe a *very* basic crud)16:06
mnaserjust to know that ok: db is running properly, api is responding properly, the service seems to be deployed correctly16:06
cdentmnaser: presumably you already have bits in place which do the deployment and add placement to the service catalog, yes? If so, we can write about a four line shell script into a playbook, point it at some gabbi files and be done. Would that work? It sounds like you'd prefer something more tempest oriented, but placement is "built in" at the moment:16:08
cdentactually, never mind, you're using tempest in the "point at pre-existing stuff" sense16:08
mnasercdent: tempest is easier but i'm not opposed to other solutions, we've in the past done something where we just call a task that talked to that cloud16:09
mnaserso yes, ideally if i can hit that shell script and point it at gabbi files and it can test that endpoint, that's perfect16:09
cdentIf you want to create the first half of that: the environment that ends with a noop  shell script, I can do the part that makes the shell script do something useful16:09
mnasercdent: https://github.com/openstack/openstack-ansible-os_heat/blob/master/tests/test-heat-functional.yml this isn't ideal, but it's almost something like that16:10
mnaserso we don't always go to tempest, sometimes we fall off to something similar to that16:10
mnaser(that probably returns 300 even if the db is failing so it's not ideal but okay, i can do that16:10
* cdent nods16:10
cdentWe've already got a huge mess of existing gabbi tests that can be cribbed from to get very complete coverage16:11
cdentmnaser: Cool, ping me when something exists and I can get right on it.16:12
cdentbrb16:12
*** ttsiouts has quit IRC16:18
*** ttsiouts has joined #openstack-placement16:18
*** ttsiouts has quit IRC16:23
mriedemumm, so do we have a placement extraction meeting in 30 minutes?16:30
edleafeyup16:31
mriedemdoes everyone else know about that?16:32
edleafessshhh - it's a secret!16:32
openstackgerritChris Dent proposed openstack/placement master: Use one ConfigOpts in placement-status  https://review.openstack.org/63359516:32
mriedemmy point is do melwitt, lyarwood and dansmith know there is an extraction follow up meeting in 28 minutes?16:32
cdentmriedem: i made an email earlier today, and mentioned it at the scheduler meeting16:32
mriedemah i see the email now16:33
mriedemsorry16:33
dansmithI hadn't noticed16:33
cdenta part of me was just going to wait and see, but that would have been churlish16:33
cdentdansmith: there's some context loading in the email, and also a bit earlier today in this channel from lyarwood16:36
dansmithack, have now read it16:39
mriedemlikely also want bauzas since it sounds like he's going to be making the updates to the libvirt reshaper patch16:40
mriedemwhich i burned out on16:40
mriedemdansmith: btw, i wanted to ask since you mentioned the cinder extraction having an overlapping release as an example in the last call we had - what was the db story with the cinder extraction? i'm assuming just copy the (single at the time) nova db and that was it? and downtime was less of a concern because people already dealt with downtime back in the essex/folsom days16:43
dansmithmriedem: I don't remember that specifically, but I think it was pretty much a dump and load, yeah..16:44
dansmiththe cinder thing is one example, which is worthy, but I think that in general nova has not removed things from its tree in the same release that you had to make a deployment change16:44
dansmithlike all the cells stuff was planned to be "move to this release, *then* sideways adjust things before the next release where something is required/dropped"16:45
dansmithwhich is definitely how I think we *should* be doing things like this16:45
mriedeme.g. move to newton when cells v2 and placement are optional but available, and required to get to ocata16:46
*** bauzas has joined #openstack-placement16:46
dansmithright16:46
efriedIs there a way to join the hangout from the hangouts app on mobile? I'm going to be on the move at the time.16:46
mriedemefried: i can maybe invite you16:46
mriedemyeah16:46
efriednoyce, thanks.16:48
melwittyeah. that model eases upgrades for deployers, in my experience from my time at yahoo. preferable to be able to deal with a code move/package change and data migration separately16:48
dansmithyeah, I've been saying that about the placement move since the beginning of the discussion, but alas.16:48
cdentdoes leaving code behind also mean not making changes in the extracted code? If not, NBD, if so, then presumably that's the reason we chose the current strategy, to not be so slow16:49
cdentbecause we are running years behind schedule...16:49
efriedIf we go that route, will it mean that we should reinstate some kind of testing around nova-in-placement?16:49
mriedemchanges = new features16:49
mriedem?16:49
dansmithcdent: yeah, that's not a reasonable excuse to me, but I understand the exhaustion16:49
cdentmriedem: yeah, I'm thinking the pending specs, for example.16:50
edleafeWe have been making changes in the extracted code already. It means that the nova-placement code will basically stay unchanged16:50
mriedemedleafe: not api changes though16:50
dansmithIMHO, you can still make changes to the new thing, nova just has to *work* with the old thing16:50
cdentthat's why we have microversions, yes?16:50
edleafecdent: I was just typing that16:50
mriedemyes16:50
cdentif so, let's just do it, and re-open extracted placemen for business16:50
efriedexcept this would mean we would need to start doing version discovery and conditionals.16:50
mriedemnova should be cool with at least n-1 of any dependent service api16:50
cdentefried: no, just be explicit16:51
mriedemand yeah, we said awhile back in nova that we weren't doing placement version discovery anymore16:51
efriedright16:51
mriedemand that placement just had to be upgraded before nova16:51
mriedemlike ironi16:51
cdentright16:51
mriedem*ironic16:51
mriedemregardless with 4 weeks to feature freeze i don't see us adding new code in nova that depends on new microversions in placement16:52
cdentyes16:52
efriedso - if we make a new microversion in extracted placement, and write nova code to exploit it, that nova code needs to be conditioned to *not* use it if we discover we're using non-extracted placement.16:52
mriedemso version discovery is likely not an issue16:52
mriedemefried: correct16:52
cdentbut we _could_ have those new features ready and waiting16:52
edleafeefried: we shouldn't be changing nova to use those new features until we can be sure that they will be there16:52
efriededleafe: which, when we were mandating upgrade-placement-before-nova, wasn't an issue.16:53
mriedemit's going to be a practice we have to get used to for placement anyway,16:53
edleafeIOW, if we keep nova-placement, nova has to rely on that code16:53
mriedemit's what we do with other services already like cinder and neutron16:53
mriedemwe = nova16:54
efriedmriedem: Not sure I understand that.16:54
edleafemriedem: sure, but if we are allowing placement to NOT be separate, then upgrading placement doesn't mean anything16:54
efriedIf the upgrade-placement-first edict remains, we should never have to do those conditionals after this bridge release.16:54
mriedemnova does not (or should not) land feature code that relies on external service apis without making sure those are available first16:54
mriedemefried: i guess...but that kind of sucks16:55
edleafemriedem: exactly. So nova code in Stein cannot depend on any of the Stein changes to placement16:55
efriedthis ^16:55
efriedand the plus side of the bridge release is what, again? Making it so deployers don't have to do two things?16:55
mriedemfwiw i don't think i was ever heavily in favor of the edict since version discovery is pretty simple and we do it for other external services already and it eases upgrades for people16:56
dansmithit's pretty common to want to just deploy new code.. tracking the config changes on each release is hard enough16:56
dansmiththen any sort of architectural changes happen once things are up and running in a separate maintenance windo16:56
dansmithotherwise you're deploying new things in a new way all at once and it's much harder to plan and react16:56
efriedwe should, like, have a hangout or something to discuss this.16:57
mriedemlet me enjoy 3 more minutes of nudity16:57
edleafemriedem: then we get enjoy it?16:57
mriedemha16:58
mriedemyes16:58
* cdent is agnostic about nova doing version discovery16:58
mriedemthe version discovery thing is a different issue anyway16:58
cdentI guess it would be fair to say I'm agnostic about nova16:58
efriedPersonally, I like The Edict. Keeps the code clean, and seems like it wasn't a terribly onerous imposition on deployers?16:58
mriedemi just know there is a perception in operator land that all of openstack must be upgraded to the same release at the same time and that makes upgrades hard and it really shouldn't be that way16:59
cdentthat's maintained by most of the deployment tools. lyarwood was making it clear to me earlier that tripleo makes it very hard to mix bits16:59
cdentvio is _very_ much the same way16:59
cdentwhich is annoying16:59
mriedemoh i'm sure17:00
* cdent headphones up17:00
mriedempublic clouds don't need to run that way though, glance could be 2 years old17:00
bauzason another meeting but will join as soon as it ends17:00
mriedemhttps://hangouts.google.com/call/C5h9TyVk5BMfSO8vGxZ_AEEE17:00
bauzascan someone point me the hangouts url ?17:00
bauzasoh snap17:00
mriedemhttps://etherpad.openstack.org/p/placement-extract-stein-517:04
bauzasthanks17:05
*** tssurya has quit IRC17:09
mriedemL41 for how grenade does post-upgrade validation of resources https://etherpad.openstack.org/p/placement-extract-stein-517:18
edleafeL42 now17:19
sean-k-mooneycdent: the funny thing is i had considerd buying the harware to test this personally in the past but i have not done so yet17:21
cdentfight corporate oppression sean-k-mooney !17:24
sean-k-mooneyhehe well the reason i did not by it was nvidas licencing17:24
bauzasnvidia licencing isn't really a problem when you work on that :)17:25
bauzasbut man, the hardware layout for V100s is insane17:25
mriedemhttps://blueprints.launchpad.net/nova/stein17:27
-openstackstatus- NOTICE: Any changes failed around 16:30 UTC today with a review comment from Zuul like "ERROR Unable to find playbook" can be safely rechecked; this was an unanticipated side effect of our work to move base job definitions between configuration repositories.17:28
*** e0ne has quit IRC17:33
cdentmriedem, melwitt: question I didn't want to ask on the call just now, for fear of it pulling us away from making immediate decisions but:17:49
cdentGiven that we've now changed the strategy of how to proceed does that have any impact on the governance agreement? It seems like it doesn't have to if we don't want it to, but it could if we do.17:50
mriedemi'm not opposed17:51
cdentbasically: do we need to revisit the agreement, or is everyone still cool as it is?17:55
cdent(I'm in the camp of: things are already a certain way, de facto, may as well de jure too)17:56
cdentmriedem: to be clear: what are you not opposed to?17:58
melwittI'm not opposed to revisiting it if people want to. I need a little time to digest the summary of what the new strategy is (extracted placement not required in stein, but I'm unclear on the data migration piece)17:58
mriedemcdent: i'm not opposed to splitting placement out in governance17:58
cdentto be completely up front about it: the only reason I'm asking/care about it that much _now_ is because of planning the rest of the year17:59
mriedemyar, my goals are due in a few weeks for the first half as well18:00
mriedemlooking at http://lists.openstack.org/pipermail/openstack-dev/2018-September/134541.html really the only thing that likely won't be done in stein is the deployment tool upgrade part (item 3) but that's basically no biggy in stein now because we aren't going to delete the placement-in-nova code in stein18:01
mriedemper the call18:01
cdentaye18:01
mriedemmelwitt: the data migration piece remains the same as far as i know - the difference is you can upgrade nova to stein without having to worry about deploying extracted placemetn and doing the data migration during the upgrade *to* stein,18:04
mriedemyou can deal with after you've upgraded to stein but before upgrading to train18:04
mriedemif you'rea new deploy in stein, then you just use extracted placement from the start since there is no upgrade18:04
melwittok, right. I have a hard time following all of it in my head18:07
mnasercdent: https://github.com/openstack/placement/blob/master/placement/conf/database.py is there plans to move to [database] instead of [placement_database] ?18:15
mnasergiven that there hasnt been 1.0.0 yet and everyone will be configuring things from scratch18:15
cdentmnaser: not any immediate plans no18:16
mnaserok cool18:16
cdentthere have been various discussions about it in the past but they resolved to "meh, keep it as is"18:16
cdentI actually like it being named, as it makes sense if there's a future with more than one database18:17
mnasercdent: fair enough, i think i have a fully cleanly deployed placement over here so i should be able to push it up shortly18:17
cdentrad18:17
edleafeoh geez - please no more multiple databases18:18
mnaserim just trying to do a simple curl request18:18
mnaserto see what happens and if it responds correctly18:18
cdent / should give you a 1.30 microversion18:19
cdent(version doc)18:19
mnasercdent: yeah but i dont know if that guarantees a successful db connection, thats what im trying to figure out :>18:20
mnaser(i get a successful response on /)18:20
cdentit doesn't, but it is a good starting point18:20
mnaser# curl http://127.0.0.1:878018:20
mnaser{"versions": [{"status": "CURRENT", "min_version": "1.0", "max_version": "1.30", "id": "v1.0", "links": [{"href": "", "rel": "self"}]}]}18:20
cdent /resource_providers will access the db but will need auth18:21
cdentactually, if the service is able to start, it will try to sync the traits to the db, before it will answer any web requests. if that fails, the service fails18:24
cdentso if you got the version doc, you should be going18:24
cdenteven though that request itself does not need it18:24
cdentmnaser: you can see that traits sync in the logs of the placement service for you own human sanity check (but not great for automated)18:37
cdentmriedem, melwitt, efried : we didn't schedule another check in. Do we need one, or should we wait and see?18:38
melwittI think it doesn't hurt to plan to have one. I've found them to be nice for clearly looking at where we are. either way sounds fine to me18:42
cdentmelwitt: since I already wrote it this way in the notes, I'm gonna say that we'll book one when the time seems right18:45
melwittsounds good to me18:46
cdentcool18:47
*** e0ne has joined #openstack-placement19:29
mnasercdent: usefulness of a small test, db works but python-memcache was not isntalled of import fails for keystonemiddleware cache19:39
mnaseri think i'm almost done and i added a small test that grabs a keystone token, grabs the placement api url from service catalog and then hits /resource_providers expecting a 200 ok -- which should be ok in the interim until we get something more upstreamy.19:48
mnaserhttps://review.openstack.org/#/c/618820/ ok, this passes for me locally on centos, so we'll wait for CI to report for other OS (it'll probably work fine)19:55
cdentmnaser: cool, thanks, I'll mess with that a bit more tomorrow, I'm past my sell by date for today20:03
mnasercdent: cool as the state it is, we can probably merge it because we get basic validation, so i guess all that to say it's probably not that high of a prio now that i have that in place20:04
cdentWhen I look at it tomorrow if I see something obvious/simple to do to make it more robust, I'll do a followup patch to check it out20:04
cdentbut for now: good night all20:06
*** cdent has quit IRC20:06
*** e0ne has quit IRC21:00
*** takashin has joined #openstack-placement21:51
*** bodgix has joined #openstack-placement22:17
*** bodgix has left #openstack-placement22:17
*** alex_xu has quit IRC22:38
*** alex_xu has joined #openstack-placement22:40
*** ttsiouts has joined #openstack-placement23:08
*** mriedem has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!