Tuesday, 2016-03-15

*** angdraug has quit IRC00:08
*** dims has joined #openstack-meeting-cp00:23
*** docaedo has quit IRC00:31
*** dims has quit IRC00:32
*** docaedo has joined #openstack-meeting-cp00:32
*** markvoelker has joined #openstack-meeting-cp00:40
*** vilobhmm11 has quit IRC00:53
*** sdake has joined #openstack-meeting-cp00:55
*** dims has joined #openstack-meeting-cp00:56
*** sdake has quit IRC01:10
*** markvoelker has quit IRC01:14
*** vilobhmm11 has joined #openstack-meeting-cp01:22
*** vilobhmm111 has joined #openstack-meeting-cp01:27
*** vilobhmm11 has quit IRC01:29
*** cshahani has quit IRC01:38
*** cshahani has joined #openstack-meeting-cp01:39
*** cshahani has quit IRC01:55
*** vilobhmm111 has quit IRC02:00
*** vilobhmm11 has joined #openstack-meeting-cp02:03
*** cshahani has joined #openstack-meeting-cp02:28
*** dims has quit IRC02:38
*** markvoelker has joined #openstack-meeting-cp03:11
*** vilobhmm11 has quit IRC03:19
*** markvoelker has quit IRC03:44
*** vilobhmm11 has joined #openstack-meeting-cp03:45
*** sdake has joined #openstack-meeting-cp04:06
*** sdake_ has joined #openstack-meeting-cp04:09
*** sdake has quit IRC04:11
*** sdake_ has quit IRC04:19
*** sdake has joined #openstack-meeting-cp05:16
*** askb has joined #openstack-meeting-cp05:33
*** askb has quit IRC05:33
*** sdake has quit IRC05:33
*** askb has joined #openstack-meeting-cp05:34
*** markvoelker has joined #openstack-meeting-cp05:41
*** markvoelker has quit IRC06:14
*** askb has quit IRC06:24
*** belmoreira has joined #openstack-meeting-cp07:58
*** markvoelker has joined #openstack-meeting-cp08:12
*** vilobhmm11 has quit IRC08:32
*** markvoelker has quit IRC08:44
*** sdague has joined #openstack-meeting-cp10:01
*** markvoelker has joined #openstack-meeting-cp10:42
*** dims has joined #openstack-meeting-cp11:12
-openstackstatus- NOTICE: Gerrit is going to be restarted11:13
*** markvoelker has quit IRC11:15
*** dims has quit IRC11:20
*** dims has joined #openstack-meeting-cp11:21
*** dims has quit IRC11:21
*** dims has joined #openstack-meeting-cp11:21
-openstackstatus- NOTICE: Gerrit had to be restarted because was not responsive. As a consequence, some of the test results have been lost, from 08:30 UTC to 10:30 UTC approximately. Please recheck any affected jobs by this problem.11:33
-openstackstatus- NOTICE: Gerrit had to be restarted because was not responsive. As a consequence, some of the test results have been lost, from 09:30 UTC to 11:30 UTC approximately. Please recheck any affected jobs by this problem.11:36
*** belmoreira has quit IRC12:07
*** raildo-afk is now known as raildo12:16
*** cshahani has quit IRC12:21
*** dims has quit IRC12:25
*** dims has joined #openstack-meeting-cp12:26
*** ninag has joined #openstack-meeting-cp12:53
*** markvoelker has joined #openstack-meeting-cp12:53
*** sdake has joined #openstack-meeting-cp13:08
*** ninag_ has joined #openstack-meeting-cp13:23
*** ninag has quit IRC13:26
*** sdake has quit IRC13:38
*** askb has joined #openstack-meeting-cp13:59
*** sdake has joined #openstack-meeting-cp14:25
*** askb has quit IRC14:51
-openstackstatus- NOTICE: Launchpad OpenID SSO is currently experiencing issues preventing login. The Launchpad team is working on the issue14:58
*** ChanServ changes topic to "Launchpad OpenID SSO is currently experiencing issues preventing login. The Launchpad team is working on the issue"14:58
*** markvoelker has quit IRC15:00
*** ChanServ changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"15:31
-openstackstatus- NOTICE: Launchpad SSO is back to normal - happy hacking15:31
*** askb has joined #openstack-meeting-cp15:33
*** askb has quit IRC15:33
*** askb has joined #openstack-meeting-cp15:33
*** ninag_ has quit IRC16:09
*** ninag has joined #openstack-meeting-cp16:12
*** ninag has quit IRC16:18
*** ninag has joined #openstack-meeting-cp16:19
*** kazuIchikawa has joined #openstack-meeting-cp16:20
*** markvoelker has joined #openstack-meeting-cp16:25
*** sdake_ has joined #openstack-meeting-cp16:31
*** sdake has quit IRC16:34
*** ninag has quit IRC17:44
*** kazuIchikawa has quit IRC17:57
*** ninag has joined #openstack-meeting-cp17:59
*** dims_ has joined #openstack-meeting-cp18:03
*** dims has quit IRC18:03
*** askb has quit IRC18:16
*** vilobhmm11 has joined #openstack-meeting-cp18:20
*** dims_ has quit IRC18:29
*** sdake_ has quit IRC18:34
*** thinrichs has joined #openstack-meeting-cp18:36
*** thinrichs has quit IRC19:02
*** sdake has joined #openstack-meeting-cp19:07
*** dims has joined #openstack-meeting-cp19:12
*** dims has quit IRC19:18
*** sdake_ has joined #openstack-meeting-cp19:21
*** sdake has quit IRC19:23
*** dims has joined #openstack-meeting-cp19:26
*** tyr_ has joined #openstack-meeting-cp19:42
*** dims has quit IRC19:45
*** thinrichs has joined #openstack-meeting-cp19:51
*** sdake_ has quit IRC19:51
*** diablo_rojo has joined #openstack-meeting-cp19:52
*** annegentle has joined #openstack-meeting-cp20:01
*** aspiers has joined #openstack-meeting-cp20:05
*** fhermeni has joined #openstack-meeting-cp20:08
*** dims has joined #openstack-meeting-cp20:11
*** sdake has joined #openstack-meeting-cp20:11
*** thinrichs has quit IRC20:17
*** samueldmq has joined #openstack-meeting-cp20:23
*** bswartz has quit IRC20:29
*** bswartz has joined #openstack-meeting-cp20:30
*** thinrichs has joined #openstack-meeting-cp20:32
*** thinrichs has quit IRC20:32
*** thinrichs has joined #openstack-meeting-cp20:41
*** SamP has joined #openstack-meeting-cp20:47
*** sdake has quit IRC20:50
*** SamP has left #openstack-meeting-cp20:52
*** sdake has joined #openstack-meeting-cp20:53
*** cdent has joined #openstack-meeting-cp20:58
*** ddeja has joined #openstack-meeting-cp21:00
*** kfox1111 has joined #openstack-meeting-cp21:00
thingeecourtesy ping for Qiming TravT gordc dirk mriedem SergeyLukjanov21:00
thingeecourtesy ping for daemontool jroll boris-42 redrobot flaper87 rhochmuth21:00
thingeecourtesy ping for fungi flwang dims vipul johnthetubaguy rakhmerov21:00
thingeecourtesy ping for docaedo stevemar mtreinish bswartz adam_g adrian_otto21:00
jroll\o21:00
thingeecourtesy ping for zigo Piet sdake mugsie sheeprine thinrichs21:00
thingeecourtesy ping for jklare loquacities smelikyan Daisy skraynev odyssey4me21:00
aspierso/21:00
kfox1111o/21:00
*** samapthP has joined #openstack-meeting-cp21:00
dimso/21:00
elmikoo/21:00
nikhilo/21:00
fungihey-hey, kids!21:00
thingeesamueldmq ping21:01
stevemaro/21:01
thinrichsHi all21:01
samueldmqhi all21:01
samueldmqthingee: hi21:01
ddejahello21:01
thingee#startmeeting crossproject21:01
openstackMeeting started Tue Mar 15 21:01:25 2016 UTC and is due to finish in 60 minutes.  The chair is thingee. Information about MeetBot at http://wiki.debian.org/MeetBot.21:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:01
*** openstack changes topic to " (Meeting topic: crossproject)"21:01
openstackThe meeting name has been set to 'crossproject'21:01
nikhilthingee: am still not on the list21:01
* thingee adds nikhil now :(21:01
fhermenihello21:01
thingeemissed ya'll21:02
adam_ghiya21:02
nikhilthingee:  thx21:02
cdento/21:02
david-lyleo/21:02
*** annegentle has quit IRC21:02
thingeeagenda today https://wiki.openstack.org/wiki/Meetings/CrossProjectMeeting#Proposed_agenda21:02
thingeepretty full so lets get to it!21:02
samapthPo/21:02
*** kencjohnston has joined #openstack-meeting-cp21:02
*** sdake_ has joined #openstack-meeting-cp21:02
thingee#topic Compute node HA (a.k.a. automatic evacuation of instances21:02
*** openstack changes topic to "Compute node HA (a.k.a. automatic evacuation of instances (Meeting topic: crossproject)"21:02
*** mriedem has joined #openstack-meeting-cp21:02
bswartz.o/21:03
mriedemo/21:03
kencjohnstono/21:03
thingee#link http://specs.openstack.org/openstack/openstack-user-stories/user-stories/proposed/ha_vm.html21:03
*** angdraug has joined #openstack-meeting-cp21:03
thingeeaspiers: hi!21:03
aspiershey :)21:03
EmilienMo/21:03
aspiersconscious that my slot doesn't have too much time21:03
angdraugo/21:03
*** rockyg has joined #openstack-meeting-cp21:03
*** bauzas has joined #openstack-meeting-cp21:03
lifeless\o/21:03
rockygThanks thingee !21:03
*** dtroyer has joined #openstack-meeting-cp21:03
aspiersthingee: shall I kick things off?21:03
lifeless\_/o/+21:04
thingeeaspiers: yes21:04
dtroyero/21:04
aspiersok21:04
aspiershi everyone :) I'm new to cross-project meetings so please bear with me :)21:04
aspierssince Tokyo I've been moderating weekly meetings of the #openstack-ha IRC community21:04
aspierswe've been collaborating on various approaches to implementing automatic "evacuation" of VM instances21:04
thingee#link https://review.openstack.org/#/c/257809/ cross-project spec21:04
aspiers"evacuation" is in quotes, because so far we've really been dealing with resurrection of dead VMs, via nova's evacuate API21:04
*** sdake has quit IRC21:05
aspiershowever in the future, proactive evacuation of live VMs based on (say) alarms from ceilometer is certainly a possibility too21:05
EmilienMalarms from Aodh*21:05
aspiersright21:05
aspiers:)21:05
aspierswe've been trying to promote awareness of the various approaches out there21:05
*** carolbarrett has joined #openstack-meeting-cp21:05
*** raildo is now known as raildo-afk21:05
*** gordc has joined #openstack-meeting-cp21:05
aspierswith the ultimate goal of facilitating convergence on a single implementation, if possible21:06
aspiersyou can find a summary of all known existing approaches (to resurrection) here:21:06
aspiers#link https://etherpad.openstack.org/p/automatic-evacuation21:06
aspiersfirst question from my side is: how cross-project is this, really? which projects do we expect to be affected?21:06
*** shamail has joined #openstack-meeting-cp21:06
thingeeaspiers: do you mind telling us here and identifying in the spec the projects this involves?21:06
thingeeyes21:07
thingeeheh21:07
aspierssure21:07
aspiersIMHO the answer is "probably less than 10 projects":21:07
aspiers1. obviously nova is involved :)21:07
mriedemheh21:07
aspiers2. ceilometer21:07
aspiers3. aodh21:08
aspiers4. senlin is clearly aiming to be involved21:08
aspiers5. mistral - Intel have been working on a PoC where mistral orchestrates evacuation (ddeja who is here with us :)21:08
*** _gryf has joined #openstack-meeting-cp21:08
aspiers6. heat has been aired in the spec linked above as a possible alternative to mistral, not sure how realistic that is21:08
aspiers7. congress: I expect this to be needed in the future, to accommodate different recovery policies21:09
samueldmqmay the subset be defined by "any project involving quota management" + what triggers and what effectively does the ressurection ?21:09
aspiersand maybe 8. cinder?21:09
*** sdague has quit IRC21:09
rockygwhat about manila?21:09
jrollI suspect someone wants this for ironic eventually, based on ideas I've seen, but this would just use nova live-migrate or something, right?21:09
aspiersrockyg: I hadn't thought about manila yet :)21:09
aspiersoh yeah of course: also ironic and Triple-O21:10
jroll(so ironic, as a virt driver, would just work)21:10
elmikoi'm curious if there are some criteria for determining if our project is affected? (something akin to what samueldmq said)21:10
jroll(once it has migrate)21:10
aspiersthere's the whole question of fencing of compute nodes21:10
thingeegordc, cdent ^21:10
mriedem25. watcher?21:10
aspierselmiko: not yet21:10
aspiersyes quite possibly watcher too :)21:10
gordcthingee: just reading spec now (missed some of earlier chatter)21:10
alaskijroll: I think it's very likely this would use public apis in Nova, so ironic should "just work"(tm)21:10
jrollalaski: cool, that's what I hoped21:11
aspiersif congress is in charge of deciding what triggers workflows, then other projects don't necessarily have to be aware that data collected from them is being used in this way21:11
mriedemyeah, nova already has the APIs for this as far as i know21:11
gordcaspiers: have you look at the events ceilometer captures currently?21:11
thinrichsaspiers: agreed21:11
bauzasI don't see what Nova needs21:11
aspiersgordc: not nearly as much as I should21:11
samueldmqaspiers: elmiko: interesting, I'd expect that we had defined keywords to help identifying involved projects21:11
aspiersbauzas: currently the nova evacuation process is not robust enough (I'm told)21:11
mriedemfwiw, there is an ibm product that already does something like this, event-driven automatic evacuation, i could get names of people involved in that, they might also be invovled in this,21:12
mriedembut they had some nova api extensions i believe21:12
mriedemlike rate limiting the migrations off the compute node21:12
mriedemrather than just a huge batch21:12
aspiersmriedem: yes that is one of the problems which needs tackling21:12
elmikosamueldmq: yea, i'm very curious as i could a hypervisor dropout affecting sahara in a negative way21:12
aspiersscalable scheduling21:12
elmiko*i could see21:12
gordcaspiers: i think quite a few of this is already captured in Ceilometer via messages sent from services or the polling done21:13
samueldmqelmiko: ++21:13
shamailaspiers: +1, the current prototype i’ve seen for Masakari specifies nova hosts for evacuation versus using the scheduler21:13
shamailnova host*21:13
gordcyou can tell when a host or instance is up or down and provisioning failures i believe are already sent as events by nova21:13
bauzasaspiers: well, evacuate for instance HA means that you resurrect your instance, and then impacting the users21:13
jrollso, for the sake of time, what's the goal for this meeting? "should we pursue this" or?21:13
aspiersthere are lots of architectural considerations21:14
bauzasjroll: ++21:14
aspiersand jroll is right, we can't discuss them now21:14
shamailI personally see it as instance recovery vs. availability21:14
samueldmqjroll: ++21:14
mriedembauzas: also, you can't evacuate unless the compute is donw21:14
mriedem*down21:14
aspiersin the short time we have, I just want to understand how this relates to the cross-project team (if it's called a team, I'm not sure)21:14
mriedemmethinks anyway21:14
mriedemaspiers: i'm not sure it does21:15
bauzasyup, I was just explaining the problem about using evacuate for HA21:15
gordcfor those interested, comment on spec i guess? i have a feeling most of this is already covered across various projects21:15
thingeeaspiers: the spec was proposed in the repo21:15
thingee:)21:15
mriedemaspiers: if you have API needs in nova, then propose those as specs for nova21:15
aspiersthingee: yeah, not by me ;-)21:15
thingeeok, so I think we're saying, sure do this and use existing apis?21:15
cdentthingee: that is what it sounds like21:15
thingeeand it shouldn't have been proposed spec as sdague mentioned earlier21:15
mriedemthingee: and if there are gaps in the API, propose project-specific specs to fill those gaps21:15
_gryfmriedem, I think all the api are in place, if we talking about nova21:16
thingeemriedem: +121:16
alaskimriedem: +121:16
aspiersyup, I agree too21:16
jrollmriedem: ++21:16
thingeeok will leave a comment to have this spec abandoned21:16
aspierswell21:16
thingee#topic super scheduling21:16
*** openstack changes topic to "super scheduling (Meeting topic: crossproject)"21:16
mriedemaspiers: hunt down jwcroppe if you haven't already21:16
aspiersok, thanks21:16
thingeefhermeni: hi!21:16
fhermenihi there21:16
samueldmqit'd be great to have a single bp to use cross-projects (in the topic of changes)21:16
thingee#link https://review.openstack.org/#/c/210549/ spec21:16
fhermeni(first OS meeting, be cool)21:17
*** doffm has joined #openstack-meeting-cp21:17
thingeegot a lot of first timers today :)21:17
rockygfhermeni, we're generally a friendly bunch21:18
fhermeniwe’ll see21:18
elmikolol21:18
thingeeso we have no josh harlow, so I want to understand where are we going with this spec. I guess a little intro and what you need from the general cross-project team21:18
fhermeniok. Indeed, josh is afk (6am local time)21:19
fhermeniso there is 2 objectives21:19
fhermeni1) expose a single entry for every scheduling aspect to allow to exhibit a higher perspective on scheduling expectations21:19
fhermeni2) abstract the scheduler a bit to decouple from existing implementation21:20
fhermeniin that sense, you support the spec, you are suitable21:20
bswartzI haven't read the spec yet21:20
bswartz(but I will)21:20
fhermeniso we designed the spec for being extensible, with a very few orthogonal concepts21:21
bswartzdoes it at least acknowledge that the problem it's trying to solve is really hard?21:21
fhermeniyes of course21:21
fhermeni(I am writing VM placement algorithm since 10 years, I know it is hard ;))21:21
bswartzokay21:21
bswartzthe exception cases are likely to be where it gets tangled up21:22
fhermeniso we drafted a spec and wanted to have feedback from the involved communities21:22
thinrichsfhermeni: I couldn't tell from the spec if you were proposing having a scheduler that makes decisions using all the state/context/data/resources of many different services, or if you were proposing an abstraction that all (local) schedulers would implement.21:22
bauzasfhermeni: so we had kind of this discussion in the past during previous summits21:22
lifelessevery summit21:22
gordclol21:23
bauzasfhermeni: that's why something popped up called Gantt21:23
lifelessI don't think we can call it a summit without a scheduler session21:23
fhermeniit is an abstraction. The backend may or may not fullfil all the expectation (the extensibility concern)21:23
bauzaswith the clear intent of splitting out the nova scheduler to be a separate project that others could consume21:23
fhermenilifeless: it is not a surprise to me21:23
*** sdake_ is now known as sdake21:23
bauzasit was a dead-end for technical reasons at least, but also valid design concerns21:24
aspiersthis sounds a bit like a blog post I wrote last year http://blog.adamspiers.org/2015/05/17/cloud-rearrangement/21:24
gordcbauzas: gnatt isn't active anymore is it?21:24
bauzasgordc: nope, defunct21:24
gordckk. good to know21:24
fhermenias an academic, I observe there is about 50 VM placement algorithm that are proposed yearly. No one can propse or test a validation with OS because of the thigh coupling with Nova21:24
bauzasmostly because we weren't having those stable interfaces we were needing to have21:25
*** diablo_rojo has quit IRC21:25
fhermenidecoupling might help21:25
gordcfhermeni: is your scheduler completely original or does it leverage nova's scheduler?21:25
bauzasfhermeni: see, the real problem of that is 2 words : tech debt21:25
fhermenibauzas: that is one of the point that require nova output21:25
fhermenibauzas: does the language wrap nova features21:26
*** mugsie has joined #openstack-meeting-cp21:26
thingeeso in the interest of time here, I agree with sdague and cdent that probably engagement should happen with nova here? I'm not really sure why this needs a cross-project spec.21:26
fhermenibauzas: and ofc I cannot reply objectively because I am out of nova and an OS newbie21:26
jrollwe keep saying vm placement, but this is about general resource placement, correct?21:26
samueldmqthingee: ++21:26
cdentthingee: just to be clear sdague and I have different ideas on how that engagement should happen21:26
rockygjroll, ++21:26
jrollthingee: ++, I think if something like this gets pulled into nova, then it may require some cross-project work, but it needs to begin with nova21:26
bauzascdent: and I agree with sdake21:27
bauzasdamn21:27
bauzassdague21:27
thingeecdent: sorry21:27
cdenthoisted on your own tab key bauzas !21:27
cdent:)21:27
bswartzto do correct global placement is far larger than just nova21:27
rockygbswartz, ++21:27
bswartzyou need information about storage, networking, compute, and information that exists nowhere in openstack today21:28
rockygCongress would play a part21:28
fhermeniI am ok discussing with nova ofc. The language is quite simple so I am not affraid about its suitability but it is quite different from the pending spec I read about the proposed generic api for nova21:28
rockygbswartz, you copied my keyboard!21:28
samueldmqthingee: I think it could be useful to add 'keywords' in a spec, that'd identify what projects are affected21:28
nikhilthis will need to know about the placement of services too in such a case?21:28
bauzasbswartz: that's exactly why we need stable interfaces and a correct model for describing resources21:28
thingeesamueldmq: sure21:28
samueldmqthingee: so it will be easier to identify proposals that only relate (mostly) to a single project21:29
bauzasboth were not the case in nova a couple of months before, and that's gradually improving21:29
rockygbauzas, which is why this is crossproject.  More resources than just vms and compute nodes21:29
samueldmqthingee: I will propose a change to the spec template21:29
bswartzyeah I'm only arguing that while nova involvement is essential, this is in no-way nova-specific21:29
thingeefhermeni: so I would say figure out what happened with gnatt, see if there is a need to revive that. I'm not really hearing much people here having an interest with global placement scheduler though21:29
bauzasrockyg: so, see, I had this convo a lot of times before, and had no clear commitment on resource placement for others than vms21:29
fhermenithingee: you don’t, don’t worry we have21:30
* jroll is curious how many people here know the details of the nova scheduler rework21:30
fhermenithingee: watcher is an example21:30
jrolland the plans for the future21:30
bauzaspeople want x-project affinity for placing instances, but I haven't seen a clear interest for placing something else but instances21:30
bswartzthingee: I think we're all interested, but we've been burned by past failures and fear getting burned again21:30
mriedemjroll: take jaypipes out for lunch21:30
jrollmriedem: oh, I know the plans, but curious about others :)21:30
elmikojroll: i don't21:30
jrolle.g. "is there a need to revive gaant"21:31
rockygI'm aware but not in deep21:31
jrollit's too much to describe here, but I think it's good background reading for this topic21:31
* jroll finds email21:31
jroll#link http://lists.openstack.org/pipermail/openstack-dev/2016-February/086371.html21:31
rockygSo, there are all sorts of issues in multiregion clouds that include other resources besides vm21:31
jrollfolks may want to read that for some context21:31
elmikojroll++21:32
rockygThe ops community wants placement related to networking capabilities21:32
bauzasrockyg: I'm certainly sure, I just said I haven't seen user stories yet21:32
fhermeniindeed. For testbed for example, it is a prime feature21:32
rockygAnd HPC want placement wrt data storage21:33
jrollso anyway, I agree that the super scheduler folks should chat more with the nova scheduler folks21:33
bauzasrockyg: that's affinity for vms21:33
jrolland figure out how this fits in, if at all21:33
fhermeniHPC as well indeed. network and cache affinity21:33
elmikoyea, i've seen some user asking question about data and compute affinity with respect to hadoop21:33
rockygbauzas, but global rather than local21:33
fhermenielmiko: indeed. VM close to the storage for data intensive workload21:33
thingeecdent: can we get some folks interested in engaging with this spec? I think you and sdague are the only from nova that have responded to the spec21:33
elmikofhermeni: right, and asking for more options in sahara to control the affinity21:33
bauzascould we stop making confusion between x-project affinity for placing vms, vs. placing things like volumes ?21:34
bauzasthingee: I made a very clear statement too21:34
cdentI think it is pretty clear that bauzas should be actively contributing to the discussion on the spec21:34
cdentespecially to clarify some of the thing he just clarified here21:34
rockygcdent, ++21:34
bauzasI don't see the need for that being a x-project spec honestly21:35
rockygI'd love to see a session on this at the design summit21:35
bauzasat least now21:35
cdentthere's a vast amount of assumptions that people make about the use cases and nobody seems to really agree21:35
*** thinrichs has quit IRC21:35
thingeeok, anyone disagree with bauzas?21:35
jrollbauzas: ++ this needs to start in nova and expand21:35
thingeegoing21:35
jrollthe nova work is making rooms for other resources like network and volumes21:35
thingee...21:35
thingeegoing21:35
cdentfdafda21:35
fhermeniok to me21:35
rockyggone!21:35
jrollcdent has opinions it seems?21:35
cdentjroll: I do, but not really on where the talk happens21:36
thingee#agreed fhermeni and harlow to work with nova-scheduler folks to further this along to later propose cross-project21:36
*** thinrichs has joined #openstack-meeting-cp21:36
jrollcdent: right on21:36
thingeefhermeni: thanks21:36
fhermeniwith pleasure21:36
thingee#topic rolling upgrades21:36
*** openstack changes topic to "rolling upgrades (Meeting topic: crossproject)"21:36
thingeekencjohnston: hi21:36
bauzasfhermeni: http://eavesdrop.openstack.org/#Nova_Scheduler_Team_Meeting21:36
kencjohnstonhowdy21:36
thingee#link https://review.openstack.org/#/c/290977/1 spec21:36
kencjohnstonthird first timer...21:37
elmikonice =)21:37
kencjohnstonThis spec was copied from a Product Work Group developed User Story21:37
kencjohnstonwith the intent of providing common problem description and use cases for upgrade improvements21:38
bknudsonwe didn't get rolling schema upgrades into keystone last release.21:38
kencjohnstonrealizing that work is underway in many projects on this front21:38
*** dansmith has joined #openstack-meeting-cp21:38
*** thinrichs has quit IRC21:38
thingeeso I think dansmith and sdague bring up a good point that this should reflect current use cases that are already in flight and in progress.21:39
thingeewe should start there21:39
kencjohnstonthingee yeah happy to do so and I wlll incorporate their feedback21:39
kfox1111this is similar to that ops spec.21:39
thingeehowever, this spec needs ownership of someone technical that can surface the problem, and agreed concepts of a solution as well as existing shared libraries like oslo.versionobjects21:39
kfox1111but more narrow in scope.21:39
thingeeto be clear, the product working group is trying to engage more with the community in the use cases they receive from things like the user survey.21:40
kencjohnstonthingee +121:40
thingeebut this was received the wrong way I think. mostly because it was looked as a wish list since some of the stuff is not the current direction today.21:41
gordcwe already have a tag for this? https://governance.openstack.org/reference/tags/assert_supports-rolling-upgrade.html21:41
thingeeHowever, I do think it would be good if the product working group was able to find someone who wants to own these repos from the technical side21:41
mriedemgordc: yeah i was going to point that out21:41
kencjohnstonthingee Correct, and that is on me. I misunderstood some important direction thingee provided me on that front.21:41
gordcare we trying to find owners for each project?21:42
mriedemif a project doesn't have the rolling upgrades tag, work with each project on what it takes to add rolling upgrade support21:42
thingeegordc: no, and I need to document this better from cross-project perspective...21:42
nikhilis this a guideline spec?21:42
nikhilor technical guideline spec? (first one being generic guideline)21:43
thingeeso cross-project specs I feel like should be general problem, general solutions and linking to existing libraries if they exist.21:43
*** samapthP has quit IRC21:43
nikhilor a use these existing tools and stop adding tech debt to my prj spec?21:43
thingeeindividual projects can do their own spec if they need to hash out particular details that involve their individual services.21:43
nikhilso all three then21:43
jrollseems like most CP specs specify an end goal and recommended solutions21:43
jrollthe end goal here is clear - the tag21:43
thingeebut we need these general ideas and solutions surfaced up somewhere, instead of what mriedem suggested of having to figure things out and dig for this information.21:43
samueldmqwhat is a cross-project spec?21:43
jrollthe recommended solutions is less clear, as we can't just copy/paste code21:44
samueldmqwhat does define something as a cp spec?21:44
nikhilyes, then we'd remove the Proposed solution Heading from the spec IMHO21:44
thingeesamueldmq: I think it's something that involves more than one project.21:44
jrollbut rather should have things that get you there - if you use RPC, you should version that properly. you should do expand/contract style migrations. etc.21:44
dansmithpersonally, I really don't want us to have a spec to cover all of upgrades21:44
dansmithif we need a spec to distill best practices for versioned RPC, then we can do that21:44
nikhilI just saw the previous 2 specs and the comment from Dan on this one and was confused21:44
dansmith(for example)21:45
jrolldansmith: ++21:45
gordcdansmith: +21:45
thingeedansmith: sure, if you want to call it that, that's fine with me21:45
jrollthere's a number of things (e.g. RPC) that projects do that make upgrades hard21:45
thingeedansmith: that's not different from the logging spec best practice21:45
thingeeor eventlet21:45
samueldmqthingee: I got that question after reading cdent's comment in patchset 2 at https://review.openstack.org/#/c/257809/21:45
fungiyeah, "upgrades" seems like a blanket idea, which can mean many different facets of design to different projects21:45
jrollwe could clarify what things those are, and how a project might solve them21:45
mriedemlike log guidelines i guess? http://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html21:45
jroll(whether that's one spec or many, I'm not opinionated)21:45
dansmithjroll: right, versioned rpc applies to a lot of projects.. versionedobjects will be picked up by fewer of those, and some will use neither21:45
thingeemriedem: exactly what I was mentioning above :) thanks21:46
jrollyep21:46
mriedemthingee: ah you beat me to logging :)21:46
jrolldansmith: there's other solutions too, like "don't change the rpc wire protocol" - crazy but it totally would work21:46
jroll:P21:46
thingeedansmith: I think a best practice is good.21:46
dansmithjroll: right, which is why I kinda hesitate to do any of this, but I think of all the topics, versioned RPC best practices is probably the widest applicability21:46
gordcjroll: that's what we use :)21:46
jrollI do question if a spec is needed for best practices, or if this should be more like a (project guide?) docs thing21:47
rockygAlso the upgrade tag is not the final resolution here, but the indicator that it was reached21:47
jrollor a blog, or a wiki21:47
fungietching the protocol in stone without possibility of future revision encourages unsavory sorts of data serialization with nasty security vulnerability surface area though21:47
nikhiljroll: ++21:47
jrollfungi: I was about 10% serious :)21:47
dansmithjroll: yeah, I've tended to put this kind of stuff in blogs21:47
fungiheh21:47
thingeejroll: as mriedem pointed out earlier, this isn't different from best practice of logging or eventlet which we cover today in our repo21:47
dansmithjroll: I'd rather leave specs and TC-blessed things for required stuff21:47
jrollthingee: okay, that's fair precedent21:48
jrollI don't have a strong opinion, I guess. writing things down is what's important to me21:48
nikhilWell I feel that many developers pick cp spec and just assume it to be source of truth21:48
* thingee thinks if there is disagreement of what should go in this repo, he should propose what should go in this repo in the project team guide21:48
thingeedansmith: ^21:48
nikhilI feel we need to decouple the technical details and emphasize more on best practices21:48
fungii feel like a lot of the disagreements over cross-project specifications have to do with meaning people are reading into the term "specification"21:48
dansmithnikhil: right, I'd hate for someone to read a spec, think it doesn't work for them, and decide not to pursue upgrades because that's the required solution21:48
rockygnikhil, if not source of truth, director to the various project sources of truth21:48
thingeeand then we can hash it out there.21:48
thingeedansmith: I mean we have precedent of best practices, but we don't have to keep doing that.21:49
jrollfungi: that's a good question21:49
thingeebut I think rolling upgrades is a common problem people are trying to solve21:50
dansmiththingee: yeah, I'm not super familiar with the PTG, so I can't really say, but sounds reasonable21:50
rockygfungi, ++21:50
nikhilthingee: I like CP specs, but it would be pretty good survey white paper sort of thing that has pointers to best articles or something of such sorts21:50
gordci don't think it's necessarily a bad thing we realised everything so far is probably overreaching. we're just confirming they are not required for everyone21:50
thingeedansmith: what's ptg here? :P21:50
funginova added specifications, then neutron, then others followed suit. then someone thought we should maybe have something like specs to coordinate stuff across multiple projects, but still called that a "specification"21:50
thingeedansmith: oh pwg?21:50
thingeeheh21:50
dansmiththingee: project team guide you mentioned above?21:50
samueldmqfungi: yes, either meaning of specification or cross-project :)21:50
thingeedansmith: oh gotcha21:51
dansmiththingee: I think I misread,21:51
dansmithyou meant use the PTG to describe what CP specs are for?21:51
rockygCan't remember which project, but they have RFEs for non developers to describe their needs21:51
thingeedansmith: yes, because there are disagreements here it seems.21:51
nikhilrockyg: ack and I agree to what you said21:51
jrollrockyg: neutron and ironic21:51
thingeesean dague isn't present, I know he had disagreements here.21:52
rockygAnd this doc is pretty much an RFE.21:52
samueldmqwhat does RFE stand for ?21:52
fungirfe is also known broadly in free software as a "wishlist item"21:52
dansmithrequest for enhancement21:52
rockygRequest for Enahancement21:52
samueldmqthanks21:53
fungias in "here's something we'd like to see happen, but we don't know how it should be done or who will actually do it"21:53
samueldmqand I agree it makes sense to have them21:53
samueldmqif we're going to get requirements from non-dev people21:53
rockygand once the RFE is fleshed out with the stuff we've talked about, either individual project specs or cross project specs or both can be generated by devs21:53
thingee#action thingee to define cross-project specs in project team guide21:53
dansmithfungi: right, usually only happens if money is exchanged somewhere :)21:53
samueldmqrockyg: ++21:53
thingeekind of a bummer that we have to go back to this, but I don't want us to keep getting blocked here.21:54
samueldmqthingee: nice! should our spec template include something as well?21:54
carolbarrettrockyg +121:54
kencjohnstonfungi I do want to clarify there is commitment from representatives of companies int he PWG to work on this (this being upgrades)21:54
bknudsonI think part of the problem is that as a volunteer org we need devs to be able to work on it21:54
carolbarrettkencjohnston: +121:54
thingee#topic stale specs21:55
*** openstack changes topic to "stale specs (Meeting topic: crossproject)"21:55
*** mriedem has quit IRC21:55
rockygI think we have devs, but not senior devs.  We need the cores and drivers to architect the right directions to go with this21:55
samueldmqbknudson: agreed, starting on getting RFEs and writting specs down21:55
jrollso nobody is arguing that we should or should not document best practices for projects that want to do upgrades, rather where it should be21:55
thingee#link http://lists.openstack.org/pipermail/openstack-dev/2016-March/089115.html21:55
carolbarrettbknudson: I believe there are devs working on it today and we can continue to ask them to work on this21:55
fungikencjohnston: carolbarrett: awesome. though it's generally helpful to have the people who will be working on it write up their proposal for how they'd do it so there's more to review21:55
thingeeI've listed some stale specs here.21:55
thingeelifeless has a bit :)21:55
thingeejroll: I'm just going to define what the cross-project specs should be for. We can start there.21:56
jrollthingee: cool21:56
gordcthingee: i would abandon them and let them reopen if someone actually cares about it21:56
lifelessthingee: I do :(21:56
thingeeso this involves the TC to abandon them. I don't have powers.21:56
thingeebut I wanted people to be aware21:57
lifelessbackwards compat is ongoing21:57
thingeeand revive whatever you care about21:57
gordcthingee: ah i see. :(21:57
samueldmqgordc: ++21:57
lifelessI need to update it21:57
thingeelifeless: ok!21:57
nikhilthingee: you open to running the bot thing that auto-abandons if not active for 2/3 weeks?21:57
lifelesstestr isn't on the map right now, though it is still something I want to do21:57
rockygSo, I'm willing to abandon mine if I can't update it by the summit.  I can always come back to it.21:57
nikhilthingee: ah, nvm21:57
thingee#topic open discussion21:57
*** openstack changes topic to "open discussion (Meeting topic: crossproject)"21:57
thingeecdent: ^21:57
fungiare there really enough cp specs that someone can't just manually abandon them when they seem to be going nowhere any more?21:57
samueldmqthingee: fyi I've talked to ayoung and he agrees with abandoning No global admin (Adam Young) - https://review.openstack.org/#/c/20562921:58
cdentthanks thingee: I just wanted to point people at this thread: http://lists.openstack.org/pipermail/openstack-dev/2016-March/088658.html21:58
jrollfungi: only the owner and the TC has abandon rights, afaik21:58
jrollcdent: this one is so much fun21:58
kfox1111I've got two specs that are still very important to our group, but I've not had time this cycle to follow up on, since its been a huge amount of work to try and get through.21:58
cdentthe death (or dying) of wsme impacts several projects and there's been some discussion about how to formalize or guide people in doing the a better thing21:58
kfox1111I'm about to deliver a production system so I'm hoping I'll have time coming up to push some more.21:58
cdentthere's been some discussion of announcing that projects _must_ migrate21:58
fungijroll: we _can_ fix that with acls, though either way the tc needs to be involved in the discussion of who should abandon changes for the repository they control and how21:59
nikhilheads up: We're having a discussion for quotas impl to be separate service or a lib and folks should expect a ML email from someone from the WG soonish.21:59
cdent(in say N months)21:59
elmikocdent: must migrate, as in away from wsme?21:59
cdenta thing like that seems like it would be vaguely cross-project21:59
jrollfungi: yeah, just pointing out why thingee hasn't done it I guess :)21:59
cdentelmiko: yes, but it is just an idea, not a decision21:59
jrollcdent: must? yikes21:59
elmikocdent: ack, just trying to understand better21:59
cdenta strawman21:59
jrollI don't remember seeing must in that thread, rather should21:59
thingeeone minute21:59
jroll(I agree, fwiw, I just don't want to force people to do it)21:59
fungiif the tc wants to grant thingee the ability to abandon/restore changes on that repo, it's easy to solve in gerrit with a line in an acl22:00
fungii'm happy to write the change to implement it22:00
bknudsonI vote to give thingee the ability to abandon changes22:00
rockygWell, if no one is maintaining it, there will be bitrot.  So migration is a question of when, not if22:00
elmikomust migrate does seem strong, but providing some nice guides on how to migrate would be awesome! (i know, more work)22:00
cdentelmiko: well exactly22:00
thingeeok that's time22:00
thingeethanks everyone for attending!22:00
nikhilthanks22:01
elmikothanks thingee22:01
thingee#endmeeting22:01
cdentthat was a biggun22:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"22:01
openstackMeeting ended Tue Mar 15 22:01:05 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/crossproject/2016/crossproject.2016-03-15-21.01.html22:01
samueldmqthanks22:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/crossproject/2016/crossproject.2016-03-15-21.01.txt22:01
jrollrockyg: well, those projects would need to be responsible for keeping it up (e.g. lucas is only helping because ironic uses it)22:01
openstackLog:            http://eavesdrop.openstack.org/meetings/crossproject/2016/crossproject.2016-03-15-21.01.log.html22:01
*** harlowja has joined #openstack-meeting-cp22:01
rockygjroll, yup.22:01
elmikocdent: imo, good case for improved api-wg example code ;)22:01
cdent:)22:01
rockyg++22:01
*** raildo-afk is now known as raildo22:02
rockygSo, really, spend the time to document how to migrate rather than maintaining the code.22:02
elmikoi should have something fun to show off at summit22:02
*** gordc has left #openstack-meeting-cp22:03
*** shamail has quit IRC22:04
*** fhermeni has left #openstack-meeting-cp22:04
*** raildo is now known as raildo-afk22:06
*** sdake has quit IRC22:09
*** rockyg has quit IRC22:10
*** dims_ has joined #openstack-meeting-cp22:11
*** david-lyle has quit IRC22:13
*** dims has quit IRC22:14
*** cdent has quit IRC22:15
*** dims has joined #openstack-meeting-cp22:16
*** dims_ has quit IRC22:17
*** beekhof has joined #openstack-meeting-cp22:23
*** ninag has quit IRC22:47
*** dims has quit IRC23:06
*** carolbarrett has quit IRC23:25
*** tyr_ has quit IRC23:26
*** dtroyer has left #openstack-meeting-cp23:36
*** sdake has joined #openstack-meeting-cp23:45
*** harlowja has quit IRC23:47
*** guessi has joined #openstack-meeting-cp23:51
*** guessi has left #openstack-meeting-cp23:55
*** sdake_ has joined #openstack-meeting-cp23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!