Wednesday, 2016-05-04

*** fragatin_ has joined #openstack-operators00:11
*** shakamunyi has quit IRC00:12
*** markvoelker has joined #openstack-operators00:14
*** fragatin_ has quit IRC00:14
*** fragatina has quit IRC00:14
*** fragatina has joined #openstack-operators00:15
*** SimonChung has quit IRC00:18
*** dminer has joined #openstack-operators00:22
*** dminer has quit IRC00:35
*** mageshgv has joined #openstack-operators00:40
*** VW has joined #openstack-operators00:47
*** VW has quit IRC00:52
*** mageshgv has quit IRC00:54
*** saneax_AFK is now known as saneax01:02
*** csoukup has joined #openstack-operators01:04
*** csoukup has quit IRC01:08
*** armax_ has joined #openstack-operators01:11
*** fragatin_ has joined #openstack-operators01:12
*** armax has quit IRC01:12
*** armax_ is now known as armax01:12
*** armax has quit IRC01:14
*** fragatin_ has quit IRC01:14
*** karl907 has quit IRC01:14
*** fragatina has quit IRC01:16
*** armax has joined #openstack-operators01:26
*** jamesdenton has joined #openstack-operators01:47
*** jamesden_ has joined #openstack-operators01:48
*** jamesdenton has quit IRC01:51
*** apoorvad has quit IRC01:56
*** cdelatte has quit IRC02:15
*** jamesden_ has quit IRC02:21
*** ducttape_ has joined #openstack-operators02:30
*** fragatina has joined #openstack-operators02:31
*** jaypipes has quit IRC02:40
*** ducttape_ has quit IRC02:45
*** VW has joined #openstack-operators02:46
*** mriedem has quit IRC02:47
*** VW has quit IRC02:48
*** VW has joined #openstack-operators02:48
*** markvoelker has quit IRC02:52
*** ducttape_ has joined #openstack-operators02:54
*** ducttape_ has quit IRC03:00
*** lhcheng has joined #openstack-operators03:01
*** saneax is now known as saneax_AFK03:02
*** ducttape_ has joined #openstack-operators03:03
*** saneax_AFK is now known as saneax03:10
*** ducttape_ has quit IRC03:46
*** markvoelker has joined #openstack-operators03:52
*** fragatina has quit IRC03:56
*** fragatina has joined #openstack-operators03:56
*** markvoelker has quit IRC03:57
*** liverpooler has quit IRC04:09
*** mageshgv has joined #openstack-operators04:26
*** amit213 has quit IRC04:44
*** mageshgv has quit IRC04:56
*** saneax is now known as saneax_AFK05:01
*** VW has quit IRC05:05
*** saneax_AFK is now known as saneax05:13
*** fragatina has quit IRC05:15
*** mageshgv has joined #openstack-operators05:17
*** mageshgv has quit IRC05:23
*** mnaser has quit IRC05:41
*** VW has joined #openstack-operators05:53
*** liverpooler has joined #openstack-operators05:55
*** GheRivero has left #openstack-operators06:02
*** mnaser has joined #openstack-operators06:03
*** david-lyle has quit IRC06:05
*** openstackgerrit has quit IRC06:17
*** openstackgerrit has joined #openstack-operators06:18
*** simon-AS5591 has joined #openstack-operators06:27
*** eddima has joined #openstack-operators06:31
*** lhcheng has quit IRC06:35
*** VW has quit IRC06:39
*** markvoelker has joined #openstack-operators06:54
*** markvoelker has quit IRC07:00
*** tesseract has joined #openstack-operators07:01
*** tesseract is now known as Guest2331907:01
*** matrohon has joined #openstack-operators07:14
*** zeih has joined #openstack-operators07:21
*** cfriesen has quit IRC07:35
*** dmk0202 has joined #openstack-operators07:54
pilgrimstack1j^2: what does it mean? how it could be big tent but not ATC?08:07
*** matrohon has quit IRC08:11
*** simon-AS559 has joined #openstack-operators08:37
*** simon-AS5591 has quit IRC08:38
*** zeih has quit IRC08:43
*** markvoelker has joined #openstack-operators08:50
*** saneax is now known as saneax_AFK08:54
*** markvoelker has quit IRC08:55
*** zeih has joined #openstack-operators08:55
*** derekh has joined #openstack-operators08:56
*** arnewiebalck has joined #openstack-operators09:00
*** electrofelix has joined #openstack-operators09:01
*** zeih has quit IRC09:12
*** zeih has joined #openstack-operators09:25
*** saneax_AFK is now known as saneax09:30
*** zeih has quit IRC09:35
*** fragatina has joined #openstack-operators09:59
*** dmk0202 has quit IRC09:59
*** haukebruno has joined #openstack-operators10:08
*** zeih has joined #openstack-operators10:20
*** david-lyle has joined #openstack-operators10:58
*** admin0 has joined #openstack-operators11:10
*** belmoreira has joined #openstack-operators11:15
*** admin0 has quit IRC11:16
*** xavpaice has quit IRC11:35
yankcrimepilgrimstack1: i think the plan is to credit osops-related contributions with something that isn't just a reflection of code committed11:38
*** xavpaice has joined #openstack-operators11:39
*** clickboom has joined #openstack-operators11:39
*** ducttape_ has joined #openstack-operators11:46
*** ducttape_ has quit IRC11:50
*** cdelatte has joined #openstack-operators12:02
*** markvoelker has joined #openstack-operators12:04
*** matrohon has joined #openstack-operators12:10
*** jaypipes has joined #openstack-operators12:11
*** VW has joined #openstack-operators12:16
*** clickboom has quit IRC12:27
*** VW has quit IRC12:33
*** admin0 has joined #openstack-operators12:34
*** admin0 has quit IRC12:39
*** julim has joined #openstack-operators12:47
RaginBajinyankcrime pilgrimstack1:  That's right. There were a few sessions at the Summit about providing a different type of recognition instead of ATC.  That way others who may not contribute code are recognized for their work.13:08
RaginBajinThis was the session - https://etherpad.openstack.org/p/AUS-ops-Ops-Recognition13:09
*** krotscheck_dcm is now known as krotscheck13:10
*** VW has joined #openstack-operators13:11
pilgrimstack1ping pfreund, this should concern yourself13:11
pfreundI was there.13:12
pfreund;)13:13
*** mageshgv has joined #openstack-operators13:16
*** jamesdenton has joined #openstack-operators13:16
*** mriedem has joined #openstack-operators13:16
*** armax has quit IRC13:19
*** armax has joined #openstack-operators13:20
mrhillsmanmorning13:20
*** ZucanGK has joined #openstack-operators13:23
*** ZucanGK is now known as Zucan13:25
*** electrofelix has quit IRC13:28
*** esker has quit IRC13:29
*** krobzaur has joined #openstack-operators13:30
*** electrofelix has joined #openstack-operators13:31
*** geseib has joined #openstack-operators13:33
*** fwdit has joined #openstack-operators13:38
RaginBajinmorning13:42
*** d0ugal has quit IRC13:46
*** d0ugal has joined #openstack-operators13:47
*** zioproto has quit IRC13:47
*** csoukup has joined #openstack-operators13:48
*** electrofelix has quit IRC13:52
*** zioproto has joined #openstack-operators13:52
*** electrofelix has joined #openstack-operators13:52
*** electrofelix has quit IRC13:57
*** electrofelix has joined #openstack-operators13:57
*** pilgrimstack1 has quit IRC14:01
*** zioproto has quit IRC14:02
*** jeh has quit IRC14:03
*** zioproto has joined #openstack-operators14:05
*** dtrainor has joined #openstack-operators14:05
*** VW has quit IRC14:08
*** jeh has joined #openstack-operators14:09
*** jeh has quit IRC14:09
*** jeh has joined #openstack-operators14:09
*** VW has joined #openstack-operators14:09
*** pilgrimstack has joined #openstack-operators14:10
*** VW has quit IRC14:12
*** dmk0202 has joined #openstack-operators14:14
*** zeih has quit IRC14:14
*** liverpooler has quit IRC14:15
*** Zucan has quit IRC14:16
*** jamesdenton has quit IRC14:22
*** jamesdenton has joined #openstack-operators14:22
*** fwdit has quit IRC14:25
*** julim has quit IRC14:32
*** dmk0202 has quit IRC14:33
*** zeih has joined #openstack-operators14:33
*** julim has joined #openstack-operators14:35
*** zeih has quit IRC14:36
*** zeih has joined #openstack-operators14:36
*** Zucan has joined #openstack-operators14:38
*** cfriesen has joined #openstack-operators14:41
*** VW has joined #openstack-operators14:42
*** VW has quit IRC14:42
*** VW has joined #openstack-operators14:42
*** vinsh_ has quit IRC14:47
*** vinsh has joined #openstack-operators14:47
*** wvera has joined #openstack-operators14:51
*** esker has joined #openstack-operators14:54
*** esker has quit IRC14:58
*** esker has joined #openstack-operators14:58
*** bvandenh has joined #openstack-operators15:00
*** zeih has quit IRC15:00
*** armax has quit IRC15:17
*** armax has joined #openstack-operators15:27
*** belmoreira has quit IRC15:34
*** fragatina has quit IRC15:37
*** armax has quit IRC15:38
*** armax has joined #openstack-operators15:38
*** mdorman has joined #openstack-operators15:46
*** Guest23319 has quit IRC15:46
*** geseib has quit IRC15:47
*** geseib has joined #openstack-operators15:47
*** matrohon has quit IRC15:49
*** apoorvad has joined #openstack-operators15:53
*** saneax is now known as saneax_AFK16:01
*** bvandenh has quit IRC16:04
*** eddima has quit IRC16:06
mdormananybody out there making use of private flavors much?16:08
*** haukebruno has quit IRC16:08
mgagnemdorman: with cells? :D16:10
*** fragatina has joined #openstack-operators16:13
*** SimonChung has joined #openstack-operators16:15
*** simon-AS559 has quit IRC16:20
*** vinsh has quit IRC16:26
*** vinsh_ has joined #openstack-operators16:26
*** cloudbert has quit IRC16:26
*** britthouser has joined #openstack-operators16:29
jlkmdorman: kinda? I think some of our customers are making flavors, that may or may not be public.16:35
mdormanmgagne:   well, yeah, with cells, hehe.16:43
mdormanmgagne / jlk    yeah we’re looking at making special flavors for certain users with specific use cases that need a much larger root disk , or something like that.  but we don’t wnat to expose those out to all the users generally16:44
mdormanjust curious how well the private flavors stuff actually works as far as them only being availble to a single tenant16:44
jlkI think ti's been working fine for us16:44
mdormankk16:46
mdormangood data point, thanks16:46
automagicallyWe use them pretty regularly as well16:46
automagicallyAnd they seem to work out well16:47
*** rbridgeman has joined #openstack-operators16:48
*** rbridgeman_ has joined #openstack-operators16:49
*** rbridgeman has quit IRC16:53
jlkSo good news. It appears you can cleanly and easily upgrade from Kilo directly to Mitaka.16:53
*** derekh has quit IRC16:57
mgagnejlk: great news! :D17:05
jlkI only had to make minimal changes to our upgrade playbook we used for havana -> kilo.17:05
mgagnejlk: so this means we might be able to skip a couple of versions too :D https://twitter.com/mgagne/status/72620143101600563317:05
jlkheh17:06
jlkeven more awesome, every step of the way my cloud stayed minimally functional, testing between each major service upgrade.17:06
*** shakamunyi has joined #openstack-operators17:07
*** shakamunyi has quit IRC17:13
*** shakamunyi has joined #openstack-operators17:14
mriedemso, are there any large production openstack clouds running with 0mq that anyone is aware of?17:16
mriedemand doing it better than with rabbitmq?17:16
mriedemjlk: klindgren: j^2: ^?17:17
jlkI haven't heard of anybody at scale being successful with 0mq17:18
jlkfor... just about anything, let alone OpenStack17:18
klindgrenI am not aware of anyone doning 0mq either17:18
mriedemok, thanks for the info17:18
mriedemi didn't think so17:18
* mriedem is about to send a rant17:18
klindgrenI thought mirantis was doing some testing and showed that for openstack 0mq was slower?17:18
mriedemjaypipes told me that they are testing 0mq and have had issues17:19
klindgrenpart of that might be the way components are using 0mq.  IE Openstack's implementation of 0mq is less than ideal and no one has put the time in it to make it better.17:20
mriedemsure, and people using rabbit is what has made it better in oslo.messaging, like the heartbeat thing17:21
klindgrenexactly17:22
jlkyeah, I'd be really really curious what problem people are trying to solve with 0mq17:22
jlkCells + rabbit seems to have scaled rather well for RAX17:23
mgagneIf we really want to talk about scaling, I hope we design with 100k+ nodes in mind then.17:24
jlkWell, RAX wasn't there yet, when I left17:25
jlk60K~ between 6~ regions17:26
mgagneI'm more or less hoping that planning for much more than needed will force people to find and fix limitations in design and plan for hyperscale instead17:27
mgagne10k is enough right? then comes the next guy with 50k nodes and we have to redesign again17:28
jlksure, but some designs for super scale are overly complicated for reasonable scale.17:30
mgagnetrue and I unfortunately don't have any proposition. We already had that situation anyway with RAX writing cells on its own. So I have this weird feeling that history will repeat itself again.17:32
mgagneI'm however very curious about the implementation details of cellsv2, didn't have time to check yet17:33
mgagnestill trying to make xenial responds to ping :-/17:33
mgagnewhich isn't glamorous :-/17:33
jlkhah17:34
jlkI sat in a cellsv2 design session at summit17:34
jlkit's still a ways off.17:34
jlkand there are still concerns with scheduler bottleneck given the design of v217:34
mgagnejlk: yes, that's my main concern, scheduling performance17:34
jlkso with cellsv2, there is still just a single scheduler17:35
jlkthere is not a scheduler per-cell17:35
mgagneyea... we were able to partition cloud and baremetal in cells. now it would be gone17:35
jlkimproving scheduler performance is its own project, not necessarily tied to cells.17:35
mgagneand we already see races in baremetal scheduling due to the number of nodes and the way ironic driver in Nova compute works (lack of clustering)17:36
*** electrofelix has quit IRC17:36
jlkUpstream gate tests aren't even really testing cells yet. THere is a plan for single-cell testing, where nova-api and other services are separated across two nodes.  Multi-cell testing comes later.17:36
mgagnebut the presentation made by Intel about scheduling performance gave me great insight about the bottleneck and how to improve it17:37
jlkah I missed that one17:37
jlkglad to see people putting resources behind improving it though17:37
jlkNova has been talking about splitting scheduling off to it's own DB as well17:37
mgagnejlk: https://www.youtube.com/watch?v=BcHyiOdme2s17:38
jlkTHanks, one of hundreds of those I need to watch :/17:38
mgagneyea, just not enough time to watch them all and do work at the same time =(17:39
*** bvandenh has joined #openstack-operators17:47
*** mriedem has quit IRC17:52
*** mriedem has joined #openstack-operators17:54
*** bloatyfloat has joined #openstack-operators17:59
*** wvera has quit IRC18:16
claytonmdorman: we do private flavors pretty regularly18:23
claytonyou can give more than one tenant access to a private flavor18:23
claytonwe have some that are kind of standard that we give people access to as needed18:23
mdormancool cool, thanks for info.18:25
*** dronshaw has joined #openstack-operators18:29
*** jamesden_ has joined #openstack-operators18:30
*** jamesdenton has quit IRC18:33
*** wvera has joined #openstack-operators18:35
*** vinsh_ has quit IRC18:36
*** vinsh has joined #openstack-operators18:37
*** liverpooler has joined #openstack-operators18:38
klindgrenRe: scale - I would like to see projects be able to handle "modderate scale" without sharding - but then have a sharding story to handle massive scale.18:39
jlkso that's basically single cell vs multiple cells18:42
klindgrenmoderate scale being between 5-10k nodes - 100k-200k vm's, imho. before sharding is needed18:42
jlkand a scale point for single cell where things start to fall over18:42
jlkor do you mean sharding as in regions?18:42
klindgrencells like solution18:42
jlkbecause with your numbers, I think that's more scaling (with cells) within a single region, and then going to regions for the next level up.18:43
klindgrenbascially - I want to run a whole DC behind openstack18:43
mdormanfor varying size values of “DC” :)18:43
jlkright18:44
klindgrenWithout having to have regions like DC - server room a DC - server room b.18:44
jlkthat's what RAX did. Each DC was essentially a region.18:44
mdormanyeah same for us18:44
mgagnejlk: I don't feel regions should be seen as a scaling solution18:44
jlkeach DC would scale upwards to 30K or so HVs before I left18:44
jlkmgagne: no, it just happened to work well18:44
jlkno idea what happens if you try to throw 60K HVs at one region.18:45
jlkwe just don't have any clue what the pain points are for A) size of a single cell, B) aggregate size of all cells, and C) sheer # of cells18:45
mgagnejlk: how do you think that A provider doing scaling at the control plane? nothing the user can see for sure18:47
jlkI'm having a hard time parsing that question18:47
mgagnewhat is EC2 doing to scale?18:47
klindgrennot using openstack18:48
mdormanwouldn’t we all like to know :)18:48
klindgren>_>18:48
klindgren<_<18:48
mgagnejlk: clearly you don't have to switch "regions" in the same AZ zone and I'm sure it supports much more than 10k nodes18:48
jlkwell it really depends on how you set up your control plane18:48
jlkNothing officially scales that big18:48
jlkdumping all those nodes into a single cell is going to have some pain points. DB access contention (sheer bandwidth concerns), rabbit contention, glance bandwidth, scheduler performance, nova conductor performance, network segmentation, etc...18:49
mdormanyeah there is always going to be some other bottleneck somewhere18:50
mgagnejlk: I'm sure they have a different design than openstack :D18:50
klindgrennot to mention failure domain size18:50
*** simon-AS559 has joined #openstack-operators18:50
jlkCurrently in OpenStack there is no tech to scale out of those problems. Cells is the answer for that, for better or worse, and cellsv1 is just incomplete, and cellsv2 is also incomplete18:50
mgagne2 incomplete versions? :O18:51
mdormanhaha18:51
jlkyeah, cellsv1 was done out of tree by RAX, and it has some tradeoffs18:51
jlkit was never complete enough to go in-tree18:51
mgagne=)18:51
jlkthe cellsv2 design is loosely based on it, but significantly different in some ways18:51
jlkand, well, just isn't really complete yet.18:51
mdormangetting there18:52
klindgrenFor next summit - I am thinking of a t-shirt that says: I am the original cell0.18:52
jlkadd to this we have so few companies doing scale beyond 10K nodes18:52
jlkgetting any sort of real #s is going to be difficult, as it can vary greatly on workload18:53
klindgrenyea - that is an interesting point where per the user survey most openstack ( like > 80%, iirc) installs are under 100 nodes18:54
klindgrenwith more than 50% being under 2018:54
jlkIt takes a lot of $$ to drive 10K nodes18:54
mgagneare we considering baremetal nodes? =)18:55
jlkgenerally yes, the hypervisor count is 1 HV per bare metal computer18:55
mgagnehehe18:55
mgagnewe might get over 10k soonish then =)18:56
mdorman:)18:56
mgagnewe fun to hear about how Mirantis enrolled existing nodes to Ironic18:56
mdormanRaginBajin:  may be afew minutes late for meeting, need to go grab food real quick first.   but i’ll catch up on the convo.18:57
RaginBajinSounds good.18:57
mgagneour reaction was: Hey! we could have given that talk!18:57
jlkI've seen some really good demos from Deva using bifrost for that19:00
*** vinsh_ has joined #openstack-operators19:00
*** vinsh has quit IRC19:04
*** piet has joined #openstack-operators19:10
*** rockyg has joined #openstack-operators19:10
*** rmcall has joined #openstack-operators19:13
*** dronshaw has quit IRC19:15
mgagnejlk: this slide summarizes the enrollment process: https://www.youtube.com/watch?v=IIGKSYEOt6Y&feature=youtu.be&list=PLRkaKWXET-H6IBSoFzKA1LYJ83IKTQCVi&t=107819:18
mgagneuse Ironic fake driver, use scheduler hint in Nova to enroll Ironic node in Nova19:19
*** dronshaw has joined #openstack-operators19:20
*** lhcheng has joined #openstack-operators19:22
*** apoorvad has quit IRC19:31
*** geseib has quit IRC19:39
*** geseib has joined #openstack-operators19:39
*** piet has quit IRC19:44
*** openstackgerrit has quit IRC19:48
*** openstackgerrit has joined #openstack-operators19:48
*** piet has joined #openstack-operators19:49
*** gyee has joined #openstack-operators19:57
*** wvera has quit IRC20:01
*** lathiat has left #openstack-operators20:06
*** dminer has joined #openstack-operators20:08
*** piet has quit IRC20:08
*** harlowja has quit IRC20:09
*** wvera has joined #openstack-operators20:10
*** cdelatte has quit IRC20:12
*** bvandenh has quit IRC20:13
*** dmk0202 has joined #openstack-operators20:14
*** wvera has quit IRC20:15
*** apoorvad has joined #openstack-operators20:15
*** fawadkhaliq has joined #openstack-operators20:27
*** gyee has quit IRC20:27
*** shakamunyi has quit IRC20:29
*** gyee has joined #openstack-operators20:30
*** cdelatte has joined #openstack-operators20:31
*** wvera has joined #openstack-operators20:41
*** wvera has quit IRC20:46
*** fawadk has joined #openstack-operators20:47
*** esker has quit IRC20:48
*** fawadkhaliq has quit IRC20:50
*** SimonChung has quit IRC20:52
*** SimonChung1 has joined #openstack-operators20:52
*** jaypipes has quit IRC20:56
*** wvera has joined #openstack-operators20:58
*** julim has quit IRC20:59
*** ducttape_ has joined #openstack-operators21:00
*** mjrichardson has joined #openstack-operators21:00
*** julim has joined #openstack-operators21:01
*** shakamunyi has joined #openstack-operators21:04
*** gyee has quit IRC21:06
*** shakamunyi has quit IRC21:06
*** shakamunyi has joined #openstack-operators21:07
*** shakamunyi has quit IRC21:08
*** harlowja has joined #openstack-operators21:10
*** dronshaw has quit IRC21:12
*** mdorman has quit IRC21:17
*** dbecker has quit IRC21:20
*** gyee has joined #openstack-operators21:20
*** piet has joined #openstack-operators21:21
*** gyee has quit IRC21:27
*** Zucan has quit IRC21:30
*** VW has quit IRC21:32
*** VW has joined #openstack-operators21:35
*** gyee has joined #openstack-operators21:39
*** ducttape_ has quit IRC21:42
*** VW has quit IRC21:42
*** chlong has quit IRC21:45
*** rmcall has quit IRC21:47
*** VW has joined #openstack-operators21:48
*** fragatin_ has joined #openstack-operators21:50
*** mriedem has quit IRC21:50
*** fragatina has quit IRC21:51
*** VW_ has joined #openstack-operators21:53
*** rockyg has quit IRC21:56
*** VW has quit IRC21:56
*** fragatin_ has quit IRC21:57
*** mageshgv has quit IRC21:57
*** fragatina has joined #openstack-operators21:57
*** VW_ has quit IRC21:58
*** dmk0202 has quit IRC22:01
*** rockyg has joined #openstack-operators22:04
*** chlong has joined #openstack-operators22:05
*** julim has quit IRC22:06
*** simon-AS559 has quit IRC22:07
*** jamesden_ has quit IRC22:13
*** csoukup has quit IRC22:15
*** shakamunyi has joined #openstack-operators22:19
*** ducttape_ has joined #openstack-operators22:24
*** blair has quit IRC22:24
*** piet has quit IRC22:28
*** jvalente has quit IRC22:41
*** wvera has quit IRC22:47
*** harlowja has quit IRC22:52
*** harlowja has joined #openstack-operators22:53
*** ducttape_ has quit IRC23:00
*** sorrison_laptop has joined #openstack-operators23:20
*** ducttape_ has joined #openstack-operators23:29
*** bapalm_ has quit IRC23:30
*** fragatina has quit IRC23:39
*** fragatina has joined #openstack-operators23:40
*** piet has joined #openstack-operators23:48
*** fawadkhaliq has joined #openstack-operators23:50
*** amit213 has joined #openstack-operators23:51
*** fawadk has quit IRC23:52

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!