Tuesday, 2017-12-19

*** chyka has quit IRC00:00
*** d0ugal has joined #openstack-operators00:04
*** rcernin has quit IRC00:09
*** rcernin_ has joined #openstack-operators00:09
*** VW_ has joined #openstack-operators00:15
*** VW has quit IRC00:18
*** VW_ has quit IRC00:19
*** aojea has joined #openstack-operators00:23
*** aojea has quit IRC00:28
*** marst has quit IRC00:35
*** tuanla____ has joined #openstack-operators00:38
*** chito_ has quit IRC00:38
*** d0ugal has quit IRC00:40
*** rcernin_ has quit IRC00:50
*** d0ugal has joined #openstack-operators00:51
*** tuanla____ has quit IRC00:53
*** tuanla____ has joined #openstack-operators00:54
*** catintheroof has joined #openstack-operators00:56
*** chyka has joined #openstack-operators01:01
*** gyee has quit IRC01:01
*** chyka has quit IRC01:06
*** catintheroof has quit IRC01:21
*** Apoorva_ has joined #openstack-operators01:32
*** linkmark has quit IRC01:33
*** Apoorva has quit IRC01:36
*** Apoorva_ has quit IRC01:37
*** trungnv has joined #openstack-operators01:52
*** penick has quit IRC02:07
*** rcernin has joined #openstack-operators02:11
*** tuanla____ has quit IRC02:20
*** tuanla____ has joined #openstack-operators02:21
*** aojea has joined #openstack-operators02:25
*** Apoorva has joined #openstack-operators02:29
*** aojea has quit IRC02:29
*** trungnv has quit IRC02:38
*** AlexeyAbashkin has joined #openstack-operators02:38
*** AlexeyAbashkin has quit IRC02:43
*** mriedem has quit IRC02:56
*** markvoelker has joined #openstack-operators03:24
*** VW has joined #openstack-operators03:29
*** khyr0n has joined #openstack-operators03:45
*** VW has quit IRC03:48
*** VW has joined #openstack-operators03:52
*** markvoelker has quit IRC03:58
*** Apoorva has quit IRC04:04
*** VW has quit IRC04:07
*** fragatina has quit IRC04:20
*** fragatina has joined #openstack-operators04:20
*** aojea has joined #openstack-operators04:25
*** dmibrid_ has quit IRC04:28
*** aojea has quit IRC04:30
*** khyr0n has quit IRC04:37
*** tuanla____ has quit IRC04:54
*** tuanla____ has joined #openstack-operators04:55
*** markvoelker has joined #openstack-operators04:55
*** khyr0n has joined #openstack-operators04:58
*** markvoelker has quit IRC05:28
*** penick has joined #openstack-operators05:32
*** tuanla____ has quit IRC05:33
*** penick_ has joined #openstack-operators05:36
*** penick has quit IRC05:36
*** chyka has joined #openstack-operators05:50
*** chyka has quit IRC05:51
*** tuanla____ has joined #openstack-operators05:58
*** penick_ has quit IRC06:12
*** chyka has joined #openstack-operators06:15
*** aojea has joined #openstack-operators06:17
*** chyka has quit IRC06:20
*** markvoelker has joined #openstack-operators06:25
*** aojea has quit IRC06:28
*** gkadam has joined #openstack-operators06:38
*** trungnv has joined #openstack-operators06:43
*** aojea has joined #openstack-operators06:50
*** aojea has quit IRC06:50
*** aojea has joined #openstack-operators06:50
*** aojea has quit IRC06:52
*** yprokule has joined #openstack-operators06:53
*** markvoelker has quit IRC06:59
*** aojea has joined #openstack-operators07:02
*** aojea has quit IRC07:06
*** chyka has joined #openstack-operators07:08
*** chyka has quit IRC07:13
*** lvdombrkr has quit IRC07:22
*** rcernin has quit IRC07:31
*** ircuser-1 has joined #openstack-operators07:39
*** lvdombrkr has joined #openstack-operators07:42
*** AlexeyAbashkin has joined #openstack-operators07:52
*** rcernin has joined #openstack-operators08:02
*** d0ugal has quit IRC08:03
*** v12aml has quit IRC08:03
*** d0ugal has joined #openstack-operators08:07
*** tuanla____ has quit IRC08:10
*** tuanla____ has joined #openstack-operators08:11
*** v12aml has joined #openstack-operators08:27
*** khyr0n has quit IRC08:37
*** racedo has joined #openstack-operators08:37
*** aojea has joined #openstack-operators08:45
*** trungnv has quit IRC08:52
*** markvoelker has joined #openstack-operators08:56
*** chyka has joined #openstack-operators08:58
*** chyka has quit IRC09:02
*** aojea has quit IRC09:05
*** aojea_ has joined #openstack-operators09:08
*** jsheeren has joined #openstack-operators09:13
*** treiz has quit IRC09:17
*** treiz has joined #openstack-operators09:18
*** markvoelker has quit IRC09:29
*** derekh has joined #openstack-operators09:36
*** treiz has quit IRC09:52
*** treiz has joined #openstack-operators09:53
*** treiz has quit IRC09:57
*** treiz has joined #openstack-operators10:00
*** yprokule has quit IRC10:15
*** gfa has quit IRC10:15
*** yprokule has joined #openstack-operators10:18
*** fragatin_ has joined #openstack-operators10:22
*** fragatina has quit IRC10:22
*** markvoelker has joined #openstack-operators10:26
*** gfa has joined #openstack-operators10:42
*** electrofelix has joined #openstack-operators10:46
*** fragatina has joined #openstack-operators10:57
*** fragatin_ has quit IRC10:57
*** markvoelker has quit IRC11:00
*** AlexeyAbashkin has quit IRC11:22
*** AlexeyAbashkin has joined #openstack-operators11:38
*** markvoelker has joined #openstack-operators11:57
*** aojea_ has quit IRC12:24
*** markvoelker has quit IRC12:29
*** tuanla____ has quit IRC12:39
*** catintheroof has joined #openstack-operators13:11
*** catintheroof has quit IRC13:12
*** catintheroof has joined #openstack-operators13:12
*** d0ugal has quit IRC13:14
*** rcernin has quit IRC13:20
*** d0ugal has joined #openstack-operators13:22
*** aojea has joined #openstack-operators13:25
*** markvoelker has joined #openstack-operators13:27
*** markvoelker has quit IRC13:28
*** markvoelker has joined #openstack-operators13:29
*** d0ugal has quit IRC13:37
*** aojea has quit IRC13:38
*** d0ugal has joined #openstack-operators13:39
*** liverpooler has joined #openstack-operators13:40
*** jsheeren has quit IRC13:44
*** mihalis68 has joined #openstack-operators13:48
*** makowals has joined #openstack-operators13:53
*** yprokule has quit IRC13:56
*** mriedem has joined #openstack-operators13:58
mihalis68it is time for the openstack operators weekly meeting but i don't see anyone. I will give it some time in case there aren't enough attendees14:01
*** shintaro has joined #openstack-operators14:01
mihalis68hi!14:02
shintarohi14:03
mrhillsmano/14:03
mihalis68o/14:03
mihalis68#startmeeting Ops Meetup Team14:03
openstackMeeting started Tue Dec 19 14:03:45 2017 UTC and is due to finish in 60 minutes.  The chair is mihalis68. Information about MeetBot at http://wiki.debian.org/MeetBot.14:03
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:03
*** openstack changes topic to " (Meeting topic: Ops Meetup Team)"14:03
openstackThe meeting name has been set to 'ops_meetup_team'14:03
mihalis68#link https://etherpad.openstack.org/p/ops-meetups-team14:03
mihalis68#topic actions14:04
*** openstack changes topic to "actions (Meeting topic: Ops Meetup Team)"14:04
mihalis68I have requested that we (Bloomberg) sponsor breakfast/coffee for the Tokyo meetup14:04
mihalis68that completes my actions14:04
shintaronot much update from me this week14:04
mihalis68anyone else have pending past actions (I'm going to go on the honor system for this)14:04
mihalis68breakfast/coffee was $2k, which although not that much is a bit higher than at other meetups if I recall correctly14:05
mihalis68is that just because prices are high in Tokyo?14:05
*** emccormick has joined #openstack-operators14:05
mihalis68hey!14:05
emccormicko/14:05
mihalis68welcome back14:05
mihalis68we are just in the reviewing past actions14:05
emccormickthanks! Sorry for being so absent14:05
mihalis68part of the meeting14:05
mihalis68it's the time of year, lots of people have a lot going on14:06
emccormickmy action item was to get here. Done! :D14:06
mihalis68no other action updates14:06
mihalis68#topic Tokyo meetup, march 201814:06
*** openstack changes topic to "Tokyo meetup, march 2018 (Meeting topic: Ops Meetup Team)"14:06
shintaroI don't remember the past price for breakfast, but thats what I got from the venue and my contractor14:06
mihalis68I've requested that we sponsor breakfast, but it's not my call14:07
mihalis68I don't think it's a bad price, but I might get asked about it, so thought I'd try and be prepared14:07
mihalis68I think it's a good opportunity since we get to be a co-sponsor for a lot less than NTT :)14:07
mihalis68NTT is also doing the evening event I think you said last week?14:08
shintaroyes we will14:08
mihalis68the only things left to do then are (possibly) schwag and the technical agenda14:08
mihalis68I haven't looked at the agenda for a while, how is it going?14:09
shintaroI've asked the foundation to provide T-shirts14:09
shintaroI've added popular topics from the past events.14:09
shintaro#link https://etherpad.openstack.org/p/TYO-ops-meetup-201814:10
shintaronot much update14:10
mihalis68I think we can pick up the technical agenda in the new year. I'll email the list asking for input on that after this meeting and we can maybe review when there's more of us available14:12
mihalis68#action mihalis68 to remind the operators mailing list to look at the technical agenda for tokyo14:12
shintarook14:12
mihalis68#topic meetup #2 august 201814:12
*** openstack changes topic to "meetup #2 august 2018 (Meeting topic: Ops Meetup Team)"14:12
mihalis68I mentioned last week or the week before: Bloomberg may offer to host in NYC again August14:13
shintaro+114:13
mihalis68#action mihalis68 to push for a firm decision on hosting in august 201814:15
mihalis68anyone else have a gut feel on whether they'd attend?14:15
emccormickWell that'd be nice. Maybe I could actually make this one!14:15
emccormickI would definitely be there14:15
mihalis68Cool14:15
mrhillsmanamrith mentioned verizon may be willing to host something14:16
emccormicklast time I just got offered a boat load of money to fly to mexico and do some work and couldn't say no ;)14:16
mrhillsmani think if bloomberg offers, verizon can help or vice versa14:16
mrhillsmanprefer the former since NY last time was pretty good14:16
mihalis68it would be good to get a contact at verizon we could talk to14:16
shintaroWe have OpenStack days Tokyo on 2nd-3rd Aug so I hope it wont overlap.14:16
mrhillsmanamrith14:16
mihalis68that's a good point14:16
mrhillsmanamrith kumar14:16
mrhillsmani do not have his email, but he is quite active14:17
emccormickwhere is Amrith at?14:17
mrhillsmanlet me try to find contact info14:17
mihalis68ironically my wife works for verizon, however she's in their advertising part, no contact with the likely openstack-interested parts14:18
mrhillsmanmaybe she can get in touch or someone who can14:18
mihalis68I'll look for amrith first14:19
mrhillsmanhe is a fellow there so should be easy to find him14:19
mihalis68ok cool14:19
mihalis68#topic new logo14:19
*** openstack changes topic to "new logo (Meeting topic: Ops Meetup Team)"14:19
mihalis68there are two new logo proposals from the foundation in an email on the operators mailing list14:20
mihalis68I'm not sure everyone who is here today has seen and responded14:20
mihalis68I'd encourage everyone to do so14:20
mihalis68there's a shield-shaped one that I like and I think others did, and then a round one that's less popular so far14:20
shintarowe haven't got response from Clair, but it was clear everyone liked the first one14:20
emccormickFeedback so far seemed pretty universally in one camp.14:21
emccormick*nod*14:21
mihalis68I have a feeling the shield already won, but I did want everyone to get a chance to speak up14:21
shintaroI will confirm Clair for update14:21
mihalis68#action shintaro to check on new logo status with Clair from the foundation14:21
mihalis68#topic opsguide14:22
*** openstack changes topic to "opsguide (Meeting topic: Ops Meetup Team)"14:22
mihalis68#link https://wiki.openstack.org/wiki/OpsGuide14:22
mihalis68this is the new conversion of the OpenStack Operators Guide which was ejected from the core docs maintained with code and given to the community14:22
mihalis68I still haven't had meaningful feedback on this14:23
emccormickI think it looks great.14:24
emccormickI still have on my list to go through and start reviewing content14:24
emccormickI have gotten as far as opening it in its own browser window ...14:24
mihalis68the reason I keep asking is that right now it's as converted from the source14:24
mihalis68systematic changes could be made efficiently throughout the document via code changes14:25
mihalis68once it starts being edited by the wiki process, however, it is forever after a bunch of wiki pages14:25
mihalis68one idea I just had...14:25
mihalis68check me on this, may be premature, but...14:26
mihalis68we could logo-ify it with our "shield" !14:26
shintaro+114:27
emccormick+114:27
mihalis68ok14:27
emccormickif we gotta own it, let's own it14:27
shintarowhat was the process to give feedback on wiki? I will spread it around in Japanese community14:27
mihalis68I'll get my guy to do that and maybe we can "approve" this via email amongst us first and then unleash to operators community via the mailing list?14:27
mihalis68For now I am de facto the owner, so email to me14:28
mihalis68Bloomberg tasked a contractor of ours to do the conversipon14:28
shintarook thanks14:28
mihalis68with the aim of giving it back to the community in a high quality form14:28
mihalis68I could just say "ok good to go, have at it"14:28
emccormickJust be I don't see any reason you need to go for wider approval on a logo. Just let anyone with a problem complain. We can always take it off14:29
emccormicknothing objectionable though. Let's ping the list to get people looking at it.14:29
mihalis68however feedback from, say mccormick, shintaro etc would make me feel more confident that it's a good start14:29
emccormickI sent a link to a sample wiki before for tracking bugs and what people have reviewed...14:29
emccormickcould be a google doc also.14:29
mihalis68well this is kind of what I'm asking. By default a wiki is just open to anyone with edit privileges to do whatever14:30
mihalis68but if we launch with defined owners, we get to define a better process14:30
mihalis68I am happy to continue to be AN owner but there needs to be several or it's not at all a community thing14:31
emccormickYeah we need lots of people to dip in14:31
shintaroright14:31
mihalis68let's discuss amongst ourselves for now via email until we have consensus to broaden the scope. The hesitancy is just around having a good launch with enough process that it takes on some life of its own and is not just an abandonware doc that moved from docs to wiki14:32
emccormickhere14:32
emccormickhttps://wiki.openstack.org/wiki/Documentation/QueensDocTesting14:32
emccormickthat's just a sample tracking doc14:33
mihalis68is your proposal we make a document similar to that for launch and then maintenance of OpsGuide?14:33
mihalis68by the way med_ also planned to take a look and provide feedback14:34
mihalis68haven't heard yet14:34
mihalis68#topic any other business14:35
*** openstack changes topic to "any other business (Meeting topic: Ops Meetup Team)"14:35
mihalis68we are few in number today, so may as well zip through the agenda and give everyone some time back14:35
mihalis68any final comments?14:35
mihalis68I propose we officially NOT try to meet next week14:35
shintaro+114:36
mihalis68any other response? That would make next meeting Jan 2nd 14:00 UTC14:36
shintaroand the week after, I will not be available14:36
mihalis68unless that is seen as a bad time too?14:36
emccormick+114:37
shintaroI will be on vacation until Jan 8 so I will be back on 9th14:37
emccormickI can be here on the 2nd14:37
emccormickkida back to school *snoopy dance*14:37
mihalis68but will it be worthwhile to try on 2nd?14:38
mihalis68I guess we can meet and wrap-up quickly if not much business?14:38
mihalis68I don't know if I'll be at work or out but a quick meeting is likely fine either way14:38
mihalis68anyone else?14:38
mihalis68ok, we seem to have almost no opinion but it's slightly in favor of Jan 2nd14:40
mihalis68#agreed next meeting jan 2nd normal time14:40
mihalis68last call for any other business14:40
mihalis68#endmeeting14:41
*** openstack changes topic to "Discussion of topics related to operating Openstack infrastructures at scale | Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators | Logs: http://eavesdrop.openstack.org/irclogs/"14:41
openstackMeeting ended Tue Dec 19 14:41:19 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:41
openstackMinutes:        http://eavesdrop01.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-12-19-14.03.html14:41
openstackMinutes (text): http://eavesdrop01.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-12-19-14.03.txt14:41
openstackLog:            http://eavesdrop01.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-12-19-14.03.log.html14:41
mihalis68thanks everyone14:41
shintarothank you14:41
*** shintaro has quit IRC14:41
emccormicko/14:41
emccormickhappy holidays all of you14:42
emccormicksee you next year!14:42
*** VW has joined #openstack-operators14:43
mihalis68hey VW we already finished14:45
*** smcginnis has joined #openstack-operators14:45
mihalis68there was very little to this meeting14:45
VWok - mihalis68 - sorry14:45
mihalis68hey sean, just telling vw we already finished14:45
mihalis68you did not miss much, don't worry14:45
smcginnisBah, missed it again! :/14:45
mihalis68next meeting attempt Jan 2nd14:45
VWhad to drop all three kids off this morning14:45
VWtook extra time14:45
smcginnismihalis68: Thanks, will try to get my phone to stop giving me calendar updates an hour off. :)14:45
mihalis68I did an email reminder this time, but very late14:46
mihalis68can do better14:46
mihalis68we didn't even discuss the openstack release cadence this time,14:46
mihalis68still trying to comprehend all that discussion14:46
smcginnismihalis68: I think enough downsides have been pointed out that it isn't going to happen. At least any time soon.14:47
smcginnisBut might be good to have an opinion on some of the issues it raises and how that might impact operators.14:47
mihalis68somehow every suggestion is eventually converted into a plan that would require more effort, and so is obviously unworkable14:48
emccormickHuh. I thought it was somewhat well received. Perhaps I missed a new bit of flames on -dev14:48
mihalis68do fewer releases! More work :(14:48
mihalis68do more releases! More work :(14:48
smcginnisemccormick: I think the initial responses were well recieved, but then more issues were raised with it.14:49
emccormickI'll finish and respond on list.14:49
smcginnisThierry posted a nice recap of the dicussion on the openstack-dev list.14:49
smcginnisIf it would help, I can forward that to the openstack-operators list.14:49
smcginnisOr just give a link here.14:50
emccormickI'll get caught up.14:50
emccormickI was generally in favor of it, but if we can get fast-forward upgrades going then I care much less14:50
emccormickand LTS14:50
mihalis68I think it would be good to xpost to -operators14:50
mihalis68although I now do follow -dev14:50
smcginnisOK, will do. Just for convenience, here's a link to the original post: http://lists.openstack.org/pipermail/openstack-dev/2017-December/125688.html14:51
mihalis68somehow I missed every previous subscription confirmation email14:51
smcginnismihalis68: Spam filter?14:51
mihalis68*mutter* spam filters *mutter*14:51
mihalis68yeah14:51
smcginnis:)14:51
mihalis68not too hard to avoid if I try14:51
mihalis68but failed numerous times14:51
mihalis68finally did it14:51
mihalis68hey I can put Tokyo breakfast/coffee on my corporate Amex, just got approval14:51
smcginnisMmmm, coffee.14:53
mihalis68Bloomberg believes strongly in coffee14:53
*** sxc731 has joined #openstack-operators14:53
smcginnisA sign of a good company. :D14:54
mihalis68the kitchen coffee here is just good enough to prevent external coffee shop outings, it's uncanny14:54
mihalis68like it's not *great* but every time I think I'll go get something better I decide ok free coffee and save 10-15 minutes wins14:54
smcginnisHaha, sneaky.14:54
smcginnisNow that I work from home, one of the few things I miss about going in to an office was access to unlimited free coffee. Even though it wasn't great, it was right there. ;)14:55
mihalis68I guess release cadence is well and truly in discussion across the community so no need for the meetups team specifically to worry about any more14:55
VWsome days you get what you pay for smcginnis ;)14:55
mihalis68I have never worked from home by default14:55
mihalis68I'm like a fossil14:55
smcginnisVW: True!14:55
VWit's a bit like Russian roulette around here sometimes when it comes to coffee pots14:56
smcginnismihalis68: I never did extensively until this job. Gotta say, I kind of like it.14:56
mihalis68I'd try it14:56
*** sxc731 has quit IRC14:56
mihalis68but then I run a team of 9-12 now (depending on how you count it)14:56
mihalis68I feel like I should be here14:56
VWmihalis68: I have made thursdays work from home day for about two years14:56
smcginnisVW: Yeah, you would think making coffee was straight forward enough, but it's amazing how folks can mess it up.14:56
VWit's a nice way to get away from the desk and the walk-ups and get caught up14:57
amrithoink14:57
amrithjust saw my name here14:57
smcginnisVW: ++ I feel like I get a lot more done without the random distractions.14:57
mihalis68oh yeah, the interruptions here ...14:57
mihalis68hey  amrith14:57
*** emccormick has quit IRC14:57
amrithhello hello14:57
smcginnisamrith: o/14:57
mihalis68is verizon potentially interested in co-hosting openstack operators meet up in august 2018 in NYC?14:57
amrithoh, damn. smcginnis is here14:57
amrithsee you all later14:57
amrith;)14:57
mihalis68harsh14:57
smcginnis:)14:58
mihalis68Bloomberg is considering hosting again14:58
mrhillsmanhey amrith i mentioned discussing with you possibly verizon hosting/co-hosting meetup in fall next year14:58
amrithmihalis68 potentially yes14:58
mihalis68would be 2 years since last time we hosted14:58
mihalis68ok cool14:58
amrithyes, discussed it with amitry and a couple of others14:58
amrithbut the timing didn't work14:58
mihalis68yeah it's all mrhillsman's fault14:58
mrhillsman:)14:58
mrhillsmanwelp, now that you two know each other hehe14:58
mrhillsmanmy job is done14:59
mrhillsmanknow of each other rather14:59
mihalis68can I email you direct amrith? I'm cmorgan2@bloomberg.net14:59
amrithmrhillsman I've known smcginnis for some time now14:59
mihalis68where are you located?14:59
amrithmihalis68 email on its way14:59
amrithnear boston14:59
mihalis68ok so NYC is reasonable but not actually home14:59
amrithwe have a big team in Warren, NJ and VZ's hq is very nearby15:00
mihalis68seems we have local, boston and then DC/Reston area folks interested as a minimum, and then at least one +1 from japan15:00
amrithso I'm a remote employee15:00
mihalis68yeah my wife works for VZ in west street and goes to basking ridge occasionally15:00
mihalis68in advertising15:00
amrithah, cool. so yes, hosting in that area is something we're interested in.15:00
mihalis68very good15:01
amrithemail on the way15:01
amrithand sorry for yanking your chain smcginnis15:01
mihalis68don't scare off the developers from this meeting slot15:01
smcginnisamrith: Haha, it's only fair.15:01
mihalis68sean brings news from the scene of the crime15:01
mihalis68I mean from the developers15:01
mrhillsmanamrith are you participating in the edge meetings/discussions?15:02
mrhillsmangotta run peeps15:02
mrhillsmanthx mihalis68 for chairing15:02
mrhillsmanttyl15:02
* amrith wonders whether smcginnis proximity to crime scenes may be more than correlation :)15:02
mihalis68most welcome15:02
amrithmrhillsman I was hoping to15:02
amrithbut got tied up in something else15:02
amrithshould dial in now,15:02
smcginnisHah!15:03
amrithmrhillsman thanks for the reminder, now dialed in15:05
mrhillsman;)15:05
*** aojea has joined #openstack-operators15:07
amrithamitry ... would you please pm me an email address which I can use for the operators meetup scheduling15:08
*** aojea has quit IRC15:09
*** aojea has joined #openstack-operators15:09
*** marst has joined #openstack-operators15:10
*** smcginnis has left #openstack-operators15:12
*** chyka has joined #openstack-operators15:20
*** aojea has quit IRC15:25
*** chyka has quit IRC15:26
*** chyka has joined #openstack-operators15:26
*** chyka has quit IRC15:31
*** liverpooler has quit IRC15:38
*** aojea has joined #openstack-operators15:41
*** liverpooler has joined #openstack-operators15:48
*** gyee has joined #openstack-operators16:42
*** penick has joined #openstack-operators16:42
*** aojea has quit IRC16:42
*** AlexeyAbashkin has quit IRC17:01
mnaser(posted in nova but reposting here): Spec-ing out new compute and in the interest of deployers and doing things in open we’re planning to publish the document ... I wanted to gather some feedback at what sort of cpu overcommit you’ve ran/seen people run?17:14
*** gkadam has quit IRC17:22
*** Apoorva has joined #openstack-operators17:43
*** Apoorva has quit IRC17:43
*** Apoorva has joined #openstack-operators17:44
*** derekh has quit IRC17:47
*** catintheroof has quit IRC17:50
*** catintheroof has joined #openstack-operators17:51
penickmnaser: at Oath we've done from 16:1 in user sandbox environments to 2:1/3:1 in production17:53
mnaserpenick: sweet, thanks for that info.. what about ratios of memory / cpu to be able to right size compute nodes?17:54
penickmnaser: Unfortunately the answer for that is "It depends". What size VMs are you going to run? If you're going to offer your customers a selection of flavors, you can pretty much count on them only wanting the largest flavor. Humans are dependable like that ;)17:57
mnaserpenick: the idea is to have some sort of document to say "hey, this is what we've experienced, this is what others have experienced, let that help you decide"17:58
*** chito_ has joined #openstack-operators17:58
mnaserthe idea is to have a general amount of information so that potential deployers can make an informed decision17:58
mnaserbecause i think a lot of us tend to do this sort of 'architecting' behind our doors and i think we can all benefit from this information17:59
penickSure. Right sizing depends on workload. So I suggest you provide a formula and some strawman examples.17:59
penicklet's say you have 2 flavors, Flavor foo has 2 cpu cores and 4gb of ram; flavor bar has 8 cores and 16gb of ram.18:00
mnaserpenick: yeah i agree that usually we have to do a ratio of gb/cpu18:01
mnaserso usually that ratio obviously depends on your workloads18:01
penick(abandoning the strawman)18:02
penickOur hypervisors range from 12 cores and 64gb of ram to 24 cores and 512gb of ram. Depending on network security zone type (corp vs prod, for example) and region.18:03
penickAnother factor to consider is the level of resiliency you'd like to offer your users, in some cases it makes sense to use more and smaller hypervisors rather than fewer deeper hypervisors. Build wide vs deep.18:05
penickOur largest datacenters are in the US, so we build bigger hypervisors there. However in, say, ireland we have lower instance counts overall so we use smaller hypervisors (more likely to see 96-128gb) to ensure instances have rooms to spread around.18:06
penickIn the US we have much higher instance counts, so building deeper hypervisors makes sense, since we'll still have many hypervisors to meet demands18:07
*** catintheroof has quit IRC18:07
*** catintheroof has joined #openstack-operators18:08
penickSo, yeah. We do our hypervisor selection based on intended usage(network security zone type), geographical region, Overcommit SLA, power constraints, availability requirements, and cost.18:09
*** harlowja has joined #openstack-operators18:15
mnaserpenick: some really good information to write up18:18
*** chyka has joined #openstack-operators18:18
*** racedo has quit IRC18:20
*** AlexeyAbashkin has joined #openstack-operators18:22
penickmnaser hope it helps18:24
*** AlexeyAbashkin has quit IRC18:27
*** racedo has joined #openstack-operators18:28
*** racedo has quit IRC18:37
cloudnullmnaser: re: osic -- we did a very similar thing as penick.  The over commit was setup as 2:1 and we didn't really mess with the memory over commit ratios.18:38
cloudnullThe networking was the same throughout the cloud so we did partition on networking capability. All hypervisors were using bonded intel x710 nics, had 48 cores (with hyperthreading enabled) and 256G of ram.18:38
cloudnullWe had however have three different storage profiles so we created host aggregates and flavor meta-data so that we could offer a "good", "better", "best" option in terms of storage performance.18:39
cloudnullThe "good" option used rotational media and had the most storage, The "better" option was on s3500 ssd's and had a fair amount of storage. The "best" option was on s3700 ssd's and had the least storage.18:39
cloudnulleach flavor profile had varying amounts of ram18:39
mnasercloudnull: so there wasn't a relationship of the amount of memory / vcpus to try and maximize resources?18:40
cloudnullthe most use flavor was the medium of each size which had 8G18:40
cloudnullnope, we prioratized instance performance over density.18:41
mnasercloudnull: so you could have scenarios where you had a hypervisor that had free memory but all vcpus used up, alright18:41
cloudnullyup18:41
mnaseri'm gathering all this information to build up a nice interesting article for sizing physical infrastructure18:41
cloudnullvery cool!18:42
mnaseri feel like this is a common thing we all work to figure out18:42
cloudnullsorry, i missed this pings earlier, technically im not here right now :)18:42
mnaser(also side note i started poking around the ansible nspawn+ssh connection stuff you worked on!)18:42
mnaserno worries, thanks for the info!18:42
cloudnullcool!18:42
cloudnullI have that all running in the lab now18:43
mnaserwe've had nspawn in prod for a while now, pretty much what OSA does but nspawn containers per service that are managed via puppet18:43
cloudnullnice!18:43
cloudnullnspawn has been great in test.18:44
mnaserit's very reliable and good at what it does18:44
cloudnullI really like it with macvlan18:44
mnaserreally, i havent experimented that far18:44
cloudnullit seems bulletproof that way18:44
mnaserhost networking to avoid weird shenanigans18:44
mnaser(is what we've done so far)18:44
mnaserand make things a lot more sane in terms of addressing etc18:45
*** chyka has quit IRC18:45
cloudnullagreed18:45
mnaser(also, this is all control plane, i dont really want to run openvswitch or neutron-*-agent or cinder-volume in containers yet)18:45
cloudnullmacvlan is similar to what osa does already however it gets rid of the bridge and veth.18:45
cloudnullyea we do the same in osa18:45
mnaserwhy didnt osa just use host networking?18:45
cloudnullany service that interacts with a physical device really didn't make sense in a container for us18:46
*** AlexeyAbashkin has joined #openstack-operators18:46
mnaser++18:46
cloudnullwe didn't do host networking because we wanted to isolate network traffic a particular container could have access to18:46
mnaservalid, i guess you work with the assumption of not knowing what type of network you're sitting in18:47
cloudnullhind sight being 20/20 it probably would have been a good idea and would have saved us quite a bit of growing pains.18:47
cloudnullhowever now, it i think we've got a handle on it all.18:47
mnaseryeah its always a pain at the start18:47
cloudnullso it's kinda "meh" to go back .18:48
mnaser(if only OSA did binary packaging .. but i guess "contributions welcome" :p)18:48
cloudnullyea. its been kicked around a bunch.18:48
*** chyka_ has joined #openstack-operators18:49
cloudnullbut the from source thing has been working well for us, for now.18:49
mnaserthere's a lot of work to be done but i think the harder part is seeing how to architecturally make a shift to select one or another18:49
cloudnullwe use distro packages for all the OS bits.18:49
cloudnulljust the openstack parts are source base.d18:49
*** chyka_ has quit IRC18:49
cloudnullyea, it would be nice to give deployers that choice18:50
mnaserits easy to say "use rpms" for 1 role but harder to architect a global change so that methodology can be used across18:50
*** chyka_ has joined #openstack-operators18:50
mnaserwell, its that and also.. the fact that as much as the roles are nice, they are pretty OSA biased (venv/etc)18:50
cloudnullyea.18:50
mnaserbut if there hasn't been much role-only consumers then i guess its probably best to get the flexibility of doing your own thing18:51
cloudnullthis is very true. I mean we could make some of that conditional, but we'd still have a pretty biased OSA pov even if we did.18:51
mnaseri think one of the problems right now of ansible is it is very hard to make service specific unbiased roles18:52
cloudnullit is a pain.18:52
mnaserwith puppet you add the resources you need, with ansible you have to use variables to control the flow and skip/run things that are needed/not needed18:52
*** chyka_ has quit IRC18:54
cloudnullyea some of the mechanics could be a lot better in ansible and a lot has improved since 2.0 but like everything they all have their pros and cons.18:54
cloudnullall of todays problems will be solved tomorrow :)18:54
cloudnullok, mnaser gotta run,18:55
* cloudnull going back to not being here.18:55
mnaserits been fun chatting, enjoy your holidays cloudnull18:55
cloudnulllet me know if i can help or review things as you create your doc18:55
mnaserwill do!18:56
cloudnullI think that'd be super helpful for a lot of folks18:56
mnaser++18:56
cloudnulltake care!18:56
mnaserlater!18:56
*** catintheroof has quit IRC18:56
*** AlexeyAbashkin has quit IRC18:58
*** gkadam has joined #openstack-operators19:03
*** chyka has joined #openstack-operators19:12
*** chito_ has quit IRC19:13
*** alanmeadows has quit IRC19:15
*** alanmeadows has joined #openstack-operators19:15
*** chyka has quit IRC19:17
*** catintheroof has joined #openstack-operators19:29
*** openstack has joined #openstack-operators19:42
*** ChanServ sets mode: +o openstack19:42
*** tdasilva has quit IRC19:47
*** tdasilva has joined #openstack-operators19:47
*** electrofelix has quit IRC19:48
*** dmibrid_ has joined #openstack-operators19:50
*** fragatina has quit IRC19:56
*** drewn3ss has quit IRC20:01
*** VW_ has joined #openstack-operators20:11
*** VW has quit IRC20:14
*** drewn3ss has joined #openstack-operators20:15
*** gkadam has quit IRC20:18
*** VW_ has quit IRC20:35
*** AlexeyAbashkin has joined #openstack-operators20:37
*** AlexeyAbashkin has quit IRC20:41
*** Apoorva has quit IRC20:51
*** penick has quit IRC20:53
*** emccormick has joined #openstack-operators20:59
*** chyka has joined #openstack-operators21:02
*** catintheroof has quit IRC21:02
*** catintheroof has joined #openstack-operators21:03
*** chyka has quit IRC21:06
*** catintheroof has quit IRC21:07
*** Apoorva has joined #openstack-operators21:20
*** penick has joined #openstack-operators21:20
*** penick has quit IRC22:19
*** penick has joined #openstack-operators22:25
*** rcernin has joined #openstack-operators22:28
*** penick has quit IRC22:40
*** marst has quit IRC22:41
*** penick has joined #openstack-operators22:43
*** penick has quit IRC23:39
*** catinthe_ has joined #openstack-operators23:43
*** catintheroof has joined #openstack-operators23:52
*** catintheroof has quit IRC23:52
*** catintheroof has joined #openstack-operators23:53
*** catinthe_ has quit IRC23:55
*** catintheroof has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!