Monday, 2016-02-01

*** chlong has quit IRC00:01
*** dims_ has joined #openstack-operators00:02
*** dims has quit IRC00:05
*** alejandrito has quit IRC00:05
*** tpeoples has quit IRC00:08
*** tpeoples has joined #openstack-operators00:10
*** xarses_ has joined #openstack-operators00:12
*** xarses has quit IRC00:13
*** chlong has joined #openstack-operators00:17
*** elo has joined #openstack-operators00:38
*** ducttape_ has joined #openstack-operators00:44
*** markvoelker has joined #openstack-operators00:46
*** ducttape_ has quit IRC00:49
*** markvoelker has quit IRC00:50
*** dims_ has quit IRC01:04
*** HenryG_ is now known as HenryG01:08
*** mriedem is now known as mriedem_away01:42
*** markvoelker has joined #openstack-operators01:47
*** markvoelker has quit IRC01:52
*** dims has joined #openstack-operators02:02
*** Marga_ has joined #openstack-operators02:07
*** Marga_ has quit IRC02:08
*** Marga_ has joined #openstack-operators02:08
*** dims has quit IRC02:12
*** dims has joined #openstack-operators02:12
*** dims has quit IRC02:13
*** ducttape_ has joined #openstack-operators02:30
*** k_stev has joined #openstack-operators02:32
*** ducttape_ has quit IRC02:35
*** Marga_ has quit IRC02:35
*** Marga_ has joined #openstack-operators02:36
*** k_stev has quit IRC02:37
*** harshs has joined #openstack-operators02:48
*** ducttape_ has joined #openstack-operators03:32
*** ducttape_ has quit IRC03:37
*** ducttape_ has joined #openstack-operators03:37
*** krobzaur has quit IRC03:46
*** markvoelker has joined #openstack-operators03:47
*** mriedem_away has quit IRC03:50
*** ducttape_ has quit IRC03:51
*** markvoelker has quit IRC03:52
*** ducttape_ has joined #openstack-operators03:54
*** ducttape_ has quit IRC04:27
*** ducttape_ has joined #openstack-operators04:35
*** ducttape_ has quit IRC04:35
*** Marga_ has quit IRC04:50
*** chlong has quit IRC05:05
*** Marga_ has joined #openstack-operators05:08
*** chlong has joined #openstack-operators05:17
*** chlong has quit IRC05:25
*** ducttape_ has joined #openstack-operators05:36
*** chlong has joined #openstack-operators05:38
*** ducttape_ has quit IRC05:42
*** darrenc_ has joined #openstack-operators05:43
*** Tridde_ has joined #openstack-operators05:44
*** mjrichar1son has joined #openstack-operators05:45
*** markvoelker has joined #openstack-operators05:48
*** med_ has joined #openstack-operators05:48
*** med_ is now known as Guest8071105:49
*** mgagne has quit IRC05:49
*** Guest65103 has quit IRC05:49
*** logan- has quit IRC05:49
*** Tridde has quit IRC05:49
*** mjrichardson has quit IRC05:49
*** darrenc has quit IRC05:49
*** saneax_AFK has quit IRC05:49
*** mgagne_ has joined #openstack-operators05:49
*** logan- has joined #openstack-operators05:50
*** markvoelker has quit IRC05:53
*** saneax_AFK has joined #openstack-operators05:57
*** saneax_AFK is now known as saneax05:57
*** Marga_ has quit IRC06:04
*** Marga_ has joined #openstack-operators06:22
*** harshs has quit IRC06:29
*** chlong has quit IRC06:41
*** harshs has joined #openstack-operators06:45
*** chlong has joined #openstack-operators06:46
*** harshs has quit IRC06:58
*** eddima has joined #openstack-operators07:12
*** zul has joined #openstack-operators07:13
*** yportnova_ has joined #openstack-operators07:15
*** yportnova_ has quit IRC07:33
*** berendt has joined #openstack-operators07:38
*** ducttape_ has joined #openstack-operators07:38
*** ducttape_ has quit IRC07:43
*** chlong has quit IRC07:46
*** markvoelker has joined #openstack-operators07:49
*** zul has quit IRC07:52
*** markvoelker has quit IRC07:54
*** belmoreira has joined #openstack-operators07:55
*** zeih has joined #openstack-operators08:00
*** Marga_ has quit IRC08:06
*** zz9pzza has joined #openstack-operators08:23
*** rcernin has joined #openstack-operators08:26
*** d0ugal_ has quit IRC08:30
*** d0ugal has joined #openstack-operators08:30
*** bvandenh has joined #openstack-operators08:36
*** liverpooler has joined #openstack-operators08:46
*** zul has joined #openstack-operators09:11
*** pilgrimstack has joined #openstack-operators09:24
*** zul has quit IRC09:25
*** vinsh_ has quit IRC09:25
*** pilgrimstack has quit IRC09:26
*** zul has joined #openstack-operators09:32
*** pilgrimstack has joined #openstack-operators09:35
*** electrofelix has joined #openstack-operators09:44
*** matrohon has joined #openstack-operators09:45
*** rmsilva has joined #openstack-operators09:46
*** markvoelker has joined #openstack-operators09:50
*** rmsilva has quit IRC09:50
*** egonzalez90 has joined #openstack-operators09:50
*** zul has quit IRC09:51
*** egonzalez90 is now known as egonzalez09:52
*** markvoelker has quit IRC09:54
*** openstackgerrit has quit IRC10:02
*** openstackgerrit_ has joined #openstack-operators10:03
*** openstackgerrit_ has quit IRC10:03
*** zeih has quit IRC10:05
*** matrohon has quit IRC10:08
*** matrohon has joined #openstack-operators10:10
*** marekd has quit IRC10:34
*** egonzalez has quit IRC10:36
*** berendt has quit IRC10:41
*** marekd has joined #openstack-operators10:42
*** zul has joined #openstack-operators10:46
*** berendt has joined #openstack-operators10:58
*** dims has joined #openstack-operators11:09
*** markvoelker has joined #openstack-operators11:50
*** openstackgerrit_ has joined #openstack-operators11:54
*** openstackgerrit_ is now known as openstackgerrit11:55
*** markvoelker has quit IRC11:55
*** prithiv has joined #openstack-operators11:59
*** openstackgerrit has quit IRC11:59
*** prithiv has quit IRC12:01
*** openstackgerrit has joined #openstack-operators12:08
*** ducttape_ has joined #openstack-operators12:35
*** arcimboldo has joined #openstack-operators12:48
*** markvoelker has joined #openstack-operators12:51
*** ducttape_ has quit IRC12:52
*** markvoelker has quit IRC12:56
*** rarcea has joined #openstack-operators13:02
*** prithiv has joined #openstack-operators13:03
*** arcimboldo has quit IRC13:03
*** zul has quit IRC13:03
*** zeih has joined #openstack-operators13:04
*** dims has quit IRC13:06
*** prithiv has quit IRC13:08
*** prithiv has joined #openstack-operators13:09
*** dims has joined #openstack-operators13:09
*** markvoelker has joined #openstack-operators13:14
*** zul has joined #openstack-operators13:16
*** piet has joined #openstack-operators13:30
*** k_stev has joined #openstack-operators13:32
*** prithiv1 has joined #openstack-operators13:46
*** prithiv has quit IRC13:46
*** dminer has joined #openstack-operators14:01
*** dminer has quit IRC14:05
*** dminer has joined #openstack-operators14:06
*** ducttape_ has joined #openstack-operators14:12
*** prithiv1 has quit IRC14:13
*** prithiv has joined #openstack-operators14:13
*** zul has quit IRC14:13
*** fawadkhaliq has joined #openstack-operators14:17
*** fawadkhaliq has quit IRC14:18
*** zhangjn has joined #openstack-operators14:19
*** fawadkhaliq has joined #openstack-operators14:20
*** ducttape_ has quit IRC14:21
*** mdorman has joined #openstack-operators14:24
*** krobzaur has joined #openstack-operators14:27
*** rarcea has quit IRC14:29
*** zhangjn has quit IRC14:35
*** ducttape_ has joined #openstack-operators14:41
*** VW has joined #openstack-operators14:42
*** zul has joined #openstack-operators14:45
*** kencjohnston has joined #openstack-operators14:52
*** piet has quit IRC14:52
*** csoukup_ has quit IRC14:53
*** xarses_ has quit IRC14:55
*** rarcea has joined #openstack-operators14:57
*** VW has quit IRC14:57
*** piet has joined #openstack-operators14:58
*** kencjohnston has quit IRC15:04
*** ToMiles has joined #openstack-operators15:04
*** regXboi has joined #openstack-operators15:05
*** ducttape_ has quit IRC15:09
*** ToMiles has quit IRC15:10
*** mriedem has joined #openstack-operators15:13
*** alejandrito has joined #openstack-operators15:17
*** csoukup_ has joined #openstack-operators15:28
*** VW has joined #openstack-operators15:34
*** ducttape_ has joined #openstack-operators15:34
*** VW has joined #openstack-operators15:34
*** zeih has quit IRC15:39
*** zeih has joined #openstack-operators15:40
*** zeih has quit IRC15:44
*** xarses has joined #openstack-operators15:46
*** harshs has joined #openstack-operators15:54
pietAre any of the operators in this channel interested in participating in a study on mobile use cases?  We're kind of hurting for folks to interview.15:55
*** zul has quit IRC15:58
*** rcernin has quit IRC15:58
*** fawadkhaliq has quit IRC16:03
*** liverpooler has quit IRC16:07
*** belmoreira has quit IRC16:10
mdormanclayton:   i’m struggling to find the wsgi stuff for neutron?  did you get that one working under uwsgi?16:12
*** SimonChung1 has quit IRC16:14
*** mriedem is now known as mriedem_away16:15
*** harshs has quit IRC16:23
*** alop has joined #openstack-operators16:25
*** alop has quit IRC16:29
*** alop has joined #openstack-operators16:29
*** krobzaur_ has joined #openstack-operators16:30
*** zz9pzza has quit IRC16:31
*** krobzaur has quit IRC16:31
openstackgerritMike Dorman proposed openstack/osops-tools-contrib: Update README with details on how to contribute to contrib repo.  https://review.openstack.org/27477216:36
openstackgerritMike Dorman proposed openstack/osops-tools-generic: Update README with details on how to contribute to curated repo.  https://review.openstack.org/27477316:36
openstackgerritMike Dorman proposed openstack/osops-tools-logging: Update README with details on how to contribute to logging repo.  https://review.openstack.org/27477416:37
openstackgerritMike Dorman proposed openstack/osops-tools-monitoring: Update README with details on how to contribute to monitoring repo.  https://review.openstack.org/27477616:38
openstackgerritMike Dorman proposed openstack/osops-example-configs: Update README with details on how to contribute to example configs repo.  https://review.openstack.org/27477716:39
*** rcernin has joined #openstack-operators16:40
*** Marga_ has joined #openstack-operators16:44
*** Marga_ has quit IRC16:44
*** Marga_ has joined #openstack-operators16:45
*** zeih has joined #openstack-operators16:50
claytonno, last I looked it didn't exist16:51
claytonneutron is probably next on the list to do for us16:51
mdormangottcha, thanks.   we’re pretty interested in that one too because we get a lot of timeouts on nova->neutron calls16:52
*** rarcea has quit IRC16:52
claytonI just want to get off of canonical's packaging16:55
claytonFinally getting around to working on upgrading to liberty, found a bug in neutron that breaks flat networking, canonical apparently hasn't rolled a new neutron package since the initial release16:56
claytonare you guys on liberty now?16:56
mdormannope not yet. kilo.2 to get ready for it, though.16:59
claytonwe're on kilo.<latest> for nova, but we're not going to upgrade that until everything else is on liberty16:59
mdormankk.  yeah we typically do everything at once.16:59
claytonso when I say I'm working on liberty upgrade, that's actually just glance, cinder and neutron16:59
claytonwe used to, I'm glad we have the option not to now16:59
*** kencjohnston has joined #openstack-operators16:59
mdormanright.  we’re still in cross-dependency hell so we kinda have to17:00
claytonnod, we're at the point where we can see the light at the end of the tunnel17:00
claytonI got nova-compute working in docker a week or two ago,but that won't go to prod until after the superbowl.  We won't upgrade the other services until that's deployed17:02
mdormanmakes sense17:02
*** rcernin has quit IRC17:02
mdormanwe’ll probably go the venv route, not sure about containers yet, but we need something more sane17:02
claytonI found deploying venvs to just be a pain in the ass17:03
claytonprobably not as bad if you prebuild them and ship them as tarballs though17:03
claytonwe switched to doing that for the CLI clients a while ago17:03
claytonwe build one venv per client with exactly the version we want, then hardlink all the common files together and generate an archive17:04
*** Apoorva has joined #openstack-operators17:04
claytonthen we have puppet drop that in /usr/local under a unique directory and symlink them into /usr/bin17:04
*** armax has joined #openstack-operators17:05
claytonthe two biggest issues we have with venvs in production is the amount of time it takes to build them (if you do it with puppet) and updating them reliably is basically impossible.  you need to build a new one every time you update them, otherwise they end up with weird cruft that ends up breaking them eventually17:07
*** esker has joined #openstack-operators17:09
*** matrohon has quit IRC17:09
*** gyee has joined #openstack-operators17:17
klindgrenI am interested in doingthe container route.  Though I am getting tired of dependancy hell with redhat- where they release a new version of something into epel thats a higher version that what some of the openstack projects want.17:21
*** SimonChung has joined #openstack-operators17:21
klindgrenLatest was something with epel + ceilometer17:21
mdormangood to know, clayton17:21
claytonwell, if you do docker, you can run ubuntu in them!17:22
claytonI've thought about running alpine actually, but we'd have to mirror it locally, etc17:22
claytonwe'd get better results for smaller containers in the short run by just building the venvs we put inside of them outside of the final image17:23
claytonso we didn't have to put compilers and crap into the venv17:23
claytonerr, venv = base image17:24
*** bvandenh has quit IRC17:24
*** prithiv has quit IRC17:27
klindgrenthat was my thought as well17:29
klindgrenalso nice to remove config drift as it never seems like the versions of ancilary packages in prod are the same as dev17:30
klindgrenso it would be nice to know that everything you pushed out to prod was exactly the same in dev/test/stage17:30
*** mgagne_ has quit IRC17:32
*** mgagne_ has joined #openstack-operators17:32
*** bvandenh has joined #openstack-operators17:32
*** piet has quit IRC17:33
*** mgagne_ is now known as mgagne17:34
*** kelv has joined #openstack-operators17:35
*** harshs has joined #openstack-operators17:36
*** SimonChung has quit IRC17:39
*** SimonChung has joined #openstack-operators17:39
*** zeih has quit IRC17:39
*** zeih has joined #openstack-operators17:43
*** prithiv has joined #openstack-operators17:53
*** prithiv has quit IRC17:53
*** yportnova_ has joined #openstack-operators18:09
*** zeih has quit IRC18:13
*** Marga_ has quit IRC18:16
*** rarcea has joined #openstack-operators18:17
*** dims_ has joined #openstack-operators18:21
*** dims has quit IRC18:22
openstackgerritMike Dorman proposed openstack/osops-example-configs: Update README with details on how to contribute to example configs repo.  https://review.openstack.org/27477718:28
openstackgerritMike Dorman proposed openstack/osops-tools-contrib: Update README with details on how to contribute to contrib repo.  https://review.openstack.org/27477218:28
openstackgerritMike Dorman proposed openstack/osops-tools-generic: Update README with details on how to contribute to curated repo.  https://review.openstack.org/27477318:29
openstackgerritMike Dorman proposed openstack/osops-tools-logging: Update README with details on how to contribute to logging repo.  https://review.openstack.org/27477418:29
openstackgerritMike Dorman proposed openstack/osops-tools-monitoring: Update README with details on how to contribute to monitoring repo.  https://review.openstack.org/27477618:30
*** VW has quit IRC18:35
*** zeih has joined #openstack-operators18:37
*** liverpooler has joined #openstack-operators18:39
*** VW has joined #openstack-operators18:42
*** Marga_ has joined #openstack-operators18:51
*** VW_ has joined #openstack-operators18:52
*** liverpooler has quit IRC18:52
*** ducttape_ has quit IRC18:54
*** VW has quit IRC18:55
*** ducttape_ has joined #openstack-operators18:59
mdormananybody using something other than native python to do ssl termination for neutron api?   I’ve moved ssl termination to haproxy, but now neutron provides http:// references to itself, i need it to provide https:// references.19:06
claytonmdorman: yes, all the services except heat are stupid about this19:07
claytonthere is a new middleware that addresses this, but I think you need to be on liberty for it19:07
claytonheat will accept a X-Forwarded-Proto header and make its version responses match that19:07
mdormani think i did it with nova with enabled_ssl_apis19:08
mdormannot seeing a thing in neutron like that, but i’m looking.19:08
klindgrenclayton, whats the middlewear?19:08
claytonit's in oslo middleware, let me find it19:09
klindgrenI can see if we can fudge the versions enough to pull it into our kilo deployment stuff19:09
claytonhttps://github.com/openstack/oslo.middleware/blob/master/oslo_middleware/http_proxy_to_wsgi.py19:09
*** VW_ has quit IRC19:10
claytonthat makes wsgi lie to the app and think it's running under https, or whatever you pass in via the http headers19:10
klindgrenAlso - is any one super happy with their glance-registry monitoring?19:10
claytonthe version endpoints are constructed using the information in the wsgi environment, which is why it always returns the right hostname at least19:10
*** zeih has quit IRC19:10
*** VW has joined #openstack-operators19:10
klindgrenseems like we have cases where glance-registry goes full retard until a restart happens.19:11
claytonI've seen glance api chew up gigs of memory, but not that problem19:11
klindgrenI htink it happens when glance-registry gets cut off from its db server for a short period.  I seems to bascially never recover19:12
mdormanugh, not sure this is worth my effort to go down this middleware road19:14
klindgrenlooks like it was added in oslo-middleware 2.9.019:15
mgagnemdorman I'm sure you saw this thread back in september: http://lists.openstack.org/pipermail/openstack-dev/2015-September/074871.html19:15
mgagnemdorman not much conclusion except: just wait, it's gonna be better in the next versions19:16
mgagneregarding SSL termination19:16
klindgrenseems like liberty is going to be the requirement for that middleware19:16
claytoneveryone just using that middleware seems like the cleanest solution19:17
claytonif you're doing ssl offload, you can probably insert headers when you do it19:17
mdormanthanks mgagne i had just found that when you mentioned it :)19:17
mgagnemdorman blog post I was working on back then: https://gist.github.com/mgagne/f298c151b61d44cb5fea19:18
mgagnemdorman shows the config madness19:18
mgagneand inconsistencies19:19
mdormanthanks mgagne19:21
mdormanthat’s helpful19:21
mgagnewe have yet to fix it ourselves, not much priority on that :-/19:21
*** marekd has quit IRC19:21
claytonthe good news with liberty is that restarting OVS agent isn't quite the disaster it used to be19:22
claytonjust did a test with 33 instances behind 33 routers and restarting it caused them to miss about two pings each19:22
claytonotoh, I'm still seeing a 1-2 minute outage when doing the upgrade from kilo to liberty19:23
claytonwhich I had hoped wouldn't be  required19:23
openstackgerritSimon Leinen proposed openstack/osops-tools-generic: Region support for Neutron scripts  https://review.openstack.org/27486319:24
mdormanfun19:24
claytonoh hrm, we don't have neutron root wrap enabled for compute nodes I bet19:24
mdormanopenstack:  like ice skating up hill19:27
mdormani should totally troll the success bot with that19:27
*** nikhil_k has joined #openstack-operators19:28
*** nikhil has quit IRC19:31
*** marekd has joined #openstack-operators19:32
*** liverpooler has joined #openstack-operators19:34
*** simon-AS559 has joined #openstack-operators19:42
*** piet has joined #openstack-operators19:43
*** matrohon has joined #openstack-operators19:45
*** yportnova_ has quit IRC19:53
*** VW has quit IRC19:54
*** yportnova_ has joined #openstack-operators20:00
*** harshs has quit IRC20:03
*** yportnova_ has quit IRC20:04
*** VW has joined #openstack-operators20:05
*** VW has quit IRC20:05
*** VW has joined #openstack-operators20:05
*** yportnova_ has joined #openstack-operators20:12
*** yportnova_ has quit IRC20:14
*** harshs has joined #openstack-operators20:19
*** harshs has joined #openstack-operators20:23
*** ducttape_ has quit IRC20:26
*** yportnova_ has joined #openstack-operators20:30
*** VW has quit IRC20:33
*** harshs has quit IRC20:36
*** electrofelix has quit IRC20:37
*** ducttape_ has joined #openstack-operators20:39
*** harshs has joined #openstack-operators20:41
*** yportnova_ has quit IRC20:44
*** VW has joined #openstack-operators20:46
*** zeih has joined #openstack-operators20:48
*** lhcheng has joined #openstack-operators20:49
*** Apoorva has quit IRC20:51
*** dims has joined #openstack-operators21:02
*** dims_ has quit IRC21:03
*** zeih has quit IRC21:04
*** rocky_g has joined #openstack-operators21:08
*** esker has quit IRC21:09
*** VW has quit IRC21:11
*** Apoorva has joined #openstack-operators21:15
*** rcernin has joined #openstack-operators21:18
*** darrenc_ is now known as darrenc21:19
*** VW has joined #openstack-operators21:26
*** esker has joined #openstack-operators21:37
*** esker has quit IRC21:37
*** esker has joined #openstack-operators21:37
*** yportnova_ has joined #openstack-operators21:40
*** yportnova_ has quit IRC21:42
*** rcernin has quit IRC21:44
*** matrohon has quit IRC21:45
*** krobzaur_ has quit IRC21:49
*** eil397 has joined #openstack-operators21:49
*** bvandenh has quit IRC21:53
*** chlong has joined #openstack-operators22:10
*** harshs has quit IRC22:18
*** VW has quit IRC22:22
*** VW has joined #openstack-operators22:24
*** piet has quit IRC22:24
*** harshs has joined #openstack-operators22:25
*** VW has quit IRC22:29
*** alejandrito has quit IRC22:30
*** VW has joined #openstack-operators22:31
*** SimonChung1 has joined #openstack-operators22:35
*** SimonChung has quit IRC22:35
*** SimonChung3 has joined #openstack-operators22:35
*** SimonChung1 has quit IRC22:36
*** lhcheng has quit IRC22:44
*** k_stev has quit IRC22:44
*** SimonChung3 has quit IRC22:44
*** SimonChung has joined #openstack-operators22:44
*** lhcheng has joined #openstack-operators22:44
*** harshs has quit IRC22:44
*** harshs has joined #openstack-operators22:45
*** esker has quit IRC22:50
*** harshs has quit IRC22:51
*** liverpoo1er has joined #openstack-operators22:58
*** harshs has joined #openstack-operators22:58
*** liverpooler has quit IRC23:01
*** regXboi has quit IRC23:05
*** kencjohnston has quit IRC23:07
*** piet has joined #openstack-operators23:14
*** armax has quit IRC23:15
*** david-lyle has quit IRC23:15
*** rocky_g has quit IRC23:25
*** ducttape_ has quit IRC23:28
*** armax has joined #openstack-operators23:29
*** mriedem_away is now known as mriedem23:32
*** SimonChung1 has joined #openstack-operators23:33
*** SimonChung has quit IRC23:33
*** SimonChung1 has quit IRC23:33
*** csoukup_ has quit IRC23:34
*** dminer has quit IRC23:34
*** csoukup_ has joined #openstack-operators23:34
*** rarcea has quit IRC23:35
*** SimonChung has joined #openstack-operators23:37
*** csoukup_ has quit IRC23:40
*** markvoelker has quit IRC23:41
*** ducttape_ has joined #openstack-operators23:41
dmsimardIf someone wants to file a bug for a script in https://github.com/openstack/osops-tools-monitoring, is there somewhere he can do that ?23:54
dmsimardmdorman,j^2 ^23:54
*** kelv has quit IRC23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!