Monday, 2015-09-21

*** zhangjn has quit IRC00:04
*** chlong has joined #openstack-operators00:42
*** zhangjn has joined #openstack-operators00:55
*** zhangjn has quit IRC00:57
*** zhangjn has joined #openstack-operators00:58
*** geoffarnold is now known as geoffarnoldX01:00
*** derekh has quit IRC01:14
*** csoukup has joined #openstack-operators01:57
*** c_soukup has joined #openstack-operators02:01
*** csoukup has quit IRC02:04
*** csoukup_ has joined #openstack-operators02:19
*** c_soukup has quit IRC02:22
*** esker has quit IRC02:26
*** geoffarnoldX is now known as geoffarnold03:24
*** elo has joined #openstack-operators03:33
*** elo1 has joined #openstack-operators03:34
*** elo2 has joined #openstack-operators03:35
*** elo has quit IRC03:37
*** elo1 has quit IRC03:38
*** elo2 has quit IRC03:39
*** elo has joined #openstack-operators03:41
*** elo has quit IRC03:42
*** geoffarnold is now known as geoffarnoldX03:45
*** stanchan has joined #openstack-operators03:58
*** fawadkhaliq has joined #openstack-operators04:20
*** deepthi has joined #openstack-operators05:04
*** deepthi has quit IRC05:29
*** deepthi has joined #openstack-operators05:30
*** csoukup_ has quit IRC05:36
*** vsedelnik has joined #openstack-operators06:13
*** fawadkhaliq has quit IRC06:14
*** vsedelnik has quit IRC06:14
*** fawadkhaliq has joined #openstack-operators06:15
*** Marga_ has quit IRC06:18
*** fawadkhaliq has quit IRC06:19
*** Marga_ has joined #openstack-operators06:19
*** Marga_ has quit IRC06:21
*** fawadkhaliq has joined #openstack-operators07:08
*** chlong has quit IRC07:17
*** matrohon has joined #openstack-operators07:19
*** vsedelnik has joined #openstack-operators07:23
*** Marga_ has joined #openstack-operators07:52
*** Marga_ has quit IRC07:57
*** fawadkhaliq has quit IRC08:26
*** chlong has joined #openstack-operators08:43
*** kelv has quit IRC08:53
*** Marga_ has joined #openstack-operators08:53
*** Marga_ has quit IRC08:58
*** derekh has joined #openstack-operators08:59
*** vsedelnik has quit IRC09:01
*** vsedelnik has joined #openstack-operators09:01
*** vsedelnik has quit IRC09:22
*** vsedelnik has joined #openstack-operators09:26
*** markvoelker has quit IRC09:27
*** Marga_ has joined #openstack-operators09:30
*** Marga_ has quit IRC09:35
*** kelv has joined #openstack-operators09:37
*** Marga_ has joined #openstack-operators09:52
*** Marga_ has quit IRC09:54
*** Marga__ has joined #openstack-operators09:54
*** Marga__ has quit IRC09:59
*** fawadkhaliq has joined #openstack-operators10:02
*** zhangjn has quit IRC10:10
*** zhangjn has joined #openstack-operators10:17
*** zhangjn has quit IRC10:19
*** zhangjn has joined #openstack-operators10:26
*** markvoelker has joined #openstack-operators10:28
*** markvoelker has quit IRC10:32
*** zhangjn has quit IRC10:35
*** zhangjn has joined #openstack-operators10:35
*** vsedelnik has quit IRC10:50
*** vsedelnik has joined #openstack-operators10:52
*** vsedelnik has quit IRC10:53
*** vsedelnik has joined #openstack-operators10:53
*** DinaBelova has joined #openstack-operators10:54
*** Marga_ has joined #openstack-operators10:55
*** Marga_ has quit IRC10:59
*** bradjones has joined #openstack-operators11:05
*** bradjones has quit IRC11:05
*** bradjones has joined #openstack-operators11:05
*** bradjones has quit IRC11:08
*** bradjones has joined #openstack-operators11:08
*** bradjones has quit IRC11:08
*** bradjones has joined #openstack-operators11:08
*** bradjones has quit IRC11:14
*** bradjones has joined #openstack-operators11:14
*** bradjones has quit IRC11:14
*** bradjones has joined #openstack-operators11:14
*** markvoelker has joined #openstack-operators11:29
*** markvoelker has quit IRC11:33
*** vsedelnik has quit IRC11:40
*** vsedelnik has joined #openstack-operators11:46
*** Marga_ has joined #openstack-operators11:56
*** Marga_ has quit IRC12:00
*** markvoelker has joined #openstack-operators12:11
*** priteau has joined #openstack-operators12:12
*** ferest has joined #openstack-operators12:21
*** ferest has quit IRC12:27
*** regXboi has joined #openstack-operators12:45
*** vsedelnik has quit IRC12:46
*** vsedelnik has joined #openstack-operators12:49
*** deepthi has quit IRC12:53
*** Marga_ has joined #openstack-operators12:56
*** geoffarnoldX is now known as geoffarnold13:01
*** Marga_ has quit IRC13:01
*** fawadkhaliq has quit IRC13:10
*** mfisch has quit IRC13:18
*** mfisch has joined #openstack-operators13:19
*** fawadkhaliq has joined #openstack-operators13:19
*** mfisch is now known as Guest6124313:19
*** lascii is now known as alaski13:20
*** VW has joined #openstack-operators13:25
*** zhangjn has quit IRC13:37
*** fawadkhaliq has quit IRC13:51
*** vinsh has quit IRC13:51
*** sdake has joined #openstack-operators13:57
*** Marga_ has joined #openstack-operators13:57
*** david-ly_ is now known as david-lyle14:01
*** vsedelnik has quit IRC14:01
*** geoffarnold is now known as geoffarnoldX14:02
*** Marga_ has quit IRC14:02
*** zhangjn has joined #openstack-operators14:07
*** elo has joined #openstack-operators14:11
*** zhangjn has quit IRC14:12
*** csoukup has joined #openstack-operators14:17
*** zhangjn has joined #openstack-operators14:19
*** david-lyle has quit IRC14:22
*** vsedelnik has joined #openstack-operators14:25
*** david-lyle has joined #openstack-operators14:26
*** vsedelnik has quit IRC14:27
*** Marga_ has joined #openstack-operators14:27
*** Marga_ has quit IRC14:28
*** Marga_ has joined #openstack-operators14:28
*** vsedelnik has joined #openstack-operators14:29
*** vsedelnik has quit IRC14:30
*** vsedelnik has joined #openstack-operators14:30
*** geoffarnoldX is now known as geoffarnold14:43
*** zhangjn has quit IRC14:45
*** geoffarnold has quit IRC14:47
*** zhangjn has joined #openstack-operators14:51
*** mdorman has joined #openstack-operators14:51
klindgrenMorning all - IN case you missed it on Friday - please please please help fill out the operator patches etherpad: https://etherpad.openstack.org/p/operator-local-patches14:56
*** Fin1te has joined #openstack-operators14:57
*** albertom has quit IRC14:59
*** emagana has joined #openstack-operators14:59
*** devlaps has joined #openstack-operators15:02
*** elo has quit IRC15:05
*** zhangjn has quit IRC15:05
*** albertom has joined #openstack-operators15:05
*** zhangjn has joined #openstack-operators15:06
*** zhangjn has quit IRC15:06
*** zhangjn has joined #openstack-operators15:07
*** zhangjn has quit IRC15:08
*** zhangjn has joined #openstack-operators15:09
*** zhangjn has quit IRC15:09
*** zhangjn has joined #openstack-operators15:10
*** zhangjn has quit IRC15:11
*** SimonChung has quit IRC15:12
*** Piet has joined #openstack-operators15:12
*** zhangjn has joined #openstack-operators15:12
*** devlaps has quit IRC15:19
*** geoffarnold has joined #openstack-operators15:27
*** devlaps has joined #openstack-operators15:28
*** vsedelnik has quit IRC15:41
*** vsedelnik has joined #openstack-operators15:43
*** matrohon has quit IRC15:49
*** zhangjn has quit IRC15:50
*** zhangjn has joined #openstack-operators15:53
*** SimonChung has joined #openstack-operators15:56
*** emagana has quit IRC16:00
*** gyee has joined #openstack-operators16:02
*** emagana has joined #openstack-operators16:02
mgagneklindgren: you should rename it for https://etherpad.openstack.org/p/operator-embarassing-patches16:03
*** bvandenh has joined #openstack-operators16:03
*** alejandrito has joined #openstack-operators16:03
*** alop has joined #openstack-operators16:03
klindgrenmgagne, ha16:04
mgagne /jk ;)16:04
klindgrenmost peoples seem to be backports of bug fixes16:04
*** zhangjn has quit IRC16:14
*** vsedelnik has quit IRC16:16
*** stanchan has quit IRC16:18
*** vsedelnik has joined #openstack-operators16:23
mgagneklindgren: now I need to find a way to publish those patches in a consumable way16:33
mdormangit diff | curl -XPUT http://gist.github.com16:34
mdorman:)16:34
klindgrenmgagne, same here - figured would try to find the common ones or ones people wanted16:34
klindgrenand work on those first16:34
mgagnemorgan: my patches are against Icehouse Ubuntu packages :D16:34
mgagnemorgan: sorry, wasn't meant for you :P16:40
mgagneklindgren: my patches are against Icehouse Ubuntu packages :D16:40
morganmgagne: figured.16:40
*** VW has quit IRC16:41
*** VW has joined #openstack-operators16:42
*** VW has quit IRC16:47
*** VW has joined #openstack-operators16:48
*** chlong has quit IRC16:50
*** derekh has quit IRC16:55
*** geoffarnold is now known as geoffarnoldX16:58
*** geoffarnoldX is now known as geoffarnold16:58
*** chlong has joined #openstack-operators17:02
*** vsedelnik has quit IRC17:06
*** Fin1te has quit IRC17:06
*** devlaps has quit IRC17:12
*** elo has joined #openstack-operators17:15
*** egon has joined #openstack-operators17:27
egonDoes anyone have swap turned on in their VMs?17:28
*** subscope has joined #openstack-operators17:37
*** stanchan has joined #openstack-operators17:42
*** stanchan has quit IRC17:43
*** harlowja has joined #openstack-operators17:46
*** Fin1te has joined #openstack-operators18:02
*** stanchan has joined #openstack-operators18:06
*** fawadkhaliq has joined #openstack-operators18:11
*** harshs has joined #openstack-operators18:14
*** vsedelnik has joined #openstack-operators18:14
*** fawadkhaliq has quit IRC18:15
*** harshs has quit IRC18:23
*** fawadkhaliq has joined #openstack-operators18:25
*** harshs has joined #openstack-operators18:30
*** VW has quit IRC18:35
*** VW has joined #openstack-operators18:36
*** VW has quit IRC18:41
*** fawadkhaliq has quit IRC18:41
*** cdelatte has joined #openstack-operators18:50
*** Rockyg has joined #openstack-operators18:58
*** Marga_ has quit IRC19:00
*** matrohon has joined #openstack-operators19:03
*** bradjones has quit IRC19:04
*** SimonChung has quit IRC19:05
*** SimonChung has joined #openstack-operators19:05
*** Piet has quit IRC19:14
*** bvandenh has quit IRC19:24
*** VW has joined #openstack-operators19:27
klindgrenegon, we don't19:29
klindgrenexcept under windows - because its windows...19:29
*** Marga_ has joined #openstack-operators19:35
*** pontusf1 has quit IRC19:42
*** pontusf1 has joined #openstack-operators19:43
*** Marga_ has quit IRC19:46
*** Marga_ has joined #openstack-operators19:46
*** VW_ has joined #openstack-operators19:53
*** VW has quit IRC19:57
*** VW has joined #openstack-operators19:57
*** VW has quit IRC19:57
*** Piet has joined #openstack-operators19:57
*** VW_ has quit IRC19:57
*** VW has joined #openstack-operators19:58
*** bvandenh has joined #openstack-operators20:00
*** emagana_ has joined #openstack-operators20:08
*** emagana has quit IRC20:08
*** bvandenh has quit IRC20:17
*** csoukup has quit IRC20:18
*** elo has quit IRC20:32
egonHeh.20:34
egonI'm trying to find anyone who does. I think everyone has it disabled by default.20:34
jproulxegon I have has swop enabled on some VMs but our backing ephemeral store is network not local so we discourage it, why do you ask?20:36
*** elo has joined #openstack-operators20:38
*** markvoelker has quit IRC20:41
mgagneAbout the same here. If you *really* need a swap, create a file and enable swap on it: http://www.rackspace.com/knowledge_center/article/create-a-linux-swap-file20:41
jproulxIf you really need a class of systems with swap you can use instance types to provide that and it does work (for me anyway with ubuntu cloud images as the usual VM operating system)20:44
Rockyghttps://etherpad.openstack.org/p/mitaka-cross-project-session-planning20:44
*** Fin1te has quit IRC20:46
*** vsedelnik has quit IRC20:47
klindgrenRockyg - is the mitaka ops summit stuff been finalized somewhere?20:48
*** vsedelnik has joined #openstack-operators20:48
*** elo has quit IRC20:48
Rockygnope.  This is the work etherpad for the cross project sessions.  The intention is to schedule so ops can participate in them20:50
*** vsedelnik has quit IRC20:52
*** matrohon has quit IRC20:55
*** vsedelnik has joined #openstack-operators20:55
*** harshs has quit IRC20:56
mgagneklindgren: so I tested the AZ patch for cells: https://etherpad.openstack.org/p/PAO-LDT-cells-patches21:01
mgagneklindgren: I added a note but found that nova-api throws a KeyError for the first az-list call made, it's fine after. Any idea?21:01
klindgrenmgagne, their was somehtign that didn't specifically work with that patch as well iirc21:02
*** SimonChung has quit IRC21:02
klindgrenlike service-list or something21:02
*** harshs has joined #openstack-operators21:03
klindgrenI dont recall getting a key-error on az-list21:03
mgagneklindgren: are you using the database or json config?21:03
*** harshs has quit IRC21:03
klindgrenjson_config I *think*21:03
klindgrenjson_config21:04
klindgrenand we have weight_offset and sale delimited21:04
klindgrenerr21:04
klindgrenactually no21:04
klindgrenwe are using DB21:04
klindgrendidn't look hard enough at the cells.json file21:04
*** Fin1te has joined #openstack-operators21:04
*** harshs has joined #openstack-operators21:05
mgagnemuch more simple IMO to use with a masterless puppet setup21:05
mgagnebut we used to have to patch Nova to make JSON works :-/21:06
klindgren302 - mdorman he kinda lead the design on that - I just get to deal with the outcome21:06
klindgrenha!21:06
klindgrenthe shift to cells was agreed upon - but happened when I was on vacation - so I am pretty fuzzy on some of the exact implementation details21:07
mgagnemdorman: you got some explaining to do young man! /jk21:07
egonjproulx: Yeah, we have swap disabled too, but 3 years later, someone is arguing about it. I was just wondering if anyone did anything different.21:13
egonmgagne: swapfiles have been as fast as swap partitions for a long time now, so that's definitely the modern approach.21:14
klindgrenthe real question is - do you want to deal with someone swapping a compute node to death21:15
klindgrenor if you are ok with the end users app getting oomkilled21:15
klindgrenwe are ok with oomkilled21:15
klindgrenbut we also run instances on ephemeral storage - so if they want to enable swap via swap  file we can't stop them21:15
*** vsedelnik has quit IRC21:17
*** Fin1te has quit IRC21:18
*** markvoelker has joined #openstack-operators21:20
mdormanjust saw notifications for my name here21:22
mdormansorry, wasn’t paying attention21:22
klindgrenblaming you for cells21:22
klindgrenmuahahaha21:22
mdormanah, yep, pretty much21:22
klindgrenmgagne, was asking for our cell setup - we use db vs's cells.json21:22
mdormanwe are using db config, not json21:23
mgagnemdorman: ok, I'm testing AZ support for cells and it's not totally working with JSON config21:23
klindgrendo we have some puppet stuff to automate that all together?21:23
mgagnemdorman: first call always fail. debugging now21:23
mdormanno, no puppet config for setting up cells in db.  it has to be done manually21:23
mdormaniirc the AZ stuff was part of the patches we got from sorrison and i think the AZ components of it more or less just worked for us21:24
mgagnemdorman: I'm suspecting patch only support db config and first call always fails and caches empty result somewhere until nova-api is restarted21:24
mdormanit was the flavors i had a harder time getting to work for kilo and i think i eventually just gave up.21:24
mdormanmgagne:  yeah that seems like a good hypothesis21:24
mgagnemdorman: logs are in nova-api log of api cell, nothing in nova-cells of api or compute cells21:25
*** Rockyg has quit IRC21:25
mdormanhmm.  yeah i donno i’d hvae to look at it.  it seems odd that there should be a difference between db vs. json config at that level though.21:28
mgagnemdorman: here is the logs I get: http://paste.openstack.org/show/473312/21:29
mgagnebut no trace of the error in the compute cell which doesn't help :-/21:29
*** priteau has quit IRC21:31
*** Piet has quit IRC21:33
mdormanwhat’s in your cells.json?21:36
*** SimonChung has joined #openstack-operators21:37
mgagnewhich one?21:38
mgagneIn API cell: all compute cells. In compute cell: the API cell.21:38
mgagneno luck adding weight_scale to api cell config in compute cell. (just in case)21:39
mgagnewill continue debug with LOG.debug and friends21:39
mdormani was thinking in the api cell21:40
mdormanbut i’m sure you have it right.21:40
mdormani donno man21:40
mdormani don’t really have any great ideas21:41
mgagnehehe21:41
mgagneI added weight_ to API cell configs already. I'm trying to debug where the error comes from. so far, it looks to not come from the compute cell21:41
mgagneoh hahaha21:44
mgagneI had a rogue nova-cells service21:45
mgagnemdorman: ^21:45
mdormanhahah21:48
mdormanluckily something simple then :)21:48
mgagneyea, I should get ride of that nova-cells service21:48
*** harshs has quit IRC21:48
mgagnerunning it somewhere I shouldn't and always forget about it21:49
mgagnemdorman: so patch is working "fine" so far. hosts is empty so will need to debug again21:49
mgagnethanks for your moral support :P21:49
mdormananytime!21:50
*** elo has joined #openstack-operators21:54
*** harshs has joined #openstack-operators21:55
mgagnemdorman: when listing AZ as admin, do you have info about services and compute nodes?21:56
*** regXboi has quit IRC22:05
*** alejandrito has quit IRC22:05
mdormanlet me see22:11
mdormanlike ‘nova availability-zone-list’ yo umean?22:11
*** vsedelnik has joined #openstack-operators22:19
*** elo has quit IRC22:20
*** elo has joined #openstack-operators22:20
*** Piet has joined #openstack-operators22:23
*** vsedelnik has quit IRC22:25
mgagneyea22:27
mgagnemdorman: ^22:27
mgagneklindgren, mdorman: The script I use to sync Nova flavors across cells: https://gist.github.com/mgagne/6d9dfd23b7be0f380726 :D22:30
mdormancool22:30
mdormanwe had originally set it up with mysql triggers, since our api and cell dbs happen to be in the same mysql instance22:31
mdorman(that was intended to just be a temporary thing)22:31
mdormanand we rarely create new flavors22:31
*** VW has quit IRC22:33
*** VW has joined #openstack-operators22:33
mdormangotta run now, mgagne, but will be back tomorrow.  have a good one.22:38
*** VW has quit IRC22:38
mgagnemdorman: alright, have a good one!22:38
mdormanthanks22:39
*** mdorman has quit IRC22:39
klindgrenso just noticed this.  Nova with neutron setup. When booting a vm with a secuirty group that doesn't exist you will get a 400 error saying the securitygroup is not found.22:46
klindgrenHowever, if you attempt to boot a vm where a security group is created, but has a different case, the request goes through but the vm throws an error later on22:46
klindgrenIE security group is "Testing" but you supplied "testing" or vise-versa22:47
klindgrenSeems like if both nova and neutron treat security groups with case sensitivity - we should probably catch errors like this without blowing up way down in the stack, iwht an error that isn't resurfaced to the end-user22:48
*** markvoelker has quit IRC22:48
*** cdelatte has quit IRC22:49
egonyeah, there's a lot of case sensitivity in openstack, even in cases where it probably doesn't make sense.22:49
egonsomeone explained that there were performance reasons for doing that22:49
egonand that changing it now was Too Hard.22:50
egonBut, you still end up with strange case sensitivity cases.22:50
*** stanchan has quit IRC22:54
mgagneklindgren: don't forget that MySQL might be case insensitive22:55
mgagneklindgren: last time I know of that someone tried to fix a similar case, it broke our cloud: https://bugs.launchpad.net/python-novaclient/+bug/142388522:56
openstackLaunchpad bug 1423885 in python-novaclient "nova flavor-show inconsistent with mixed case in names" [Low,Fix released] - Assigned to Abhishek Talwar (abhishek-talwar)22:56
*** derekh has joined #openstack-operators23:00
klindgrenmgagne, awsome23:02
klindgrenENEEDMOREALCOHOL23:03
*** emagana_ has quit IRC23:11
*** elo has quit IRC23:23
*** derekh has quit IRC23:32
*** zhangjn has joined #openstack-operators23:54
*** zhangjn has quit IRC23:54
*** zhangjn has joined #openstack-operators23:54
*** zhangjn has quit IRC23:55
*** zhangjn has joined #openstack-operators23:55
*** zhangjn has quit IRC23:55
*** zhangjn has joined #openstack-operators23:56
*** markvoelker has joined #openstack-operators23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!