Thursday, 2015-07-16

*** saneax has quit IRC00:13
*** britthouser has joined #openstack-operators00:28
*** j05hk has quit IRC00:30
*** SimonChung1 has quit IRC00:45
*** signed8bit_ZZZzz is now known as signed8bit00:51
*** j05hk has joined #openstack-operators00:52
*** j05hk has left #openstack-operators00:58
*** signed8bit is now known as signed8bit_ZZZzz01:02
*** openstack has joined #openstack-operators01:24
*** SimonChung has joined #openstack-operators01:26
*** SimonChung1 has joined #openstack-operators01:32
*** SimonChung has quit IRC01:36
*** j05hk has joined #openstack-operators01:36
*** signed8bit_ZZZzz is now known as signed8bit01:41
*** signed8bit is now known as signed8bit_ZZZzz01:52
*** signed8bit_ZZZzz is now known as signed8bit01:56
*** j05hk has quit IRC01:57
*** j05hk has joined #openstack-operators01:57
*** zul has joined #openstack-operators02:00
*** signed8bit has quit IRC02:06
*** markvoelker has quit IRC02:22
*** j05hk has quit IRC02:24
*** j05hk has joined #openstack-operators02:38
*** sorrison_laptop has joined #openstack-operators02:46
*** hakimo_ has joined #openstack-operators02:53
*** hakimo has quit IRC02:54
*** j05hk has quit IRC02:59
*** j05hk has joined #openstack-operators03:07
*** j05hk has quit IRC03:12
*** bradjones has quit IRC03:14
*** j05hk has joined #openstack-operators03:14
*** j05hk has quit IRC03:18
*** j05hk1 has joined #openstack-operators03:18
*** bradjones has joined #openstack-operators03:19
*** bradjones has quit IRC03:19
*** bradjones has joined #openstack-operators03:19
*** j05hk1 has quit IRC03:29
*** britthou_ has joined #openstack-operators03:30
*** britthouser has quit IRC03:33
*** saneax has joined #openstack-operators03:33
*** markvoelker has joined #openstack-operators03:33
*** markvoelker has quit IRC03:34
*** markvoelker has joined #openstack-operators03:34
*** markvoelker_ has joined #openstack-operators03:37
*** markvoelker has quit IRC03:39
*** jraim has quit IRC03:45
*** serverascode has quit IRC03:45
*** kelv has quit IRC03:45
*** bhunter71 has quit IRC03:47
*** kelv has joined #openstack-operators03:51
*** maishsk has quit IRC03:52
*** jraim has joined #openstack-operators03:56
*** serverascode has joined #openstack-operators03:58
*** mahito has quit IRC04:00
*** mahito has joined #openstack-operators04:03
*** britthouser has joined #openstack-operators04:15
*** belmoreira has joined #openstack-operators04:15
*** britthou_ has quit IRC04:16
*** belmoreira has quit IRC04:28
*** markvoelker has joined #openstack-operators04:49
*** markvoelker_ has quit IRC04:53
*** mahito has quit IRC04:55
*** saneax has quit IRC04:56
*** saneax has joined #openstack-operators04:59
*** mahito has joined #openstack-operators04:59
*** dboik has joined #openstack-operators05:57
*** simon-AS5591 has joined #openstack-operators05:58
*** dboik has quit IRC06:02
*** sw3 has left #openstack-operators06:20
*** spligak has quit IRC06:25
*** belmoreira has joined #openstack-operators06:25
*** simon-AS5591 has quit IRC06:31
*** sorrison_laptop has quit IRC07:37
*** simon-AS559 has joined #openstack-operators07:50
*** maishsk has joined #openstack-operators07:59
*** markvoelker has quit IRC08:04
*** markvoelker has joined #openstack-operators08:11
*** markvoelker has quit IRC08:16
*** markvoelker has joined #openstack-operators08:17
*** markvoelker has quit IRC08:25
*** derekh has joined #openstack-operators08:27
*** markvoelker has joined #openstack-operators08:29
*** mahito has quit IRC08:33
*** markvoelker has quit IRC08:33
*** simon-AS559 has quit IRC08:40
*** markvoelker has joined #openstack-operators08:44
*** simon-AS559 has joined #openstack-operators08:45
*** markvoelker has quit IRC08:48
*** belmoreira has quit IRC08:51
*** markvoelker has joined #openstack-operators08:58
*** markvoelker has quit IRC09:03
*** belmoreira has joined #openstack-operators09:05
*** maishsk has quit IRC09:52
*** maishsk has joined #openstack-operators10:04
*** simon-AS5591 has joined #openstack-operators10:32
*** simon-AS559 has quit IRC10:35
*** maishsk has quit IRC11:02
*** maishsk has joined #openstack-operators11:03
*** belmoreira has quit IRC11:15
*** bhunter71 has joined #openstack-operators11:26
*** saneax has quit IRC11:33
*** belmoreira has joined #openstack-operators11:40
*** vinsh has quit IRC11:58
*** simon-AS559 has joined #openstack-operators12:16
*** simon-AS5591 has quit IRC12:17
*** simon-AS5591 has joined #openstack-operators12:19
*** simon-AS559 has quit IRC12:21
*** j05hk has joined #openstack-operators12:22
*** saneax has joined #openstack-operators12:23
*** j05hk has quit IRC12:25
*** markvoelker has joined #openstack-operators12:29
*** markvoelker has quit IRC12:34
*** markvoelker has joined #openstack-operators12:43
*** saneax has quit IRC12:45
*** markvoelker has quit IRC12:48
*** cpschult has joined #openstack-operators12:48
*** saneax has joined #openstack-operators12:53
*** markvoelker has joined #openstack-operators12:57
*** markvoelker has quit IRC13:02
*** markvoelker has joined #openstack-operators13:05
*** markvoelker_ has joined #openstack-operators13:06
*** markvoelker has quit IRC13:10
*** cpschult has quit IRC13:17
*** bhunter71 has quit IRC13:17
*** jmckind has joined #openstack-operators13:22
*** j05hk has joined #openstack-operators13:25
*** jaypipes has joined #openstack-operators13:35
*** dminer has joined #openstack-operators13:40
*** ferest has joined #openstack-operators13:44
*** ferest has quit IRC13:47
*** dboik has joined #openstack-operators13:55
*** dboik has quit IRC13:55
*** dboik has joined #openstack-operators13:55
*** signed8bit has joined #openstack-operators13:57
*** maishsk has quit IRC13:57
*** dminer has quit IRC13:59
*** dboik_ has joined #openstack-operators14:00
*** csoukup has joined #openstack-operators14:00
*** dboik has quit IRC14:04
*** cpschult has joined #openstack-operators14:08
*** dminer has joined #openstack-operators14:08
*** mdorman has joined #openstack-operators14:15
*** fifieldt_ has quit IRC14:19
*** markvoelker_ has quit IRC14:24
*** hiddentoken has joined #openstack-operators14:32
*** bhunter71 has joined #openstack-operators14:32
*** maishsk has joined #openstack-operators14:32
*** jmckind has quit IRC14:34
*** radez_g0n3 is now known as radez14:44
*** hiddentoken has quit IRC14:51
*** jmckind has joined #openstack-operators14:52
*** markvoelker has joined #openstack-operators14:53
*** j05hk has quit IRC15:01
*** j05hk has joined #openstack-operators15:12
*** belmoreira has quit IRC15:13
*** regXboi has joined #openstack-operators15:27
*** maishsk has quit IRC15:27
*** ig0r_ has quit IRC15:40
*** SimonChung1 has quit IRC15:42
*** jmckind has quit IRC15:50
*** dboik_ has quit IRC16:03
*** alop has joined #openstack-operators16:06
*** simon-AS5591 has quit IRC16:07
*** dboik has joined #openstack-operators16:20
balajinis there a regular operators syncup?16:27
balajinif yes, where can i find more details16:27
mdormando you mean like an irc meeting?  or an in-person meet up?16:31
*** alop has quit IRC16:32
*** alop has joined #openstack-operators16:33
*** SimonChung has joined #openstack-operators16:48
*** SimonChung has quit IRC16:51
*** SimonChung has joined #openstack-operators16:53
*** mdorman has quit IRC16:56
*** derekh has quit IRC17:00
*** dabernie has quit IRC17:00
*** dabernie has joined #openstack-operators17:01
*** pasquier-s has quit IRC17:01
*** simon-AS559 has joined #openstack-operators17:02
*** simon-AS5591 has joined #openstack-operators17:03
*** simon-AS559 has quit IRC17:06
*** pasquier-s has joined #openstack-operators17:08
*** pasquier-s has quit IRC17:15
*** signed8bit has quit IRC17:19
*** signed8bit has joined #openstack-operators17:20
*** bhunter71 has quit IRC17:20
*** pasquier-s_ has joined #openstack-operators17:22
*** bhunter71 has joined #openstack-operators17:31
mgagneanyone having issues with Nova Kilo where the flavor is not properly updated after a resize?17:32
mgagnewith cells =)17:39
mgagnefound the issue. the instance_extra.flavor field is not properly updated although the instances.instance_type_id field is.17:43
mgagneso the compute cell is properly updated with the new? flavor object but api cell still has what looks like an old version of the flavor model.17:57
*** dboik has quit IRC18:02
*** maishsk has joined #openstack-operators18:04
*** Marga_ has quit IRC18:06
*** Marga_ has joined #openstack-operators18:07
*** signed8bit is now known as signed8bit_ZZZzz18:16
mgagnejaypipes: any hint about that one? I'm looking for input from nova without much success ^18:21
jaypipesmgagne: unfortunately, I don't know about that. alaski might be able to help with cells stuff.18:21
mgagnejaypipes: as long as I have someone knowledgeable to poke, I'm happy thanks18:22
mgagnealaski: ping ^18:22
alaskimgagne: hi18:22
mgagnealaski: I'm having issue with cells + resize (flavor)18:23
alaskiI haven't seen that before, but we limit resizes so they're not done often18:23
mgagnethe new flavor looks to not be properly updated in the instance_extra.flavor field at the api cell18:23
*** dboik has joined #openstack-operators18:23
mgagneI saw some bugs about it but they all got fixed in kilo :-/18:24
alaskiare you running stable/kilo?18:25
mgagnerunning from 2015.0.1 tag18:25
mgagnedue to me using ubuntu packages18:25
balajinha, mdorman isnt around18:25
balajinin real life is very difficult but even if there is an irc meetup once a month, it would be good18:26
balajinfeeding into the topics for mid cycle summit / summit18:26
alaskimgagne: I'm not sure what that corresponds to, but let me look at kilo code18:27
mgagnealaski: I tested with boot from volume. We retry with local disk and check if it changes anything18:30
alaskimgagne: I'm pretty sure you're going to need https://review.openstack.org/#/c/176518/, or parts of it18:30
mgagnealaski: sure, will try now18:31
alaskimgagne: what's likely happening is the instance update goes to the parent cell, but instance_update_at_top in Kilo doesn't handle flavor updates18:31
mgagnealaski: bug title sure reflects what it tries to fix :D https://bugs.launchpad.net/nova/+bug/146035018:31
openstackLaunchpad bug 1460350 in OpenStack Compute (nova) "Cells: Race deleting instance can lead to instances "undeleted" at the top" [Low,Fix released] - Assigned to melanie witt (melwitt)18:31
alaskimgagne: :)  it addressed a number of things but modestly mentioned only one bug18:32
mgagnehehe, trying to patch, lot of offsets and rejections :D18:33
alaskithere were a lot of cells improvements that happened leading up to that change18:33
mgagneis it a dead-end for me? :D18:34
alaskiwell, if you have the option of pulling code from git at or after that commit that would be best18:35
mgagnewell, I wouldn't be here if it was of that. master already has the patch =)18:35
mgagnewe are not geared for that yet18:36
alaskiotherwise you may need to find a few patches that happened in liberty, or try addressing your specific issues locally18:36
mgagneyea, doing that atm18:36
mgagneand then I end up with 90% of the code of liberty anyway :D18:36
alaskiheh18:37
alaskihttp://git.openstack.org/cgit/openstack/nova/tree/nova/cells/messaging.py?h=stable/kilo#n105218:37
alaskithe issue is very likely that instance there has flavor info in it, but the instance_update call isn't going to update the db for it18:37
mgagnewell, my instance is not deleted, it's resized :-/18:37
alaskiyeah.  and the resize will generate some instance_update_at_top messages, but they're only updating the instance, not the flavor18:38
mgagneyou mean the instance_extra.flavor field?18:39
alaskimgagne: yeah.  you would need a call like http://git.openstack.org/cgit/openstack/nova/tree/nova/objects/instance.py?h=stable/kilo#n707 in there18:47
*** signed8bit_ZZZzz is now known as signed8bit18:47
klindgrenwe were running into something with flavor stuff being broken with cells as well - but I think it was flavor create - I am pretty sure it was related to the the objectification of flavors18:47
mgagneklindgren: didn't get the chance to test flavor create. I upgraded packages and ran the migration.18:47
mgagneklindgren: now that you talk about it, will have to test that as well18:48
klindgrenmdorman has been handling that bit18:48
klindgrenI have been vusy packaging kilo into a venv + SCL for nova-compute + neutron so that we can run kilo18:48
klindgrenon our last few cent6 boxes18:48
klindgrenwhile we work on upgrading them18:49
mgagneklindgren: hehe. same here but with ubuntu packages. forward porting our patches atm18:49
* klindgren checks out patchset stuff18:49
klindgrenoh also I think something is broken with the x509 keypair stuff in cells too18:50
klindgrenunder kilo18:50
*** melwitt has joined #openstack-operators18:51
mgagnethe patch is mixed with objectification, it makes backporting hard...18:51
mgagne1.35 - Make instance_update_at_top, instance_destroy_at_top and instance_info_cache_update_at_top use instance objects18:52
mgagneone of our cell is still running Juno :-/18:52
klindgrenmgagne, the flavor create stuff was a patch that we are using from NeCTAR - so if you didn't use that - then flavor create is just as broken as always :-)18:55
mgagneklindgren: I don't know that patch. we are using a custom script to populate/sync flavors from a yaml file18:56
*** maishsk has quit IRC18:56
*** maishsk has joined #openstack-operators18:57
klindgrenmgagne, https://github.com/NeCTAR-RC/nova/commit/5abc8847dc89b162b6ae678176a5cfe4989144a918:58
mgagneklindgren: oh. do you have to sync your auto increment and pray it will never go out of sync?19:00
klindgrenmgagne, we had the api and child cell database on the same db servers so mike setup a sql trigger to keep that stuff insync19:01
klindgrenbut with that patch - it jsut seemed to work19:01
klindgrenwe also create new flavors - like bascially never19:01
*** rbrooker has joined #openstack-operators19:01
*** maishsk has quit IRC19:02
mgagneklindgren: main challenge for us was to make it simple enough that you don't need the local shaman to create it19:02
mgagneklindgren: and allow automated provisioning in our dev env.19:03
*** cpschult has quit IRC19:03
*** maishsk has joined #openstack-operators19:04
*** regXboi has quit IRC19:06
*** kencjohnston has joined #openstack-operators19:13
*** belmoreira has joined #openstack-operators19:18
*** regXboi has joined #openstack-operators19:21
*** cpschult has joined #openstack-operators19:22
*** mdorman has joined #openstack-operators19:28
*** mdorman_ has joined #openstack-operators19:39
*** mdorman has quit IRC19:40
*** mdorman_ is now known as mdorman19:40
*** simon-AS559 has joined #openstack-operators19:44
*** simon-AS5591 has quit IRC19:47
*** jproulx has joined #openstack-operators19:47
*** harlowja has joined #openstack-operators19:49
*** harlowja_ has quit IRC19:49
*** harlowja has quit IRC19:52
*** harlowja has joined #openstack-operators19:53
*** regXboi has quit IRC20:09
*** cpschult has quit IRC20:10
*** belmoreira has quit IRC20:11
*** cpschult has joined #openstack-operators20:23
*** cdelatte has quit IRC20:37
*** delattec has quit IRC20:37
*** cdelatte has joined #openstack-operators21:05
*** signed8bit is now known as signed8bit_ZZZzz21:11
*** signed8bit_ZZZzz is now known as signed8bit21:12
*** jproulx has quit IRC21:15
*** alop has quit IRC21:36
*** kencjohnston has quit IRC21:37
*** SimonChung has quit IRC21:45
*** bhunter71 has quit IRC21:49
*** SimonChung has joined #openstack-operators21:57
*** SimonChung1 has joined #openstack-operators21:59
*** SimonChung has quit IRC22:02
*** dboik has quit IRC22:02
*** cpschult has quit IRC22:05
mgagnealaski: so I managed to backport the fix and it works22:06
mgagnealaski: I had to backport and adapt ca908166dbfc311715e9e10449eb5e93add66a30 6d6fdeeff880cc5fe8edcc24ef94d3bc4ca924e2 eaaa659333c7586a71155c065dfb0f7b7e3758fc22:07
*** signed8bit is now known as signed8bit_ZZZzz22:19
*** radez is now known as radez_g0n322:19
klindgrenmgagne, was the migration stuff  an issue with straight kilo+cells or kilo+juno child cell?22:32
mgagneklindgren: I'm running kilo all across the board22:32
klindgrenI thought you had a juno cell?22:32
mgagneklindgren: I do have serious issue with juno cells but I'm not sure it's related22:32
mgagneklindgren: yep22:32
mgagneklindgren: but I disabled it and didn't test resize22:32
mgagneklindgren: so: nova resize => doesn't work with kilo+cells22:33
klindgrenmgagne, awesome22:33
mgagneklindgren: api cell kilo + api compute juno: still working on it.22:33
klindgrendo you have a patch set for kilo to fix that? :-)22:33
mgagneklindgren: it's badly broken, even more with backported patches22:33
mgagneklindgren: yea, will post them now22:34
klindgrenmgagne, <322:34
mdorman:)22:34
mgagneklindgren: keep in mind that it's not in production yet so a lot of tests still need to be done22:34
klindgrenmgagne, np - I will follow to see if it gets fixed up22:35
klindgrenmgagne, we don't have resize/migration enabled at all for our cloud - so its more like - if we turn it on - I kinda would like it to "just work"22:35
klindgrenand by turn it on - I mean actually add ssh keys/ssh access everywhere22:36
mgagneklindgren: that should be it: https://github.com/mgagne/nova/commits/kilo/nova_resize_cells22:36
mgagneklindgren: we already have patches messing up with cells rpc so the last patch is not the one I used as-is due to me updating rpc version with subversion (1.34.3) instead of 1.3522:37
*** csoukup has quit IRC22:37
klindgrenthats fine - we do the same thing... which kinda makes me wonder if we are all fixing up cell stuff.... maybe we in LDT - could do something so that cell v1 works better and we dont have to keep fixing it everytime to do basic tasks....22:38
mgagnewith kilo api cell and juno compute cell and with those patches, when running nova service-list: http://paste.openstack.org/show/381932/22:38
klindgrenIE do work to get cells v1 fixes upstreamed22:39
mgagneyep22:39
klindgrenand make cells "less shitty"22:39
klindgrenI will float ^^ to VW22:39
mgagneproblem is that cells v1 is considered legacy and no one will want to seriously work on it because "cells v2 is around the corner"22:40
mgagnewe are still supporting and forward porting this patch in our setup: https://review.openstack.org/#/c/59101/22:40
klindgrenexcept last I heard roadmap wise - cellsv1 was the only way forward for atleast 2 more cycles inc L22:41
klindgreniirc everyone will have one and only one cell in L22:41
klindgrenso if you have mroe than one cell you still need cells v122:41
klindgrenor maybee its "M"22:41
*** csoukup has joined #openstack-operators22:41
klindgreneitherway - I also heard that nova-network was dieing and that neutron was the way of the future....22:42
klindgren:-)22:42
mgagne:D22:42
* klindgren is on neutron so.... no comment as to that....22:42
mgagneand people are still surprised when hearing why people are still using nova-network.22:42
mgagneklindgren: yea, we have been running quantum since grizzly b1 =)22:43
klindgrenI am hopeful that the thing Carl Baldwin is working on with neutron22:43
mgagnewell, since grizzly was made available in ubuntu testing somewhere22:43
klindgrenwill solve a number of issues for deployers22:43
*** alop has joined #openstack-operators22:43
mgagneyea, I'm really looking forward to it22:44
klindgrenBeing able to define a network that is made up of multiple distinct L2 segments, where each l2 segment is limited to a set of hosts22:45
mgagneyep22:45
*** dminer has quit IRC22:45
klindgrenIE they way people build datacenters :-)22:45
mgagneno way? =)22:45
klindgrenyea - I know right - crazyness22:45
*** signed8bit_ZZZzz is now known as signed8bit22:45
mgagneso if anyone has idea on how to support kilo + juno with cells, that would be great22:47
*** signed8bit is now known as signed8bit_ZZZzz22:55
*** simon-AS559 has quit IRC22:57
*** zigo has quit IRC23:21
*** zigo has joined #openstack-operators23:22
*** csoukup has quit IRC23:23
*** mdorman has quit IRC23:24
*** britthouser has quit IRC23:47
*** britthouser has joined #openstack-operators23:50
*** jaypipes has quit IRC23:50
*** barra204 has joined #openstack-operators23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!