Monday, 2014-09-29

*** openstack has joined #openstack-ironic12:41
*** rakesh_hs has quit IRC12:53
lucas-hungrydtantsur, oh, I haven't read the reply12:54
lucas-hungrywill take a look12:54
*** lucas-hungry is now known as lucasagomes12:54
*** pensu has joined #openstack-ironic13:01
*** rloo has joined #openstack-ironic13:07
derekhlucasagomes: I've kicked off CI for https://review.openstack.org/#/c/124061/1 , assuming it passes your good to merge13:21
lucasagomesderekh, w00t! thanks!13:21
lucasagomesI will keep an eye on jenkins and remove the -2 if it's all good13:22
*** ifarkas has quit IRC13:23
Shrewslucasagomes: do you have the etherpad link with the ongoing hash ring work?13:23
Shrewsalso, morning13:23
lucasagomesShrews, morning, hmm lemme find13:24
lucasagomesShrews, https://etherpad.openstack.org/p/68IcuXSqzz13:25
Shrewslucasagomes: ah yes. thank you sir13:25
lucasagomesyvw :)13:26
* Shrews updates whiteboard with that link13:26
*** ifarkas has joined #openstack-ironic13:31
Shrewsumm, did review.o.o go away?13:34
GheRiveronot for me13:39
GheRiverobut etherpad is... sometimes13:39
Shrewsi'm working from the car shop this morning. not sure if it's their wifi or not13:40
*** pensu has quit IRC13:55
*** teju has left #openstack-ironic13:57
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449313:59
*** jistr has quit IRC14:00
rloohappy Monday Ironickers :)14:00
Shrewsmorning rloo14:00
rloolucasagomes: you OK with approving this? : https://review.openstack.org/#/c/124258/14:01
lucasagomesrloo, oh haven't looked into that14:01
rloomorning Shrews14:01
lucasagomesrloo, yes it's ok14:01
lucasagomesI agree with deva, if needed we can remove later14:01
rloolucasagomes: thx14:01
lucasagomesrloo, thanks and morning :)14:01
*** jistr has joined #openstack-ironic14:01
dtantsurrloo, Shrews, morning14:01
Shrewshi dtantsur14:02
rlooAfternoon lucasagomes, dtantsur.14:02
GheRiveromorning all14:02
rlooafternoon GheRivero14:03
*** pensu has joined #openstack-ironic14:09
openstackgerritDmitry Tantsur proposed a change to openstack/ironic: Fix typo in agent validation code  https://review.openstack.org/12467514:11
Shrewslucasagomes: nice catch on the missing FK on 124493. fixed that14:21
lucasagomesShrews, cool, I'm still working on the periodic task thing14:21
lucasagomesI'm getting a 401 when prepare try to cache the image :(14:21
NobodyCamGood morning Ironic14:22
rloomorning NobodyCam14:25
NobodyCammorning rloo :)14:26
jrollmorning ironic :)14:29
jrolldtantsur: oopssssssss14:29
dtantsur:)14:29
*** jrist has joined #openstack-ironic14:29
jrollyeah, I would rc1 that14:30
dtantsurmorning jroll, NobodyCam, GheRivero14:30
jrolldtantsur: +2'd14:31
NobodyCammorning jroll and dtantsur; Shrews and GheRivero too :)14:31
openstackgerritA change was merged to openstack/ironic: Updated from global requirements  https://review.openstack.org/12425814:31
GheRiverohi there14:31
lucasagomesNobodyCam, morning14:32
NobodyCamd'oh14:32
NobodyCamand ofc lucasagomes too!!14:32
NobodyCam:-p14:33
NobodyCamits a monday14:33
lucasagomes:)14:33
*** rwsu has joined #openstack-ironic14:33
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449314:33
ShrewsNobodyCam: morning14:33
NobodyCam:)14:33
romchegMorning Shrews, NobodyCam and everyone else14:34
*** k4n0 has quit IRC14:34
romcheg!14:34
Shrewso/14:34
NobodyCammornign romcheg :)14:34
dtantsurromcheg, o/14:34
ShrewsHas anyone received their email for PTL voting yet?14:41
NobodyCamonly for TripleO14:41
openstackgerritDmitry Tantsur proposed a change to openstack/python-ironicclient: Small fixes for utils/{common_filters,common_params_for_list}  https://review.openstack.org/12404014:42
ShrewsNobodyCam: odd they wouldn't all go out at the same time14:42
dtantsurrloo, ^^^14:42
NobodyCamShrews: many project only had one ptl canadate14:42
NobodyCamno need to vote on only one14:43
lucasagomesShrews, I did for TripleO only14:43
lucasagomesbut as NobodyCam said, I think Ironic won't need a vote14:43
NobodyCambrb14:44
dtantsurheh I've seen only one candidate for Ironic PTL, guess who :D14:44
jrollhm, I didn't get any14:46
jrolldid nova have multiple candidates?14:46
*** pensu has quit IRC14:47
*** pcrews has joined #openstack-ironic14:48
*** foexle has quit IRC14:49
rloothx dtantsur14:52
lucasagomesjroll, I think only michael still applied for nova ptl14:54
jrollaha, cool14:54
lucasagomeshaven't check tho, but as far as I remember14:54
lucasagomesand I didn't get any pool for the nova ptl etc... so I'm assuming that's correct :)14:54
*** hemna has quit IRC14:55
jrollyeah, figured :)14:55
*** todd_dsm has joined #openstack-ironic14:56
*** todd_dsm has quit IRC14:56
NobodyCamfyi: https://wiki.openstack.org/wiki/PTL_Elections_September/October_2014#Candidates14:56
jrollnice, ty NobodyCam :)14:58
jrollonly two elections, heh14:58
lucasagomesI wonder what happens if nobody is willing to be ptl15:00
jrollrm -rf $project15:00
lucasagomeslol15:02
lucasagomesidk, it kinda sounds like they should have a strategy for that15:02
jrollI'm sure they do15:02
dtantsurchoose a random person from ATC's15:02
dtantsurwill be fun15:02
lucasagomesheh15:03
rlooI asked last week, and devananda said they had no policy wrt no one running for ptl of a project.15:03
Shrewsthat would be an odd situation15:03
Shrewsi think it should be, of the ATC, the one with the oldest commit  :)15:04
* Shrews jokes, obviously15:04
lucasagomesheh15:06
lucasagomesit would be an anarchy project :P15:06
NobodyCamno ptl would seem like a dead or die-ing project :(15:07
lucasagomeshmm I don't see it that way15:07
NobodyCam:-p hehehe15:07
devanandamorning, all15:08
rlooI was thinking that no PTL would default to the previous/current PTL, thereby saving the PTL a few minutes or whatever from doing the paperwork or whatever ;)15:08
NobodyCamgood mornig devananda15:08
lucasagomessomeone have seem it from glance client http://paste.openstack.org/show/116745/?15:08
lucasagomesdevananda, morning15:08
Shrewsmorning devananda15:08
rlooahh, all hail the new PTL;) Morning devananda.15:08
lucasagomesdevananda, have  a min to talk about the hash_ring take_over()15:08
devanandalol. morning, all :)15:08
devanandalucasagomes: yep. just saw your comment15:08
lucasagomesdevananda, right there's more, didn't commented yet15:08
devanandaonly have a minute tho - gotta drive in a bit15:08
devanandaoh, k15:08
lucasagomesdevananda, 1) the thing about storing the condutor.id15:09
lucasagomeseverytime the conductor rejoins the cluster, even if it has an old entry we are unregistering it and registering again15:09
lucasagomesso take_over() will be trigged on the same conductor15:09
lucasagomesidk if it's desirable, we may want to make the rejoin of the conductor to just update the list of drivers and update_at fields15:10
devanandalucasagomes: so that's one of the things I changed by using the online field15:10
lucasagomes2) I'm having a hard time with glanceclient to cache the images, if I use the context coming from the periodic task I get a 401, and with an admin token I'm getting a http://paste.openstack.org/show/116745/15:10
lucasagomeslooking into that15:10
devanandait no longer deletes itself on shutdown15:10
devanandain part because of the FK, we actually can't "delete" the old conductor record15:11
lucasagomesdevananda, hmm... it still does, I'm using all the patches, I can check it15:11
lucasagomesso it shouldn't?15:11
*** romcheg has left #openstack-ironic15:11
devanandait calls unregister, which should be calling "set online=False"15:11
devanandaand making it appear offline eg to the hash ring15:11
devanandahowever15:11
devanandain a restart, the window there should be small15:11
devanandaso yes, also checking the heartbeat would help15:12
lucasagomesdevananda, oh... yeah15:12
lucasagomesdevananda, I commented also in another review about the "online"15:12
devanandathe difference between just heartbeat and heartbeat+online is an indication of intention15:12
lucasagomesI think it's a bit redudant, now I see one use for it, but isn't the heartbeat alone enough?15:12
lucasagomessince in a crash online may not be updated15:12
devanandano recent hearbeat? => maybe crashed, net split, or something else?15:12
lucasagomesand will be a misleading field15:13
lucasagomesright15:13
devanandano recent heartbeat + online=False => it definitely went away and isnt' coming back15:13
lucasagomesso online is more about "the conductor was gracefully stopped/started"15:13
devanandarecent heartbeat + online=False => restart15:13
devanandayes15:13
devanandathat may not be the best column name15:13
devanandaand clearly i didn't document THAT very wel inline15:13
lucasagomeshah yeah15:13
lucasagomesI was really like hmmm15:13
lucasagomesright it makes more sense now15:13
devanandaI also didn't implement any of the heartbeat+online checking bits15:14
dtantsurstepping afk, see you later15:14
*** dtantsur is now known as dtantsur|afk15:14
lucasagomesright, ok... I will see if I can sort the glance thing fix15:14
lucasagomessince that is the main problem right now15:15
lucasagomesand then we can sort that online bits15:15
*** vinbs has joined #openstack-ironic15:15
lucasagomesdevananda, thanks15:15
dtantsur|afkfolks, easy one for your stats: https://review.openstack.org/#/c/124675/  4x +2, +1 for Mr. J15:16
lucasagomes+15:16
lucasagomes+a15:16
dtantsur|afkthnx15:16
devanandalucasagomes: stepping afk for about an hour, but still available on gtalk15:17
lucasagomesdevananda, ack, I will continue taking a look into it15:17
lucasagomestalk to u there if needed15:17
lucasagomescheers15:17
*** Poornima has joined #openstack-ironic15:17
*** comstud has joined #openstack-ironic15:20
*** jistr has quit IRC15:22
*** jistr has joined #openstack-ironic15:22
openstackgerritA change was merged to openstack/python-ironicclient: Add unit tests for "ironic node-show" shell cmd  https://review.openstack.org/12000115:25
*** Guest73730 is now known as mgagne15:29
*** mgagne has quit IRC15:29
*** mgagne has joined #openstack-ironic15:29
*** vinbs has quit IRC15:32
*** dlaube has joined #openstack-ironic15:34
*** shakamunyi has joined #openstack-ironic15:36
openstackgerritA change was merged to openstack/ironic: Fix typo in agent validation code  https://review.openstack.org/12467515:49
*** jcoufal has quit IRC15:56
*** foexle has joined #openstack-ironic15:59
*** marcoemorais has joined #openstack-ironic16:14
devanandaback16:20
*** eghobo has joined #openstack-ironic16:27
lucasagomesdevananda, wb16:28
lucasagomesI will post what I've so u take a look16:28
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: WIP: add periodic task to rebuild hash rings  https://review.openstack.org/12461016:30
lucasagomesdevananda, https://github.com/openstack/ironic/blob/master/ironic/common/images.py#L289-L292 is failing with http://paste.openstack.org/show/116753/16:31
*** rushiagr is now known as rushiagr_away16:31
lucasagomesif I comment https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/deploy_utils.py#L420-L423 things works16:32
lucasagomes>.<16:32
lucasagomesit fetchs the image and all, but fail to look at the images metadata16:32
lucasagomes:/16:32
devanandathis is happening from prepare() ?16:33
devanandayea ... bad context perhaps?16:34
lucasagomesyeah from prepare16:34
lucasagomesyup, the thing is16:34
devanandanormally this is called with the user's context16:34
lucasagomesthe context from the periodic task was giving me a 40116:34
devanandabut that's nota vailable from the periodic task16:34
lucasagomesso I started using the admin context16:34
lucasagomesnow I'm hitting that other problem16:34
lucasagomesyeah16:34
devanandaright16:34
lucasagomesI think it should work with the admin context, idk if it's a glance bug16:35
lucasagomesI'm trying to check it now16:35
devanandaironic may now require a different privilege with glance to do this16:35
*** comstud has quit IRC16:35
devanandacheck the service catalog for ironic's permissions16:35
devanandadevstack sets up an account for ironic, eg. to talk to neutron16:35
lucasagomesoh right, lemme check that16:35
devanandait might not be auth'd for glance, since ironic didn't need to do that (outside of the context passed from nova)16:35
devanandawell - it needs to. but it wasn't16:36
lucasagomesit makes sense, lemme check here my devstack and catalog16:37
*** jistr has quit IRC16:38
*** comstud has joined #openstack-ironic16:41
*** ifarkas has quit IRC16:46
*** eghobo has quit IRC16:46
*** chuckC has quit IRC16:54
*** ChuckC_ has joined #openstack-ironic16:56
*** jrist has quit IRC16:58
*** pensu has joined #openstack-ironic16:59
NobodyCamquick run to the store... brb17:02
lucasagomesI will brb too, will be back for the meeting17:02
*** lucasagomes is now known as lucas-afk17:03
*** derekh has quit IRC17:06
*** rushiagr_away is now known as rushiagr17:09
*** pensu has quit IRC17:12
*** athomas has quit IRC17:13
*** ndipanov is now known as ndipanov_gone17:15
NobodyCamand back17:16
*** harlowja has joined #openstack-ironic17:16
*** harlowja has quit IRC17:22
Shrewsdevananda: i think we need to name the new FK in 124493 to match what the alembic upgrade would use. otherwise, we won't be able to downgrade correctly if 'ironic-dbsync create-schema' is used instead of 'upgrade'. trying to confirm that now...17:22
Shrewss/name the new FK/name the new FK in models.py/17:23
devanandaShrews: ++17:23
*** jrist has joined #openstack-ironic17:24
*** ChuckC_ has quit IRC17:24
*** harlowja has joined #openstack-ironic17:24
*** marcoemorais has quit IRC17:30
*** athomas has joined #openstack-ironic17:31
*** marcoemorais has joined #openstack-ironic17:33
*** marcoemorais has quit IRC17:33
*** marcoemorais has joined #openstack-ironic17:34
*** pensu has joined #openstack-ironic17:35
Shrewsyup. confirmed17:38
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449317:38
NobodyCamhumm  Keystone open'd for Kilo development... when are we opening up17:40
devanandaNobodyCam: after RC1 is tagged17:43
devanandaNobodyCam: which is when ever these bugs are fixed17:43
NobodyCam:) should we open the spec repo?17:44
devanandaNobodyCam: we could? though I'm going to remain focused on getting these bugs fixed until they're actually done17:44
NobodyCamdevananda: oh ya ++, just figured it would allow folks to start preping their specs17:45
devanandaNobodyCam: let's land any preparatory changes, like the new section, etc17:46
NobodyCamdevananda: like https://review.openstack.org/#/c/103225/17:46
NobodyCam:)17:46
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add HashRingManager to wrap hash ring singleton  https://review.openstack.org/12095117:48
NobodyCamdtantsur|afk: have you tested https://review.openstack.org/#/c/12129717:48
devanandaShrews: working on 124493 right now? mind if I rebase and poke at the "online" bits?17:50
Shrewsdevananda: go ahead17:51
*** tariqne has joined #openstack-ironic17:51
*** marcoemorais has quit IRC17:52
*** marcoemorais has joined #openstack-ironic17:53
*** marcoemorais has quit IRC17:55
NobodyCamjust posting spec reviews that others may wish to look over: https://review.openstack.org/#/c/12129717:55
*** marcoemorais has joined #openstack-ironic17:56
*** marcoemorais has quit IRC17:58
*** marcoemorais has joined #openstack-ironic17:58
*** marcoemorais has quit IRC17:58
* NobodyCam calls t-mobile to yell... brb17:58
*** marcoemorais has joined #openstack-ironic17:59
*** lsmola has quit IRC18:01
*** ChuckC_ has joined #openstack-ironic18:03
*** chuckC has joined #openstack-ironic18:04
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Fix hash ring stability  https://review.openstack.org/11893218:09
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449318:09
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add HashRingManager to wrap hash ring singleton  https://review.openstack.org/12095118:09
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Fix hash ring stability  https://review.openstack.org/11893218:13
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449318:13
devanandafixed a rebase error18:13
devanandadoh. need to fix again18:13
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Fix hash ring stability  https://review.openstack.org/11893218:14
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449318:14
*** marcoemorais has quit IRC18:16
*** marcoemorais has joined #openstack-ironic18:18
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: WIP: add periodic task to rebuild hash rings  https://review.openstack.org/12461018:20
devanandaafk a few18:20
devanandai think the first 3 patches in that series are good now. going to work on the periodic task when I get back18:21
*** marcoemorais1 has joined #openstack-ironic18:28
*** Poornima has quit IRC18:30
*** marcoemorais has quit IRC18:30
devanandajroll: is teh agent-nv test passing at all yet?18:39
jrolldevananda: not yet :(18:40
devanandajroll: think it'll be passing soon?18:40
jrolldevananda: the VMs don't have enough RAM in there, was trying to find the best way to solve it, got pulled into other stuff18:40
jrolldevananda: we could quickly hack it to set up 1024MB VMs when testing agent driver, but that's not viable long term18:40
JayFmaybe more useful to have something that works at all than one that just uses resources and failing18:41
JayFeven if we know we'd have to get the ram usage back down18:41
devanandayea, the wasted test resources was my concern at the moment18:41
devanandaseems like moving taht test to the experimental pipeline for now would be better18:42
devanandauntil you have a solution that allows it to pass18:42
*** rushiagr is now known as rushiagr_away18:42
devanandaOR what JayF suggested  -- but that's somewhat misleading, and we're liable to break it again when we enable parallel testing in our jobs18:43
JayFI'd rather if we're going to push any change, we push the one to make it work :)18:43
devananda(which I hope is early in Kilo)18:43
jrollI have a solution to make it (hopefully) pass, it's set VM_RAM to 102418:44
jrollbut as you say18:44
NobodyCampopping out for quick smoke b4 meeting18:45
jrollthe only real solution, I think, is to get DIB working18:45
jrollwe'll still need to use 1024 when testing coreos image built from source18:46
devanandajroll: that rules out the parallel testing against coreos images, but i think that's OK18:47
jrollagree18:47
*** bandicot has joined #openstack-ironic18:48
*** lucas-afk is now known as lucasagomes18:50
NobodyCamlast call for agenda changes18:51
devanandajroll: i don't see a toggle for that in infra/config or devstack-gate right now. sounsd like something we'll need eventualy anyways, to support your coreos images.18:52
jrolldevananda: right, just need to add it to dsg18:52
jrolldevananda: there's already an if for the agent_ssh driver18:52
NobodyCamis yuriyz around?18:53
*** BertieFulton has joined #openstack-ironic18:54
*** dtantsur|afk is now known as dtantsur18:56
dtantsurNobodyCam, did not test, but assumed it works, as it managed to actually build docs as part of docs job :)18:57
NobodyCamyep18:57
*** romcheg has joined #openstack-ironic19:00
NobodyCamBertieFulton: want to jion us in #openstack-meeting19:02
jrollare the SAP folks working on anything specific? :)19:05
NobodyCamjroll: not yet this is their first time joining.19:06
jrollright, curious if the goal is to deploy ironic at SAP, or work on some particular integration, etc19:07
*** marcoemorais1 has quit IRC19:11
NobodyCamjroll: I have a working POC env in the SAP lab that can deploy a Sles (Suse linux Enterprize server) with the use of the tripleo seed19:13
jrollNobodyCam: neat19:14
NobodyCam:)19:14
*** marcoemorais has joined #openstack-ironic19:15
*** marcoemorais has quit IRC19:21
*** marcoemorais has joined #openstack-ironic19:21
*** dprince has quit IRC19:49
*** harlowja has quit IRC19:50
*** marcoemorais has quit IRC19:54
*** harlowja has joined #openstack-ironic19:55
*** marcoemorais has joined #openstack-ironic19:55
Shrewsdevananda: fyi, you'll need to remove your -2 for https://review.openstack.org/10322519:56
devanandaShrews: ack, ty19:58
*** marcoemorais has quit IRC19:58
*** marcoemorais has joined #openstack-ironic19:59
lucasagomesalright it's dinner time here, have a good night everyone!19:59
devanandaok - I also need to switch cafes. the wifi here is terrible19:59
*** lucasagomes is now known as lucas-dinner19:59
NobodyCamgreat meeting all Thnk you :)20:00
NobodyCamthank even20:00
devanandabbiab20:00
dtantsurg'night :)20:00
*** dtantsur is now known as dtantsur|afk20:00
NobodyCamnight dtantsur|afk and lucas-dinner :)20:00
*** BertieFulton has left #openstack-ironic20:00
NobodyCambrb20:04
*** tariqne has left #openstack-ironic20:07
*** pensu has quit IRC20:07
*** andreykurilin_ has joined #openstack-ironic20:10
*** kincl has joined #openstack-ironic20:19
*** marcoemorais has quit IRC20:21
*** igordcard has joined #openstack-ironic20:22
*** todd_dsm has joined #openstack-ironic20:26
*** stevebaker has joined #openstack-ironic20:27
*** comstud has quit IRC20:38
*** comstud has joined #openstack-ironic20:38
*** marcoemorais has joined #openstack-ironic20:39
openstackgerritJarrod Johnson proposed a change to stackforge/pyghmi: Fix issue where thread kill() can 'stick' ipmi communications  https://review.openstack.org/12487320:56
openstackgerritJarrod Johnson proposed a change to stackforge/pyghmi: Recover from kill() while in command  https://review.openstack.org/12487320:58
*** jcoufal has joined #openstack-ironic21:02
NobodyCamI'm somewhat concerned about the "This will cause all existing21:08
NobodyCamnode -> conductor mappings to be invalid" comment21:09
openstackgerritA change was merged to stackforge/pyghmi: Recover from kill() while in command  https://review.openstack.org/12487321:09
ShrewsI think we need some documentation on the hash ring. The fact that the 'help' text for the HR options is several sentences long is a good indicator of that.21:10
rlooShrews: ++21:10
ShrewsI'm having a devil of a time trying to wrap my head around how it's supposed to work21:10
NobodyCamdo we expose reblance in the cli?21:11
Shrewseewww. i hope not21:11
Shrewsthat seems like the wrong thing to do21:12
jrollyes21:12
jrollself-healing ftw21:12
NobodyCamShrews: how do we handle the comment on 11893221:12
ShrewsNobodyCam: what comment?21:13
*** todd_dsm has quit IRC21:14
NobodyCamand unless a rebalance is triggered immediately will cause existing deployed nodes to be unable to reboot21:14
*** todd_dsm has joined #openstack-ironic21:14
ShrewsNobodyCam: oh. i don't know what the proposal is for the retriggering21:15
Shrewskill or add a conductor?21:16
NobodyCamlol.... 21:11 | Shrews > eewww. i hope not21:17
*** Guest78856 has joined #openstack-ironic21:18
Shrewswell, my "eewww" was my concern for too easily exposing a very disruptive operation21:19
*** Guest78856 is now known as annegentle21:19
NobodyCammy concern was that a operator would have to add then remove a fake conductor to rebalance after that lands21:19
ShrewsNobodyCam: yeah, valid concern that needs to be addressed21:20
jrollwhere does a rebalance need to be triggered?21:20
jrollor like, what situation, I should say21:20
NobodyCamlol well not forthe IPA folks21:21
NobodyCamajroll: https://review.openstack.org/#/c/11893221:21
ShrewsNobodyCam: i think this review will address it: https://review.openstack.org/#/c/124610/21:21
Shrewsjust need to trigger the periodic task21:21
jrollNobodyCam: but *what situation* would need a manual rebalance?21:22
*** linggao has quit IRC21:22
jrolloh, you're talking about the commit message21:23
jrollso it's only when updating an ironic environment to that code21:23
jrolljust... rolling restart?21:23
NobodyCamyea any existing env21:23
jrollprobably would need like a double rolling restart21:23
*** todd_dsm has quit IRC21:23
*** todd_dsm has joined #openstack-ironic21:24
jrolldunno if there's really a way to avoid that21:27
devanandaback21:28
NobodyCamwb devananda21:29
*** cinerama` is now known as cinerama21:30
devanandayes, https://review.openstack.org/#/c/118932/9 will require restarting all ironic services21:30
devanandaapi and conductor21:30
* devananda edits the commit message21:31
jrollI mean, deploying code in general requires that21:31
jrollunless there's some magic in ironic I don't know about :P21:31
devanandajroll: right. except this changes *where* the nodes map to21:31
devanandaso you can't just update one half (eg the API services)21:31
jrolloh, yeah21:31
jrollidk, I generally assume I need to update both21:32
devanandaalso, this patch doesn't include the periodic task yet21:32
devanandawe might want to invert the patch order if we are concerned about per-patch support21:32
NobodyCamdevananda: can we make special note in the chane log or you thinkk thats overkill21:33
NobodyCamchange log even21:33
devanandaNobodyCam: define "special note"21:33
devanandachange log == commit message history21:33
*** krtaylor has quit IRC21:34
NobodyCamhumm note sure. just concerned aboutthe change and folks knowing to restart21:35
NobodyCamsnote/not21:35
devanandaif folks are doing a release upgrade, they should restart everything21:35
devanandafolks running off of trunk will need to know21:35
devanandas/they should/we can assume they will/21:35
*** krtaylor has joined #openstack-ironic21:36
NobodyCamack :)21:36
*** todd_dsm has quit IRC21:40
rloodevananda: i was wondering about what/if we needed to do anything wrt string freeze cuz of the added string: https://review.openstack.org/#/c/124225/21:41
devanandarloo: I commented on that on rev221:43
rloodevananda: "Below here, I'm adding a new string to _LW(). I think it's important to log something here."21:44
rloodevananda: that comment. Is that all you need to do? the stringfreeze page has some points but I don't know if they apply here: https://wiki.openstack.org/wiki/StringFreeze#Medium.2FLarge_changes_and_changes_after_the_RC_release21:45
devanandarloo: it is only one word different from an existing string in the same file21:47
devanandas/provision/power/21:47
devanandarloo: so it seems like a "minor" change to me21:48
rloodevananda: ok ;) Shall I approve it then, or wait for others to provide feedback?21:49
devanandarloo: lgtm, already has 2 +221:49
rloodevananda: done.21:50
devanandaty21:51
rloodevananda: it had 2 +2 and Jenkins blessing before I even looked at it, so thought maybe folks were waiting for ... something... ;)21:51
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent  https://review.openstack.org/11893221:52
* devananda updates commit message on ^, blowing away a +221:52
devanandaactually, going to rebase it again21:52
devanandaI think this should be on top of the periodic task. Shrews - does taht make sense to you?21:53
*** krtaylor has quit IRC21:53
Shrewsdevananda: yep21:55
Shrewsdevananda: actually...21:58
Shrewsthe periodic task depends on is_active_on changes. i don't think you can do that21:58
devanandaShrews: oh? I was going to put the ring distribution fix on top of the other three21:59
Shrewsoh wait. that's a separate change.21:59
devanandaI think we have two scenario21:59
*** jcoufal has quit IRC21:59
devananda- change mapping, then add periodic task ==> an upgrade between these is unrecoverable22:00
devananda- add periodic task, then change mapping ==> an upgrade between these could yield a lot of unnecessary rebalancing22:00
devanandaso I think #2 is better22:00
*** krtaylor has joined #openstack-ironic22:01
Shrewsagreed #2 is better. but we should merge these fairly closely together. i don't see anyone running an in-between version22:01
devanandaright22:03
devanandawe need both in RC122:03
* jroll runs the in between just to spite Shrews22:06
*** todd_dsm has joined #openstack-ironic22:07
* Shrews avoids OnMetal22:09
NobodyCam118932 has a couple of pep8 issues22:09
NobodyCam:(22:09
jrollrude, Shrews22:10
*** mikedillion has joined #openstack-ironic22:10
jroll:P22:10
Shrewshehe22:11
* Shrews needs to make dinner. ciao folks22:11
NobodyCamenjoy Shrews22:11
* devananda rebases things22:12
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent  https://review.openstack.org/11893222:13
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449322:13
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state  https://review.openstack.org/12461022:13
* devananda fixes pep8 issues, hopefully22:13
devanandalucas-dinner: comments on your changes to 124610 ^22:13
*** igordcard has quit IRC22:16
*** bandicot has quit IRC22:20
devanandaadam_g: looks like we're not precaching the trusty image for DIB, so it's fetching (and sometimes failing to do so) from the 'net22:21
devanandaadam_g: http://logs.openstack.org/51/120951/10/check/check-tempest-dsvm-ironic-pxe_ssh-postgres/832b1ce/logs/devstacklog.txt.gz22:21
adam_gdevananda, yeah i saw that just a bit ago22:22
devanandaknow if there's an open bug?22:22
NobodyCamhumm anyone seen this error: http://logs.openstack.org/51/120951/10/check/check-tempest-dsvm-ironic-pxe_ssh-postgres/832b1ce/logs/screen-g-api.txt.gz#_2014-09-29_18_39_15_25922:22
NobodyCamlol22:22
adam_gdevananda, i haven't seen one. ill get one opened. theres likely some image caching stuff in place already  that we can use.22:23
* NobodyCam should read b4 posting22:23
devanandaadam_g: awesome, ty22:23
*** mikedillion has quit IRC22:23
devanandaNobodyCam: i think that's the issue i was just looking at22:24
NobodyCamdevananda: yep22:24
devanandaNobodyCam: seems like it should have failed sooner22:24
NobodyCamI have not seen that before22:24
adam_ghmm22:24
openstackgerritA change was merged to openstack/ironic: Conductor changes target_power_state before starting work  https://review.openstack.org/12422522:26
openstackgerritA change was merged to openstack/ironic-specs: Add a section for RPC API impact  https://review.openstack.org/10322522:30
*** mikedillion has joined #openstack-ironic22:31
openstackgerritA change was merged to openstack/ironic-specs: Stop using intersphinx  https://review.openstack.org/12129722:31
NobodyCamdevananda: :( check_uptodate.sh: ironic.conf.sample is not up to date. (118932)22:35
devanandayurg. i fixed it in some of the rebases22:35
* devananda wants to kill that check22:35
NobodyCamkill the check but keep the sample file?22:36
NobodyCamor remove both22:36
devanandakill the check. see the ML.22:36
devanandagoing to post a fix shortly22:38
*** penick has joined #openstack-ironic22:40
*** todd_dsm has quit IRC22:43
*** achanda has joined #openstack-ironic22:43
devanandawhat do ya'll think -- need a new CONF option for the timing of the periodic task to recheck hash rings?22:48
devanandaor not?22:48
devanandajroll, NobodyCam, rloo ^22:48
jrolldevananda: as opposed to?22:49
jrollthe power timeout option?22:49
jrollerr s/timeout/loop/22:49
devanandaas opposed to using the default global timing22:49
jrollvery yes please22:49
devanandayes what?22:49
devanandayes new CONF? yes default?22:49
jrollwe have our power loop pretty high because it can be slow22:49
jrollyes new conf22:49
devanandahm, ok22:50
devanandaso this has the same properties as the power loop22:50
devanandait only fires off max_periodic_workers at once22:50
jrollone property it doesn't have is using the crappy ipmi protocol :)22:50
devanandasure, but this is liable to pull a bunch of data from glance22:51
devanandarunning a lot of workers in parallel might not be what you want22:51
*** todd_dsm has joined #openstack-ironic22:51
jrollmmm22:51
devanandagranted, pxe images, not user images22:51
jrollright...22:52
jrollI meant we have our power loop set to a high interval, not high number of workers22:52
*** ChuckC_ has quit IRC22:52
jrollthough I see your point22:52
*** ChuckC_ has joined #openstack-ironic22:54
*** todd_dsm has quit IRC22:54
* devananda fixes conf file22:56
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent  https://review.openstack.org/11893222:56
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state  https://review.openstack.org/12461022:56
*** penick has quit IRC23:00
*** marcoemorais has quit IRC23:01
*** marcoemorais has joined #openstack-ironic23:01
*** marcoemorais has quit IRC23:02
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Stop running check_uptodate in the pep8 testenv  https://review.openstack.org/12491923:02
*** marcoemorais has joined #openstack-ironic23:02
devanandaalso, I'd be forever grateful if anyone has time to test the hash-ring patch series - especially the periodic tas - on a multi-node cluster23:03
devanandawith the pxe driver23:03
devanandaI will do it, ofc, but the mroe folks taht play with it the better23:03
NobodyCamdevananda: is that a check that the gate should run too?23:04
NobodyCamit would costly23:04
NobodyCam*would be23:04
devanandaNobodyCam: yes. see my commit message23:05
devanandait should run in the check pipeline, not the gate pipeline23:05
NobodyCamhummm py 26 failing on 11893223:05
NobodyCam:)23:05
NobodyCamhummm https://jenkins06.openstack.org/job/gate-ironic-python26/850/console23:06
NobodyCamAttributeError: 'HashRingTestCase' object has no attribute 'assertLess'23:07
devanandaoh, interesting23:08
NobodyCam27 passes23:08
devanandasome times I wish we weren't supporting old versions of redhat ...23:08
devanandahttps://docs.python.org/2/library/unittest.html23:09
devanandaassertLess was added in 2.723:09
NobodyCamnice23:09
jrolland rhel didn't backport it like OrderedDict?23:09
devanandaNobodyCam: I'm inclined, for right now, not to bother adding another test that non-votingly posts comments on our patches saying whether the config file is out of date23:14
NobodyCamjroll: hummm https://pypi.python.org/pypi/unittest223:14
devanandalet's go back to the good ol' days of just doing that manually now and then23:14
devanandait's as easy as "tox -egenconfig"23:14
jrollNobodyCam: something something global requirements23:14
NobodyCamlol23:15
NobodyCamdevananda: I'm ok with that..23:15
NobodyCamwe could also auto build it in to docs/ and remove it from the repo23:16
devanandaNobodyCam: this is the standard location that all the projects share23:18
NobodyCamyea, though I thought several have already removed the sample23:19
* NobodyCam is probibly wrong23:19
NobodyCamoh /me never knew about assertAlmostEqual and assertNotAlmostEqual.23:20
NobodyCamfor fuzzy math ?23:20
*** andreykurilin_ has quit IRC23:21
devanandanova removed teh sample23:21
devanandaseveral other projects got (-) feedback when they went t o23:22
devanandaas a consumer of other projects, i appreciate when they have a smple config for me23:22
devanandasample23:22
devanandaso I don't want to remove it from ironic23:22
NobodyCamI also like having it! for in doc or repo is fine, just so long as I have access to it23:23
* devananda rebases and fixes assertLess23:23
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent  https://review.openstack.org/11893223:23
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449323:23
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state  https://review.openstack.org/12461023:24
NobodyCambut ack +123:24
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add HashRingManager to wrap hash ring singleton  https://review.openstack.org/12095123:24
devanandaok, gonna head home before it starts raining too much more23:24
NobodyCamok stay dry23:24
NobodyCam:)23:24
*** mikedillion has quit IRC23:25
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Stop running check_uptodate in the pep8 testenv  https://review.openstack.org/12491923:28
*** romcheg has left #openstack-ironic23:39
*** lucas-dinner has quit IRC23:42
*** shakamunyi has quit IRC23:51
*** achanda has quit IRC23:55
*** harlowja is now known as harlowja_away23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!