Tuesday, 2014-09-23

« Monday, 2014-09-22 Index Wednesday, 2014-09-24 »
*** r-daneel_ has quit IRC00:03
greghaynesdevananda: Currently honeymoon vacationing so couldnt meeting / am pretty afk00:08
yjiang5jroll: NobodyCam: does this patch mean currently devstack will not do the ml2 thing for us automatically and we have to do it ourself? If yes, I will try to create a patch for it.00:13
yjiang5s/we have to do it ourself/we have to do it manually/00:13
jrollyjiang5: you want ironic/devstack to work with real switches or?00:13
yjiang5jroll: what do you mean of real switch?00:14
jrollI'm not sure what your question is00:14
jrollyjiang5: oh, you mean: ovs-vsctl add-br br-int00:14
jrolletc?00:14
yjiang5jroll: something like that, so that I can use the devstack to work on real physical machine (Currenlty I think it's for virtual machine, right?)00:15
*** penick has quit IRC00:15
jrollthe document in that patch is for real machines, yes00:15
jrolljust need to follow those directions00:16
devanandagreghaynes: ah! that's much more important :)00:16
jrollgreghaynes: congrats :)00:16
yjiang5jroll: cool. Possibly I can try to enhance devstack so that it can works for real machine also.00:16
jrollyjiang5: it does work for real machines, just with extra steps :)00:17
jrollyjiang5: not sure if this can be totally automated00:17
*** achanda has quit IRC00:19
*** yuanying has joined #openstack-ironic00:26
*** yjiang5 has quit IRC00:30
* devananda dinners00:32
*** HenryG has quit IRC00:36
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Adding support for decommissioning  https://review.openstack.org/10437900:42
*** marcoemorais has quit IRC00:44
*** yuanying has quit IRC00:59
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Add versioning to Agent decommission  https://review.openstack.org/10685901:00
*** ChuckC_ has quit IRC01:02
*** derekh has joined #openstack-ironic01:03
*** ChuckC_ has joined #openstack-ironic01:19
jrollJayF: dunno if you're still around, but this is interesting: http://logs.openstack.org/19/122519/3/check/check-tempest-dsvm-ironic-agent_ssh-nv/0f6fbcd/logs/ironic-bm-logs/baremetalbrbm_0_console.txt.gz01:21
jrollJayF: (go to the bottom)01:21
jroll[   33.816755] systemctl[293]: Failed to switch root: Specified init binary /sysroot/usr/lib/systemd/systemd does not exist.01:22
*** rloo has quit IRC01:22
*** killer_prince is now known as lazy_prince01:30
*** derekh has quit IRC01:38
*** Haomeng has quit IRC01:49
*** chenglch has joined #openstack-ironic01:55
openstackgerritOpenStack Proposal Bot proposed a change to openstack/ironic-python-agent: Updated from global requirements  https://review.openstack.org/11405801:55
*** yuanying has joined #openstack-ironic01:56
*** shakamunyi has quit IRC01:57
*** lazy_prince is now known as killer_prince01:59
*** yuanying has quit IRC02:04
*** dlaube has quit IRC02:42
*** ramineni has joined #openstack-ironic02:44
*** HenryG has joined #openstack-ironic02:48
*** dividehex has quit IRC02:55
*** achanda has joined #openstack-ironic02:59
*** yuanying has joined #openstack-ironic03:01
*** dividehex has joined #openstack-ironic03:01
*** yuanying has quit IRC03:09
*** killer_prince is now known as lazy_prince03:09
openstackgerritRuby Loo proposed a change to openstack/ironic: Raise IronicExceptions in node_power_action()  https://review.openstack.org/12247603:18
*** pcrews has quit IRC03:22
*** pcrews has joined #openstack-ironic03:25
*** pcrews has quit IRC03:41
*** vinbs has joined #openstack-ironic03:45
*** pensu has joined #openstack-ironic03:48
*** marcoemorais has joined #openstack-ironic03:50
*** marcoemorais1 has joined #openstack-ironic03:52
*** marcoemorais has quit IRC03:55
*** pcrews has joined #openstack-ironic03:59
*** yuanying has joined #openstack-ironic04:06
vinbsMorning Ironic!! :)04:09
*** Poornima has joined #openstack-ironic04:11
*** vinbs_ has joined #openstack-ironic04:14
*** yuanying has quit IRC04:15
*** vinbs has quit IRC04:17
*** vinbs_ is now known as vinbs04:17
*** vinbs_ has joined #openstack-ironic04:19
*** rameshg87 has joined #openstack-ironic04:21
*** rakesh_hs has joined #openstack-ironic04:22
*** vinbs has quit IRC04:22
*** nikunj2512 has joined #openstack-ironic04:23
*** vinbs_ has quit IRC04:24
*** vinbs has joined #openstack-ironic04:25
*** enterprisedc has joined #openstack-ironic04:28
*** lazy_prince is now known as killer_prince04:31
*** pensu has quit IRC04:31
*** achanda has quit IRC04:33
*** achanda has joined #openstack-ironic04:34
*** pcrews has quit IRC04:34
*** achanda_ has joined #openstack-ironic04:36
*** achanda has quit IRC04:38
*** pensu has joined #openstack-ironic04:57
*** Haomeng has joined #openstack-ironic05:03
*** yuanying has joined #openstack-ironic05:11
*** ChuckC_ has quit IRC05:15
*** yuanying has quit IRC05:20
*** achanda_ has quit IRC05:21
*** achanda has joined #openstack-ironic05:21
*** achanda_ has joined #openstack-ironic05:24
*** achanda has quit IRC05:26
*** harlowja is now known as harlowja_away05:31
*** Haomeng has quit IRC05:40
*** rushiagr_away is now known as rushiagr05:52
*** ChuckC_ has joined #openstack-ironic05:54
*** agordeev has quit IRC05:56
*** romcheg has joined #openstack-ironic06:06
openstackgerritOpenStack Proposal Bot proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/12334206:07
*** romcheg has quit IRC06:07
*** romcheg has joined #openstack-ironic06:08
*** romcheg has quit IRC06:08
*** jcoufal has joined #openstack-ironic06:10
openstackgerritRamakrishnan G proposed a change to openstack/ironic: Pass ipa-driver-name to agent ramdisk  https://review.openstack.org/12304706:11
*** yuanying has joined #openstack-ironic06:16
*** pensu has quit IRC06:18
*** pensu has joined #openstack-ironic06:21
*** k4n0 has joined #openstack-ironic06:23
*** yuanying has quit IRC06:25
*** achanda_ has quit IRC06:43
*** ifarkas has joined #openstack-ironic06:48
*** killer_prince is now known as lazy_prince06:49
*** romcheg has joined #openstack-ironic07:01
*** pradipta_away is now known as pradipta07:07
*** jcoufal has quit IRC07:18
*** jcoufal has joined #openstack-ironic07:20
*** yuanying has joined #openstack-ironic07:21
*** yuanying has quit IRC07:29
*** ndipanov_gone is now known as ndipanov07:31
*** marcoemorais1 has quit IRC07:33
*** rameshg87 has quit IRC07:38
*** Haomeng has joined #openstack-ironic07:39
*** derekh has joined #openstack-ironic07:47
*** agordeev has joined #openstack-ironic07:52
*** pensu has quit IRC07:59
*** pensu has joined #openstack-ironic07:59
*** andreykurilin has joined #openstack-ironic08:03
*** jistr has joined #openstack-ironic08:04
*** pradipta is now known as pradipta_away08:18
*** athomas has joined #openstack-ironic08:23
*** yuanying has joined #openstack-ironic08:26
*** lucasagomes has joined #openstack-ironic08:27
*** dtantsur|afk is now known as dtantsur08:28
dtantsurLate morning Ironic08:29
lucasagomesdtantsur, morning08:29
*** MattMan has quit IRC08:30
dtantsur:)08:30
*** MattMan has joined #openstack-ironic08:31
vdrokmorning Ironic, dtantsur, lucasagomes !08:33
*** foexle has joined #openstack-ironic08:34
*** pensu has quit IRC08:34
*** yuanying has quit IRC08:34
*** pensu has joined #openstack-ironic08:34
openstackgerritVladyslav Drok proposed a change to openstack/ironic: Add parameter to override locale to utils.execute  https://review.openstack.org/12274308:36
*** mkerrin has quit IRC08:41
yuriyzmorning Ironic08:44
dtantsuryuriyz, morning08:53
*** andreykurilin has quit IRC08:58
*** jcoufal has quit IRC08:58
*** mkerrin has joined #openstack-ironic09:11
*** pensu has quit IRC09:14
*** jcoufal has joined #openstack-ironic09:15
*** chenglch has quit IRC09:15
*** pensu has joined #openstack-ironic09:18
derekhlucasagomes: I rebuilt the testenvs last night , so if https://review.openstack.org/#/c/122360/1 passes your good to go09:19
openstackgerritDmitry Tantsur proposed a change to openstack/ironic: EXPERIMENTAL Implement hardware discovery in PXE driver  https://review.openstack.org/11003109:19
lucasagomesderekh, awesome!09:19
lucasagomesderekh, ta much09:19
derekhlucasagomes: np09:19
*** sabah has joined #openstack-ironic09:24
*** subah has joined #openstack-ironic09:25
lucasagomesderekh, the tripleo test still failing tho, but that's a diff error seems?09:28
*** sabah has quit IRC09:29
derekhlucasagomes: it hasn't run yet, hit "toggle Ci" on the bottom left to see comments from jenkins09:29
*** yuanying has joined #openstack-ironic09:31
lucasagomesoh gotcha09:31
* lucasagomes was looking at the last results09:31
lucasagomeslast time it ran09:31
*** datajerk has joined #openstack-ironic09:35
*** datajerk has quit IRC09:37
*** yuanying has quit IRC09:39
*** subah has quit IRC09:56
*** dtantsur is now known as dtantsur|brb09:57
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Do not use the context parameter on refresh()  https://review.openstack.org/12244109:58
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Spelling and Grammar mistakes fixed  https://review.openstack.org/11873310:07
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Spelling and Grammar mistakes fixed  https://review.openstack.org/11873310:08
*** Poornima has quit IRC10:28
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Remove redundant context parameter  https://review.openstack.org/12341210:29
lucasagomesdevananda, reading the meeting logs now, I had to solve some problems yesterday evening and couldn't make it to the meeting :(10:33
*** yuanying has joined #openstack-ironic10:36
lucasagomesNobodyCam, https://review.openstack.org/#/c/122360/ when you get some time10:43
*** yuanying has quit IRC10:44
*** Poornima has joined #openstack-ironic10:45
*** nikunj2512 has quit IRC10:51
*** lazy_prince is now known as killer_prince11:03
openstackgerritRamakrishnan G proposed a change to openstack/ironic: Pass ipa-driver-name to agent ramdisk  https://review.openstack.org/12304711:03
*** ramineni has quit IRC11:06
*** romcheg has quit IRC11:13
*** igordcard has joined #openstack-ironic11:19
*** Poornima has quit IRC11:20
*** killer_prince is now known as lazy_prince11:32
*** lucasagomes is now known as lucas-hungry11:34
*** romcheg has joined #openstack-ironic11:37
*** yuanying has joined #openstack-ironic11:41
*** dtantsur|brb is now known as dtantsur11:45
*** rakesh_hs has quit IRC11:48
*** yuanying has quit IRC11:49
*** pensu has quit IRC12:03
*** k4n0 has quit IRC12:09
*** vinbs has quit IRC12:17
*** vinbs has joined #openstack-ironic12:17
*** vinbs has quit IRC12:18
*** vinbs has joined #openstack-ironic12:18
*** Haomeng|2 has joined #openstack-ironic12:21
*** Haomeng has quit IRC12:22
*** lazy_prince is now known as killer_prince12:28
*** dprince has joined #openstack-ironic12:42
*** yuanying has joined #openstack-ironic12:46
*** romcheg1 has joined #openstack-ironic12:50
*** romcheg has quit IRC12:50
*** jcoufal has quit IRC12:52
*** killer_prince has quit IRC12:54
*** yuanying has quit IRC12:55
*** jcoufal has joined #openstack-ironic12:55
*** romcheg1 has quit IRC12:56
*** lucas-hungry is now known as lucasagomes12:56
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Do not use the context parameter on refresh()  https://review.openstack.org/12244112:59
*** rloo has joined #openstack-ironic12:59
*** vinbs has quit IRC13:09
openstackgerritRuby Loo proposed a change to openstack/ironic: Raise IronicExceptions in node_power_action()  https://review.openstack.org/12247613:12
*** pensu has joined #openstack-ironic13:14
*** igordcard has quit IRC13:33
*** pensu has quit IRC13:42
*** pensu has joined #openstack-ironic13:43
*** yuanying has joined #openstack-ironic13:51
*** r-daneel_ has joined #openstack-ironic14:00
*** yuanying has quit IRC14:00
jrollmorning all :)14:03
rloomorning jroll14:03
dtantsurjroll, rloo, morning14:07
rloohi dtantsur14:07
*** penick has joined #openstack-ironic14:11
*** penick has quit IRC14:12
NobodyCamgood morning Ironic14:13
NobodyCammorning dtantsur jroll rloo14:14
jrollmorning NobodyCam :)14:14
NobodyCam:)14:14
lucasagomesmorning jroll dtantsur rloo14:14
lucasagomesNobodyCam, too14:14
NobodyCammorning lucasagomes14:14
rloomornin' NobodyCam, lucasagomes14:14
jrollheya lucas14:14
NobodyCamlucasagomes: have you tested that agenst https://review.openstack.org/#/c/12079914:16
lucasagomesNobodyCam, yup, well the tripleo-CI was rebuilt and now the tests pass14:17
lucasagomesfor that change14:17
lucasagomesNobodyCam, see https://review.openstack.org/#/c/122360/14:17
NobodyCamlooks like rloo just approved14:18
rlooNobodyCam: yeah. I wasn't supposed to?14:18
NobodyCamrloo: nope.. but I'm always very careful with ssh changes as they can affect the gate14:19
jrollNobodyCam: how?14:20
rlooNobodyCam: but we already merged that change once.14:20
rlooNobodyCam: so by 'careful', what extra steps do you take? Do you test it yourself?14:20
jrollI'm not sure how they can break our gate but pass all of our jobs14:21
NobodyCamI always get the syntax agenst https://github.com/openstack-infra/tripleo-ci/blob/master/elements/testenv-worker/bin/ci_commands#L65-L7314:21
jrollNobodyCam: it passed tripleo CI, that's a good enough check :P14:21
NobodyCam:)14:21
rlooNobodyCam: so the syntax needs to be checked against that, but also tripleo have to rebuild their env, right?14:23
lucasagomesyup derek also +1'ed he's the main one in the ooo ci stuff14:23
lucasagomesso I think we are good to go with that14:23
jrollright, derek said he rebuilt, +1'd the change, it passed their CI14:23
NobodyCamrloo: yep that how I understand it14:24
derekhYup, test env's have been rebuilt so it should be good to go14:24
rlooNobodyCam: ok. I think we're good on this one. Now, let's see if we can do as good a job the next time... ;)14:24
jrollright, this one was reverted due to their CI breaking, this is the re-revert14:25
NobodyCamoh ya14:25
jrollso it's fine14:25
jrollbut us reviewers should read the big THIS WILL BREAK TRIPLEO CI comment :)14:25
NobodyCamit's all good now14:25
lucasagomesjroll, yup that's it14:25
rloothx derekh for staying on top of this :)14:25
NobodyCam:)14:25
derekhrloo: no prob14:25
rloohi dtantsur. Do you have a minute? wrt your comment: https://review.openstack.org/#/c/122224/3/ironic/drivers/modules/agent.py14:27
rloodtantsur: what did you mean by 'caller side'?14:27
openstackgerritYuriy Zveryanskyy proposed a change to openstack/ironic: Do not return 'id' in REST API error messages  https://review.openstack.org/12315014:29
*** jcoufal has quit IRC14:30
dtantsurrloo, there is some place where we call driver.vendor_passthru(taskm **kwargs), maybe it should be done there?14:31
rloodtantsur: probably the conductor calls that. But the whole idea of this exercise was for the drivers to NOT raise Exceptions, only IronicExceptions.14:32
dtantsurah right14:32
dtantsurhmm... Im afraid we'll be copy-pasting this part to all drivers...14:33
jrolloh hey, just so y'all know14:33
jrollthe openstack gate is mostly broken14:33
jrolldue to a bad apt mirror14:33
* jroll won't say whose apt mirror it is.... *cough*14:33
NobodyCam:(14:34
rloodtantsur: yeah, if we want all the vendor passthru's to do a similar thing.14:34
dtantsurperfect timing, just before devananda starts retargetting bugs :)14:34
jrollNobodyCam: hint: it's not HP's mirror14:34
NobodyCamjroll: dbl :( :(14:34
rloojroll: ouch. I was wondering what would happen given the RC1 deadline this week.14:34
jrollrloo, dtantsur: do other passthrus raise non-Ironic exceptions?14:34
jrollrloo: it's not even due to load, we just have broken mirrors ALL THE &$*^#&* time14:35
dtantsurrloo, well... now all our vendor-passthru look very similar, we may even want to move it to the base class later. but I'm ok with the code for now14:35
jroll++14:35
rloojroll: other passthrus *might* raise non-Ironic exceptions. I cannot guarantee that they won't. But I did go through the code and tried to handle all exceptions.14:35
jrollsince those are async we should be capturing those errors14:35
rloojroll: yes, that makes sense.14:36
rloodtantsur: should i open a bug about all vendor-passthrus should handle non-IronicExceptions "just in case", and that the soln might be to move some common code to base class?14:37
dtantsurrloo, yes please14:37
jrollI would move it to the conductor, I think14:37
rloojroll: that goes back to: the whole point of this exercise is that the drivers should not be (knowingly) raising non-Ironic Exceptions14:38
jrollrloo: indeed14:38
rloojroll: unless we decide that the conductor should not be raising non-Ironic exceptions, but OK below that.14:39
*** yuriyz has quit IRC14:40
jrolleh, I like the former14:40
rloojroll: ok, good ;)14:40
rloojroll: wrt the gate being broken, is there a bug or something, or basically have to wait for an all clear from someone/infra?14:41
jrollI think just wait for all clear, I'll give you a heads up when I hear about it14:41
jrollI would keep approving things, they just likely won't pass14:42
rloojroll: thx. Would you mind updating the etherpad with the gate status?14:42
jrollsure14:42
rloojroll: but we shouldn't do recheck, or is the problem sporatic?14:43
jrollit's sporadic, but <20% of jobs are passing14:43
jrollor something like that14:43
*** yuriyz has joined #openstack-ironic14:43
rloojroll: ugh, probably not work doing rechecks then. Unless we have a HIGH bug fix I guess.14:44
jrollright.14:44
jrollI mean, it shouldn't take too long14:44
jrolljust waiting on our mirrors team14:44
jrollmain point of contact is out today but they've contacted the rest14:45
*** pensu has quit IRC14:48
*** yuanying has joined #openstack-ironic14:56
rloodtantsur: (and anyone else). Thx for waiting, but I'm fine if I -1 a patch, and others approve it, esp if I don't get back in a timely fashion. (wrt 121923)14:57
rloodtantsur: I'm not always 'here' and don't want to feel like I'm a bottleneck ;)14:57
*** jistr has quit IRC14:58
*** dlaube has joined #openstack-ironic15:00
dtantsurack :)15:03
*** yuanying has quit IRC15:04
devanandamorning, all15:15
NobodyCamgood morning devananda15:15
jrollheya devananda15:15
NobodyCamdevananda: was just looking for your PTL Email so I could +1 it15:16
jrollmorning devananda15:16
jrollNobodyCam: PTL election doesn't work like that :P15:16
NobodyCamlol15:16
devanandaNobodyCam: please do not +1 it. That is expressly discouraged. See anteaya's email which opened the PTL nominations.15:16
jrolloh wow, I said good morning twice15:16
* jroll drinks more coffee15:16
devanandajroll: nice to see you too :)15:16
jroll:P15:16
devanandaalso, is the gate broken?15:17
rloomorning devananda :)15:17
NobodyCamyep!15:17
jrollyeah15:17
NobodyCamproken mirror (somewhere)15:17
jrolldevananda: bad apt mirror at rackspace15:17
NobodyCam:-p15:17
jrollhow this still isn't fixed is beyond me15:17
devanandajroll: oh15:18
jrollmirror team was poked over an hour ago15:18
* jroll embarrassed face15:18
JayFjroll: you should contact IM15:19
dtantsurdevananda, morning15:19
jrolllikely the wrong channel to make these decisions, but yeah15:20
anteayayes no +1ing candidate emails15:20
anteayadevananda: thanks for the discouragement15:21
anteayait is assumed candidates have backing15:21
anteayaotherwise why would they run?15:21
devanandaanteaya: indeed :)15:21
anteayaand some non-english type folks might think +1ing the emails is actual voting15:21
anteayaso then we just get more15:21
anteayawhich make tristan and myself unhappy15:22
anteayasince we can't find the actual candidate email15:22
anteayaand we can't miss any15:22
NobodyCamack :) thank you for the info15:23
NobodyCamand mornign anteaya15:23
lucasagomesdevananda, morning15:23
anteayaNobodyCam: morning :D15:24
*** ghe_ has joined #openstack-ironic15:26
*** dtantsur is now known as dtantsur|afk15:27
devanandawe didn't jump on the hash ring discussion after the meeting yesterady ...15:30
devanandasorry about that. I got side tracked by several other discussions15:31
openstackgerritDmitry Tantsur proposed a change to openstack/ironic: EXPERIMENTAL Implement hardware discovery in PXE driver  https://review.openstack.org/11003115:31
lucasagomesdevananda, if we are planning a rc2, I think we may should give it a go to get it fixed in J15:33
*** jistr has joined #openstack-ironic15:35
devanandaI think it's very important for anyone using the PXE driver at non-trivial scale15:36
lucasagomesyeah, plus, the pxe driver is the one we usually recommend people to use15:36
NobodyCamit is our refference driver15:37
lucasagomesyeah, I have to look more into greghaynes patches15:37
lucasagomesif we decide to fix it I'm happy to help with whatever needed code and reviews15:37
* NobodyCam is just affraid its late in cycle for such a changge15:37
JayFNobodyCam: I mean, I'm not sure we can break it any worse15:38
jrollNobodyCam: well, how much worse could it get?15:38
jrollheh15:38
NobodyCam^^^ is also very true15:38
lucasagomes:)15:38
rlooyesterday, devananda mentioned that there had been discussions wrt how to fix the hash ring. Is there a summary anywhere? Does anyone have a handle on how much is involved wrt fixing it?15:38
devanandaheh15:38
lucasagomesrloo, I read the meeting logs15:38
devanandarloo: outside of myself or greghanes, i'm not sure15:39
lucasagomesand btw, sorry I didn't get home in time for the meeting :(15:39
lucasagomes(I'll probably be moving home soon, so I went to see the new apt)15:39
devanandalucasagomes: no worries. life happens :)15:39
NobodyCamoh very kewl... is the new pad nice15:39
*** viktors is now known as viktors|afk15:40
*** linggao has joined #openstack-ironic15:44
devanandai'm chasing a few other bugs at the moment15:45
devanandaand will turn my attention to the hash ring shortly, unless someone else wants to start digging in there15:46
*** jasondotstar has joined #openstack-ironic15:51
lucasagomesdevananda, right, I can give it a go... so the patches we have right now are the greg ones right?15:52
* lucasagomes will take a look... 15:52
lucasagomesyou know if there's something missing from those?15:52
devanandalucasagomes: i haven't looked through them in a bit, so, not sure15:54
NobodyCamrloo: gots a second to take a look at https://review.openstack.org/#/c/11861415:54
lucasagomesdevananda, ack... so I see two... one exposing the hashring oursite rpcapi15:54
lucasagomesand one add conductor_id to the nodes table15:54
lucasagomesdb table*15:54
*** romcheg has joined #openstack-ironic15:55
lucasagomeslooks like there's only plumbing stuff proposed yet15:55
rlooNobodyCam: oh yeah. That was on my list to do one day. Guess I'll do it now ;)15:55
NobodyCam:) thank you :)15:55
lucasagomesdevananda, apart from that, we should look at lifeless patches to fix the ring rebalancing15:56
lucasagomesare you looking at those too?15:56
devanandawanyen: hi! is Syed on IRC? I would like to discuss https://bugs.launchpad.net/ironic/+bug/1370466 with him15:56
devanandalucasagomes: was just digging into the ilo bug that wanyen raised yesterday. I'm confused by some things in the driver ... like using ilo power with ipmitool management interface in the same driver15:57
lucasagomesdevananda, yeah, there's a known problem with the credentials there15:58
devanandalucasagomes: i'm starting a new pad to collate thoughts on the hash ring15:58
devanandahttps://etherpad.openstack.org/p/68IcuXSqzz15:58
lucasagomestl;dr we ma want to have generic credentials for drivers on K15:58
devanandalucasagomes: yea. it seems really odd to mix different OOB drivers15:58
lucasagomesdevananda, cool, yeah I have to dig more into the hash ring code to be able to understand how to fix, I will take a look at the etherpad15:59
lucasagomesdevananda, yeah it is15:59
devanandalucasagomes: also generic credentials don't really make sense, since different drivers require different /types/ of credentials16:01
JayFWhile credentials are on the mind :)16:01
*** penick has joined #openstack-ironic16:01
JayFme and JoshNang and others here at Rackspace have been talking about how we want a way to hide secrets int he API16:01
*** yuanying has joined #openstack-ironic16:01
devanandauser/pass is ubiquitous. but what about ssh key? SNMP MIB? the set of four things needed for IPMI double bridging?16:01
JayFand possibly even decide who can see the secrets based on policy.json16:01
devanandaetc...16:01
devanandaJayF: yes! please!16:01
lucasagomesdevananda, we don't have a proper solution for that yet... but drivers needs at least address, username, password16:02
JayFIt came up because we wanted to ship the agent more information for decom16:02
devanandaJayF: stick that in the gdoc for planning and please propose a spec for discussion16:02
lucasagomesthe thing is, we may should not be using the namespaces for those parameters16:02
lucasagomesilo_address, ipmi_address... maybe we should only use address16:02
jrollwe're not supposed to propose kilo specs yet :P16:02
lucasagomeshave like a convention or something16:02
* jroll thinks we should do it anyhow16:02
JayFso it can do more of the validation stuff (like, here's the switchport we say you're on), but we did *not* want to ship the bmc credentials to the agent as it clearly violates the oob|ib separation16:02
devanandajroll: i don't mean today() :)16:03
jrollno! today!16:03
*** ndipanov is now known as ndipanov_gone16:03
jrollNOW16:03
jroll:P16:03
devanandajroll: oh. you mean it's not done yet?16:03
devananda:)16:03
lucasagomesJayF, +1... would make sense to have another field in the nodes to put the credentials (instead of driver_info)16:03
jrollhehe16:03
lucasagomesso we could hide it based on policy?16:03
lucasagomesor even just hidding driver_info16:04
JayFdevananda: Do you want JoshNang and I to run a session on decommissioning?16:04
jrolldevananda: we're going to spam this project with so much code when k opens :P16:04
lucasagomessince the instance stuff was already separated from there16:04
devanandaactually we should use barbican for key/credential storage, eventually16:04
lucasagomesdevananda, +1 yeah that too16:04
JayFdevananda: We're presenting on what our vision for it is, in the main sessions (by "vision" I mean, what we're actually doing today, hehe)16:04
JayFdevananda: so I didn't know if you wanted a design session on the gdoc for it too16:04
JayFI actually think validation and decom are similar topics16:04
JayFand might be worthy of being combined16:04
devanandathough I spoke with barbican devs at the OSSG midcycle, and they were happy for us to wait another cycle or two before consuming it for an internal service like this16:05
lucasagomesthere's keystone v3 too16:05
JayFbecause you're saying 'validate this machine is ready for a tenant'... which could include cleaning up after the previous tenant16:05
devanandaJayF: I assume ya'll have a presentation in the main track, which everyone is going to go see16:05
lucasagomesbut it all should be plugabble16:05
JayFdevananda: yeah16:05
devanandaok -- so16:05
devanandawe'll definitely talk about these things :)16:05
devanandabut let's get back to hash rings and bugs and the release ;)16:05
lucasagomeshttps://docs.google.com/spreadsheets/d/1XBKdeDeGfaRYaThjIIoYRwe_zPensECnxsKUuqdoVmQ/edit?pli=1#gid=016:05
lucasagomes^ JayF  add it there16:06
lucasagomesplease16:06
JayFThe future is more exciting than fixing bugs now :P16:06
* NobodyCam guesses he'll have to ware his shades16:08
lucasagomesJayF, the session about wiping data from the previous tenant will be included in ur decom one no?16:08
lucasagomesso we may not need that other16:08
JayFI didn't even see that one16:09
JayFit's not even marked with a name though16:09
jrollagree16:10
*** yuanying has quit IRC16:10
* JayF is +1 to nuking the entry with no information that's useful16:10
* NobodyCam shared song stuch in his head: https://www.youtube.com/watch?v=8qrriKcwvlY16:12
*** marcoemorais has joined #openstack-ironic16:16
devanandajroll: https://bugs.launchpad.net/ironic/+bug/1300939 is assigned to you16:16
jrolldevananda: yeah16:16
jrolldevananda: I've poked at it; I have a plan; haven't written code yet16:17
devanandajroll: likelyhood of said non-code landing in RC1?16:18
jrolldevananda: heh, you tell me :P when are you cutting it?16:18
jrolldevananda: it's low on my priority list, I probably shouldn't have assigned that to myself16:20
devanandajroll: checking with ttx on such things to confirm my understandings16:20
jrolldevananda: but I could try to push it through if you think that's the right thing to do16:20
devanandajroll: I am sad when there are folks putting effort into translating strings for Ironic and a bug is preventing users from benefitting from that work16:21
*** ifarkas has quit IRC16:21
jrolldevananda: agree, however it's only broken when the request header is different than the server locale; as ironic is an internal API, I don't know how often that will happen16:22
jrolldevananda: but I can bump it up on my list16:22
openstackgerritA change was merged to openstack/ironic: Do not use the context parameter on refresh()  https://review.openstack.org/12244116:29
lucasagomes\o/16:30
*** rushiagr is now known as rushiagr_away16:31
devanandaanswer from ttx is that, if we know there are bugs we want to fix, we need to postpone RC116:31
devananda"believe it is the final RC"16:32
*** rushiagr_away is now known as rushiagr16:32
devanandawe can open a new RC if we find new bugs16:32
devanandajroll: ah - you have a good point there16:32
*** jrist has quit IRC16:35
*** rushiagr is now known as rushiagr_away16:36
devanandaryanpetrello: hi! regarding the wsme XML support issue (https://bugs.launchpad.net/ironic/+bug/1271317) -- mrda has been away, and this isn't release blocking16:39
devanandaryanpetrello: but I wanted to check on your comments on the patch. do you think there's a way to fix this purely in ironic at this point, or will it still need a change to wsme?16:39
ryanpetrellohrm16:40
ryanpetrellolet me take a look again16:40
*** ghe_ has quit IRC16:40
devanandaShrews: working on https://review.openstack.org/#/c/122802/ still?16:42
devanandaShrews: looks like the code is complete, and just docs remain. I'd like to close it soon :)16:42
*** jrist has joined #openstack-ironic16:42
devanandarloo: it looks like there are three patches open for https://bugs.launchpad.net/ironic/+bug/1267693 still, is that correct?16:44
rloodevananda: yes. I think two just need reviews. I was going to ask you about one of them.16:45
rloodevananda: https://review.openstack.org/#/c/122526/16:45
devanandarloo: actually i'm looking at https://review.openstack.org/#/c/122224/3/ironic/drivers/modules/agent.py now16:45
rloodevananda: ok, let me know when/if you get to 122526.16:46
*** pensu has joined #openstack-ironic16:48
devanandarloo: also, this is adding a new translated string16:49
rloodevananda: ugh.16:49
devananda*adding two16:49
rloodevananda: exception strings are considered user-visible?16:50
lucasagomesdevananda, why 1314732 should not be target to rc1?16:51
rloodevananda: what if we don't _() for now?16:51
devanandarloo: https://wiki.openstack.org/wiki/StringFreeze#What_changes_are_affected_by_the_string_freeze.3F16:51
rloodevananda: yeah, I'm reading that.16:52
*** todd_dsm has joined #openstack-ironic16:52
devanandalucasagomes: oh. I had an old copy of the page up that didn't show "fix committed" yet16:53
lucasagomesdevananda, oh... right, because the lsat patch merged min ago16:53
devanandalucasagomes: also, that's not a release blocking bug16:53
devanandalucasagomes: it's an internal refactoring, which is great, but doesn't represent a user-impacting problem16:53
lucasagomesyeah that's true16:54
devanandai'm untargeting anything which doesn't directly impact users of the released code16:54
devanandabasically, up till now, I used rc1 target as a "we'd like to fix this soon", and now it is, "we can't tag until these are closed"16:55
rloodevananda: for 122224, are you OK if I remove the _()? I'm also OK if these patches don't get into RC16:56
lucasagomesalright, no bothers :) just asked for clarification because I wanted it fixed by rc, so we can start K with a organized rpc objects in place to be able to at least try to oslofy it in K16:56
lucasagomesdevananda, +1, agreed yeah16:56
Shrewsdevananda: sorry, was at an appt. re: 122802, i was arguing that explaining the difference b/w "upgrade" and "create_schema" in the quickstart and install guide was not the correct place. these guides assume a fresh setup (no pre-exisitng db). i think a separate doc describing dbsync is the more appropriate place.16:58
devanandarloo: https://review.openstack.org/#/c/122526/ probably warrants a bit more thought, since this is a behavior change16:58
Shrewsdevananda: do you have thawts on that?16:58
devanandaShrews: quickstart should be exactly that. a doc on the db stuff should also be that.16:59
rloodevananda: yeah, that's what I wanted to discuss with you. Will punt that to K unless you want it addressed before that.16:59
lucasagomesalright folks, I will call it a day here17:00
Shrewsdevananda: exactly my point, too. i'd like to leave 122802 as-is and create a new doc describing the ironic-dbsync command17:00
lucasagomeshave a good night everyone!17:00
rloonight lucasagomes17:00
devanandalucasagomes: cheers, g'night!17:01
*** lucasagomes is now known as lucas-dinner17:01
lucas-dinnerShrews, yeah... I think u right, I will change my vote there17:02
Shrewslucas-dinner: thanks. and good night17:02
lucas-dinnerShrews, btw, the idea of having diff behaviors with dbsync... idk, I was thinking about it and it sounds confuse17:02
Shrewslucas-dinner++17:02
ryanpetrellodevananda: I'm working on a WSME change that should give you what you need17:03
ryanpetrellotesting now17:03
*** derekh has quit IRC17:03
devanandaryanpetrello: oh! neat. though I am wondering how we get that in if it would require a version bump17:03
devanandaryanpetrello: since we're past dep freeze17:03
ryanpetrelloyea :\17:03
ryanpetrellothe only other option outside of a dep freeze17:04
ryanpetrello*the dep freeze17:04
ryanpetrellowould be to yank out wsme's @wsmeexpose and have your own that does what my WSME patch does17:04
devanandarloo: the VendorPassthruException string in https://review.openstack.org/#/c/122224/3/ironic/drivers/modules/agent.py should really be in ironic/common/exception.py, no?17:04
ryanpetrelloprobably not a big deal since you're stuck on feature freeze anyways17:04
devanandarloo: that doesn't address the i18n question, though17:05
devanandaryanpetrello: we're still fixing bugs, and possibly have one that's large enough to push RC back several days17:05
*** yuanying has joined #openstack-ironic17:06
rloodevananda: Yes, it could be in common/exception.py. Not sure who added that in. But yeah, it still doesn't address the i18n question. I think if we don't translate the string it would be OK (if we want this fix in and don't want to bother the xlation folks with any exceptions.)17:06
devanandarloo: I'm thinking that the impact of not converting to an ironicException in vendor_passthru is minimal, since that will always be happening in an RPC thread17:07
*** rushiagr_away is now known as rushiagr17:08
rloodevananda: yes, that's true, but not consistent. The first question is do we want the change regardless of the string issue. And if so, do we want it now, or in K.17:09
devanandaright17:10
rlooI think the point was not to raise non-IronicExceptions from drivers.17:10
rlooor err for drivers to only raise IronicExceptions ;)17:10
*** christopheraedo has joined #openstack-ironic17:11
devanandarloo: indeed. the KeyError change is definitely good17:11
devanandarloo: I was just thinking through the larger issue of https://review.openstack.org/#/c/122526/ and that unhandled exceptions matter most if they hit the main conductor thread17:12
rloodevananda: fwiw, seamicro uses that VendorPassthruException, and heh, they just use message=<originalexception>.message17:12
rloodevananda: wrt the conductor. I agree with Lucas. Cuz I'm paranoid.17:13
devanandarloo: heh. well. repeating that here isn't terrible.17:13
devanandarloo: that cuts it down to 1 string change17:14
devanandarloo: fwiw, duplicating the  LOG.exception(_LE('vendor_passthru failed with method %s') line is not a string change -- transifex will see that it's a dup and reuse it17:15
*** yuanying has quit IRC17:15
*** loki184 has joined #openstack-ironic17:16
rloodevananda: right. it's the exception strings that are the issue. So the VendorPassthruException could just be (message=e) instead of the string that is there now17:17
rloodevananda: but the MissingParameterValue one. hmm.17:17
*** rwsu has quit IRC17:17
*** harlowja_away is now known as harlowja17:19
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add Commands section to documentation  https://review.openstack.org/12351817:20
Shrewsdevananda: lucas-dinner: I was thinking of something like this for our docs ^^^^17:20
*** Poornima has joined #openstack-ironic17:21
devanandarloo: yea, we either land that now and accept that it's during string freeze, or we leave the bug open, which i'm not thrilled about17:22
devanandarloo: but let's ask jroll if that's an important bug to fix in their driver now17:22
rloodevananda: wrt the agent, how about i take out the KeyError one for now. It'll be caught by vendor_passthru() above it anyway.17:23
devanandarloo: ++17:23
devanandarloo: so on https://review.openstack.org/#/c/122476/3/ironic/conductor/utils.py ...17:24
devanandarloo: my recollection of the issue here is that some drivers were "bleeding" underlying failures up to the conductor17:25
devanandarloo: so eg, if we know that calling 'ipmitool' might fail with some ProcessUtilsException, we should handle that in ipmitool driver17:25
rloodevananda: yeah, so I 'handled' as much as I could, all non-Ironic exceptions in the drivers.17:26
devanandarloo: but simply masking every exception coming from any power driver? right now, I'm not recaling why that's beneficial17:26
rloodevananda: but dtantsur|afk and lucas-dinner wanted me to still catch Exceptions here.17:26
*** Haomeng|2 has quit IRC17:26
devanandasure - cause we cant know that a driver will never in the future raise something else17:27
rloodevananda: so you think they shouldn't be masked by IronicExceptions?17:27
ryanpetrellodevananda: https://review.openstack.org/#/c/123520/17:27
devanandarloo: https://review.openstack.org/#/c/122476/3/ironic/conductor/utils.py is destroying information that would be valuable in debugging it17:28
*** Haomeng|2 has joined #openstack-ironic17:28
devanandarloo: it is hiding the actual exception,a nd all its context, and raising a new exception from this point in the code, while only preservign the message17:29
devanandawe've lost the traceback, the error code, the context ...17:29
devanandarloo: I'm not seeing the benefit. what am I missing?17:29
rloodevananda: hmm. we could LOG.exception here too.17:30
rloodevananda: so drivers call this. Something needs or not? to deal with these exceptions.17:30
devanandarloo: yup. but then we'll also LOG it when the new IronicException is caught at a higher layer17:30
rloodevananda: if they raise non-IronicExceptions, then the drivers will need to handle them, or if they don't handle them, then we're back to drivers raising non-IronicExceptions.17:31
devanandarloo: yup, and then we'll file a new bug against that driver :)17:31
rloodevananda: ok, so you're fine with the code as is then (w/o the patch).17:32
devanandarloo: if these are the last places where drivers are obviously raising non-ironic exceptions, yep17:33
rloook, I'll abandon it then.17:33
devanandaryanpetrello: nice! that's a really simple change :)17:34
ryanpetrello*should* give you all what you need17:34
ryanpetrellojust provide the argument wherever you like in Ironic17:34
ryanpetrelloor if you want to do it globally, just make your own decorator that wraps this and defaults17:34
devanandaryanpetrello: we are using  @wsme_pecan.wsexpose()17:35
ryanpetrelloright, so you should just be able to provide `content_types=['json']` with this patch17:35
devanandaawesome17:35
ryanpetrelloif WSME feature freeze is a problem, you *could* copy the wsmeexpose into Ironic17:35
ryanpetrellothat's kind of gross, though17:35
*** rwsu has joined #openstack-ironic17:36
openstackgerritAdam Gandelman proposed a change to openstack/ironic: Update devstack docs, require Ubuntu 14.04  https://review.openstack.org/12251917:36
devanandadhellmann: is it too late to get a new version of WSME out, and into ironic's requirements.txt? (I think so, but want to check)17:36
*** romcheg has quit IRC17:39
*** athomas has quit IRC17:41
*** loki184 has quit IRC17:43
*** Haomeng|2 has quit IRC17:44
*** athomas has joined #openstack-ironic17:45
*** Haomeng|2 has joined #openstack-ironic17:46
*** rushiagr is now known as rushiagr_away17:46
openstackgerritRuby Loo proposed a change to openstack/ironic: Don't reraise Exceptions from agent driver  https://review.openstack.org/12222417:49
*** dprince has quit IRC17:54
*** pcrews has joined #openstack-ironic17:54
*** Poornima has quit IRC17:55
*** jistr has quit IRC17:57
*** rushiagr_away is now known as rushiagr17:57
*** romcheg has joined #openstack-ironic18:02
*** ChuckC_ has quit IRC18:10
*** yuanying has joined #openstack-ironic18:11
devanandazigo: question on packaging of ironic's client lib18:19
devanandazigo: current global requirements states >=0.2.118:19
devanandazigo: are you packaging ==0.2.1, or latest tag as of RC ?18:20
*** yuanying has quit IRC18:20
*** dprince has joined #openstack-ironic18:21
*** linggao_ has joined #openstack-ironic18:25
*** linggao__ has joined #openstack-ironic18:28
*** linggao_ has quit IRC18:28
*** linggao has quit IRC18:29
*** rushiagr is now known as rushiagr_away18:30
*** linggao__ has quit IRC18:30
*** ghe_ has joined #openstack-ironic18:36
adam_gdevananda, in ubuntu land, i found that we'd usually ship the minimum until requirements changed and its constraints broke package build (specifying a min. version > whats available in the archive), or a newer client was uploaded for another reason (ie, another openstack project requires a newer version, security fix, etc)18:44
devanandaadam_g: hmm, well, I may have screwed up by not proposing a bump to pythonironicclient last week18:48
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add Commands section to documentation  https://review.openstack.org/12351818:50
NobodyCambrb quick walkies18:53
adam_gdevananda, requirements are frozen now, right?18:55
devanandayep18:55
* devananda moves to infra18:55
* adam_g wonders about an exception process there18:56
*** derekh has joined #openstack-ironic19:00
*** andreykurilin_ has joined #openstack-ironic19:00
*** jistr has joined #openstack-ironic19:06
*** tatyana has joined #openstack-ironic19:08
*** athomas has quit IRC19:09
ShrewsAnyone know what the 'stamp' command to ironic-dbsync does?19:10
* Shrews guesses "stamp all over my database until it is destroyed!"19:11
*** pensu has quit IRC19:12
*** ChuckC_ has joined #openstack-ironic19:12
adam_gShrews, i believe it sets the current migration version without running any migrations19:12
Shrewsnm. found it in the alembic docs: "‘stamp’ the revision table with the given revision; don’t run any migrations."19:12
devanandaadam_g: upon deeper digging, it looks like /nodes/detail support was in 0.2.1, but /chassis/CCC/nodes/detail and /nodes/NNN/ports/detail were added later19:13
devanandaadam_g: thankfully neither of those are needed by Nova19:13
devanandain a quick local test with the client == 0.2.1 Nova seems to work19:14
adam_gdevananda, cool.. are driver requirements just not managed by requirements?19:16
adam_g(in nova)19:16
*** yuanying has joined #openstack-ironic19:16
*** Damaris_Zulauf has joined #openstack-ironic19:17
devanandaadam_g: ironicclient is in test-reuirements in nova19:18
devanandanova core didn't want it requirements since it's not a requirement all the time19:18
adam_ghmm19:18
adam_gmakes sense. i guess vmware's suds requirement is grandfathered19:19
jrollrloo, devananda: is what an important bug to fix in the agent driver?19:19
jrollrloo, devananda: what specifically, I should say, I just see lots of exception-catching talk19:20
devanandajroll: agent driver raising KeyError instead of MissingParameter19:20
rloojroll: https://review.openstack.org/#/c/122224/19:20
jrollthis one? driver_info['agent_url'] = kwargs['agent_url']19:21
rloojroll: it is Ok. we punted it til Kilo. See the FIXME comment I added.19:21
rloojroll: yeah.19:21
jrollthe agent *always* sends an agent_url19:21
jrollthere has never been an agent (at least in the openstack/ repo) that has not done that19:21
jrollif the agent gets that far, it has the url and will send it19:21
rloojroll: it should but really, can we trust all agents out there? :-)19:21
jrollif it can't decide what its url is, it will bail19:21
jrollyou can trust all ironic-python-agents out there19:22
jrollwhich is what we certify that this driver works with19:22
jrollit's documented as a parameter to heartbeat (at least with a docstring)19:22
jrollso if someone writes an agent that doesn't do this, I'd say they are doing it wrong19:22
jroll(tm)19:22
rloojroll: it isn't a big deal since the exception (if any) will be caught.19:22
rloojroll: so you're saying do NOT try to catch any KeyError there?19:23
jrollI'm saying I don't think it's valuable to catch that; however it can't hurt to raise a MissingParameterValue instead19:23
rloojroll: ok, then we'll do that in Kilo (raise MissingPV)19:24
jrollrloo: more clearly, I would +2 that change but wouldn't file a bug for it :)19:24
jrollsounds good :)19:24
rloothx jroll. (couldn't do the change now cuz of string freeze)19:24
jrollno problem, thank you!19:25
rloojroll: feel free to review that patch too ;)19:25
*** yuanying has quit IRC19:25
*** Damaris_Zulauf has quit IRC19:25
jrollheh, will do19:25
*** jrist has quit IRC19:26
*** HenryG is now known as HenryG_afk19:27
jrolldevananda: do we have a bug open for how slow power state sync periodic taks is?19:33
devanandajroll: nova's or ironic's?19:34
JayFIronic's, in this case19:35
jrollironic, yeah19:35
jrolldevananda: the "check power state" loop where it pokes ipmi power status19:35
devanandahm, maybe? nothing on my radar at the moment19:36
devanandait's configurable, right?19:36
JayFIt's all serial19:36
JayFand can easily get clogged up if you have BMCs that are broken / slow to respond19:36
devanandaah, right19:36
devanandaneeds to spawn some parallel threads n stuff19:36
devanandaI'd like that to be done as part of a larger thinking about how we handle async periodic tasks, which we may get more of19:37
devanandabut yes, I would like that to be parallelized19:37
jrollmmmkay19:37
jrollwe were thinking the same; haven't done it downstream but that might happen soon19:38
*** penick has quit IRC19:49
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add documentation for ironic-dbsync command  https://review.openstack.org/12351819:52
openstackgerritJarrod Johnson proposed a change to stackforge/pyghmi: Remove overly aggressive packet processing  https://review.openstack.org/12356519:53
openstackgerritJarrod Johnson proposed a change to stackforge/pyghmi: Remove overly aggressive packet processing  https://review.openstack.org/12356519:54
*** jjohnson2 has joined #openstack-ironic19:55
NobodyCamShrews: I am assuming that you forsee other commands being added to the cmds/ folder RE: 12351819:57
ShrewsNobodyCam: yeah. ironic-api and ironic-conductor, at least19:57
*** foexle has quit IRC19:57
jjohnson2Well, I have good feedback now (after a few bugs) about pyghmi's behavior against a couple of ~500 node clusters19:58
jjohnson2(in conjunction with confluent for using the SOL facility)19:58
NobodyCam:)19:58
jjohnson2And I am doning my 'looks like my lenovo name' already...19:58
NobodyCamjjohnson2: ???? 'looks like my lenovo name'19:59
jjohnson2NobodyCam, jbjohnso is my ibm self19:59
jjohnson2jbjohnso at us.ibm.com19:59
NobodyCamahhh19:59
jjohnson2jjohnson2 is my at lenovo.com self19:59
NobodyCamahhh ... :)20:00
jjohnson2jbjohnso shall perish september 30th, long live jjohnson2 as of october 1st20:00
* NobodyCam was about to get nobodycam@hp.com :-p20:00
NobodyCams/about/able/20:00
ShrewsNobodyCam: and possibly ironic-nova-bm-migrate20:00
jjohnson2my last change to pyghmi was 500 nodes rebooting with lots of SOL output... managed to hit the default max recursion limit20:01
NobodyCamShrews: ++ though that was *started else where: (kinda) https://wiki.openstack.org/wiki/Ironic/NovaBaremetalIronicMigration20:02
ShrewsNobodyCam: it would be nice to have all of our docs in one place, rather than scattered about... eventually20:06
NobodyCam+++++20:06
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add documentation for ironic-dbsync command  https://review.openstack.org/12351820:06
*** derekh has quit IRC20:09
*** dprince has quit IRC20:11
*** marcoemorais has quit IRC20:11
*** andreykurilin_ has quit IRC20:12
*** andreykurilin_ has joined #openstack-ironic20:12
*** jjohnson2 has quit IRC20:12
*** jistr has quit IRC20:13
*** jjohnson2 has joined #openstack-ironic20:13
rlooShrews: qq about 122802: https://review.openstack.org/#/c/122802/1/doc/source/dev/dev-quickstart.rst20:14
rlooShrews: the comment about 'If using MySQL, you need to...20:14
rlooShrews: is that still needed? or does the updated ironic-dbsync cmd work for MySQL too?20:14
Shrewsrloo: yeah, you need to first create the schema in mysql before doing anything20:15
Shrewsotherwise your connection string url is invalid20:15
rlooShrews: isn't that confusing then, that you have to create it, then do ironic-dbsync ... create_schema?20:16
Shrewsrloo: yes, but i didn't name the 'create_schema' option.  :)20:16
rlooyeah, I know, you're just updating the docn Shrews. Just wondered...20:17
Shrewsit really creates tables, in mysql. not sure schema matters with sqlite20:17
rlooShrews: so it should have been called 'create_tables' instead of 'create_schemas'?20:17
Shrewsrloo: that would at least be more accurate20:17
rlooShrews: ok, I'm sure your detailed docn on that will describe it ;) I wonder if we should alias 'create_schema' to 'create_tables'. I suppose it is too late to change it?20:18
*** yuanying has joined #openstack-ironic20:21
*** tatyana has quit IRC20:22
Shrewsprobably not20:23
*** penick has joined #openstack-ironic20:25
jrollit would save us a whole lot of work if we just told people not to use sqlite :P20:27
*** yuanying has quit IRC20:30
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add documentation for ironic-dbsync command  https://review.openstack.org/12351820:33
* Shrews finally happy with 12351820:34
Shrews... for now ...20:34
rlooShrews, I -1'd 122802. Let me know what you think.20:34
*** todd_dsm has quit IRC20:34
Shrewsrloo: i considered totally removing that comment, tbh. i'll remove it20:35
rlooShrews: thx20:36
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Update docs with new dbsync command  https://review.openstack.org/12280220:38
Shrewsrloo: better?  ^^^20:38
rlooShrews: +2 ;)20:39
*** todd_dsm has joined #openstack-ironic20:45
*** marcoemorais has joined #openstack-ironic20:46
*** jrist has joined #openstack-ironic20:52
*** andreykurilin_ has quit IRC20:54
openstackgerritA change was merged to openstack/ironic: Revert "Revert "Search line with awk itself and avoid grep""  https://review.openstack.org/12236020:55
JayFjroll: ^ hah20:56
jroll:P20:57
NobodyCam:-p20:57
JayFWe were just talking about should we use awk for something20:57
JayFand then, awk20:57
openstackgerritA change was merged to openstack/ironic: Pass ipa-driver-name to agent ramdisk  https://review.openstack.org/12304720:58
Shrewsthe high temp today was 65. that's crazy for NC at this time of year21:00
* Shrews puts on the flannel21:00
NobodyCamlol21:01
NobodyCamraining here now :(21:01
*** penick has quit IRC21:03
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Adding support for decommissioning  https://review.openstack.org/10437921:04
*** marcoemorais has quit IRC21:06
*** marcoemorais1 has joined #openstack-ironic21:06
*** ghe_ has quit IRC21:09
*** penick has joined #openstack-ironic21:12
openstackgerritRuby Loo proposed a change to openstack/ironic: Don't reraise Exceptions from agent driver  https://review.openstack.org/12222421:17
*** jjohnson2_ has joined #openstack-ironic21:19
*** jjohnson2 has quit IRC21:23
*** yuanying has joined #openstack-ironic21:26
*** jjohnson2_ has quit IRC21:28
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Add versioning to Agent decommission  https://review.openstack.org/10685921:32
*** andreykurilin_ has joined #openstack-ironic21:33
*** yuanying has quit IRC21:35
*** todd_dsm has quit IRC21:50
*** todd_dsm has joined #openstack-ironic21:52
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add documentation for ironic-dbsync command  https://review.openstack.org/12351821:54
*** todd_dsm has quit IRC21:56
*** jasondotstar has quit IRC21:57
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add documentation for ironic-dbsync command  https://review.openstack.org/12351821:58
Shrewsalright. goodnight all21:59
rloobye Shrews21:59
NobodyCamahh I was just going to ask why all the commands dont have an example22:01
NobodyCambut I'll ask tomorrow.. havew a good night Shrews22:01
*** harlowja is now known as harlowja_away22:01
rloowell, if Shrews is gone, I'm going too. ciao :)22:02
NobodyCamokay night rloo :)22:02
*** rloo has quit IRC22:02
*** harlowja_away is now known as harlowja22:03
NobodyCamso I too will step away (for a couple of minutes) ... LOL .... brb22:04
adam_ghmm22:07
adam_gcheck-tempest-dsvm-ironic-parallel-nv has been running since yesterday, but failing a percentage of the time22:07
*** romcheg has quit IRC22:08
*** andreykurilin_ has quit IRC22:14
*** killer_prince has joined #openstack-ironic22:15
*** killer_prince is now known as lazy_prince22:15
NobodyCamadam_g: failing with same error?22:16
adam_gNobodyCam, ive found one common thread22:16
* jroll guesses apt errors :P22:16
adam_gwhich looks eerily similar to a big gate bug https://bugs.launchpad.net/nova/+bug/132365822:16
adam_gjroll, also, added check-tempest-dsvm-ironic-parallel-nv to http://no-carrier.net/~adam/openstack/ironic_gate_status.html22:18
adam_gneed to find a way to make that look less crowded. no design skillz here22:18
jrolladam_g: thanks :D22:18
jrollyeah, and I need to make that job work :/22:19
jrollI think it's something environmental but not 100% sure22:19
adam_gjroll, its getting thru devstack just failing the baremetal scenario?22:20
jrolladam_g: yeah, our ramdisk is panicking on boot22:20
jrollit looks like a coreos bug, but idk22:21
JayFjroll: I thought we ruled that out because it didn't repro locally?22:21
adam_gah22:21
jrollJayF: right, it *looks* like a coreos bug22:21
jrollbut is likely not22:21
jrollI wonder if we just have a bad upload somehow22:21
adam_ghave you tried booting the same ramdisk locally with the same libvirt domain specs?22:21
JayFHmm. I wonder if there's a race22:21
jrolladam_g: it appears to work for me when building ramdisk from source22:21
JayFi.e. if two IPA changes merge one right after the other22:21
JayFif one starts uploading over the top of another one22:22
jrolladam_g: idk anything about libvirt domain specs :(22:22
adam_gcheck-tempest-dsvm-ironic-agent_ssh-nv just grabs the tarball'd agent ramdisk from tarballs.o.o right?22:22
jrolladam_g: but all the dev for this, I've done with devstack on rackspace cloud servers22:22
jrollyes22:22
jrollwell, it's not tarballed any more but yes22:22
adam_gjroll, i just meant the same memory/cpu/etc.. the xml that devstack creates22:22
jrollyeah, I've done this all in devstack22:23
jrolloh fffffffff22:24
JayFUh-oh22:24
JayFwhat?22:24
jrollI think I've been using 1024mb ram22:24
jrolldefault is 51222:24
JayFThat would probably cause a fail22:24
jrollyes22:24
adam_gjroll, yeah, thats what i was getting at. we were hitting issues with vm node memory  at one point with the pxe ramdisk22:24
JayFand a failure to find an init file would be a sensible place to see it22:24
jrolladam_g: yep, I overlooked that22:25
JayFthis is why we said we wanted to have a DIB-built IPA image for the gate22:25
JayFbut nobody ever finished it22:25
jrollyeah22:25
NobodyCamoh cr*p22:25
JayFIs there a way to raise the allowable ram in the check/gate pipelines?22:25
jrollsomeone in infra is going to kill me for having yet another dsg change22:26
NobodyCamI think that was me22:26
jrollyes22:26
adam_gfor the pxe ramdisk22:26
adam_gto get it to boot /w 51222:26
adam_g'rootfstype=ramfs' was added to the kernel parameters in pxe_config.template22:26
adam_gi dont know much about coreos or its kernel to know if thats honored22:26
JayFI mean, that doesn't really apply so much with CoreOS?22:26
JayFjroll: we could possibly get it to boot with a larger root fs and less ram22:27
adam_glike i said, i dunno enough about coreos22:27
jrollJayF: if you want to give that a shot, gopher it22:27
jrollin the meantime, I'm putting up a dsg change22:27
JayFjroll: rootflags=size=300M or something similar22:27
adam_gjroll, you may not want to do that22:27
JayFjroll: we run with rootflags=size=20G in prod fwiw22:27
JayFlol22:27
jrollwhat other fileststems are there?22:27
JayFyou cannot use ramfs as backing for coreos at all22:28
JayFnor would it be helpful to do so in this case22:28
jrolladam_g: because y'all ran into memory issues with it?22:28
adam_gjroll, atm our jobs aren't running in parallel and only have 1 vm node, but the parallel job requires 3 nodes.22:28
jrolladam_g: right... :|22:29
jrolladam_g: how far out do you see that happening?22:29
jrolladam_g: and are we going to make the agent job do that as well?22:29
adam_gi dont know how well the slave nodes will handle 3x1024G22:29
adam_gjroll, https://review.openstack.org/#/c/123253/is a WIP that just makes the global ironic macro do everything in parallel22:30
* jroll tris a thing22:31
adam_gjroll, the goal is to get rid of the regex and just run the entire tempest suite, letting it skip things as necessary. we could make TEMPEST_CONCURRENCY=1 for you, but that would increase test time significantly. the new parallel job takes ~60M with TEMPEST_CONURRENCY=2 as opposed to 3022:31
jrolltries, even22:31
jrolladam_g: right, going to try this first22:31
*** openstackgerrit has quit IRC22:31
*** yuanying has joined #openstack-ironic22:31
adam_gis running coreos /w <1024 memory a known thing ?22:32
jroll(the rootflags= thing)22:32
jrollthis ramdisk, specifically, has given us issues at 512mb22:32
adam_gjroll, IIRC the chatter at the sprint, it sounded like the same thing we hit22:32
JayFadam_g: basically, the CoreOS PXE image will split your ram in half by default: half for ram, half for ramdisk22:32
adam_gramfs vs initramfs22:33
jrolladam_g: indeed22:33
*** openstackgerrit has joined #openstack-ironic22:33
JayFadam_g: we need enough room in the ramdisk for the entire image + a cirros image22:33
adam_goh22:33
adam_gwhy cirros?22:33
JayFadam_g: so we can do something like rootflags=size=384M and give the root fs 384M and only have 128M of ram22:33
JayFadam_g: because we write the image to the local FS in the agent before writing it to disk22:33
jrollcirros is what devstack tests with22:33
JayFadam_g: no streaming support yet22:33
jrolloh yeah, that22:34
adam_goh22:34
JayFwhich is nbd in our prod environment where some of our machines have half a terabyte of ram22:34
JayFnot so minor when you're trying to run it tons of times on every commit22:34
adam_gah, gotcha22:34
JayFjust low-ram is never a case we optimized the agent for22:34
NobodyCamoh wow: https://review.openstack.org/#/c/10310522:35
JayFeven now doing that optimization seems fake because I can't imagine anyone ever using ironic to deploy a box with less than a gb of ram22:35
JayFhell, I can't even remember the last time I used a computer of any type with that little ram22:35
adam_gIIRC initramfs will split your ram like that, but ramfs will just happily let you eat up everything22:35
JayFmy cell phone has more than 1GB of ram22:35
JayFadam_g: it gets... weird because of the way coreos does their rootfs... I think they actually are using btrfs not tmpfs at all22:36
JayFadam_g: I don't fully understand it but it was required in order to get docker to run in ram22:36
adam_goh, of course the are! :)22:36
* JayF trying to replicate it so he can run docker in ram on onmetal-memory1 nodes22:36
jrollyuriyz: I like your agent DIB thing22:38
jroll:)22:38
*** yuanying has quit IRC22:40
NobodyCam^{j|J}* : have you seen these https://review.openstack.org/#/c/110487 & https://review.openstack.org/#/c/103105 and do you still think the IPA repo is the bast place for the element?22:41
jrollgood highlight22:41
JayFI don't highlight on regexp, jfyi22:41
NobodyCamlol22:41
JayFNobodyCam: I think IPA imagebuild/ is the best place for them *now*22:42
NobodyCamJayF: jroll JoshNang ^^^^^^^^^^^22:42
jrolllol22:42
jrollyou forgot jrist :)22:42
JayFNobodyCam: I suspect for K we'll move all those builders (including DIB element for pxe ramdisk) into their own repo22:42
NobodyCamd'on22:42
NobodyCamd'oh22:42
* JayF remembers to note this on the googledoc22:42
jrollNobodyCam: those do the same thing, right?22:42
NobodyCamI think so, but would luv it a IPA person could lok them over and see if one is closer to done then the oter22:43
JayFjroll: I think the answer is "Yes, nothing"22:43
NobodyCam*other22:43
JayF:P22:43
jrollNobodyCam: looking at that code tells me nothing, I don't know DIB well enough to tell you if it's good22:43
jrollNobodyCam: but happy to try to run one :)22:44
JayFNobodyCam: I'm not 100% sure, but I think Yuriy's is if I had to guess22:44
JayFNobodyCam: but I know jack and squat about dib22:44
NobodyCamyea22:44
NobodyCamI think so too22:44
NobodyCamgoing to pop out for a somke then dig in to both22:44
jrollyuriy says that his pxe boots22:44
NobodyCamoh then I'm sure his is closer22:50
NobodyCamoh wow: https://review.openstack.org/#/c/10527522:52
* NobodyCam need to keep a closer eye on the dib repo22:52
jrolllol22:53
JayFdracut is cool :)22:53
*** todd_dsm has joined #openstack-ironic22:53
jrist:)22:55
NobodyCamjroll: checking are you in fact testing yuriyz' dib element?22:56
jrollNobodyCam: not at the moment22:59
NobodyCamack22:59
*** penick has quit IRC23:03
*** marcoemorais1 has quit IRC23:07
*** marcoemorais has joined #openstack-ironic23:08
* JayF added quite a bit to https://docs.google.com/a/oldos.org/spreadsheets/d/1XBKdeDeGfaRYaThjIIoYRwe_zPensECnxsKUuqdoVmQ/edit#gid=0 (Kilo summit planning) if anyone wants to look over it23:11
jrollJayF: does this look wrong at all? rootflags=size=384M23:13
jrollJayF: it's not even trying to boot :(23:13
NobodyCamlooks tiny23:13
jrollNobodyCam: out of 512?23:14
NobodyCamoh thats also tiny23:14
JayFI mean... I'm not surprised that not-booting would be the failure case23:14
jrollsuch is life in the gate23:14
jrollJayF: yeah :/23:14
JayFIt may be that half the available ram is an upper bound of some kind23:14
jrollJayF: but like... no console output23:14
jrollmm23:14
*** yuanying has joined #openstack-ironic23:18
*** dhellmann has quit IRC23:21
*** dhellmann has joined #openstack-ironic23:21
openstackgerritA change was merged to stackforge/pyghmi: Remove overly aggressive packet processing  https://review.openstack.org/12356523:38
*** ChuckC_ has quit IRC23:49
*** ChuckC_ has joined #openstack-ironic23:49
*** ryanpetrello has quit IRC23:52
*** ChuckC_ has quit IRC23:54
*** Nicklaus_Zulauf3 has joined #openstack-ironic23:55
*** ryanpetrello has joined #openstack-ironic23:57
*** todd_dsm has quit IRC23:57
*** marcoemorais has quit IRC23:58
* NobodyCam wounder afk looking for food stuffs23:58
NobodyCamwanders afk even23:59
NobodyCamlol23:59
*** marcoemorais has joined #openstack-ironic23:59
« Monday, 2014-09-22 Index Wednesday, 2014-09-24 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!