Friday, 2016-04-15

*** ayoung has joined #openstack-ironic00:03
*** piet has quit IRC00:12
openstackgerritStephanie Miller proposed openstack/bifrost: Unify testing scripts  https://review.openstack.org/29742400:12
*** Sukhdev has joined #openstack-ironic00:15
*** sambetts|afk has quit IRC00:22
*** rloo has quit IRC00:23
*** alex_xu has quit IRC00:26
*** hoangcx has joined #openstack-ironic00:26
*** alex_xu has joined #openstack-ironic00:27
*** yarkot1 has joined #openstack-ironic00:29
*** yarkot1 has quit IRC00:30
*** sambetts has joined #openstack-ironic00:31
*** akrish9 has joined #openstack-ironic00:31
*** yarkot1 has joined #openstack-ironic00:34
*** akrish9 has quit IRC00:44
*** yarkot1 has quit IRC00:44
*** akrish9 has joined #openstack-ironic00:45
openstackgerritGonĂ©ri Le Bouder proposed openstack/ironic: ipxe: retry on failure  https://review.openstack.org/30619600:59
*** ptx0 has joined #openstack-ironic01:00
*** ptx0 has joined #openstack-ironic01:00
*** akrish9 has quit IRC01:02
*** akrish9 has joined #openstack-ironic01:02
*** akrish9 has quit IRC01:04
*** akrish9 has joined #openstack-ironic01:06
*** hoangcx_ has joined #openstack-ironic01:08
*** baoli has joined #openstack-ironic01:15
*** amotoki_ has quit IRC01:16
*** Guest17323 has quit IRC01:20
*** Sukhdev has quit IRC01:25
*** yarkot1 has joined #openstack-ironic01:33
*** baoli has quit IRC01:35
*** piet has joined #openstack-ironic01:35
*** baoli has joined #openstack-ironic01:35
*** mtanino has quit IRC01:36
*** suro-patz has joined #openstack-ironic01:47
*** Marga__ has joined #openstack-ironic01:48
*** baoli has quit IRC01:48
*** baoli has joined #openstack-ironic01:48
*** suro-patz1 has joined #openstack-ironic01:50
*** Marga_ has quit IRC01:51
*** suro-patz has quit IRC01:51
openstackgerritOpenStack Proposal Bot proposed openstack/bifrost: Updated from global requirements  https://review.openstack.org/30516701:52
*** Marga__ has quit IRC01:52
openstackgerritTan Lin proposed openstack/ironic: Typo  https://review.openstack.org/30620901:54
*** Haomeng has joined #openstack-ironic02:00
*** cameron_C has joined #openstack-ironic02:01
*** alex_xu has quit IRC02:02
*** baoli has quit IRC02:07
*** suro-patz1 has quit IRC02:08
*** alex_xu has joined #openstack-ironic02:09
*** suro-patz has joined #openstack-ironic02:09
*** chlong has joined #openstack-ironic02:20
*** amotoki has joined #openstack-ironic02:25
*** kwazar has quit IRC02:26
*** thrash|g0ne has quit IRC02:26
*** kwazar has joined #openstack-ironic02:26
*** Haomeng|2 has joined #openstack-ironic02:26
*** Haomeng has quit IRC02:26
*** suro-patz has quit IRC02:31
*** hoangcx has quit IRC02:42
*** yuanying has quit IRC02:48
*** e0ne has joined #openstack-ironic02:51
*** Haomeng|2 has quit IRC02:52
*** Haomeng|2 has joined #openstack-ironic02:53
*** amotoki has quit IRC02:55
*** e0ne has quit IRC02:56
*** amotoki has joined #openstack-ironic02:59
*** e0ne has joined #openstack-ironic03:00
*** hoangcx_ has quit IRC03:04
*** piet has quit IRC03:07
*** e0ne has quit IRC03:08
*** baoli has joined #openstack-ironic03:10
*** e0ne has joined #openstack-ironic03:11
*** e0ne has quit IRC03:13
*** e0ne has joined #openstack-ironic03:17
*** harlowja_at_home has joined #openstack-ironic03:18
*** thrash has joined #openstack-ironic03:22
*** thrash has quit IRC03:22
*** thrash has joined #openstack-ironic03:22
*** Haomeng|2 has quit IRC03:22
*** ijw_ has joined #openstack-ironic03:23
*** pleia2_ has joined #openstack-ironic03:27
*** izaakk_ has joined #openstack-ironic03:27
*** Mr_T has quit IRC03:28
*** akrish9 has quit IRC03:28
*** piet has joined #openstack-ironic03:29
*** akrish9 has joined #openstack-ironic03:29
*** harlowja_at_home has quit IRC03:30
*** sambetts_ has joined #openstack-ironic03:31
*** baoli has quit IRC03:32
*** sambetts has quit IRC03:32
*** ijw has quit IRC03:32
*** xavierr has quit IRC03:32
*** dhellmann has quit IRC03:32
*** izaakk has quit IRC03:32
*** pleia2 has quit IRC03:32
*** izaakk_ is now known as izaakk03:32
*** Sukhdev has joined #openstack-ironic03:33
*** xavierr has joined #openstack-ironic03:38
*** Mr_T has joined #openstack-ironic03:45
*** pleia2_ is now known as pleia203:46
*** Haomeng has joined #openstack-ironic03:46
*** yuanying has joined #openstack-ironic03:47
*** e0ne has quit IRC04:00
*** vmud213 has joined #openstack-ironic04:01
*** e0ne has joined #openstack-ironic04:04
*** dhellmann has joined #openstack-ironic04:05
openstackgerritDebayan Ray proposed openstack/proliantutils: Add proper error message for extraction failure  https://review.openstack.org/30533104:13
*** vishwanathj_away has quit IRC04:15
*** Marga_ has joined #openstack-ironic04:16
*** Marga_ has quit IRC04:17
*** Marga_ has joined #openstack-ironic04:18
*** e0ne has quit IRC04:22
*** e0ne has joined #openstack-ironic04:26
*** ijw_ has quit IRC04:29
*** e0ne has quit IRC04:30
*** piet has quit IRC04:38
*** piet has joined #openstack-ironic04:39
*** chlong has quit IRC04:48
*** harlowja_at_home has joined #openstack-ironic04:50
*** vinaym213 has joined #openstack-ironic04:53
*** vishwanathj has joined #openstack-ironic04:57
*** piet has quit IRC04:57
*** vmud213 has quit IRC04:57
*** vishwanathj is now known as vishwanathj_zzz04:57
*** harlowja_at_home has quit IRC05:00
*** chlong has joined #openstack-ironic05:01
*** alex_xu_ has joined #openstack-ironic05:10
*** rcernin has joined #openstack-ironic05:11
*** alex_xu has quit IRC05:12
*** piet has joined #openstack-ironic05:14
*** causten_ has joined #openstack-ironic05:24
*** smoriya has quit IRC05:26
*** amotoki_ has joined #openstack-ironic05:26
*** smoriya_afk has joined #openstack-ironic05:26
*** mat128 has quit IRC05:26
*** mkoderer__ has quit IRC05:26
*** smoriya_afk is now known as smoriya05:26
*** amotoki has quit IRC05:27
*** xavierr has quit IRC05:28
*** ChrisAusten has quit IRC05:28
*** jlvillal has quit IRC05:28
*** dkehn has quit IRC05:28
*** bcornec has quit IRC05:28
*** bcornec has joined #openstack-ironic05:29
*** xavierr has joined #openstack-ironic05:30
*** Marga_ has quit IRC05:33
*** daemontool has joined #openstack-ironic05:33
*** jlvillal has joined #openstack-ironic05:34
*** mkoderer__ has joined #openstack-ironic05:35
*** piet has quit IRC05:35
*** mat128 has joined #openstack-ironic05:36
*** Marga_ has joined #openstack-ironic05:38
*** davidlenwell has quit IRC05:44
*** davidlenwell has joined #openstack-ironic05:46
*** dkehn has joined #openstack-ironic05:53
openstackgerritDebayan Ray proposed openstack/ironic: Add note on prerequisite of 'rpm' file extraction  https://review.openstack.org/30624705:53
*** hoangcx has joined #openstack-ironic05:58
*** Sukhdev has quit IRC05:59
*** chlong has quit IRC06:02
*** amotoki_ has quit IRC06:07
*** Fdaisuke has quit IRC06:19
*** zhenguo_ has quit IRC06:20
*** tesseract has joined #openstack-ironic06:29
*** tesseract is now known as Guest6322906:29
*** divya has joined #openstack-ironic06:43
*** Marga__ has joined #openstack-ironic06:55
*** irf has joined #openstack-ironic06:56
irfHello folks .. i am back06:56
irfi have verified my hardware and it has Intel core I5 CPU06:57
irfIn the boot order , i have UEFI option available06:57
*** jlvillal has quit IRC06:58
irfmotherboard does not have IPMI controller , i have confirmed that so cant use IPMI driver06:58
irfcan someone suggest , which driver i can use ?06:59
irfi am reffering to this doc : http://docs.openstack.org/developer/ironic/deploy/drivers.html06:59
*** davidlenwell_ has joined #openstack-ironic07:01
*** mkoderer__ has quit IRC07:03
irfplz suggest07:03
*** ohamada has joined #openstack-ironic07:03
*** davidlenwell has quit IRC07:04
*** Marga_ has quit IRC07:04
*** mat128 has quit IRC07:05
*** mkoderer__ has joined #openstack-ironic07:09
*** jlvillal has joined #openstack-ironic07:11
*** mat128 has joined #openstack-ironic07:12
Haomengirf: depends on how can we control your server poweron/off, is there api interface to be called, or are you use PDU to control your server's power?07:13
Haomengirf: is your machine pc or server?07:14
Haomengirf: almost x86 server has ipmi card07:14
*** mkovacik has joined #openstack-ironic07:15
irfi have checked that ..07:15
irfon my motherboard, i dont have IPMI controller07:15
irfmine is just desktop PC07:16
Haomengirf: if it is pc, you can check if it supports AMT07:16
irfok07:17
*** ifarkas has joined #openstack-ironic07:17
irflet me read this : http://docs.openstack.org/developer/ironic/drivers/amt.html07:17
Haomengirf: and PC supports wol - http://docs.openstack.org/developer/ironic/drivers/wol.html07:19
*** achanda has joined #openstack-ironic07:20
*** athomas has joined #openstack-ironic07:22
*** divya has quit IRC07:22
*** amotoki has joined #openstack-ironic07:25
*** mgould has joined #openstack-ironic07:25
*** athomas has quit IRC07:27
mgouldmorning Ironic!07:28
*** pcaruana has joined #openstack-ironic07:31
*** athomas has joined #openstack-ironic07:31
*** karimb has joined #openstack-ironic07:47
*** e0ne has joined #openstack-ironic07:58
pas-hamorning mgould and all07:58
*** jtomasek_ has joined #openstack-ironic08:00
*** jcoufal has joined #openstack-ironic08:00
*** achanda has quit IRC08:06
*** yuriyz has quit IRC08:06
*** openstackgerrit has quit IRC08:08
*** yuikotakada has joined #openstack-ironic08:08
*** yuriyz has joined #openstack-ironic08:10
*** stendulker has joined #openstack-ironic08:10
*** mbound has joined #openstack-ironic08:11
*** b_bezak has joined #openstack-ironic08:15
*** alex_xu_ has quit IRC08:16
*** kromanenko has joined #openstack-ironic08:18
*** b_bezak has left #openstack-ironic08:18
*** openstackgerrit has joined #openstack-ironic08:18
*** alex_xu has joined #openstack-ironic08:23
openstackgerritSofiia Andriichenko proposed openstack/python-ironicclient: Add test for 'port-show --field' command  https://review.openstack.org/30400808:28
*** jistr has joined #openstack-ironic08:30
*** e0ne has quit IRC08:43
*** ppiela has quit IRC08:43
*** ppiela has joined #openstack-ironic08:44
*** dims_ has quit IRC08:46
*** dims has joined #openstack-ironic08:49
*** e0ne has joined #openstack-ironic08:53
*** dtantsur|afk is now known as dtantsur08:58
dtantsurMorning Ironic08:58
-openstackstatus- NOTICE: Gerrit is going to be restarted because is not processing new changes09:05
dtantsursambetts_, ifarkas, aarefiev, hi folks, please review https://review.openstack.org/305960 as soon as gerrit is back up09:06
dtantsurit's a pretty bad bug, I want to have it fixed as soon as possible09:06
*** alex_xu has quit IRC09:07
*** alex_xu has joined #openstack-ironic09:09
sambetts_Morning all09:10
*** sambetts_ is now known as sambetts09:11
vdrokmorning ironic, Haomeng dtantsur and sambetts !09:17
dtantsuro/09:17
*** amotoki has quit IRC09:18
*** yuikotakada has quit IRC09:19
-openstackstatus- NOTICE: No jobs are being processed by gerrit and zuul . We are working to solve the problem, please be aware that no changes have been sent to the queue in the last hour, so you will need to recheck jobs for that period.09:20
*** ChanServ changes topic to "No jobs are being processed by gerrit and zuul . We are working to solve the problem, please be aware that no changes have been sent to the queue in the last hour, so you will need to recheck jobs for that period."09:20
sambettsHi dtantsur, vdrok, nice to see Gerrit and Zuul continuing with their Friday tradition09:22
dtantsurwonderful.....09:23
*** alex_xu_ has joined #openstack-ironic09:25
*** alex_xu has quit IRC09:26
*** divya has joined #openstack-ironic09:29
divyaironic node-set-provision-state --config-drive /tmp/$NAME $NAME active09:30
divyacommand to deploy image using ipmi agent?09:31
*** manfredp has joined #openstack-ironic09:32
*** manfredp_ has joined #openstack-ironic09:33
*** manfredp has quit IRC09:36
*** ndipanov has joined #openstack-ironic09:40
openstackgerritDmitry Tantsur proposed openstack/ironic-specs: [WIP] Driver composition reform spec  https://review.openstack.org/18837009:42
irfok thanks09:47
*** alex_xu_ has quit IRC09:47
*** ChanServ changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/developer/ironic/ | Bugs: https://bugs.launchpad.net/ironic"09:48
-openstackstatus- NOTICE: Zuul and gerrit are working normally now. Please recheck any jobs that may have been affected by this failure.09:48
*** alex_xu has joined #openstack-ironic09:49
stendulkerdtantsur: Hi, have a minute... wanted to discuss your comment for spec https://review.openstack.org/#/c/230274/7/specs/approved/secure-baremetal-deploy-in-uefi.rst09:50
*** amotoki has joined #openstack-ironic09:55
openstackgerritvinay kumar muddu proposed openstack/ironic-python-agent: Wait for all interfaces to be up before node lookup  https://review.openstack.org/30350309:56
dtantsurstendulker, yep. even though it's better to leave a reason on the spec itself09:58
stendulkerdtantsur: Basically I did not understand your comment :(09:58
dtantsurstendulker, my comment is "why"09:59
dtantsuryou introduce an option, which does not seem required to me. you don't explain why you do it.09:59
stendulkerdtantsur: I do not find any other option to enable this feature...10:00
divya2016-04-15 15:28:14.297 ^[[01;31mERROR ironic.conductor.manager [^[[01;36mreq-1c001fa8-4a52-4a30-8a2e-ff86cafdef49 ^[[00;36mNone None^[[01;31m] ^[[01;35m^[[01;31mError while preparing to deploy to node e2fb844c-1af6-47ae-9ab7-5b651262afd9: No VIFs found for node e2fb844c-1af6-47ae-9ab7-5b651262afd9 when attempting to update DHCP BOOT options.10:00
divyaGetting this error while deploying bare metal node10:00
dtantsurstendulker, why do you need to *disable* it?10:00
dtantsuralso sorry, running for lunch, see you in a few10:00
*** dtantsur is now known as dtantsur|lunch10:00
stendulkerdtantsur: you mean all the deploys should use secure boot?10:01
stendulkerdtantsur|lunch: Will discuss this later...10:01
*** nathan-h has joined #openstack-ironic10:04
*** amotoki has quit IRC10:05
*** yolanda has quit IRC10:12
*** alex_xu has quit IRC10:14
*** alex_xu has joined #openstack-ironic10:16
*** ndipanov has quit IRC10:22
*** ndipanov has joined #openstack-ironic10:24
*** daemontool has quit IRC10:28
*** divya_ has joined #openstack-ironic10:32
divya_hi Ironics10:38
*** chopmann has joined #openstack-ironic10:40
*** chopmann has quit IRC10:41
openstackgerritTan Lin proposed openstack/ironic: [PoC]Ironic live upgrade framework - Make service compatible  https://review.openstack.org/30635710:45
openstackgerritTan Lin proposed openstack/ironic: [PoC] Refactor configdrive into a new field  https://review.openstack.org/30635810:45
*** vinaym213 has quit IRC10:52
*** jtomasek_ has quit IRC10:56
*** stendulker has quit IRC11:04
*** daemontool has joined #openstack-ironic11:06
*** dtantsur|lunch is now known as dtantsur11:09
*** irf has quit IRC11:10
*** irf has joined #openstack-ironic11:10
irfhi11:11
irfdo i need to use both the drivers ?11:11
irfpxe_wol and agent_wol?11:11
*** electrofelix has joined #openstack-ironic11:11
sambettsirf: One node can only be using one driver at a time11:12
*** hoangcx has quit IRC11:12
sambettsthe pxe_ and agent_ define different methods that ironic can do the deployment, pxe_ will use iscsi to write the image to the node, agent_ uses the Ironic Python agent's http API to transfer the image to the node11:13
*** jtomasek_ has joined #openstack-ironic11:14
aarefievdtantsur: nice catch, will you add release note in follow11:15
*** divya_ has quit IRC11:15
dtantsuraarefiev, oh, a release note... I should probably add it in this patch, as a plan to backport it11:15
dtantsurwill do, if you guys can quickly re-review11:15
sambettsdtantsur: sure :)11:15
*** divya has quit IRC11:16
irfthanks11:17
dtantsurhmm, sambetts, aarefiev, wdyt about testing rerunning introspection in the discovery job (it's a bit less busy)?11:17
*** jtomasek_ has quit IRC11:18
*** cppforlife_ has quit IRC11:19
*** yonglihe has quit IRC11:19
*** sergek has quit IRC11:19
*** ifarkas has quit IRC11:19
*** jtomasek_ has joined #openstack-ironic11:20
*** cppforlife_ has joined #openstack-ironic11:20
*** divya has joined #openstack-ironic11:20
*** yonglihe has joined #openstack-ironic11:20
*** sergek has joined #openstack-ironic11:20
divyaHi Sam11:21
*** clif_h has quit IRC11:21
sambettsdtantsur: I ultimatly don't mind, although I'd like for us to maybe try to add a separate job for it for ease of debugging, so we can be like ok, main inspection passed, discovery failed, but reinstpection passed etc, maybe that'll be easier once we've got tempest going ;/11:21
sambettsHi11:21
divya Failed to prepare to deploy. Error: No VIFs found for node cb24e0b9      | |                        | -f19b-4c27-b287-58d6133973ff when attempting to update DHCP BOOT         |11:21
sambettsdivya: How did you try to deploy?11:22
dtantsursambetts, fine, lets wait for tempest, should be there soon :)11:22
*** xek has joined #openstack-ironic11:22
divya| last_error             | Failed to prepare to deploy. Error: No VIFs found for node cb24e0b9-f19b-4c27-b287-58d6133973ff when attempting to update DHCP BOOT options.11:22
aarefievyeah I can do it in tempest11:22
dtantsurgreat11:22
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Make sure to clean the blacklist cache when disabling the firewall  https://review.openstack.org/30596011:23
dtantsursambetts, aarefiev, ^^^11:23
dtantsurwith the release note11:23
divyai used this command to deploy11:23
divya#ironic node-set-provision-state --config-drive /tmp/$NAME $NAME active11:23
TheJuliaNobodyCam: Sorry was asleep11:23
TheJuliaalso, good morning everyone!11:23
sambettsdivya: Are you using Ironic in OpenStack?11:23
divyayes11:23
*** clif_h has joined #openstack-ironic11:23
sambettsthen you don't use ironic to deploy a node, you use nova11:24
jrollgood morning TheJulia, sambetts, dtantsur, aarefiev, any other friends I have missed :)11:24
jrolland happy friday \o/11:24
sambettso/ jroll11:24
dtantsurmorning jroll, \o/11:24
sambettsTGIF11:24
TheJuliagood morning jroll11:24
divyais this steps required before deploying node. https://github.com/jriguera/ansible-ironic-standalone/wiki/Cloud-Init-and-Config-Drive11:24
dtantsurjroll, gate fix for stable/mitaka: https://review.openstack.org/#/c/306323/11:25
divyai used this doc to deploy bare metal node https://github.com/jriguera/ansible-ironic-standalone/wiki/Using-Ironic-client-with-agent_ipmitool-driver11:25
aarefievmorning jroll11:25
sambettsdivya: its in the URL, thats for Ironic standalone11:26
sambettsdivya: I'd suggest taking a look through this: http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html11:26
jrolldtantsur: +A, do we need that on liberty as well?11:28
dtantsurjroll, likely, not sure. I got hit by it in mitaka right now11:28
jrolldtantsur: okay, I'll jfdi11:28
irfhi11:28
sambettsjroll: I would expect it to affect all branches that have that script11:28
irfi am not able to delete the port using port-delete command11:29
irfironic port-create -n da7e87a0-230a-4ba6-bd11-0f44e652d69f -a 00:00:00:00:00:8811:29
jrollmerge conflict :|11:29
dtantsurjroll, ah yeah, that's why I didn't propose it11:29
irfit says mac address already in use11:30
irfand gives this error11:30
irfv11:30
irfNo valid host was found. Reason: No conductor service registered which supports driver pxe_ssh. (11:30
irfi am not able to delete the port11:30
sambettsirf: It sounds like you have a node enrolled that has the driver pxe_ssh, but you've removed support for pxe_ssh from your conductor11:31
jrolldtantsur: https://review.openstack.org/30636911:32
sambettsirf: you need to add pxe_ssh to the enabled_drivers, then restart the conductor, then delete the node or update it to use a different driver , then you can remove the pxe_ssh from enabled drivers again11:32
irfohh ok11:32
*** jtomasek_ has quit IRC11:33
*** dprince has joined #openstack-ironic11:33
*** Marga__ has quit IRC11:34
sambettsdtantsur: I've got a slight english nit with your release note to make it read better, do you have time to push a quick fix?, if not I'm happy to push it through11:35
sambettsjroll: woops... should have refrshed that page before commenting :-p11:38
irfdone11:38
irfthanks11:38
sambettsirf: :)11:39
jrollsambetts: ?11:39
*** baoli has joined #openstack-ironic11:40
*** baoli_ has joined #openstack-ironic11:41
jrolloh I see it, heh11:42
*** dencaval has quit IRC11:43
sambettsjroll: Sorry about the lack of context on that patch I meant to create a bug for it then got distracted11:45
jrollsambetts: nah it's all good11:45
*** baoli has quit IRC11:45
*** dims has quit IRC11:46
openstackgerritMerged openstack/bifrost: Updated from global requirements  https://review.openstack.org/30516711:51
*** openstack has quit IRC11:51
*** openstack has joined #openstack-ironic11:53
dtantsursambetts, sure, gimme 5 mins11:55
*** dims has joined #openstack-ironic11:56
openstackgerritSofiia Andriichenko proposed openstack/python-ironicclient: Add test for chassis-node-list  https://review.openstack.org/29948211:57
*** athomas has quit IRC11:58
*** athomas has joined #openstack-ironic11:58
openstackgerritMathieu Mitchell proposed openstack/ironic-specs: Container-based display drivers  https://review.openstack.org/30607811:59
mat128good morning ironic!12:02
sambettso/ mat12812:02
mat128\o12:02
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Make sure to clean the blacklist cache when disabling the firewall  https://review.openstack.org/30596012:03
dtantsursambetts, aarefiev, one more update :) ^^^12:04
sambettsdtantsur: :) thanks12:04
vdrokmorning TheJulia jroll and mat12812:04
dtantsursambetts, have a minute chatting about driver composition?12:05
dtantsursambetts, tl;dr I don't believe that "CIMC power with IPMI management" is something we should support. and I don't think it's something we've ever supported to be honest12:05
sambettsdtantsur: Sure, I don't know why we shouldn't support that12:06
dtantsursambetts, I'd rather ask why should we? like, getting the worst of both worlds? :)12:06
dtantsurI see people often coding drivers in assumption that power == management, and it's a sane thing to assume, provided that the same interface is usually used12:07
dtantsurjroll, you might want to follow our discussion ^^^12:07
jrollooooo12:08
sambettsthat was just an example, I just don't want to limit the flexibilty that we could offer12:08
* jroll listens12:08
mat128dtantsur: I'm with sambetts on this. If operators could chose all drivers independently CMIC power with IPMI management would be possible, wouldn't it?12:09
sambettsjroll: context is my comment: https://review.openstack.org/#/c/188370/12:09
dtantsursambetts, it's fine, but it complicates the matter. I mean, I can make all the drivers pluggable, but who will guarantee that all possible permutations work?12:09
dtantsurmat128, sambetts, also, who would support that? Sam, as a Cisco representative, will you work on supporting this combination? establish CI for it?12:09
sambettsdtantsur: thats what the comcept of the hardware_type is for12:10
mat128Nova has this concept of supported / untested and warns if you are using something untested: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L61812:10
sambettsdtantsur: I think thats what I address in the last part, its down to the vendors to gurantee support for their hardware type12:10
dtantsursambetts, my question is about human resources and vendor buy-in. if no vendors will declare official support for this, what's the use in complicating everything?12:11
dtantsursambetts, will Cisco do that? I mean support CIMC + IPMI, IPMI + CIMC, etc?12:11
mat128dtantsur: if we have a compatibility map, how will we manage 3rd party drivers, especially, how will they hook into the map if they require?12:12
dtantsurmy point here is that supporting such things is a burden on vendors12:12
dtantsurmat128, entry points are extensible by 3rd parties. that's why we don't hardcode things in python code any more12:13
dtantsurI'd like vendors to stand up and say "yes, we want to support this" before we go into complicating the spec even more12:14
dtantsurhence my question: is it something planned by someone here from vendors side?12:15
jrolldtantsur: fwiw, I actually imagined getting rid of node.driver as well12:17
jrollbut didn't think about the mixing of power/mgmt much12:17
dtantsurjroll, that's a separate topic, kind of12:17
jrollkind of :)12:17
sambettsI don't really get how this would be any different from a vendors perspective than the current proposed solution except that the current solution doesn't define supported purmutations12:17
openstackgerritVladyslav Drok proposed openstack/ironic: Fix API node name updates  https://review.openstack.org/30098312:17
jrollrather, I think I agree with sambetts in general about how this should work, however I also agree I'm not sure we should allow mixing eg. cimc/ipmi12:18
jrollbut if the vendor *does* want to support that, fine12:18
jrollI just don't understand why a user would desire that12:18
dtantsurjroll++ for why a user would desire that12:18
sambettsif we don't allow them to be customised whats the point in having them as separate interfaces?12:19
dtantsursambetts, good question :)12:20
jrollright, I think we should build the system to enable it12:21
sambettsI think there are cases when you can only provide power e.g. AMT/WOL, and no mgnt, but someone might have some custome vendor mgnt solution, so then they create a WOL + <some custom thing> and that wouldn't be possible if we lock them together12:21
jrollbut very critically ask "what's the use case" when someone does something crazy in-tree12:22
mat128for an example of operator config, we have a power driver that does IPMI and falls back to SNMP/PDU (Fallback Power). We use this power driver alongside it's equivalent management driver, FallbackManagement.12:22
mat128if you had a list of supported options, I would be SOL because the list wouldn't contain any of my stuff12:22
dtantsursambetts, yeah, this example is more interesting12:22
jroll++12:23
dtantsurhmm, thanks folks, this makes much more sense at a second glance12:23
sambettsthen that vendor can add a new hardware_type, and add a new mgnt interface, then just has to add his hardware type to both the WOL interface and his mgnt interface12:23
dtantsurnow, do we want to get rid of driver/hardware_type completely?12:23
dtantsursambetts, why do we need a hardware_type then?12:23
jrollidk about the whole hardware_type thing, but I don't have a better proposal12:24
sambettswell how to define supported/unsupport interfaces12:24
jrollAIUI, it's meant to say "hardware of this type can use these drivers"12:24
dtantsursambetts, how you do this? (in python code is NOT an answer)12:24
mat128if all drivers respect the interface, why wouldn't it be possible to mix anything?12:24
jroll^12:24
*** trown|outtypewww is now known as trown12:24
jrollthat's my next question, you'd need that to be pluggable12:24
jrollfor out of tree things12:24
sambettsso originally I started by defining a Hardware type class that defined what interfaces were valid to use with that hardware12:25
dtantsursambetts, no hardcoding in Python please :) you will totally exclude 3rd parties12:26
sambettsthey would be stevedored12:26
sambettsbut the probelm I ran into with that was that often you want to support all interfaces, except a few, e.g. CIMC works with agent, iscsi and maybe others in the future, but it'll never support ILO Virutal media12:27
*** b_bezak_ has joined #openstack-ironic12:27
dtantsursambetts++12:27
dtantsurso12:27
sambettsthats why I wanted to move the relationship the other way around12:27
mat128"This spec proposes redefining a driver as power + management + passthru interfaces with other interfaces being changeable by an operator." < if you do this, and provide new fields for deploy, console, etc. I guess your goal is met12:28
dtantsurwe need an entry point like: ironic.hardware-types.cimc.boot listing all available boot interfaces. then 3rd parties can add to it12:28
dtantsurOR12:28
dtantsurwe can leave it up to the configuration file like I have right now in the proposal12:28
dtantsur(then potentially operators can screw themselves)12:28
sambettsI see it more like: ironic.power.interface.ipmipower.supported_hardware_types: ....12:29
sambettsand have a ironic.hardwre_types one too12:29
sambettsironic.interface.power.ipmitool.support_hardware_types: ...12:30
sambettsthats better ^12:30
dtantsuryeah. maybe even s/support_//12:30
*** ndipanov has quit IRC12:31
dtantsurnow a tricky question for you folks: what defines the vendor passthru?12:31
dtantsursambetts, mat128 ^^12:32
jrolloh my12:32
mat128sambetts, dtantsur: ok different idea (bear with me for a sec): hardware_types specify all fields (like the current driver), but they can also be defined individually. All hardware_types should be CI tested by vendor, but if you mix individual options you're on your own.12:32
sambettsits should be just another interface12:32
dtantsursambetts, in 99% cases it depends on management/power12:32
* jroll thinks we should kill vendor passthru with fire12:32
dtantsurjroll, vendors will cry12:32
dtantsurmat128, I don't quite get how it's different from what sam proposes12:33
* sambetts thinks this wouldn't be a probelm if deployment didn't rely on it 12:33
dtantsurwell...12:33
dtantsursambetts, if you mix drac vendor with ipmi mgmt, it won't work12:33
jrolldtantsur: I don't think there's many useful passthrus in tree12:33
jrollright?12:33
mat128you dont need to provide any list of supported X, you just provide "bundles" that are known to work12:34
mat128like we do right now12:34
dtantsurjroll, drac bios settings iirc; something in ipmi drivers12:34
dtantsursambetts, or maybe it will...12:34
jrolldtantsur: they should move to cleaning :)12:34
*** keedya has joined #openstack-ironic12:34
*** irf has quit IRC12:34
jrollanyway, we can't kill it for now12:34
jrolland remember we mix in multiple passthru classes12:34
dtantsuroh yeah, moar fun12:35
jrollmaybe passthrus=agent,ipmitool12:35
*** Goneri has joined #openstack-ironic12:35
dtantsurjroll, lets kill the agent passthru finally. or rather, make it generic. I can file an RFE12:35
*** b_bezak_ has left #openstack-ironic12:35
sambettsa passthru can have supported_hardware_types too, and also there is no reason why a particular interface couldn't add something in their validate call to say hey I don't work with your current purmutation of interfaces12:36
dtantsurfair12:36
mat128sambetts: does that mean we don't need any supported* in stevedore?12:37
dtantsurmat128, why?12:38
sambettsmat128: you still need that so that you can customise the support_hardware_types12:38
sambettsfor each interface12:38
*** irf has joined #openstack-ironic12:38
mat128hmm, I fail to see why we should limit the operators in any way12:38
jrolldtantsur: +1, good point12:39
sambettsmat128: we're not?12:39
mat128we define hardware types that have supported interfaces (deploy/mgmt/power/etc.)12:39
sambettsno the other way around12:39
mat128ok yes, the other way around12:39
mat128what are we going to do with this information?12:40
mat128raise on unsupported setups?12:40
mat128warn?12:40
*** manfredp_ has quit IRC12:40
sambettsraise, mat128: https://review.openstack.org/#/c/188370/ my full explanation is here12:40
mat128'(mat128) hmm, I fail to see why we should limit the operators in any way'12:40
openstackgerritGonĂ©ri Le Bouder proposed openstack/ironic: ipxe: retry on failure  https://review.openstack.org/30619612:41
mat128so we limit them in their choices, don't we?12:41
dtantsurjroll, https://bugs.launchpad.net/ironic/+bug/1570841 wanna a spec for that?12:42
openstackLaunchpad bug 1570841 in Ironic "[RFE] Promote agent vendor passthru to core API" [Wishlist,Confirmed] - Assigned to Dmitry Tantsur (divius)12:42
sambettsas a vendor I want to define my support for Ironic, so I don't get emails e.g. I've configured this interface with this interface and it doesn't work12:42
mat128so it's more of declaring what is supported12:42
jrolldtantsur: I don't, but the "this is an api change and will have deprecation things" makes me think yes, a quick one12:42
dtantsurjroll, sure12:43
jrolldtantsur: more about how we do the deprecation period and such12:43
mat128which goes back to my nova example of 'you're on your own now': https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L61812:43
*** keedya has quit IRC12:43
mat128I feel this would be much simpler if we just allowed independent power/deploy/mgmt/console/etc. drivers12:43
sambettsI think we could have a hardware_type which is ANYTHING, and just allow any interface permutation if that hardware_type is being used12:43
dtantsursambetts, aka "fake" :)12:43
sambettsdtantsur: exactly12:43
sambettsbut if you declare your hardware as my hardware, then I'm going to tell you what I support12:44
mat128so ``node.hardware_type`` is the new ``node.driver``12:44
*** ndipanov has joined #openstack-ironic12:46
sambettsnot exactly, node.driver is a fixed relationship between a set of interfaces, hardware_type lets us define support for a wide range on interfaces that can all be mixed and matched, and if it validates you shoule be able to guarentee it works12:46
mat128good point, yes12:47
mat128ok then, I think it makes sense12:47
mat128so I can stevedore configure a new type that allows whatever permutations I want, it's just that when I call you up and tell you this, you can tell me that I'm on my own :)12:47
sambettsexactly12:48
mat128good stuff, sorry for disturbing the peace on this beautiful morning12:48
openstackgerritJulia Kreger proposed openstack/ironic: Add config option for ATA erase fallback in agent  https://review.openstack.org/30281912:48
sambettsthats all good, its a work in progress idea, so brian storming like this is the best thing for it12:48
sambettsbrain *12:48
sambettsman it must be Friday12:48
TheJuliaThat it is :)12:49
jrollhttp://www.reactiongifs.us/wp-content/uploads/2014/02/jumping_high_five_family_guy.gif12:50
jrolltried to find brian storming through a door but no luck :(12:50
sambettshahahaha12:50
sambettsMorning TheJulia12:50
*** mgoddard__ has joined #openstack-ironic12:51
TheJuliajroll: I think that means you just have to spend the day watching family guy :)12:51
*** alaski is now known as lascii12:52
jrollTheJulia: lol12:52
vdrokjroll: about that nova rebuild bug, wdyt about just adding if recreate before this line - https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L278612:52
vdrokthe comment says about that, so we could just set scheduled_node to instance.node12:53
vdrokthen migration won't be created12:53
*** ndipanov has quit IRC12:53
*** mgoddard_ has quit IRC12:54
jrollvdrok: idk, we probably want to hit this block: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L277112:54
jrollno sense in making more conditionals, just pass the node in if it's a real rebuild call12:55
vdrokjroll: hm, I'm not a nova expert, but I thought that this rebuild claim creates migration in nova db12:55
vdrokand after that, after instance deletion update_available_resource complains about this migration, telling that instance is deleted12:56
vdrokso maybe nopclaim is better?12:56
*** irf has quit IRC12:56
jrollvdrok: rebuild does rebuild_instance on the same host, migrate/evacuate does rebuild_instance on a different host12:56
vdrokyep, I get that12:57
jrollidk about migration records, if they're meant to be created for rebuild or not12:57
vdrokwell, they are, I'll paste a log12:57
jrollbut really, the problem here is that the rebuild api doesn't take instance.node into account12:57
jrollidk12:57
jrollseems like you should ask nova folks :)12:57
jrollI mean for normal rebuilds, not ironic rebuilds12:57
vdrokjroll: http://paste.openstack.org/show/494215/12:59
vdrokhere is what happens after instance deletion12:59
vdrokjroll: yep, will ask around in nova channel, thanks! :)12:59
vdrokbrb12:59
jrollyeah np13:00
*** derekh has joined #openstack-ironic13:03
*** ndipanov has joined #openstack-ironic13:14
*** cdearborn has joined #openstack-ironic13:17
openstackgerritDmitry Tantsur proposed openstack/ironic-specs: Promote agent vendor passthru to core API  https://review.openstack.org/30641813:19
dtantsurjroll, any quick comments on ^^^?13:19
* dtantsur didn't finish the RPC part yet, but wants some sanity check13:19
* jroll reads13:19
openstackgerritMerged openstack/ironic-inspector: Make sure to clean the blacklist cache when disabling the firewall  https://review.openstack.org/30596013:22
*** jjohnson2_ has joined #openstack-ironic13:22
* sambetts wonders if we should look into creating an API extension framework similar to that provided in neutron, so that the agent deploy interface could define the lookup and heartbeat API end points 13:23
dtantsursambetts, that's how we use vendor passthru now, I guess :)13:24
jrolldtantsur: done, good stuff, just a few things13:24
jrollno pls no extensions13:24
sambettsI guess its sort of like vendor passthru, except defined by the interfaces themselves not a separate thing13:24
dtantsurjroll, thanks, updating soon13:25
jrollcool13:25
jrolloh I forgot one13:25
jrollthere13:26
dtantsuroh, this is a good one :)13:26
sambettsI guess lookup and heartbeat are specfic to use IPA, but not specific to a particuar deploy process, so unless where going to support people creating their own ramdisk with its own API then we don't need to make this extensible13:27
sambettsbig +1 from me13:28
dtantsursambetts, even 3rd party ramdisk will probably have to follow the same logic: lookup, heartbeat, then do whatever they want13:28
sambettsyeah, I was just thinking that13:28
sambettstbh I'd really like it if the ramdisks didn't talk to the Ironic API13:29
sambettsseems like a layer break to me13:29
dtantsursambetts, well, it's not mandatory to call these API.. and not mandatory to implement in the deploy interface13:30
jrollthe other option is have them register via zookeeper/etc13:30
dtantsurhmm, yeah. are we talking about DLM on the summit?13:30
*** piet has joined #openstack-ironic13:31
jrollI think so13:32
sambettsOnce the bash ramdisk is killed with fire, the deploy interface should be more or less transparent to the ramdisk being used right?13:33
jrolldefine transparent :)13:35
sambettsif we define a standard interface for the ramdisks to talk e.g. lookup/heartbeat, then the deploy interfaces should have a standard API for functions that are called when one of those events happen13:35
jrolldtantsur: oh, we aren't discussing it, we more or less have a plan13:36
openstackgerritDmitry Tantsur proposed openstack/ironic-specs: Promote agent vendor passthru to core API  https://review.openstack.org/30641813:36
dtantsurupdate ^^^13:36
dtantsurjroll, aha, remembered13:36
jrollsambetts: yeah, sounds right13:36
jrollthis spec reminds me, going to try to finish search api spec today13:38
dtantsurhehe :) so many specs, this is the 3rd from me in 2 days13:38
jrollnice, let's land them all next week :)13:38
sambettsDo you think I need to write a spec for the remove *_ssh drivers RFE?13:38
dtantsurspeaking of which, sambetts, please check https://review.openstack.org/305864 when you have some time13:38
jrollsambetts: link?13:39
sambettsjroll: https://bugs.launchpad.net/ironic/+bug/157030113:39
openstackLaunchpad bug 1570301 in Ironic "[RFE] Deprecate and remove the *_ssh drivers" [Wishlist,Confirmed]13:39
jrollhmm13:40
jrollsambetts: can you lay out the todo list for that, then I think we could approve it13:41
jrollwith timelines for deprecation, preferably13:41
jroll(e.g. add vbmc to ironic project)13:41
*** piet has quit IRC13:41
sambettsin a full spec or in the RFE?13:41
dtantsuryay for vmbc in ironic :)13:41
jrollsambetts: in the rfe should be fine13:42
sambettsCool :D sure I'll try to update that13:42
jrollthanks, feel free to ping me13:42
jrollI want to go through open rfe's this afternoon13:42
*** piet has joined #openstack-ironic13:42
sambettswill do, I'm not sure what lucasagomes's plan is for vbmc13:42
sambettslucasagomes: you here?13:42
dtantsursambetts, he's on travel today13:43
dtantsursambetts, but moving vbmc to ironic was on his plans :)13:43
dtantsurjroll, a concerning thing: http://tarballs.openstack.org/ironic-python-agent/coreos/files/ shows the newest images from Apr 513:43
jrolldtantsur: https://review.openstack.org/#/c/305988/ fixes13:44
jrollbut being a pain in the gate for some reason :/13:44
dtantsurdamn13:44
jrolllooks like this round is debian mirrors :|13:45
jrolland timeouts13:45
jrollurgh13:45
dtantsur.....13:46
sambettssigh...13:46
sambettsdtantsur: yeah, not sure what the timeline is on it though :/13:47
jrollwe could get it done next week easy13:47
dtantsuryep, not so hard. just wait for Lucas to come back from Brno13:47
sambettsawesome :D13:47
*** [1]cdearborn has joined #openstack-ironic13:49
*** piet has quit IRC13:51
*** piet has joined #openstack-ironic13:51
sambettshmmm, deprecate for removal in OpenStack Ocata or P..?13:51
sambettsremoval in Ocata leaves them  deprecated for the whole of Newton13:52
*** jayc__ has joined #openstack-ironic13:52
jrollotaca imo13:55
jrollocata*13:56
*** Nisha_away has joined #openstack-ironic13:57
sambettscool, thats what I was thinking :)13:57
dtantsur++ for ocata13:58
*** daemontool_ has joined #openstack-ironic13:58
dtantsurpeople should not use them in production (but they do)13:58
*** ametts has joined #openstack-ironic13:59
jrollO_o13:59
*** cdearborn has quit IRC14:00
sambettsjroll: Something like https://bugs.launchpad.net/ironic/+bug/157030114:00
openstackLaunchpad bug 1570301 in Ironic "[RFE] Deprecate and remove the *_ssh drivers" [Wishlist,Confirmed]14:00
*** dims has quit IRC14:01
*** daemontool has quit IRC14:01
openstackgerritVasyl Saienko proposed openstack/ironic: Add unit tests for ClusteredComputeManager  https://review.openstack.org/30644514:03
jrollsambetts: lgtm, dtantsur are you good with that as-is?14:03
jrollI added a couple words14:03
sambettsjroll: we've already got IPMItool driver gate tests running and voting by the Ocata cycle14:04
*** dims has joined #openstack-ironic14:04
jrollsambetts: yeah, but not for all cases14:04
sambettsoh?14:04
*** mgould has quit IRC14:04
dtantsurjroll, sambetts, lgtm14:05
jrollwe just have it for pxe_ipa I believe14:05
*** yolanda has joined #openstack-ironic14:05
sambettsjroll: we have separate tests for *_ipmitool14:05
sambettshttps://review.openstack.org/#/c/305868/14:06
sambettsfor both pxe and  agent14:06
jrolloh we do have agent, nice14:06
sambettswe'd need them for tinyipa and postgres ones I guess14:06
sambettstoo14:06
jrollya14:06
jrollI also no longer see an ipxe test?14:07
sambettspxe_ipa is ipxe14:07
jrollaha, cool14:07
jrollso yeah, just need to make sure we have the same coverage14:07
sambettseither dtantsur or lucasagomes merged them14:07
*** dansmith is now known as superdan14:07
sambetts++14:07
jrollbut we can just switch them over in ocata14:07
dtantsuryep, pxe_ipa uses ipxe14:08
jrollno need to run duplicate jobs for all of newton14:08
jrollat any rate, rfe is now approved14:08
sambettscool :D14:08
jrollthanks for that14:08
dtantsurawesome14:09
*** divya has quit IRC14:09
* sambetts did it so that he doesn't have deal with confused people any more ;)14:09
*** mtanino has joined #openstack-ironic14:14
NobodyCamgood morning Ironicers...and of course...14:16
NobodyCamTGIF!!!14:16
sambettshey hey NobodyCam14:16
NobodyCammorning sambetts :)14:16
*** maurosr has quit IRC14:19
*** [1]cdearborn has quit IRC14:20
*** maurosr has joined #openstack-ironic14:20
jrollmorning NobodyCam \o14:21
*** [1]cdearborn has joined #openstack-ironic14:22
*** jayc__ has quit IRC14:22
*** jayc_ has joined #openstack-ironic14:23
NobodyCammorning jroll how goes the house hunt?14:23
*** cdearborn has joined #openstack-ironic14:23
*** Nisha_away has quit IRC14:23
jrollNobodyCam: good, we're signing on one sunday unless a couple more we see tonight are amazing :)14:24
NobodyCamjroll: awesome :) and congratz :)14:24
jrollthanks!14:25
TheJuliawow, congrats14:25
*** jayc_ is now known as jayc14:25
jrollsuper excited, has a fenced in yard for the dog and less than a mile walk to a small little downtown area14:25
jrollvery american dream-y14:25
*** e0ne_ has joined #openstack-ironic14:25
*** e0ne has quit IRC14:26
NobodyCamanyone seeing timeout in gate checks https://review.openstack.org/#/c/272658 has had several14:26
jrollyep, another patch is being problematic too14:26
jrollhaven't dug into if something changed14:26
*** vishwanathj_zzz has quit IRC14:27
*** vishwanathj_zzz has joined #openstack-ironic14:27
NobodyCamya I haven't dug in to much: http://logs.openstack.org/58/272658/38/check/gate-tempest-dsvm-ironic-pxe_ipa/bcf088f/console.html#_2016-04-15_07_13_40_43814:28
*** vishwanathj_zzz has quit IRC14:28
NobodyCambut looks like the old standard timeout at least on the surface14:28
*** vishwanathj_zzz has joined #openstack-ironic14:29
*** achanda has joined #openstack-ironic14:30
jrollyep14:30
dtantsurmorning NobodyCam14:31
NobodyCammorning dtantsur ... happy Friday :)14:31
*** achanda has quit IRC14:32
*** mkovacik has quit IRC14:33
*** mkovacik has joined #openstack-ironic14:33
dtantsurjroll, stable backport please: https://review.openstack.org/#/c/305916/14:33
*** athomas has quit IRC14:34
dtantsursambetts, aarefiev, and 2 backports for one of you guys: https://review.openstack.org/306421 and https://review.openstack.org/306429 :)14:34
jrolldtantsur: done14:36
sambettsdtantsur: also done :-P14:36
sambettsdtantsur: I forgot we backported that firewall stuff into liberty14:36
*** [1]cdearborn has quit IRC14:37
dtantsurwe did14:38
dtantsurnow we have a fix for the fix :)14:38
sambettsyay :D14:38
*** cdearborn has quit IRC14:40
NobodyCamjroll: https://review.openstack.org/#/c/304578 has 2 +2's wanted a IPA expert to hit the button..14:40
*** cdearborn has joined #openstack-ironic14:40
*** electrofelix has quit IRC14:43
* NobodyCam grabs more coffee14:44
*** mgoddard has joined #openstack-ironic14:46
*** mgoddard__ has quit IRC14:46
*** mkovacik has quit IRC14:49
*** mkovacik has joined #openstack-ironic14:50
*** electrofelix has joined #openstack-ironic14:51
*** vishwanathj_zzz is now known as vishwanathj14:51
*** daemontool__ has joined #openstack-ironic14:54
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Store ramdisk logs on all processing failures, not only reported by the ramdisk  https://review.openstack.org/30001114:55
*** mbound has quit IRC14:55
*** daemontool_ has quit IRC14:56
*** ptx0 has quit IRC14:57
openstackgerritDmitry Tantsur proposed openstack/ironic-specs: Promote agent vendor passthru to core API  https://review.openstack.org/30641814:58
dtantsuryuriyz, ^^^14:58
NobodyCamso ya seeing timeouts on more patches, /me suspects something bigger is afoot. :(14:58
sambetts:'(14:58
vdrokmorning NobodyCam15:01
vdrokyep, something strange happens - http://logs.openstack.org/83/300983/10/check/gate-tempest-dsvm-ironic-pxe_ssh/fae7ec2/console.html#_2016-04-15_12_58_40_68415:01
vdrok12:45:48,323 and the next log line is 12:58:17,74515:01
NobodyCammorning vdrok15:01
sambettsjlvillal: Btw I've had our recheck command adapted to make sure you can do a single Cisco recheck or a full recheck15:03
*** mbound has joined #openstack-ironic15:06
*** pcaruana has quit IRC15:06
jrollNobodyCam: I'll let JayF have that one given sambetts comments15:08
*** superdan is now known as _-_15:08
*** _-_ is now known as superdan15:09
*** MattMan has quit IRC15:10
*** MattMan has joined #openstack-ironic15:10
*** kozhukalov has quit IRC15:11
*** serverascode has quit IRC15:11
*** JoshNang has quit IRC15:11
*** ChrisMead has quit IRC15:11
*** vdrok has quit IRC15:11
*** boris-42 has quit IRC15:11
*** devananda has quit IRC15:12
*** jmccrory has quit IRC15:12
*** fyxim_ has quit IRC15:12
*** JayF has quit IRC15:12
*** odyssey4me has quit IRC15:13
*** anush has quit IRC15:13
*** lekha has quit IRC15:13
*** [1]cdearborn has joined #openstack-ironic15:16
*** mgoddard_ has joined #openstack-ironic15:16
*** daemontool_ has joined #openstack-ironic15:16
*** ppiela_ has joined #openstack-ironic15:16
*** odyssey4me has joined #openstack-ironic15:17
*** electrofelix has quit IRC15:18
*** maurosr has quit IRC15:18
*** anush has joined #openstack-ironic15:18
*** MattMan__ has joined #openstack-ironic15:18
*** JayF has joined #openstack-ironic15:19
*** JayF has joined #openstack-ironic15:19
*** sambetts_ has joined #openstack-ironic15:19
*** boris-42 has joined #openstack-ironic15:19
*** mat128_ has joined #openstack-ironic15:19
*** sturivnyi_ has joined #openstack-ironic15:20
*** MattMan has quit IRC15:20
*** daemontool__ has quit IRC15:20
*** mgoddard has quit IRC15:20
*** jayc has quit IRC15:20
*** vdrok_ has joined #openstack-ironic15:20
jrollJayF: icymi in the netsplit, your eyes are appreciated on https://review.openstack.org/#/c/30457815:20
*** tonyb_ has joined #openstack-ironic15:20
*** pleia2_ has joined #openstack-ironic15:21
*** fyxim_ has joined #openstack-ironic15:23
*** baoli has joined #openstack-ironic15:24
*** davidlenwell has joined #openstack-ironic15:24
*** jlvillal_ has joined #openstack-ironic15:24
*** vdrok has joined #openstack-ironic15:24
*** vdrok_ has left #openstack-ironic15:25
*** maurosr has joined #openstack-ironic15:25
*** ChrisMead has joined #openstack-ironic15:25
*** e0ne_ has quit IRC15:25
*** dims has quit IRC15:25
*** ametts has quit IRC15:25
*** ndipanov has quit IRC15:25
*** derekh has quit IRC15:25
*** baoli_ has quit IRC15:25
*** dprince has quit IRC15:25
*** ppiela has quit IRC15:25
*** mat128 has quit IRC15:25
*** jlvillal has quit IRC15:25
*** ohamada has quit IRC15:25
*** davidlenwell_ has quit IRC15:25
*** xavierr has quit IRC15:25
*** dhellmann has quit IRC15:25
*** sambetts has quit IRC15:25
*** pleia2 has quit IRC15:25
*** cameron_C has quit IRC15:25
*** kbyrne has quit IRC15:25
*** hamzy has quit IRC15:25
*** tonyb has quit IRC15:25
*** EmilienM has quit IRC15:25
*** sturivnyi has quit IRC15:25
*** pleia2_ is now known as pleia215:26
*** johnthetubaguy has quit IRC15:26
*** xavierr has joined #openstack-ironic15:26
*** dims has joined #openstack-ironic15:26
*** e0ne has joined #openstack-ironic15:26
openstackgerritAnton Arefiev proposed openstack/ironic-inspector: [WIP] Tempest: add basic test  https://review.openstack.org/29392615:26
JayFjroll: looking15:26
*** dhellmann has joined #openstack-ironic15:26
*** amotoki has joined #openstack-ironic15:27
*** jlvillal_ is now known as jlvillal15:27
*** cdearborn has quit IRC15:27
*** ptx0 has joined #openstack-ironic15:27
JayFjroll: mat128_ added a conductor side check that complains loudly if zeroize=false and iterations=015:27
*** jlvillal is now known as Guest8106215:27
JayFjroll: IMO we don't need to scream in IPA logs as well, which nobody would see anyway15:28
*** EmilienM has joined #openstack-ironic15:28
*** dprince has joined #openstack-ironic15:28
*** greghaynes has quit IRC15:28
*** mkovacik has quit IRC15:29
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Add tests for baremetal subcommands in openstackclient  https://review.openstack.org/30168715:29
*** ametts has joined #openstack-ironic15:29
*** ndipanov has joined #openstack-ironic15:30
*** ohamada has joined #openstack-ironic15:30
*** jmccrory has joined #openstack-ironic15:30
*** jistr has quit IRC15:30
*** rcernin has quit IRC15:30
*** derekh has joined #openstack-ironic15:30
*** hamzy has joined #openstack-ironic15:31
JayFsambetts_: https://review.openstack.org/#/c/304578 I just +2'd this with a comment re: your concern, can you have another look? If you feel OK with that I'll aldn it15:31
JayFland15:31
*** Guest63229 has quit IRC15:31
*** kbyrne has joined #openstack-ironic15:32
*** jayc has joined #openstack-ironic15:33
*** devananda has joined #openstack-ironic15:33
*** JoshNang has joined #openstack-ironic15:33
dtantsurhave a nice weekend everyone!15:34
*** dtantsur is now known as dtantsur|afk15:35
NobodyCamhave a great weekend dtantsur|afk15:35
vdrokdtantsur|afk: you too! good night15:35
*** cameron_C has joined #openstack-ironic15:36
*** [1]cdearborn has quit IRC15:36
*** lekha has joined #openstack-ironic15:37
*** [1]cdearborn has joined #openstack-ironic15:38
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Add tests for openstackclient plugin  https://review.openstack.org/30168715:40
*** jistr has joined #openstack-ironic15:42
*** johnthetubaguy has joined #openstack-ironic15:46
*** kozhukalov has joined #openstack-ironic15:47
*** greghaynes has joined #openstack-ironic15:47
*** vishwana_ has joined #openstack-ironic15:47
*** serverascode has joined #openstack-ironic15:47
*** vishwana_ is now known as vishwanathj_15:48
*** vishwanathj has quit IRC15:50
*** rama_y has joined #openstack-ironic15:53
*** jistr is now known as jistr|afk16:00
*** mgoddard__ has joined #openstack-ironic16:01
*** mgoddard_ has quit IRC16:04
sambetts_JayF: Make sense to me, I was looking for a coresponding patch that might do it, but the bug report seems to not have updated with patch info :(16:07
sambetts_o/ dtantsur|afk16:07
JayFsambetts_: yeah, something has been ... weird since they moved the gerrit server16:07
JayFsambetts_: like emails coming in late, bugs getting updated later than usual16:07
*** sambetts_ is now known as sambetts16:07
*** jcoufal has quit IRC16:08
JayFsambetts_: you wanna flip your vote then I'll alnd it?16:08
sambettsJayF: done :)16:08
devanandamorning, all16:09
JayFgm16:09
sambettshey devananda16:09
*** jayc has quit IRC16:10
*** serverascode has quit IRC16:13
*** gabriel-bezerra has quit IRC16:13
*** serverascode has joined #openstack-ironic16:14
*** gabriel-bezerra has joined #openstack-ironic16:15
*** yarkot has quit IRC16:15
*** anush has quit IRC16:15
*** anush has joined #openstack-ironic16:16
*** sergek has quit IRC16:17
*** mbound has quit IRC16:17
*** yonglihe has quit IRC16:17
*** yarkot1 has quit IRC16:17
*** akrish9 has quit IRC16:17
*** agordeev has quit IRC16:17
vdrokmorning devananda and JayF16:19
JayFgm16:19
*** agordeev has joined #openstack-ironic16:20
*** sergek has joined #openstack-ironic16:20
NobodyCammorning devananda16:20
*** yonglihe has joined #openstack-ironic16:20
vdrokdevananda: wdyt about https://review.openstack.org/306010 if you have some time?16:20
NobodyCammorning JayF16:20
*** Sukhdev has joined #openstack-ironic16:23
*** amotoki has quit IRC16:24
*** jayc has joined #openstack-ironic16:24
*** achanda has joined #openstack-ironic16:26
*** kromanenko has quit IRC16:27
devanandavdrok: interesting. what is the goal // what are you trying to fix?16:30
*** achanda has quit IRC16:30
vdrokdevananda: after I do nova rebuild the first time, instance.node is being replaced by the first ironic node associated with nova compute, and subsequent rebuilds fail16:32
*** jayc_ has joined #openstack-ironic16:32
devanandaoh16:33
*** jayc_ has quit IRC16:33
devanandaah, because of this? compute_node = self._get_compute_info(context, self.host)16:33
vdrokyep16:33
devanandaugh. yea, that's definitely not righ16:34
*** ohamada has quit IRC16:34
devanandait is assuming that self.host has only one hypervisor_hostname16:34
vdrokscheduled_node is passed in only in case of evacuate16:34
devanandawhich is not true for the nova.virt.ironic driver16:34
vdrokyeah16:34
*** jayc has quit IRC16:34
vdrokdevananda: another fix I thought about it this one - https://review.openstack.org/#/c/306011/16:34
vdrokbut if done that way, migration is created in nova, which then creates troubles in update_available_resource16:35
*** suro-patz has joined #openstack-ironic16:36
*** cfarquhar has quit IRC16:37
devanandavdrok: that would also prevent an admin from using nova's force-hosts capabilities16:37
devanandaactually, question16:37
devanandathe passed "instance" parameter -- this is the current instance, which is to be rebuilt, isn't it?16:37
vdrokyes16:37
devanandayou're forcing it to be rebuilt on the same ironic Node16:37
devanandawhich prevents evacuation or rebuilding on a different host from working -- even when that is desired16:38
devanandain this one, rebuild onto a new Node would work, because scheduled_node would not be None16:38
devanandahttps://review.openstack.org/#/c/306010/2/nova/compute/manager.py16:38
devanandaso yea, that's better16:38
vdrokyep, thanks, will add tests and more details there16:39
devanandavdrok: also, reading the doc block on that method, I think you want it to be "if migration:"16:41
*** cfarquhar has joined #openstack-ironic16:43
*** cfarquhar has quit IRC16:43
*** cfarquhar has joined #openstack-ironic16:43
*** harlowja has quit IRC16:44
*** harlowja has joined #openstack-ironic16:45
vdrokdevananda: I thought they are equivalent in our case? looking at compute.api.evacuate, migration is created there and recreate set to true16:46
JayFIs there some way to assert nothing happened at all?16:46
JayFTrying to write a test to confirm that nothing happens, and it's surprisingly hard16:47
JayFhttps://review.openstack.org/#/c/263842/12/ironic/drivers/modules/pxe.py line 52116:47
JayFtrying to test if CONF.pxe.manage_ramdisk_boot is false16:47
JayFbut it literally just noops the whole method16:47
JayFis there some kind of "assert this did nothing whatsoever" trick I don't know about, lol16:47
JayFI guess I could just negative-check against the other path, that's what I'll do unless there is a better suggestion here16:48
sambettsJayF: assrt that dhcp_options_for_instance isn't called, but try_set_boot_device is called?16:48
JayFthis is for clean_up_ramdisk16:48
JayFwhere literally I do nothing if it's set false16:49
sambettsassert _get_deploy_image_info is never called?16:49
JayFyeah, that's what I'm going to do16:49
devanandaJayF: assert call count is 0 ?16:49
devanandayea16:49
JayFcopy the test for the other settings, then assert all those calls didn't happen16:49
*** trown is now known as trown|lunch16:51
JayFI just kinda wish there was a assertNoop() or similar, but I guess that'd be kinda hard to write :)16:51
*** jayc has joined #openstack-ironic16:52
sambettsJayF: you'd have to assert that the stack frame didn't change or something *mind melt*16:53
JayFyeah, exactly16:53
sambettsdevananda: I've got to head off but I'd like to hear your thoughts on the idea I left in the comments for the Driver comp reform spec https://review.openstack.org/#/c/188370/ when you get a moment, it sort of changes up the way we were thinking about it16:54
*** anush has quit IRC16:55
*** anush has joined #openstack-ironic16:55
*** sambetts is now known as sambetts|afk16:56
*** Sukhdev has quit IRC16:57
*** Sukhdev has joined #openstack-ironic16:57
*** derekh has quit IRC16:58
*** piet has quit IRC16:59
*** jistr|afk is now known as jistr17:00
*** piet has joined #openstack-ironic17:00
Guest81062JayF, Do you have a link to actual tests to this point?17:03
*** achanda has joined #openstack-ironic17:04
JayFGuest81062: might wanna /nick :)17:04
Guest81062Ah. darn17:04
JayFGuest81062: I fixed them, fixed them so well they're failing now17:04
*** Guest81062 is now known as jlvillal17:04
JayFso I have to find out why it's failing, lol17:04
*** Sukhdev has quit IRC17:04
jlvillalJayF, Okay back to being jlvillal17:05
JayFlike the test is correct now; it's showing my code may be incorrect17:05
JayFso digging into that17:05
JayFalthough not that test in particular, another test bug ruby found17:05
jlvillalJayF, Okay. Happy to help if I can. Though in an all-day meeting which have to pay attention to for the most part.17:06
JayFI think I'm good for now, except for the fact my code is broken, lol17:06
JayFI think I'm going to simplify it, instead of trying to print the deprecation once per driver, just going to print it once overall17:07
jlvillalJayF, :)  Okay.17:07
*** achanda has quit IRC17:08
*** mgoddard__ has quit IRC17:10
*** mgoddard has joined #openstack-ironic17:10
devanandasambetts|afk: will take a look, thanks17:11
sambetts|afkdevananda: thanks17:12
sambetts|afkjlvillal: cisco-ironic-recheck is our new recheck command17:13
* sambetts|afk is really AFK now :D 17:13
*** yuanying_ has joined #openstack-ironic17:18
*** vdrok has quit IRC17:18
*** mat128_ has quit IRC17:18
*** nathan-h has quit IRC17:19
*** suro-patz1 has joined #openstack-ironic17:20
*** fyxim_ has quit IRC17:20
*** kbyrne has quit IRC17:21
*** daemontool__ has joined #openstack-ironic17:22
*** jmccrory has quit IRC17:22
*** pleia2 has quit IRC17:22
*** mat128 has joined #openstack-ironic17:23
*** yonglihe_ has joined #openstack-ironic17:23
*** boris-42 has quit IRC17:24
*** pleia2 has joined #openstack-ironic17:24
*** akrish9 has joined #openstack-ironic17:26
*** piet has quit IRC17:26
*** anush has quit IRC17:26
*** jayc has quit IRC17:26
*** cfarquhar has quit IRC17:26
*** suro-patz has quit IRC17:26
*** yonglihe has quit IRC17:26
*** sergek has quit IRC17:26
*** agordeev has quit IRC17:26
*** serverascode has quit IRC17:26
*** yonglihe_ is now known as yonglihe17:27
*** gabriel-bezerra has quit IRC17:27
*** hamzy has quit IRC17:27
*** ndipanov has quit IRC17:27
*** ametts has quit IRC17:27
*** EmilienM has quit IRC17:27
*** dhellmann has quit IRC17:27
*** ChrisMead has quit IRC17:27
*** sambetts|afk has quit IRC17:27
*** ppiela_ has quit IRC17:27
*** daemontool_ has quit IRC17:27
*** Goneri has quit IRC17:27
*** cppforlife_ has quit IRC17:27
*** alex_xu has quit IRC17:27
*** openstackgerrit has quit IRC17:27
*** bcornec has quit IRC17:27
*** smoriya has quit IRC17:27
*** yuanying has quit IRC17:27
*** Haomeng has quit IRC17:27
*** thrash has quit IRC17:27
*** EmilienM has joined #openstack-ironic17:29
*** bcornec has joined #openstack-ironic17:29
*** maurosr has quit IRC17:31
*** e0ne has quit IRC17:31
*** JayF has quit IRC17:31
*** sambetts has joined #openstack-ironic17:32
*** smoriya_afk has joined #openstack-ironic17:33
*** smoriya_afk is now known as smoriya17:34
*** ndipanov has joined #openstack-ironic17:34
*** b_bezak has joined #openstack-ironic17:34
*** david-lyle has quit IRC17:34
*** david-lyle has joined #openstack-ironic17:34
*** jayc has joined #openstack-ironic17:36
*** ppiela has joined #openstack-ironic17:36
*** jmccrory has joined #openstack-ironic17:37
*** maurosr has joined #openstack-ironic17:37
*** yarkot has joined #openstack-ironic17:37
*** b_bezak has quit IRC17:38
*** cfarquhar has joined #openstack-ironic17:39
*** cfarquhar has quit IRC17:39
*** cfarquhar has joined #openstack-ironic17:39
devanandasambetts: comments posted17:39
*** kbyrne has joined #openstack-ironic17:40
devanandaI like your idea, but am suggesting a different approach17:40
*** anush has joined #openstack-ironic17:40
*** daemontool__ has quit IRC17:40
*** ChrisMead has joined #openstack-ironic17:40
*** alex_xu has joined #openstack-ironic17:41
*** sergek has joined #openstack-ironic17:42
*** ndipanov has quit IRC17:44
*** ametts has joined #openstack-ironic17:44
*** dhellmann has joined #openstack-ironic17:44
*** fyxim_ has joined #openstack-ironic17:44
*** JayF has joined #openstack-ironic17:45
*** JayF has joined #openstack-ironic17:45
mat128JayF: mock objects always have a .called, you could assert that this is false17:45
*** Goneri has joined #openstack-ironic17:45
*** dhellmann_ has joined #openstack-ironic17:46
JayFmat128: yeah; I alreayd pushed up a new patch with the test fixes17:47
*** dhellmann_ has quit IRC17:47
JayFjust think it got eaten by the netsplit17:47
*** openstackgerrit has joined #openstack-ironic17:48
*** agordeev has joined #openstack-ironic17:49
*** dhellmann has quit IRC17:50
*** trown|lunch is now known as trown17:50
*** hamzy has joined #openstack-ironic17:50
*** Sukhdev has joined #openstack-ironic17:51
*** serverascode has joined #openstack-ironic17:51
*** yarkot1 has joined #openstack-ironic17:51
*** jistr has quit IRC17:51
*** boris-42 has joined #openstack-ironic17:53
*** sergek has quit IRC17:54
*** agordeev has quit IRC17:54
*** dhellmann has joined #openstack-ironic17:55
*** yonglihe has quit IRC17:55
*** sergek has joined #openstack-ironic17:56
*** dhellmann has quit IRC17:57
*** raginbajin has quit IRC17:57
openstackgerritGonĂ©ri Le Bouder proposed openstack/ironic: iscsi: wipe the disk before deployment  https://review.openstack.org/30360517:57
*** dhellmann has joined #openstack-ironic17:57
*** ndipanov has joined #openstack-ironic17:58
jrolldtantsur|afk: forgot to tell you, we can't have multiple release liaisons in newton, so I'll have to ack inspector releases17:58
jrollhttp://lists.openstack.org/pipermail/openstack-dev/2016-April/092459.html17:58
*** agordeev has joined #openstack-ironic17:58
jrollspeaking of releases, I think I'd like to release some things next week before summit, to get us in the habit of frequent releases17:58
*** vdrok has joined #openstack-ironic17:59
jrolljust throwing that out there :)17:59
*** cppforlife_ has joined #openstack-ironic17:59
*** yonglihe has joined #openstack-ironic18:00
*** ravig_ has joined #openstack-ironic18:00
*** jayc has quit IRC18:01
*** causten__ has joined #openstack-ironic18:01
*** raginbajin has joined #openstack-ironic18:02
ravig_NobodyCam, TheJulia: Can we discuss on this spec: https://review.openstack.org/#/c/294995/18:03
*** Marga_ has joined #openstack-ironic18:03
*** vishwanathj has joined #openstack-ironic18:03
NobodyCamravig_: I'm on a conference call ATM.18:03
devanandajroll: ++ to frequent point releases18:04
ravig_NobodyCam: Can you please let me know, whenever you are free18:04
*** alex_xu has quit IRC18:04
*** causten_ has quit IRC18:04
devanandajroll: also, I'd like to chat about client version stuff when you have time. perhaps at the summit, perhaps before18:05
*** gabriel-bezerra has joined #openstack-ironic18:05
*** alex_xu has joined #openstack-ironic18:07
*** vishwanathj_ has quit IRC18:07
vdrokgood night everyone!18:08
NobodyCamnight vdrok18:08
*** Goneri has quit IRC18:09
TheJuliaravig_: NobodyCam and I should be off the phone in about 45 minutes18:13
ravig_TheJulia: Cool thanks :)18:14
*** robcresswell has quit IRC18:14
*** robcresswell has joined #openstack-ironic18:15
*** e0ne has joined #openstack-ironic18:19
*** thrash has joined #openstack-ironic18:23
*** thrash has quit IRC18:23
*** thrash has joined #openstack-ironic18:23
JayFhttps://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L301 is there a reason we don't check for node_resources_unavailable() before node_resources_used() in the nova driver/18:24
JayFI think it'd resolve a bug we're seeing downstream18:24
JayFhttps://bugs.launchpad.net/nova/+bug/1503453 bug  to be specific18:24
openstackLaunchpad bug 1503453 in OpenStack Compute (nova) "unavailable ironic nodes being scheduled to" [Medium,Triaged]18:24
JayFjroll: ^ wdyt18:24
JayFMadasi: ^ that's your bug iirc18:25
*** sinval has joined #openstack-ironic18:26
Madasithat's a good question18:29
Madasinothing jumps out at me as obvious why that wouldn't work18:29
Madasihmm, the code has changed enough that the reference from the bug isn't right anymore. Should have linked to a specific commit instead of master I suppose18:32
Madasithat, or it's been so long since I looked at that code that is doesn't look familiar anymore. Not sure which18:34
*** Sukhdev has quit IRC18:37
MadasiJayF: I think swapping the order might catch cases where the orphaned node is also in maintenance, but not others18:38
Madasiheh, I actually considered that18:38
Madasihttps://bugs.launchpad.net/nova/+bug/1503453/comments/118:38
openstackLaunchpad bug 1503453 in OpenStack Compute (nova) "unavailable ironic nodes being scheduled to" [Medium,Triaged]18:38
JayFMadasi: what's the harm in -$total_ram / 018:39
JayFMadasi: does it actually try to provision it back to 0?18:40
MadasiI think the concern was that swapping the order might also cause https://bugs.launchpad.net/bugs/150217718:40
openstackLaunchpad bug 1502177 in OpenStack Compute (nova) "Existing Ironic instances report negative available RAM for the node after upgrade " [High,Fix released] - Assigned to Jim Rollenhagen (jim-rollenhagen)18:40
JayFlame :(18:40
JayFjcook: ^18:40
Madasinot following where the / 0 comes in18:41
jcookNova won't see an instance on the node if it's an orphaned instance uuid18:42
openstackgerritAndre Aranha proposed openstack/python-oneviewclient: Extend python-oneviewclient to Uplink Set  https://review.openstack.org/29598118:43
Madasiright, and that causes nova to "fix" the reported resources available, and try to schedule to that node, which ironic will fail b/c it sees an instance already there18:43
anteayadoes anyone know if any ironic summit sessions with be discussing third party ci?18:44
anteayaI didn't see any on the ironic schedule18:44
jcookMadasi: it'll set the available and used? I thought only the available18:44
jcookMadasi: oh then it'll say it's using more than available and thusly total negative?18:44
openstackgerritAndre Aranha proposed openstack/python-oneviewclient: Extended client to modify Connections  https://review.openstack.org/30236518:45
jcookMadasi: wouldn't it be more appropriate to set available to min of memory_mb and size of server?18:46
TheJuliaravig_: So, brains are now available, whats up?18:46
anteayaI'm trying to sort out my schedule and at present only nova seems to have a session where they plan on discussing third party ci and i just don't believe it18:46
anteayathat they would be the only project18:46
NobodyCamYes /me is also back18:47
Madasijcook: I think the usage gets changed here - https://github.com/openstack/nova/blob/a3da334656bac94967af58d1b9a95dbb26ebf44b/nova/compute/resource_tracker.py#L823-L82418:48
Madasibut it's been quite a while, I'd have to sit down and try to wrap my head around the code again to give good answers18:48
jcookMadasi: memory is set to 0 for error states?18:49
ravig_TheJulia: I have a question regarding the https://review.openstack.org/#/c/294995/ : Regarding the scenario 3, where we are booting from rbd, are we planning to have rbdroot parameter in kernel parameters in pxefile18:49
jcookMadasi: neutron_failed == no schedule, neutron_failed with instance uuid == schedule; neutron_failed is preventing build alone by, I'm guessing, setting memory to 018:50
*** Goneri has joined #openstack-ironic18:50
TheJuliaravig_: That was how I anticipated it working18:50
TheJuliaravig_: since the only other way I think that could work would be to assemble ramdisk to bootstrap the machine into using a value burned into the ramdisk18:51
Madasijcook: by nova or by ironic? ironic does it here for error https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L18118:51
jcookMadasi: also just set this to min(0, self.compute_node.memory_md - self.compute_node.memory_mb_used): https://github.com/openstack/nova/blob/a3da334656bac94967af58d1b9a95dbb26ebf44b/nova/compute/resource_tracker.py#L84518:51
jcookMadasi: what you just linked18:52
Madasigood luck getting nova to go along with that change :)18:52
Madasiand i'm not positive it fixes both cases18:52
Madasiwould have to think about it more18:52
jcookeither that or set self.compute_node.memory_mb_used to max(self.compute_node.memory_mb_used, self.compute_node.memory_mb)18:53
Madasiwouldn't that always show all the ram used?18:54
ravig_Since we rely it on kernel parameter, wouldn't that cause problem - recently we faced a problem, when we are using RHEL with syslinux as bootloader after the update, it stopped updating few parameters18:55
jcooksorry, that should be a min18:55
anteayaNobodyCam: do you know of any ironic design summit sessions that will be talking about third party ci?18:55
MadasiI suspect that would break several things around reserved memory for hypervisor overhead in nova18:56
*** Goneri has quit IRC18:56
Madasibut might be wrong18:56
*** ametts has quit IRC18:56
jcookMadasi: what does memory look like for error state?18:56
jcookavailable = 0 used = 0?18:56
*** dprince has quit IRC18:57
TheJuliaravig_: So, the downside of pxe booting is that the kernel/initial ramdisk may go out of sync what is on the actual "disk" as time goes on, could that be the cause of what you experienced?  I'm only thinking that we wouldn't be loading another bootloader really, we would be loading the intended kernel and ramdisk, and wouldn't be able to try and read the ceph volume to load the bootloader that m18:57
ravig_TheJulia: Instead can we update the bootdisk name in ipxe file. Do you think it makes sense - But this may cause one ipxe file for each node...18:57
TheJuliaay be on the volume18:57
jcookMadasi: https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L313 memory_mb is definitely 018:57
TheJuliaravig_: there would be an ipxe file per noe anyway18:57
TheJulias/noe/node/18:57
jcooknot sure what used is18:57
NobodyCamanteaya: not specially on that topic. here is the planning ether pad: https://etherpad.openstack.org/p/ironic-newton-summit18:58
anteayaNobodyCam: thank you18:58
Madasiif there is an instance, we will return all the memory as used, regardless of error state18:58
jcookMadasi: so that negative RAM scenario is true right now for errored nodes?18:59
Madasiif there isn't an instance, then we look at maintenance, power state, and provision state and report 0 present for the node18:59
anteayaNobodyCam: apparently it is an item on the Gate session19:00
Madasijcook: no, was only true when we switched over from looking at node resources to instance_info resources, and only for pre-existing builds19:00
NobodyCamah yes that does have a section fot 3rd party test19:01
Madasibut the concern was the swapping to order of checks from instance and then error/maintenance state to the other way around could introduce the same effect19:01
Madasiwhere we report 0 resources present for the node, the nova sees the instance and subtracts what it is using from the present amount, causing -instance size resources available19:01
anteayaNobodyCam: thank you19:01
anteayaI figured you would have something, I just didn't want to find it at the last minute19:02
NobodyCamanteaya: see the linked ehterpad in 3.1.6.319:02
JayFMadasi: I wonder if we could get the 0 resources available special-cased in the generic RT code19:02
JayFMadasi: that seems like an easier change to get in than changing the driver api signatures19:02
MadasiJayF: how? by having nova check ironic states?19:03
jcookMadasi, JayF: well, I think then putting a min in here might be the right way to handle it: https://github.com/openstack/nova/blob/a3da334656bac94967af58d1b9a95dbb26ebf44b/nova/compute/resource_tracker.py#L84519:03
JayFyep19:03
anteayaNobodyCam: yeah that was from Tokyo19:03
anteayaNobodyCam: thanks19:03
JayFMadasi: jcook ^ just suggested what I was suggesting, in a more understandable way19:03
JayFMadasi: tl;dr: if self.compute_node.memory_mb == 0; don't erase that stuff, dammit!19:03
JayF:)19:03
JayFMadasi: or making _used min(0)19:04
ravig_TheJulia: Let me talk through the scenario. Please correct me if I am wrong - iPXE loads kernel and initramdisk with some parameters related to rbdroot and rbd volume will be booted from sanboot or sanhook19:04
Madasiwouldn't a min always return 0 or negative?19:04
MadasiI'm not following your logic here19:04
jcookmin(0, memory - used)19:04
jcookerm max19:04
TheJuliaravig_: ipxe does not know about rdb volumes, so it wouldn't use sanboot19:04
JayFmake 0 the minimum that value can ever be set to19:05
JayFwhich I guess means using max like jcook said19:05
jcookyeah, w/e that code is lol19:05
JayFthat code is like, the source of 50% of our scheduling errors19:05
Madasiyeah, that makes more sense, essentially just setting 0 as the floor19:05
jcookyes19:05
JayFI mean, lets try to get that change in then?19:05
jcook+119:05
Madasibut that assumes nobody else is using that for something special, but who knows19:05
Madasithat seems like it would be silly19:05
JayFI mean, Madasi, let code review handle that19:05
jcooksubtracting and getting a negative number seems wrong in this case, and if it's being used like that it's a hack19:06
JayFI mean, does someone wanna push that up? I'll give jcook or Madasi the chance since you all had the idea / wrote the original bug19:06
Madasibut not sure that setting 0 as a min fixes the orphan problem19:06
JayFMadasi: you swap the checks AND force that as a min19:06
Madasiah, ok19:06
jcookit doesn't fix the orphan it fixes scheduling to an orphan19:06
Madasiso a cross project code fix :)19:07
*** Sukhdev has joined #openstack-ironic19:07
JayFno19:07
JayFthe ironic driver is in nova19:07
ravig_TheJulia: So, thats what I was wondering too, so in a way, will it be inside the OS's bootloader which is on rbd volume?19:07
Madasioh, true19:07
JayFso the whole patch would be in nova19:07
Madasiforgot that for a min19:07
TheJuliaravig_: It loads the kernel into ram along with the ramdisk and initiates boot of those two items, the command line parameter that gets passed along to the kernel is read by the ramdisk, and it's initscripts, if they support it, would then mount the rdb volume as root19:07
JayFMadasi: giving you the chance to write the upstream patch, I'll help with review comments if you can get it up19:07
TheJuliaravig_:So, the RDB volume's bootloader, if it has one, cannot be used, only the ramdisk can bridge the gap for instance, if that makes sense19:08
MadasiWould like to, but I won't have bandwidth for that for a while19:09
*** lennyb has quit IRC19:09
jcookJayF: I'll make the change19:09
*** afaranha has quit IRC19:10
TheJuliaravig_: sorry, RBD19:10
JayFjcook: awesome; I'll offer you the same deal, if you want I can help fixup after review comments and all19:11
jcookJayF: cool, I'll try to knock that out right mewo19:12
jcookmeow*19:12
TheJuliaspeaking of cats... *looks down to an angry cat in her lap*19:13
*** mgoddard has quit IRC19:14
*** keedya has joined #openstack-ironic19:15
*** rama_y has quit IRC19:15
*** sinval has quit IRC19:15
*** aginwala has joined #openstack-ironic19:17
*** anush has quit IRC19:18
*** akrish9 is now known as anush19:18
*** Marga__ has joined #openstack-ironic19:19
*** aginwala_ has joined #openstack-ironic19:19
*** suro-patz has joined #openstack-ironic19:20
*** serverascode_ has joined #openstack-ironic19:22
*** bcornec1 has joined #openstack-ironic19:22
*** serverascode has quit IRC19:22
*** e0ne has quit IRC19:22
*** ChrisMead has quit IRC19:22
*** yarkot has quit IRC19:22
*** maurosr has quit IRC19:22
*** bcornec has quit IRC19:22
*** suro-patz1 has quit IRC19:22
*** maurosr has joined #openstack-ironic19:22
*** Marga_ has quit IRC19:22
*** aginwala has quit IRC19:22
*** aginwala_ has quit IRC19:23
*** maurosr has quit IRC19:23
*** maurosr has joined #openstack-ironic19:23
*** serverascode_ is now known as serverascode19:23
*** e0ne has joined #openstack-ironic19:24
*** piet has joined #openstack-ironic19:24
*** xavierr has quit IRC19:25
*** aginwala has joined #openstack-ironic19:27
*** raildo is now known as raildo-afk19:27
*** ChrisMead has joined #openstack-ironic19:31
*** e0ne has quit IRC19:31
jcookJayF: is there an existing bug somewhere for this in launchpad?19:32
JayFjcook: I linked it above19:32
JayF150345319:33
JayFhttps://bugs.launchpad.net/nova/+bug/1503453 bug  to be specific19:33
openstackLaunchpad bug 1503453 in OpenStack Compute (nova) "unavailable ironic nodes being scheduled to" [Medium,Triaged]19:33
JayFthere's the full url19:33
*** ravig_ has quit IRC19:33
* JayF afk for a couple of hours19:34
jcookgot it thanks19:35
*** Sukhdev has quit IRC19:36
*** ametts has joined #openstack-ironic19:42
*** achanda has joined #openstack-ironic19:42
*** baoli has quit IRC19:45
*** aginwala has quit IRC19:48
*** piet has quit IRC19:50
NobodyCamvdrok: are you by chance look'n to that time out issue ?19:52
*** rloo has joined #openstack-ironic19:52
*** aginwala has joined #openstack-ironic19:53
*** raildo-afk is now known as raildo19:55
*** Goneri has joined #openstack-ironic19:55
*** aginwala has quit IRC20:01
*** cfarquhar has quit IRC20:03
*** aginwala has joined #openstack-ironic20:04
*** aginwala has quit IRC20:04
*** aginwala has joined #openstack-ironic20:05
*** achanda has quit IRC20:05
*** cfarquhar has joined #openstack-ironic20:06
*** cfarquhar has quit IRC20:06
*** cfarquhar has joined #openstack-ironic20:06
*** baoli has joined #openstack-ironic20:07
*** vishwana_ has joined #openstack-ironic20:09
*** vishwanathj has quit IRC20:11
*** raildo is now known as raildo-afk20:11
*** suro-patz has quit IRC20:12
*** raildo-afk is now known as raildo20:13
*** vishwanathj has joined #openstack-ironic20:15
*** vishwanathj has quit IRC20:15
*** vishwanathj has joined #openstack-ironic20:16
*** vishwanathj has quit IRC20:16
*** vishwanathj has joined #openstack-ironic20:17
*** vishwana_ has quit IRC20:20
*** Nisha_away has joined #openstack-ironic20:20
*** aginwala has quit IRC20:20
*** aginwala has joined #openstack-ironic20:21
*** ChrisMead has quit IRC20:21
*** aginwala has quit IRC20:22
jrolljcook: JayF: sounds like y'all got your question sorted-ish?20:23
*** aginwala has joined #openstack-ironic20:23
jrollI only skimmed scrollback20:23
jcookjroll: well, I think there is an existing bug20:24
*** Nisha_away has quit IRC20:24
*** ChrisMead has joined #openstack-ironic20:24
*** trown is now known as trown|outtypewww20:24
jrolljcook: sure, I get that20:25
jcookjroll: if you maintenance nodes with instances on them, you'll have memory = 0 and used = all20:25
jcookjroll: no, I mean another bug in this code20:25
*** aginwala has quit IRC20:25
jrolljcook: does that cause problems or is just incorrect>20:26
jroll?20:26
jcookI *think* you could end up with negative memory just like https://bugs.launchpad.net/nova/+bug/150217720:26
openstackLaunchpad bug 1502177 in OpenStack Compute (nova) "Existing Ironic instances report negative available RAM for the node after upgrade " [High,Fix released] - Assigned to Jim Rollenhagen (jim-rollenhagen)20:26
*** aginwala has joined #openstack-ironic20:27
jcookIt looks like the resource tracker takes the total memory and then subtracts every instance from it (+ overheads which for ironic should generally be 0, I would think)20:27
jrolloooo20:27
jrolland if your total for all nodes is negative it gets real problematic20:27
jcookI was trying to avoid doing that with the fix for scheduling to orphan instances, but I think it's already possible20:27
jrollso the original question is what if we swap the used/unavailable check20:27
jrollwhich makes sense at a glance20:28
jrollyeah, sounds like it20:28
jcookwell, I just droped the 'el' from 'elif', but yeah20:28
jcookdropped*20:28
jrollmmm20:28
*** mkovacik has joined #openstack-ironic20:28
jcookso, I'm still looking at it20:28
jrollcool20:29
jrollthis is easy to test :)20:29
*** Goneri has quit IRC20:29
jcookI'm not positive where total memory is added up, but it looks like it's pulled from db20:29
*** ijw has joined #openstack-ironic20:29
*** achanda has joined #openstack-ironic20:29
NobodyCamso fyi looking into the timeout issue I'm seeing things come to a strange end here https://github.com/openstack/ironic/blob/master/ironic_tempest_plugin/tests/scenario/test_baremetal_basic_ops.py#L78 at least in this test run: http://logs.openstack.org/58/272658/38/check/gate-tempest-dsvm-ironic-pxe_ssh/9462e59/console.html#_2016-04-15_15_17_49_95020:30
jrolljcook: well, I'm not sure if a regular deployment of nova+ironic would be broken here, but in a cells configuration, it would drop at the cells level when it sees that cell has no capacity20:31
jrollNobodyCam: oooooo20:32
jrollvdrok mentinoed something similar earlier20:32
jcookin ironic the compute memory == the memory of every node under that compute?20:33
jrollNobodyCam: so it's somewhere here: https://github.com/openstack/ironic/blob/master/ironic_tempest_plugin/tests/scenario/test_baremetal_basic_ops.py#L122-L12620:33
jrolljcook: erm, "it depends"20:33
*** aginwala has quit IRC20:34
vdrokNobodyCam: nope I have not looked yet, but it seems to be something during SSH command run/initialization20:34
jcookthe sum of the reported by every node*20:34
*** alex_xu has quit IRC20:34
jrolljcook: so a resource tracker object is actually created for every (host, node) combination (where host == compute service, node == "compute node" e.g. ironic node)20:34
*** achanda has quit IRC20:34
jrolljcook: and there's a row in the compute_nodes table for every (host, node) combination20:34
jcooko_o20:34
jrollwhich are iterated over during scheduling20:35
jrollyeah dude, nova be cray20:35
jcookthat's super helpful20:35
*** ijw has quit IRC20:35
jcookand answers a huge '?' I had20:35
jrollheh, yeah it was enlightening when I learned that20:35
*** suro-patz has joined #openstack-ironic20:35
jcookcool, so this makes this waaaaaaay easier to fix20:35
*** alex_xu has joined #openstack-ironic20:36
jrolljcook: just remember in cells, it does evaluate the sum of all nodes for choosing a cell to schedule to20:36
*** mkovacik_ has joined #openstack-ironic20:36
jcookjroll: resource tracker runs at global level too?20:37
*** mkovacik has quit IRC20:37
jcookjroll: or just in the scheduler?20:37
jcookerm20:37
jcooknova-cells global not scheduler20:38
jrolljcook: nope, only in computes. RT is what populates compute_nodes table. scheduler and cells services just read that table20:38
jcookso nova cells does a sum(memory) - sum(used)?20:38
jcookI'm guessing in the filters20:38
jrollso child cells service reads that table for totals, and passes up a dict of {ram_for_flavor: number_of_slots, (continue for each flavor)}20:39
jrolland global cells just uses number of slots for the requested flavor in each cell, to decide which cell20:39
jcookI'm familiar with that bit20:40
jrollhttps://github.com/openstack/nova/blob/master/nova/cells/state.py#L20920:40
jrollright20:40
jcookme trying to think how that is calculated for ironic20:40
jrollhere's the code that sums things: https://github.com/openstack/nova/blob/master/nova/cells/state.py#L25620:40
jrollthat iterates over each row in compute_nodes20:41
jcookah20:41
jcookso *maybe* that needs a change20:41
*** baoli has quit IRC20:41
jrollI guess I'm lost as to what bug you're fixing at this point :P20:41
jcookso if a node has an instance it reports all used20:42
jcookand if it's in a "bad" state i returns none available20:42
jcook0 - used = negative number20:42
jcookIronic uses bad state to prevent scheduling to it20:43
NobodyCamvdrok: see mine and jroll's comments just above20:43
jrolljcook: right, so you want to make it such that it reports x total, x used, 0 available?20:44
jcookultimately I want it not to schedule orphans :P20:44
jrollaha20:44
jrollorphan being ironic thinks it has an instance, but it does not?20:44
jcookyeah20:45
jrollright, so this is the really fun part20:45
jcookreservations20:45
jcook?20:45
jrollif we tell the resource tracker that the resources are used20:45
jrollbut there is no instance in nova related to that node20:46
jcookit "fixes" it20:46
jcookyeah..20:46
jrollit will correct us and say there are ... yeah20:46
jrollthere is a virt driver method to help correct this20:46
jrollI forget what it's called20:47
jrollbut it does not take node into account, only host20:47
jrolland so is completely useless to us20:47
jrolland nova is not interested in adding node to this20:47
jcookyeah, I heard a bit about that >_>20:48
jrollMadasi originally tracked that down20:48
jcookI *think* I *might* know how to fix this **maybe**20:48
jcookwithout changing apis etc20:48
jrollI'd be very curious to see that :)20:48
jcookwe'll see :)20:48
*** suro-patz1 has joined #openstack-ironic20:49
*** jjohnson2_ has quit IRC20:49
*** suro-patz has quit IRC20:49
*** jjohnson2_ has joined #openstack-ironic20:50
Madasijcook: are you saying you think maintenanced nodes w/ instance could have negative ram reported now, or after swapping that if/elif logic?20:55
Madasithey shouldn't now20:55
*** Sukhdev has joined #openstack-ironic20:56
NobodyCamah ha20:56
NobodyCamjroll: vdrok: TheJulia: http://paste.openstack.org/show/WGdfWq34V3KQFtvLOXIh/  <- this could be what's going on20:57
*** ametts has quit IRC20:58
NobodyCamthis is the line we are failing on: https://github.com/openstack/ironic/blob/master/ironic_tempest_plugin/tests/scenario/test_baremetal_basic_ops.py#L7820:58
*** lascii is now known as alaski21:01
jrollNobodyCam: why would that make it hang, though? are we sure client.exec_command uses os.system under the hood?21:01
NobodyCamI am not sure was just testing things21:02
jrollgotcha21:02
jrollI'm going to poke my head in -qa as well21:03
*** ijw has joined #openstack-ironic21:03
NobodyCam:)21:04
*** jjohnson2_ has quit IRC21:06
jcookMadasi: when I started convo, I thought now; but yeah after changing that logic21:11
Madasiah, yeah21:11
jcookMadasi: I'm gonna ponder while I push my little one on the swing. I think I know how to fix.21:12
Madasithat was the thing that made just swapping the order not an easy fix. The floor to the available ram should mitigate that though21:12
jcookyeah, I think that might be sufficient, wanna make sure though21:12
Madasibut I'm not sure if there are any other consequences from marking 0 ram on nodes with instances that we don't know about yet21:12
Madasibut yeah, it's certainly farther along to a fix than I got21:13
jcookwe'll see21:13
Madasiand doesn't take an api change, which is great21:13
jcookyeah +1 to that21:13
* JayF glad to have a manager that can be nerd-sniped into fixing longstanding bugs21:13
MadasiJayF: definitely a feature21:14
JayFreminder: if you're going to the summit and want to be a part of group hangouts for ironic, etc, please add your info here: https://etherpad.openstack.org/p/ironic-summit-arrivals21:15
MadasiIf I wasn't so busy standing up new servers, I'd have loved to run with it, but it's better to get it done than wait around for me to get free time21:15
MadasiGoing to have to miss the mini pub crawl on Sunday, a bit bummed about that21:15
JayFI basically get in at 1am monday mornign / sunday night21:16
JayFbut in exchange I got a nonstop flight both ways21:16
JayF#worthit21:16
Madasimay have to make time to hit up austin eastciders on my own time21:16
Madasinonstops are certainly nice21:16
Madasii must agree21:16
Madasihmm, should maybe make a trip to Meridian Hive Meadery too. Too many good places I wouldn't mind hitting up21:17
Madasiwhich is ironic, since I didn't go to them while I lived there :)21:18
JayFno, that's not ironic21:18
JayFironic is software for provisioning bare metal nodes as part of an openstack cloud21:18
JayFor a great song without any irony (see, it's ironic?) from the 90s21:18
*** raildo is now known as raildo-afk21:18
JayFjcook: paranoid about spam much? lol21:20
jcookyes21:20
jcook:P21:20
jrollit blows my mind that people using gmail are worried about spam :P21:21
jrollfilters are pretty good these days21:21
jrollNobodyCam: so turns out that create_timestamp() thing calls sync and umount21:22
jrollwhich is probably what's hanging21:22
jrollbut disk io looks okay on that particular run, unless there's a neighbor crushing the disks21:22
openstackgerritChris Krelle proposed openstack/ironic: Test fix for tests  https://review.openstack.org/30663421:22
NobodyCamjroll: ahh21:22
jrollI need to head out for now, have a good weekend y'all21:30
NobodyCamnight jroll21:31
Madasio/21:32
*** lenny has joined #openstack-ironic21:32
devanandag'night, jroll o/21:32
*** Sukhdev has quit IRC21:52
*** piet has joined #openstack-ironic21:53
*** raginbajin has quit IRC22:00
*** vishwanathj has quit IRC22:01
*** raginbajin has joined #openstack-ironic22:05
*** jjohnson2_ has joined #openstack-ironic22:06
*** dhellmann has left #openstack-ironic22:07
*** alex_xu has quit IRC22:13
*** alex_xu has joined #openstack-ironic22:15
*** KennethWilke has quit IRC22:16
*** karimb has quit IRC22:16
*** KennethWilke has joined #openstack-ironic22:17
*** odyssey4me has quit IRC22:22
*** rloo has quit IRC22:25
*** odyssey4me has joined #openstack-ironic22:27
*** piet has quit IRC22:30
*** baoli has joined #openstack-ironic22:30
*** jjohnson2_ has quit IRC22:41
*** baoli has quit IRC22:48
*** suro-patz1 has quit IRC22:49
jlvillalNobodyCam: What kind of bug are you seeing in the test?22:52
jlvillalSeems like exec_command should already be returning a string.22:52
NobodyCamhey hey jlvillal22:52
* jlvillal is just being curious22:53
NobodyCamya I think jroll found the real answer22:53
jlvillalOkay good :)22:53
NobodyCamsee his comment at 2:22 pst22:53
jlvillalNobodyCam: Thanks22:53
*** Sukhdev has joined #openstack-ironic22:55
*** suro-patz has joined #openstack-ironic23:04
*** ppiela has quit IRC23:12
*** [1]cdearborn has quit IRC23:14
*** Sukhdev has quit IRC23:25
*** davideag_ has quit IRC23:31
*** tonyb_ is now known as tonyb23:33
*** piet has joined #openstack-ironic23:36
*** piet has quit IRC23:52

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!