Thursday, 2014-07-03

openstackgerritJosh Gachnang proposed a change to openstack/ironic: Allow noauth for Neutron  https://review.openstack.org/9892200:01
JoshNangrloo_: updated that patch :) sorry for the lag, was in a meeting. i think i did lose things in that spec (local got out of sync it seems). i'll fix it tonight/tmr morning. thanks for the reviews!00:01
faizanDevananda: Its regarding nova ironic virt driver changes, where we need to pass the boot options to the ironic node via instance_info field.00:05
faizanThere is one more change where nova scheduler should pick the ironic node which support a specified boot mode.00:07
faizanDevananda: wanted to know how best we can make these changes.00:09
faizanDevananda: I can share you the code changes required in nova virt dirver. I have described them in the uefi spec patch00:13
NobodyCamdevananda: you want to land that00:15
NobodyCam??00:15
NobodyCamoh brb00:16
devanandaNobodyCam: it's a bug fix for a High impact bug (rebuild unable to update image, tripleo can't upgrade nodes)00:17
devanandafaizan: if you have something I can look at later, that'd be great -- perhaps post a link to code samples on the spec review00:17
devanandafaizan: and I'll try to get to it again later this week00:18
faizandevananda: thanks I will post a link to the code samples later today.00:19
*** chuckC has quit IRC00:19
faizanDevananda: Is it ok if I bug you for one more review comment?00:21
*** ramineni has joined #openstack-ironic00:21
faizanDevananda: Its regarding - "IPMI does not support changing/querying the boot type"00:24
NobodyCamdevananda: +2 but not +a'd00:24
NobodyCamnow off to Dairy Queen to pick up dinner00:27
devanandafaizan: what's the question? I'll answer when possible00:28
*** overlayer has quit IRC00:32
raminenidevananda: Hi00:33
raminenidevananda: did you get a chance to look at firmware update design spec - https://review.openstack.org/#/c/100842/00:34
devanandaramineni: if I did, I would have posted a review on it. otherwise, no, I haven't had time to review everything00:35
raminenidevananda: any comments would be helpful.00:35
raminenidevananda: ok00:37
faizandevananda: ipmitool dont have the capability to change the boot mode, but other vendors can do this with their tools like iLO, Thats why I have suggested to add dummy vendor pass through in ipmitool vendor passthrou, which will return failure if current and next boot modes are different.00:39
devanandafaizan: can ipmitool *check* the current boot mode?00:40
devanandafaizan: if not, then it seems like that won't work, and you should clearly state that this isn't supported by ipmitool00:40
faizanThis way we can have different class of drivers, like pxe_ilo, where we can use ilo capability to change boot mode and use pxe deploy00:40
jrollwhat do we mean by 'boot mode' here?00:41
devanandajroll: BIOS // UEFI00:41
jrollah00:41
* jroll hides again00:41
devanandaheh00:41
faizanipmitool does not have the capability to query the bootmode either, but we can add the current boot mode and the next boot mode of the node as the node property00:43
devanandafaizan: that doesn't make sense00:43
devanandafaizan: if the driver doesn't support it, a request to get the current boot mode should error00:44
faizanand pxe driver can check this node property before sending a request to vendor pass through for making the switch00:44
devanandafaizan: and similarly a request to change the boot mode should error00:44
faizanI agree with that,00:50
faizandevananda: Instead of doing get/set of bootmode, why can't we make these node properties static? and let admin fill those fields while creating the nodes00:56
*** ccrouch has joined #openstack-ironic00:59
faizandevananda: and also leave it to the respective vendor driver, if they have the capability to get/set the boot mode let them implement and update the node properties on the fly00:59
*** rloo_ has quit IRC01:05
*** chuckC has joined #openstack-ironic01:07
*** eghobo has quit IRC01:13
*** nosnos has joined #openstack-ironic01:13
*** ramineni has quit IRC01:19
*** zdiN0bot has joined #openstack-ironic01:20
*** Haomeng has joined #openstack-ironic01:20
*** nosnos has quit IRC01:20
*** Haomeng|2 has quit IRC01:21
*** faizan has quit IRC01:26
*** zdiN0bot has quit IRC01:57
*** radsy has quit IRC01:59
*** zdiN0bot has joined #openstack-ironic01:59
*** blamar has joined #openstack-ironic02:01
*** killer_prince has quit IRC02:26
*** Sam_S has left #openstack-ironic02:36
*** ramineni has joined #openstack-ironic02:52
*** eghobo has joined #openstack-ironic03:01
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Allow noauth for Neutron  https://review.openstack.org/9892203:02
*** killer_prince has joined #openstack-ironic03:10
*** killer_prince is now known as lazy_prince03:10
*** dwalleck has joined #openstack-ironic03:11
*** zdiN0bot has quit IRC03:25
*** Poornima has joined #openstack-ironic03:35
*** Nisha has joined #openstack-ironic03:52
*** nosnos has joined #openstack-ironic03:52
*** max_lobur has joined #openstack-ironic03:54
*** max_lobur has quit IRC04:05
*** eghobo has quit IRC04:05
*** ajc_ has joined #openstack-ironic04:17
*** yongli has quit IRC04:21
*** blamar has quit IRC04:28
*** k4n0 has joined #openstack-ironic04:39
*** pcrews has quit IRC04:42
*** bvivek has joined #openstack-ironic04:43
*** rakesh_hs has joined #openstack-ironic04:46
*** blamar has joined #openstack-ironic04:50
*** amitpp has joined #openstack-ironic04:55
*** sabah has joined #openstack-ironic05:02
*** blamar has quit IRC05:05
*** rameshg87_afk is now known as rameshg8705:05
*** rameshg87 has quit IRC05:05
*** rameshg87 has joined #openstack-ironic05:06
*** pcrews has joined #openstack-ironic05:11
*** loki184 has joined #openstack-ironic05:14
*** stendulker has joined #openstack-ironic05:18
*** pcrews has quit IRC05:28
*** Penick has joined #openstack-ironic05:45
*** harlowja is now known as harlowja_away05:47
*** dwalleck has quit IRC05:58
*** dwalleck has joined #openstack-ironic05:58
openstackgerritOpenStack Proposal Bot proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/10329106:01
*** wanyen has quit IRC06:02
*** max_lobur has joined #openstack-ironic06:02
*** coolsvap|afk is now known as coolsvap06:09
*** dwalleck_ has joined #openstack-ironic06:22
*** dwalleck has quit IRC06:26
*** lazy_prince is now known as killer_prince06:28
*** jcoufal has joined #openstack-ironic06:31
*** pradipta_away is now known as pradipta06:33
*** killer_prince is now known as lazy_prince06:39
*** Nisha has quit IRC06:40
*** Mikhail_D_ltp has joined #openstack-ironic06:53
*** takadayuiko has joined #openstack-ironic07:20
*** Penick has quit IRC07:23
*** lsmola__ has joined #openstack-ironic07:24
*** ifarkas has joined #openstack-ironic07:26
openstackgerritMichael Davies proposed a change to openstack/python-ironicclient: Expose auth_ref in ironicclient client object  https://review.openstack.org/10446707:28
*** max_lobur has quit IRC07:32
*** lucasagomes has joined #openstack-ironic07:36
*** Mikhail_D_ltp has quit IRC07:37
*** Mikhail_D_ltp has joined #openstack-ironic07:39
*** overlayer has joined #openstack-ironic07:41
*** dwalleck_ has quit IRC07:41
*** dwalleck has joined #openstack-ironic07:42
*** ndipanov has joined #openstack-ironic07:47
*** bvivek has quit IRC07:47
*** dwalleck has quit IRC07:52
openstackgerritDaryl Walleck proposed a change to openstack/ironic: Updates the Ironic on Devstack dev documentation  https://review.openstack.org/10446907:53
*** nosnos has quit IRC08:05
*** nosnos has joined #openstack-ironic08:05
*** derekh_ has joined #openstack-ironic08:05
*** martyntaylor has joined #openstack-ironic08:12
*** athomas has joined #openstack-ironic08:14
*** bvivek has joined #openstack-ironic08:19
*** foexle has joined #openstack-ironic08:24
openstackgerritMichael Davies proposed a change to openstack/ironic: Ironic nova driver to cache ironic client calls  https://review.openstack.org/10269508:31
*** loki184 has quit IRC08:32
mrdadevananda: Just FYI - new version of the Ironic Nova driver auth patch up, along with a dependent review on ironicclient. Chat to you in 13 hours :)08:38
mrdaNight Ironic!08:39
*** mrda is now known as mrda-away08:39
*** jcoufal has quit IRC08:41
*** jcoufal has joined #openstack-ironic08:44
*** pelix has joined #openstack-ironic08:55
*** bvivek has quit IRC09:01
openstackgerritMichael Davies proposed a change to openstack/ironic: Ironic nova driver to cache ironic client calls  https://review.openstack.org/10269509:02
*** loki184 has joined #openstack-ironic09:03
*** loki184 has quit IRC09:04
*** romcheg has joined #openstack-ironic09:11
*** loki184 has joined #openstack-ironic09:17
*** loki184 has quit IRC09:22
*** mitz has quit IRC09:22
*** mitz has joined #openstack-ironic09:26
*** Alexei_9871 has joined #openstack-ironic09:34
*** rakesh_hs has quit IRC09:38
*** rakesh_hs has joined #openstack-ironic09:39
*** lazy_prince is now known as killer_prince09:40
*** killer_prince is now known as lazy_prince09:48
*** bvivek has joined #openstack-ironic09:52
*** overlayer has quit IRC09:55
*** martyntaylor has quit IRC09:55
*** Nisha has joined #openstack-ironic09:59
openstackgerritImre Farkas proposed a change to openstack/ironic-specs: DRAC power driver  https://review.openstack.org/9935210:01
Nishalucasagomes: hi10:12
romchegMorning all!10:13
*** rakesh_hs has quit IRC10:14
*** rakesh_hs has joined #openstack-ironic10:15
*** pradipta is now known as pradipta_away10:22
*** lazy_prince has quit IRC10:55
*** lazy_prince has joined #openstack-ironic10:56
*** geekyogi1 has quit IRC10:57
*** Haomeng|2 has joined #openstack-ironic10:57
*** geekyogi has joined #openstack-ironic10:58
*** Haomeng has quit IRC10:58
*** derekh_ has quit IRC10:59
*** ramineni has quit IRC11:00
*** derekh_ has joined #openstack-ironic11:00
*** geekyogi has quit IRC11:10
*** sabah has quit IRC11:22
*** Poornima has quit IRC11:26
*** stendulker has quit IRC11:27
*** jcoufal has quit IRC11:30
*** jcoufal has joined #openstack-ironic11:30
*** aswadr has joined #openstack-ironic11:47
lazy_princelifeless: hi, there..?11:51
*** mkerrin has quit IRC11:51
*** geekyogi has joined #openstack-ironic11:53
*** nosnos has quit IRC11:55
*** nosnos_ has joined #openstack-ironic11:58
*** mkerrin has joined #openstack-ironic11:59
*** jcoufal has quit IRC12:01
*** jcoufal has joined #openstack-ironic12:04
*** proffalken has joined #openstack-ironic12:16
proffalkenhi all, is https://review.openstack.org/#/c/98506/3/specs/juno/agent-driver.rst the best/most relevant documentation to read when considering how to auto-discover a rack of equipment and get it into the correct state for deployment via TripleO/HEAT?12:18
*** jcoufal has quit IRC12:24
*** jcoufal has joined #openstack-ironic12:24
*** Haomeng has joined #openstack-ironic12:24
*** dtantsur has quit IRC12:24
*** dtantsur has joined #openstack-ironic12:24
*** Haomeng|2 has quit IRC12:25
*** linggao has joined #openstack-ironic12:29
*** ajc_ has quit IRC12:30
*** coolsvap is now known as coolsvap|afk12:31
*** athomas_ has joined #openstack-ironic12:34
*** athomas has quit IRC12:34
*** bvivek has quit IRC12:35
*** zigo has quit IRC12:37
*** zigo has joined #openstack-ironic12:37
*** jbjohnso has joined #openstack-ironic12:38
*** zigo has quit IRC12:41
*** zigo has joined #openstack-ironic12:42
*** k4n0 has quit IRC12:44
*** zigo has quit IRC12:45
*** avozza has joined #openstack-ironic12:50
*** Nisha has quit IRC12:56
*** martyntaylor has joined #openstack-ironic12:58
*** martyntaylor has left #openstack-ironic13:02
*** ramineni has joined #openstack-ironic13:04
*** amitpp has quit IRC13:17
*** pcrews has joined #openstack-ironic13:18
*** overlayer has joined #openstack-ironic13:26
*** matty_dubs|gone is now known as matty_dubs13:28
*** Haomeng|2 has joined #openstack-ironic13:36
*** jdob has joined #openstack-ironic13:36
jrollproffalken: discovery doesn't really exist yet... that will help, this will also help: https://review.openstack.org/#/c/102565/13:37
*** Haomeng has quit IRC13:37
jrollproffalken: but the code doesn't exist13:37
*** nosnos_ has quit IRC13:37
*** Haomeng has joined #openstack-ironic13:40
NobodyCamgood morning Ironic13:41
matty_dubsMorning13:41
jrollmornin13:42
*** Haomeng|2 has quit IRC13:42
* jroll passes around coffee13:42
NobodyCammorning matty_dubs jroll :)13:42
*** rakesh_hs has quit IRC13:42
* NobodyCam takes the coffee and hands out bagels13:42
jroll\o/13:42
NobodyCam:)13:43
NobodyCamis anyone working tomorrow?13:43
jrollnope13:44
matty_dubsNot I.13:44
*** avozza has quit IRC13:48
NobodyCam:-p was just checking :-p13:49
*** nosnos has joined #openstack-ironic13:49
proffalkenjroll: thanks.  It's something I'd be interested in helping develop so I'll give them both a good read.  What's the approach on implementation? Starting writing more specs and get them reviewed?13:53
jrollproffalken: so 102565 is for the generic bits13:54
jrollso like, implement that. then implement it for a certain driver :)13:54
jrollperhaps with another spec; idk13:54
proffalkenfairy nuff13:54
jrollthat one is a bit in flux... might want to wait until that lands or whatever13:54
jrollor chat with dtantsur (he's currently on PTO)13:55
proffalkenkk, that's cool. Just want to know current state and who to talk to before I roll up my sleaves and break out the trance music... ;)13:55
jrollheh, sounds good13:56
*** zigo has joined #openstack-ironic13:58
*** nosnos has quit IRC13:59
*** Mikhail_D_ltp has quit IRC14:00
openstackgerritAndrey Kurilin proposed a change to openstack/python-ironicclient: Remove aliases `arg` and `env` from utils  https://review.openstack.org/10455314:01
*** dkehnx has quit IRC14:02
*** andreykurilin has joined #openstack-ironic14:10
*** ndipanov has quit IRC14:11
*** ndipanov has joined #openstack-ironic14:11
*** rloo has joined #openstack-ironic14:18
*** max_lobur has joined #openstack-ironic14:23
*** dkehnx has joined #openstack-ironic14:25
*** jgrimm has joined #openstack-ironic14:27
*** overlayer has quit IRC14:29
lucasagomeshey all14:33
lucasagomessomeone knows Nisha? he pinged me but I missed it14:33
rloohey lucasagomes14:33
NobodyCammorning lucasagomes14:33
matty_dubsHeya lucasagomes14:33
lucasagomes(I'm in the mid-cycle)14:33
lucasagomeshe pinged me yesterday as well14:33
ifarkasmorning NobodyCam14:33
lucasagomesand I missed too14:33
lucasagomes:(14:33
ifarkasNobodyCam, I have a question regarding https://review.openstack.org/#/c/99352/8/specs/juno/drac-power-driver.rst14:33
NobodyCammorning ifarkas14:33
NobodyCamifarkas: sure14:34
*** jbjohnso has quit IRC14:34
lucasagomesNobodyCam, he works for HP seems14:34
ifarkasNobodyCam, about the default port and protocol, I think Ironic shouldn't set any default value for the driver_info field. If the user doesn't set it, it should not connect. Default values makes sense in case of config options but it's a bit strange for the driver_info field. What do you think?14:35
NobodyCamlucasagomes: I chattted with nisha the other day14:37
linggaomorning14:37
NobodyCamifarkas: ws-man has default ports no?14:38
linggaomorning NobodyCam lucasagomes rloo ifarkas matty_dubs proffalken jroll14:39
ifarkasNobodyCam, yeah, I guess that's a safe assumption. I haven't set up any, just using one.14:39
*** dkehn_ has joined #openstack-ironic14:39
ifarkasmorning linggao14:39
rloomorning linggao14:39
lucasagomesmorning14:39
lucasagomesrloo, I'm currently travelling, I will put another spec for the mgmt interface soon14:40
linggaomorning devananda14:40
lucasagomesrloo, probably next monday14:40
rloolucasagomes: no worries. I'm not in a hurry ;)14:40
lucasagomesaight, sorry for that14:40
NobodyCamifarkas: port = 5985 / #ssl_port = 5986 <- from https://github.com/Openwsman/openwsman14:40
linggaojust like to let you guys/gilrs know that I am going to have a 4-week vacation.14:40
NobodyCammorning linggao14:40
NobodyCammorning rloo14:41
rlooyay linggao. Enjoy!14:41
NobodyCamlinggao: wow I want one too14:41
rloomorning NobodyCam.14:41
ifarkasNobodyCam, that's the client library but not only Dell products are using openwsman and it depends on how you set it up14:41
ifarkasNobodyCam, eg. I am using it with https on port 44314:41
*** dkehn_ has quit IRC14:42
* proffalken waves to everyone who's just turned up14:42
*** dkehn_ has joined #openstack-ironic14:42
linggaoNobodyCam, rloo, devananda, lucasagomes I'll be back the Auguest 4th.14:42
*** dkehnx has quit IRC14:43
linggaoSo you guys have good time for mid-cycle meeting.14:43
*** dkehn_ is now known as dkehnx14:43
NobodyCamifarkas: so the hardware had no default config?14:43
NobodyCams/had/has/14:44
ifarkasNobodyCam, btw, are you suggesting to use a default value when the user didn't provide the proper driver_info fields?14:44
* NobodyCam does not have dell HW14:44
*** jbjohnso has joined #openstack-ironic14:44
ifarkasNobodyCam, I guess but I have never set up any. I just have an remote access to one14:44
linggaoI still have a few patches open https://review.openstack.org/#/q/owner:linggao+status:open,n,z  Please review them while I am on vacation :). I'll take care of them after I am back.14:45
NobodyCamifarkas: I was thinking the hardware would have a default value14:45
NobodyCamie with iLo There is a default port, I can change it, but out of the box 443 works14:46
ifarkasNobodyCam, yeah. should Ironic use that driver_info value even the user didn't provide it?14:46
NobodyCamifarkas: the user had to select the drac driver,14:47
NobodyCamwhy make them enter the default vault id they did change it14:47
openstackgerritMichael Davies proposed a change to openstack/ironic: Ironic nova driver to cache ironic client calls  https://review.openstack.org/10269514:47
NobodyCam*value if *14:47
ifarkasNobodyCam, okay, just wasn't sure14:47
ifarkasif I understand it correctly14:48
NobodyCamya I was asking I do not have drac hw to test with14:48
NobodyCammy thought was the mfg sets a default (or I think they should) and we too can use that default unless the user has changed it for their env14:49
ifarkasNobodyCam, ok, makes sense14:49
ifarkasNobodyCam, according to http://en.community.dell.com/techcenter/systems-management/f/4494/t/19499050.aspx the default values depend on what the user selected on the OME disc wizard14:50
NobodyCamlol14:50
NobodyCamofc14:50
ifarkasNobodyCam, if they select secure the default is 443, if not, the default is 62314:50
*** max_lobur has quit IRC14:50
NobodyCamso we could get have a default port value but not protocol14:50
*** foexle has quit IRC14:51
NobodyCams/get//14:51
ifarkasNobodyCam, I think if secure is selected, then the default port is 443 and the default protocol is https. Otherwise the port is 623 and the port is http14:52
ifarkasNobodyCam, maybe we can use the secure one as a default14:52
NobodyCam:) ok ya :)14:53
ifarkascool ;-)14:53
NobodyCami really cann;t see dell forcing some who ordered 1000 servers to run the wizard on all of them14:53
NobodyCamthou they may14:54
NobodyCam:-p14:54
lucasagomeslinggao, oh great! enjoy ur vacations14:54
linggaolucasagomes, thanks.14:54
*** coolsvap|afk is now known as coolsvap14:55
NobodyCamlinggao: is there someone who is going to take over your on going work while your out enjoying yourself?14:55
*** lazy_prince is now known as killer_prince14:59
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Rename/update common/tftp to drivers/modules/pxe_utils  https://review.openstack.org/10359515:01
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add iPXE support for Ironic  https://review.openstack.org/9931815:01
ifarkasNobodyCam, iiuc, you just need to run it on one machine and it will discover all dracs on the network and set them up15:03
ifarkasNobodyCam, but yeah, running it one by one would be terrible15:03
linggaoNobodyCam, no. I need to take care of them when I am back, if I can remeber what I did. :)15:03
NobodyCamlinggao: as your gone for a month. would you mind if we (the team) made changes / updated to your work if it becomes a blocker for others?15:05
*** max_lobur has joined #openstack-ironic15:06
NobodyCams/updated/updates/15:06
*** mmitchell_ has quit IRC15:06
linggaoNobodyCam, feel free to modify them.15:09
NobodyCamlinggao: TY :)15:10
linggaoNobodyCam, have a nice summer!15:11
devanandamorning, all15:11
NobodyCamgood morning devananda :)15:11
linggaomorning devananda15:11
*** mmitchell has joined #openstack-ironic15:11
linggaodevananda, I just told the team that I will have a 4-week vacation. Will be back on Auguest 4th.15:12
*** hemna_ has joined #openstack-ironic15:20
*** ifarkas has quit IRC15:22
*** lynxman has quit IRC15:24
*** ifarkas has joined #openstack-ironic15:24
*** lynxman has joined #openstack-ironic15:25
devanandalinggao: nice! have a great time :)15:25
linggaoDeva, thanks. enjoy the summer.15:27
openstackgerritImre Farkas proposed a change to openstack/ironic-specs: DRAC power driver  https://review.openstack.org/9935215:29
*** hemna_ has quit IRC15:30
lucasagomesmorning devananda15:30
*** jcoufal has quit IRC15:31
ifarkasmorning devananda15:33
ifarkasdevananda, a question regarding https://review.openstack.org/#/c/90126/7..12/ironic/common/disk_partitioner.py15:34
ifarkasdevananda, if you check the 7th rev, the _wait_for_disk_to_become_available method was inline and you suggested to make it an instance method on the class because it's easier to test. in the 12th revision, you suggested to convert it back to inline.15:34
ifarkasdevananda, I am a bit confused, maybe I did not understand something correctly?15:35
*** dwalleck has joined #openstack-ironic15:36
devanandaifarkas: .... oops. you're aboslutely correct. sorry about that15:37
ifarkasdevananda, ok. so it should be inline, right?15:38
devanandaifarkas: no - leave it as a separate method. it is better to test it that way.15:39
ifarkasdevananda, ack, thanks ;-)15:39
*** hemna_ has joined #openstack-ironic15:42
*** jbjohnso has quit IRC15:42
*** jbjohnso has joined #openstack-ironic15:43
*** jbjohnso_ has joined #openstack-ironic15:43
Shrewsdevananda: So, wrt disk_available_least, it looks like this value is used only for live migrations doing block migration. AFAIK, ironic doesn't support that, so it may be enough to just set this value to 0.15:44
Shrewsor set it to the same value as free_disk_gb15:44
Shrewsthe last may be more correct15:45
Shrewsor not... confusing option  :/15:45
*** jbjohnso has quit IRC15:48
*** geekyogi has quit IRC15:48
*** stendulker has joined #openstack-ironic15:48
devanandaShrews: right. but I think it has more to do with the *host* that's running n-cpu15:49
Shrewsdevananda: It's the disk space available to the compute host FOR instances. In our case, that is the disk space on the instance itself15:50
devanandaShrews: AIUI, the entire use of that option makes no sense for ironic, since there isn't a 1:1 relationship between n-cpu:node15:50
devanandaShrews: yes. but it's used to identify an available n-cpu host to which a migration could happen, based on disk space, right?15:51
devanandaShrews: if we run ironic with 1 or >1 n-cpu, in either case, that function makes no sense15:51
Shrewsdevananda: right, doesn't really relate to ironic. which is why i suggested 015:52
devanandaShrews: even though a cold migration of an instance from one node to another is, strictly speaking, possible15:52
devanandaShrews: yea... but how's that going to affect Nova?15:52
stendulkerJayF: Hello15:52
Shrewsdevananda: should only affect migrations. we have no official stance on migrations, do we?15:52
devanandaShrews: I guess my point is, what ever test is exercising that option probably should not be run if ironic is the only hypervisor available15:53
JayFstendulker: hey, I saw your comments and will try to reply today. I am busy this morning though15:53
NobodyCamJoshNang: around?15:53
devanandaShrews: official stance: we don't support it now. we *may* eventually support cold-migration.15:53
Shrewsdevananda: the tempest test is a generic test. deactivating it for ironic would be bad, imo15:54
stendulkerJayF: Thank you. Had pinged you to let you know. :)15:54
*** eghobo has joined #openstack-ironic15:54
devanandaShrews: remind me - which test?15:54
Shrewsdevananda: tempest.api.compute.admin.test_hypervisor.HypervisorAdminTestJSON.test_get_hypervisor_show_details15:54
Shrewsdevananda: my suggestion is, set it to 0 for now. when we DO need to start supporting migrations in some form, revisit it.15:55
Shrewsdevananda: https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L183-L19215:57
Shrewsif we set it to free_disk_gb, that block of code will run as-is today i think15:58
Shrewswhich may be the better choice15:59
*** eghobo has quit IRC15:59
devanandaShrews: and if disk_available_least is NONE, then we still get free_gb = compute['free_disk_gb']16:00
Shrewsright, which should be what's happening today16:00
devanandaShrews: i'm not sure I see how this relates to the tempest test, which is merely asserting that /sometehing/ was returned16:00
devanandaShrews: not that it has a value for disk_available_least16:00
Shrewsdevananda: the tempest test verifies that disk_available_least is an integer value, which it is not (it's null)16:00
Shrewsdevananda: it runs the returned json against a schema validator16:01
Shrewswhich is the failure16:01
Shrewsdevananda: https://github.com/openstack/tempest/blob/master/tempest/api_schema/compute/hypervisors.py#L2716:03
*** ndipanov has quit IRC16:03
*** dhellmann is now known as dhellmann_16:04
*** eghobo has joined #openstack-ironic16:04
Shrewsdevananda: oh! this is interesting... this schema def allows null: https://github.com/openstack/tempest/blob/master/tempest/api_schema/compute/hypervisors.py#L6116:06
Shrewswe could change tempest to allow nulls in all of the schemas for disk_available_least16:06
* Shrews tries the change16:07
devanandaShrews: :)16:07
NobodyCambbt...brb16:15
*** eguz has joined #openstack-ironic16:15
*** eguz has quit IRC16:17
*** eghobo has quit IRC16:19
*** lucasagomes_ has joined #openstack-ironic16:21
*** lucasagomes_ has quit IRC16:22
*** lucasagomes has quit IRC16:22
*** romcheg has quit IRC16:25
*** ifarkas has quit IRC16:27
*** athomas_ has quit IRC16:27
*** lsmola__ has quit IRC16:30
*** Nisha has joined #openstack-ironic16:34
Shrewsdevananda: submitted a patch to tempest for that. maybe we don't have to think about the whole migration thing for a while longer yet  :)16:35
* Shrews in dire need of lunch. bbiab16:35
NobodyCamenjoy Shrews :)16:35
JoshNangNobodyCam: yup!16:36
NobodyCamhey JoshNang wanted to see if you saw rloo's comment on the swift spec. looks like we lost a few things16:37
NobodyCam:-p16:37
JoshNangNobodyCam: yeah :/ must have gotten my local version out of sync somehow. going to fix it in about 5 minutes.16:38
NobodyCam:) woot :) ty16:38
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Allow noauth for Neutron  https://review.openstack.org/9892216:39
*** amitpp has joined #openstack-ironic16:39
*** stendulker has quit IRC16:40
*** linggao has quit IRC16:40
*** ramineni has quit IRC16:41
*** romcheg has joined #openstack-ironic16:43
*** dhellmann_ is now known as dhellmann16:44
*** dwalleck has quit IRC16:49
*** Alexei_9871 has quit IRC16:52
*** derekh_ has quit IRC16:58
openstackgerritJosh Gachnang proposed a change to openstack/ironic-specs: Swift Temporary URLs Spec  https://review.openstack.org/9890417:03
JayFJoshNang: did you patch the holes in your spec with that patch? :P17:03
JoshNangNobodyCam: ^ that should be good. FWIW, when setting up Glance to use Swift, you have to specify a single user and container for the images to go in, so it doesn't seem like it would limit you to use the config options17:04
devanandaShrews: awesome, ty17:04
JoshNangJayF: :D17:04
*** Nisha has quit IRC17:11
NobodyCamquick walkies... brb17:12
comstuddevananda: Have a few minutes to talk about a bug/race condition? :)17:13
comstudaffects the virt driver17:14
devanandacomstud: shoot17:16
comstuddevananda: https://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L626-L63017:16
*** overlayer has joined #openstack-ironic17:16
comstuddevananda: So, remember this conversation about multiple nova-computes and same host... and the synchronization on instance_uuid ? :)17:16
comstudImagine this case... a build on one compute... a delete right after on another.17:17
comstudthe delete skips the unprovision because the state is like 'building' or 'deploying' or whatever due to the code above17:17
comstud1) These checks are racey no matter what17:18
comstud2) ironic seems to allow you to unset instance_uuid on a node... while there's a target_provision_state17:18
comstudSo, what happens here is that instance ends up building and going active in ironic17:18
comstudbut it has no instance_uuid associated...17:18
comstudThere's somewhat of a separate bug here... in that the scheduler will happily try to schedule to this node again :)17:19
comstudbecause of: https://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L20017:19
comstuddoesn't also check that provision_state is NOSTATE17:19
*** blamar has joined #openstack-ironic17:20
comstudSo, it's easy to fix the scheduler to not schedule to these "broken" nodes... however, we need to solve this race properly, also17:21
devanandahuh17:21
comstudyeah, ask questions... that was a lot at once17:21
comstud:)17:21
devanandaI thought there was something preventing deletion when node.provision_state NOT IN that list17:21
devanandabut dont see it now17:21
comstudyeah, so, i'm seeing17:22
comstudhttps://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L626-L63017:22
comstudunprovision not called because state is 'deploying' or whatever17:22
devanandaright17:22
comstudand _cleanup_deploy() happily removes the instance_uuid17:22
comstudwith no error.17:22
comstudat least, that seems to be what I'm seeing :)17:23
comstudwe certainly end up with a node in 'active' with no instance_uuid associated.17:23
comstudI almost feel like because nova's state checks are always going to be racey... that it always tries to unprovision.17:23
devanandacomstud: https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L685-L68917:23
devanandaso it doesn't remove the instance_uuid17:23
devanandathat errors17:23
devanandathat should prevent nova from unprovisioning17:24
comstudhrm.17:24
jrollwell, that could race... if it goes to active between the state check and the api call17:24
devanandahowever it might be doing rather poor job and just breaking nova's instance at that point17:24
jrollbut that's a much less likely situation17:24
comstudactually.... yeah17:24
devanandajroll: true17:24
comstudnevermind, let me back up17:24
comstudI forgot17:24
devanandacomstud: :)17:24
comstudI do see nova retrying a lot during the delete :)17:25
comstudAnd then it succeeds17:25
*** Nisha has joined #openstack-ironic17:25
jrolla ha17:25
*** dwalleck has joined #openstack-ironic17:25
*** dwalleck has quit IRC17:25
devanandacomstud: question i've had for a long time on this topic, actually17:25
comstudAnd it's the ironicclient code itself that is retrying17:25
devanandaif a nova delete fails to remove the instance from the hypervisor17:25
devanandait leaves the instance in state DELETING, ERROR17:26
*** dwalleck has joined #openstack-ironic17:26
devanandadoes nova ever retry, aside from during _init_host() ?17:26
devanandaevacuate-deleted_instances is the only place I've seen that action happen from17:26
comstudIt now leaves it in ERROR, None (vm_state, task_state)17:26
devanandabut i may have missed something17:26
comstudie, it reversts DELETING to None17:26
devanandaok17:26
comstudAnd no, nova will not retry... outside of _init_host()17:26
devanandabut it doesn't allow retrying the delete17:26
devanandaugh17:26
comstudIt allows it now, yes17:26
jrollthe user can retry17:26
comstudif you manually delete again17:26
comstudthere was a bug there...17:26
comstudwhere task_state was being left17:27
devanandaroughly when did that change?17:27
comstudand that prevented a user from retrying17:27
comstudlast week or 217:27
devanandacause that's awesome. been bugging me for about a year now17:27
jrollheh17:27
comstudyes, it's been a bug since July/Aug 201317:27
comstuder, maybe i'm confusing bugs17:27
comstudbut yes, it's been broken for a while :)17:27
devanandaseems like that will help tripleo a lot, too17:27
comstudand infra... getting stuck instances17:28
devanandayea17:28
devanandasweet17:28
comstudthat's where it came up recently17:28
comstudalthough I'd found this while debugging OnMetal a couple months ago17:28
comstudheh.17:28
comstudanyway :)17:28
devanandaso. nova tries to delete the instance. ironic-client retries a bunch17:28
comstudyeah, then it succeeds17:28
devanandaif that succeeds - great. things work then, right?17:28
jrollno17:28
comstudno17:28
devanandaoh17:28
comstudsomehow ironic still ends up with 'active' with no instance_uuid17:29
jrollunprovision_instance gets skipped17:29
comstudbut it's gone from nova17:29
*** bandicot has joined #openstack-ironic17:29
comstudyeah17:29
devanandaooh right17:29
jrollthe retry is on _cleanup_whateveritscalled17:29
comstudyes17:29
comstudSo17:29
devanandathe client retry is essentially increasing the window between those checks17:29
comstudI kinda think nova should _always_ call unprovision.17:29
devanandaugh17:29
comstudand then... we deal with the races with that17:29
jroll+117:29
jrollironic should have that logic17:29
comstudLike... if it's already unprovisioned... we just say 'okay cool'17:29
comstudOtherwise perhaps nova just retries... on *that* call.17:30
devanandahmm17:30
comstudwhich will finally succeed... when the build is done on ironic side17:30
comstudThese state checks in nova are racey17:31
comstudbecause once we call ironic api and get them back, they are no longer reality.17:31
comstudwe can assume a certain order, perhaps... and i think that's what this tries to do17:31
comstudexcept it's missing that an instance can actually be in process of being built.17:31
devanandahttps://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L225-L24117:32
comstudanother interesting solution...17:32
comstudis that we could make ironic check to see if an instance_uuid is still assigned17:32
comstudat the end of provisioning17:32
comstudand automatically teardown if it's gone17:32
devanandacomstud: well. that's not possible today.17:33
comstudBut there's still room for race there without making nova always call unprovision17:33
comstudnod17:33
devanandacomstud: the lock is released when provisioning is completed. So it's not possible for the instance_uuid to change *during* provisioning17:33
devanandait could happen immediately after prov finishes (which is what you're seeing, i bet)17:33
comstudI don't particularly like that option either... was just throwing it out there.17:33
comstudyeah, net result is that i think we're seeing nova skip unprovision17:34
devanandait's an interesting idea, but the way we use locks prevents it17:34
comstudtries to unset instnace_uuid17:34
devanandaright17:34
comstudretries for a long time17:34
comstudand then it succeeds17:34
devanandai can see how this code can easily result in that17:34
comstudmaybe the non-atomic operation of unprovision and unsetting instance_uuid may be the real problem17:34
comstudmaybe that should be atomic17:34
comstudfrom nova's perspective17:35
devanandanot skipping unprovision and retrying earlier seems much better17:35
comstudor like... when we unset instance_uuid we make sure to unprovision17:35
devanandaright17:35
comstudyeah17:35
devanandafrom ironic's side, they aren't atomic17:35
devanandaone is state, one is metadata17:35
comstudyeah17:35
comstudSo anyway, I need to file a bug here... but before I do, wanted to walk through the issue17:35
devanandayou could change instance_info and instance_uuid in one PATCH call17:35
devanandabut need to use PUT for the /state/provision resource to initiate that action17:36
devanandaso here's another solution17:36
devanandahttps://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L56317:37
devanandaretry that line if it gets a 409 CONFLICT17:37
devanandawhich would be coming only from here: https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L23817:37
comstudif we make destroy() always call _unprovision(), yes17:38
devanandaright17:38
devanandathat's what i mean17:38
*** bandicot is now known as bensonmiller17:38
comstudI think the client itself will actually retry17:38
comstudand we may not need to do anything here17:38
comstudthe client itself is what is retrying for unsetting instance_uuid, at least17:39
*** blamar has quit IRC17:39
devanandaah, so it does -- https://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/client_wrapper.py#L8317:39
comstudso I think it's retrying on 409s ?17:39
comstudyeah17:39
devanandaugh. we use Conflict too much :(17:39
* devananda is increasingly frustrated with the API17:40
comstudhehe17:40
comstudI have another API thing to talk to you about :)17:40
comstudafter this17:40
devananda:)17:40
comstudso.. we think just making nova always call unprovision is the fix?17:40
comstudi mean, destroy() always call _unprovision()17:41
devanandacomstud: ++17:41
comstudOk17:41
comstudRelated...17:41
comstudIn case something else like this comes up..17:41
comstudhttps://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L20017:41
comstudI think this should be:17:41
comstudif node.instance_uuid or node.provision_state != ironic_states.NOSTATE:17:42
comstudjust as a safety net so the scheduler doesn't try to pick nodes that are in some state but don't have an instance_uuid17:42
comstud(it really should only pick nodes that are in NOSTATE and have no instance_uuid)17:42
comstudThis is probably not needed with the "real fix", however I think that makes this check technically more accurate17:43
devanandacomstud: see _node_resources_unavailable above17:43
devanandaperhaps just adding another condition there is sufficient?17:43
comstudYeah, I was going to put it there, but...17:44
comstudthat makes the node show that it has 0 resources17:44
devanandaoryou think the difference between17:44
comstudvs showing that the node is used17:44
devananda"node has 0 resources" and "all resources used" is important here17:44
comstudI'm not sure it's *that* important.17:44
devanandahow will that affect or be exposed by nova?17:44
comstudThe node having 0 resources might better indiciate there's a problem17:44
devanandaright17:44
comstudI think it has the same net effect17:44
devanandarather than showing a node fully utilized with no instance17:44
*** amitpp has quit IRC17:45
comstud0 resources should also translate to 0 free17:45
comstudalthough17:45
comstudWe do have a node that is reporting -32768M of ram free :)17:45
devanandahah17:45
comstudwhich is odd..17:45
devanandais it still getting instances scheduled on it?17:45
comstudi haven't looked into it17:46
devanandathere's definitely some races in nova that lead to negative resources17:46
comstudjust an oddity i noticed this morning17:46
comstudor someone noticed17:46
devanandabeen that way for a few years17:46
comstudokay, so I think I'll do what you suggest and put it in _node_resources_unavailable()17:47
comstudi'ma file a bug and push up a patch17:47
devanandacool17:47
comstudI assume you want this into the nova review before it lands, also17:48
devanandacomstud: w.r.t. the nova patch sets, a few questions17:48
comstudok17:48
comstudshoot.17:48
devanandaIIRC mikal's plan was to -2 the head of the ironic patch set17:48
devanandabut he hasn't, and alaski +A'd it17:48
devanandaso ... ??17:48
comstudhah17:48
jrollare you complaining? :)17:48
comstudI can add a -2 real quick if it's not landed yet17:48
devanandaalso, which is better from the nova review POV - continue to carry/merge fixes as I did this week, or wait until it lands in nova, or ?17:49
devanandajroll: there were reasons we had agreed on that process ...17:49
comstuddevananda: well, jenkins failed on it anyway :)17:49
jrolldevananda: mostly kidding17:50
comstudso it didn't merge17:50
devanandajroll: namely, being able to split the driver into patch sets that don't pass tests17:50
devanandacomstud: hah, cool17:50
devanandacomstud: if the plan changed, that's fine17:50
comstudi find it funny when jenkins fails on adding an empty directory17:50
comstudor file17:50
devanandafailed five times17:50
devanandaon adding an empty file17:50
devananda:)17:50
comstudeven better17:50
devanandalook at the retries!17:50
devananda<sadface>17:50
comstudI honestly didn't pay attention to what was said about merging these17:51
comstudI mean, the spec is approved17:51
comstudso that's all I looked at17:51
devanandacomstud: I suspect that's generally the case17:51
devanandai'll poke mikal when he's online later about it17:51
devanandacomstud: the rationale was: split the driver code into small bits which individually don't pass any tests, but are easy to review17:52
devanandacomstud: -2 the tip, and watch the tail for test results17:52
comstudok, i set this to WF -117:52
comstudfor now17:52
*** aswadr has quit IRC17:52
comstudAnyone should be able to un-WF -1  it17:52
devanandacomstud: then have someone in nova do the squash-and-resubmit when everything's approved17:52
comstudah17:52
*** dkehn_ has joined #openstack-ironic17:52
Nishajroll: JayF hi17:53
devanandabecause https://review.openstack.org/#/c/103167/ is 2500+ lines17:53
jrollhi, Nisha17:53
Nishajroll, one query17:53
* jroll listens17:54
Nishajroll: When IPA detects the link state of NIC, the OS is already there on the node (Atleast mini kernel)?17:54
Nishajroll: Can IPA detect the link state of the NIC without any OS on teh BM?17:54
Nishajroll: if yes, how?17:55
jrollNisha: not sure what you mean. why would IPA detect the link state?17:55
*** dkehnx has quit IRC17:56
Nishajroll: You mean IPA doesnt need to know whether the NIC is connected or not?17:56
jrollNisha: why would it?17:56
jrollNisha: other than to make sure it can talk to ironic...17:56
jrollNisha: but that can be detected by sending an HTTP request17:56
jrollNisha: why would anything need to detect NIC link state?17:56
Nishajroll: by broadcasting, correct?17:56
jrollwhat?17:57
* jroll is confused17:57
jrollNisha: what are you trying to do? what is the end goal?17:57
Nishajroll: This is regarding https://review.openstack.org/#/c/100951/1617:57
devanandaNisha: this question "Can IPA detect the link state of the NIC without any OS on teh BM?" does not make sense17:57
devanandaNisha: ironic-python-agent is a set of python modules that runs inside an operating system on the bare metal node17:58
jrollnone of this makes sense17:58
comstuddevananda: https://bugs.launchpad.net/ironic/+bug/1337461  assign importance to that if you want17:58
comstudbut it's in progress here17:58
jrollwhy would ironic care if a NIC has a cable attached?17:58
devanandaNisha: IPA will implement a deploy interface, not power or management interfaces.17:58
jrollNisha: what line should I be looking at?17:58
devanandaNisha: so it will be able to use eg. IPMITool or iLO interfaces for those17:58
Nishajroll: see --create-ports option17:59
Nishait has options [connected | all]17:59
devanandajroll: so the context from Nisha's side is autodiscovery17:59
jrollI don't see why 'connected' si useful17:59
jrolls/si/is/18:00
NobodyCamjroll: for hw discovery18:00
devanandajroll: could we use IPA deploy + PDU power to do inventory discovery, with absolutely no out-of-band mgmt cards? -- I think so.18:00
jrollNisha: IPA *could* detect link state, if we wrote code to do so. does that help?18:00
jrolldevananda: I agree18:00
comstudHm18:00
jrollI still don't see why only adding ports in ironic for connected NICs is useful18:01
devanandajroll: knowing link state of a NIC in general will be important as we add more interaction with SDN controllers (eg, neutron OVS)18:01
jrollbut that might just be me18:01
comstudWhat error does ironic return if we try to unprovision a node that's already unprovisioned?18:01
devanandajroll: you mean "unconnected NICs"18:01
* comstud looks18:01
jrolldevananda: I mean "only adding ports in ironic for connected NICs"18:01
devanandacomstud: https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L22818:01
jrolldevananda: or "not adding ports for unconnected NICs"18:01
devanandajroll: oh. gotcha. I agree. we should add all of htem18:01
jrollright18:01
comstudthanks18:02
devanandajroll: but stashing metadata saying "this nic is not cabled" could be helpful. I think that was the question?18:02
devanandaNobodyCam: ^ did I get that right?18:02
NobodyCamreading now18:02
jrolldevananda: that's fine. the question was about the --create-ports=[connected | all] in the spec18:02
devanandaoh18:02
NobodyCamya18:02
jrollwhich I don't think should be an option unless there's a great use case18:03
jrollanyway18:03
Nishadevananda: yes18:03
jrollNisha: did I answer your question?18:03
devanandawhy would we NOT create a port for a NIC that isn't cabled, but is in the machine and might be cabled later?18:03
Nishajroll: yes18:03
jrollNisha: :)18:03
devanandaplugging in a cable later shouldn't require RE-discovering the hardware18:03
devanandathe hardware hasnt changed18:03
devanandaonly link state18:04
jrollagree18:04
devanandaexposing a means (and perhaps then automating it) to disable NICs would be good18:04
Nishadevananda: ok , then we can have an option of link state which can be updated18:04
devanandaeg, tell Neutron "this NIC is good, give it an IP, that NIC should be offline, make sure it doesn't get on the network"18:04
Nishadevananda: not option18:04
Nishaa sub-field in inventory field18:05
devanandathat'd be a good feature IMO18:05
devananda(just not in Juno... we're already swamped :) )18:05
jrollall this discovery stuff is sooo close to making ironic an inventory system18:05
devanandajroll: i know :(18:05
jrollburn it all18:05
devanandait's a slippery slope and i've been trying to find the middle ground18:06
jrollauto-registration, don't store any info :)18:06
jroll(way easier said then done)18:06
jrolls/then/than/18:06
devanandaauto-registration, store opaque blob that an external inventory system can pull at a later date18:06
NishaNobodyCam: so is it fine if i remove the options from --create-ports18:06
NobodyCamNisha: yes18:06
jrollyeah, but you know that will get abused18:07
jrollNisha: yes, please do18:07
devanandabecause real-time request to gather invetory data is impractical/impossible18:07
devanandajroll: it already is18:07
jrollright18:07
devanandajroll: the requests to search inside that blob are just the beginning -- and i'm being clear in saing "no" even to that18:07
devanandawell, i think i'm being clear18:07
devananda:(18:07
jrollyou've been clear, just ignored18:08
devanandaawesome18:08
*** coolsvap is now known as coolsvap|afk18:08
Nishadevananda: since we are discussing this, could i request you to review the spec https://review.openstack.org/#/c/100951 again ? :)18:08
jrolldevananda: (or so it seems)18:08
*** rloo has quit IRC18:10
comstuddevananda: https://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L57018:10
comstud^ does that actually work? It seems like a BadRequest is what will be returned18:10
Nishacomstud: just remove #L570 from the URL, it will work18:11
devanandacomstud: i was wondering about that ... I suspect the #TODO has been done and that can be removed now?18:11
*** Nisha has left #openstack-ironic18:11
comstudI see no reference to InstanceDeployFailure in the API at all18:11
comstudhttps://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L22818:12
NobodyCamdo we call update_port_dhcp_opts for every port attached to a node?18:12
comstuddevananda: I'm thinking: http://paste.openstack.org/show/85443/18:14
devanandaNobodyCam: no - https://github.com/openstack/ironic/blob/master/ironic/common/neutron.py#L14918:14
devanandaNobodyCam: only for ports with a VIF18:15
devanandaNobodyCam: which mayu or may not be all of them18:15
devanandacomstud: i hate the string matching on e18:16
NobodyCammy concern with the connected status was that dhcp would be attached to a mac address that had no cable on it18:17
devanandacomstud: what happens if that exception is translated?18:17
comstudFWIW, I also hate the string matching :)18:17
comstudI'm wondering if just BadRequest will suffice on its own18:17
*** foexle has joined #openstack-ironic18:17
comstudI don't think 400 is returned for anything else18:17
comstudin here18:17
comstudworst case...18:17
comstudIf we fall through for all BadRequest..18:18
comstudnova is going to poll for target_provision_state=None anyway18:18
comstudand error there18:18
comstudso I think maybe just a 'pass' here will be fine18:18
devanandak18:19
* devananda moves to -qa to talk tempest18:19
comstuddevananda: thnx, patch shortly.18:20
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Adding support for decommissioning  https://review.openstack.org/10437918:22
*** Mikhail_D_ltp has joined #openstack-ironic18:22
openstackgerritNisha Agarwal proposed a change to openstack/ironic-specs: Discover node properties at node-create/node-update  https://review.openstack.org/10095118:23
*** Nisha has joined #openstack-ironic18:23
*** Mikhail_D_ltp has left #openstack-ironic18:31
*** dwalleck has quit IRC18:33
*** romcheg has quit IRC18:35
*** romcheg has joined #openstack-ironic18:35
*** Mikhail_D_ltp has joined #openstack-ironic18:37
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Adding support for decommissioning  https://review.openstack.org/10437918:39
*** eghobo has joined #openstack-ironic18:39
openstackgerritNisha Agarwal proposed a change to openstack/ironic-specs: Discover node properties at node-create/node-update  https://review.openstack.org/10095118:40
*** romcheg1 has joined #openstack-ironic18:41
*** overlayer has quit IRC18:42
*** romcheg has quit IRC18:43
NobodyCambrb18:47
openstackgerritNisha Agarwal proposed a change to openstack/ironic-specs: Discover node properties for iLO driver  https://review.openstack.org/10300718:48
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Adding support for decommissioning  https://review.openstack.org/10437918:52
*** Nisha has quit IRC18:54
NobodyCamdevananda: I've miss places the link for the spec's review team, happen to have that off hand? and would you mind if I added it to https://wiki.openstack.org/wiki/Ironic/Specs_Process18:56
devanandaNobodyCam: it's a gerrit group18:57
*** foexle has quit IRC18:57
*** pelix has quit IRC19:02
*** ccrouch has quit IRC19:03
openstackgerritChris Behrens proposed a change to openstack/ironic: Fix build + delete races  https://review.openstack.org/10464919:03
comstuddevananda: ^ hopefully that covers that19:04
*** rloo has joined #openstack-ironic19:16
NobodyCamdevananda: found it19:16
NobodyCam:)19:16
* NobodyCam switches to internal mode to take Standards of Business training19:18
*** rloo has quit IRC19:21
*** rloo has joined #openstack-ironic19:23
matty_dubsIs that like the ethics training we have to take? A once-a-year training video that mostly just tells you stuff you already knew, like that bribing people or trading on insider information is wrong?19:24
NobodyCamyep19:25
NobodyCam:-p19:25
devanandajroll: who is working on IPA support in devstack?19:30
jrolldevananda: literally trying to get devstack to run right now19:30
devanandaah19:30
jrolljust came back from lunch to "Cannot open network namespace "qdhcp-e23ca5f6-d06e-4015-84f4-d1d4a164629f": No such file or directory"19:31
jrolland " ovs-vsctl: tag=: argument does not end in "=" followed by a value."19:31
jroll(I hate devstack)19:32
jrolloh, maybe those are non-critical errors19:34
jrollgot a node booting19:35
* jroll adds IPA to this19:35
jrollhow do I grab a console with virsh?19:37
NobodyCamvirsh console <name>19:38
jrollthought so... error: internal error: character device <null> is not using a PTY19:38
jroll:(19:39
NobodyCamthats a vm config issues19:39
jrollI... didn't configure a vm19:39
jrollafaik19:39
NobodyCamneed to setup pty devicw19:39
jrollI'm just following http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html19:39
* jroll has no idea what he's doing19:39
jrollbrb19:39
*** enikanorov_ has joined #openstack-ironic19:43
matty_dubsdevananda: BTW, thanks for the review on the NetApp spec. Very helpful in figuring out the right way to proceed with that code.19:45
matty_dubsI haven't replied to it since I'm bugging Lucas about whether it makes sense to pursue any of that stuff through Ironic, or if it should be an entirely separate thing.19:46
*** enikanorov__ has quit IRC19:47
matty_dubs(But the latter seems likely.)19:47
devanandamatty_dubs: if you start provisioning OSs on the netapp and treat them as disposable "cloud instances" -- sure19:49
devanandamatty_dubs: part of this is also the paradigm shift from "it's physical so it's a snowflake" to "it's just an instance in a cloud, regardless of whether it's a VM or not"19:49
matty_dubsYeah. What I struggled with is that you can do some OS management, though mostly updates.19:50
matty_dubsSo it's conceivable that this would be very useful if you wanted to, say, reprovision one of these for a different client19:50
matty_dubsBut I don't think that's a common operation.19:50
devanandamatty_dubs: if you dont want to manage (deploy, delete) the instance (of an operating system on your netapp filer) through Nova, perhaps it's not a good fit19:51
*** blamar has joined #openstack-ironic19:51
*** Penick has joined #openstack-ironic19:51
matty_dubsThat's a good way to frame it. We almost certainly don't want to do that.19:52
devanandamatty_dubs: :)19:53
devanandaso how can we improve the messaging that19:53
devanandaironic is not a general tool to install an operating system19:53
devanandait's a tool for managing hardware *just like it's in a cloud*19:53
devanandacause, well, it is19:53
matty_dubsHehe19:53
matty_dubsWell, I think the problem is that I started more bottom-up here. We do power control + deploys + console in the code, and hey, we need power control + console and maybe a little deploying, so Ironic!19:54
jrollNobodyCam: hey, looks like this vm was started with: -chardev file,id=charserial0,path=/opt/stack/ironic-bm-logs/baremetalbrbm_2_console.log -chardev pty,id=charserial119:54
jrollNobodyCam: so uh, even if pty isn't working, there should be logs in those files, no?19:55
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Adding support for decommissioning  https://review.openstack.org/10437919:55
*** Penick has quit IRC19:57
NobodyCam??? I would assume so?20:00
jrollyeah :/20:01
jrollnothing there20:01
jrollhrmmmmmmm20:02
*** foexle has joined #openstack-ironic20:03
*** blamar has quit IRC20:03
*** dkehn__ has joined #openstack-ironic20:04
*** dkehn_ has quit IRC20:07
*** foexle has quit IRC20:10
openstackgerritA change was merged to openstack/ironic-python-agent: Run make with sudo when building  https://review.openstack.org/10436120:10
*** foexle has joined #openstack-ironic20:10
JayFjroll: ^20:10
jrollnice20:11
JayFso the question is if the old or new version of post job ran20:11
JayFshould be new20:11
jrollit just runs the script20:11
jrollthe latest version of the script20:11
JayFhttps://jenkins02.openstack.org/job/ironic-python-agent-buildimage-coreos/2/console20:12
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add Node decommission interfaces  https://review.openstack.org/10295320:13
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add Node decommission in conductor  https://review.openstack.org/10298420:14
openstackgerritJay Faulkner proposed a change to openstack/ironic-python-agent: Install requirements for coreos-oem-inject.py  https://review.openstack.org/10467220:17
JayFJoshNang: jroll ^^20:17
jrollhmm20:18
jrollwhat was that --external flag we were using at some point?20:18
jrollwas that for non-OS requirements20:18
jroll?20:18
JayFit was required because '-e' in requirements.txt is not allowed without allow_external20:18
jrollah20:19
jrollapproved20:19
*** dkehn__ is now known as dkehnx20:19
JayFjroll: I'm going to file a bug (low-hanging-fruit) to migrate the script off plumbum20:19
jrollcool20:19
jrollkylestev: ^ :)20:19
openstackgerritA change was merged to openstack/ironic-python-agent: Install requirements for coreos-oem-inject.py  https://review.openstack.org/10467220:21
jrolldoes everybody else's virsh logs just work? or is there configs I need to do20:23
jroll'getting started with ironic' isn't very easy20:23
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add Node decommission in conductor  https://review.openstack.org/10298420:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Factor out TFTPImageCache  https://review.openstack.org/10073420:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Factor out deploy info from PXE driver  https://review.openstack.org/10073520:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add methods to ipmitool driver  https://review.openstack.org/10036420:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add Node decommission interfaces  https://review.openstack.org/10295320:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding swift temp url support  https://review.openstack.org/8139120:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add ironic-python-agent deploy driver  https://review.openstack.org/10102020:24
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add a decommission command for the agent  https://review.openstack.org/10369420:24
jrollwhoa20:24
*** ccrouch has joined #openstack-ironic20:25
JoshNangrebase all the things!20:25
JoshNang(but seriously, didn't realize it would do that whole giant patch tree)20:26
jroll-R ftw20:26
JayFkylestev: https://bugs.launchpad.net/ironic/+bug/1337551 -- if you wanted a low hanging IPA bug :)20:26
JoshNangI was fixing a merge issue in the last patch on that chain20:27
JayFheh.20:27
JayFhttp://tarballs.openstack.org/ironic-python-agent/coreos/ipa-coreos.tar.gz/ipa-coreos.tar.gz20:27
* devananda goes out in search of food20:27
JayFlooks like we have an infra fix, but it appears this should be our IPA CoreOS image, built fresh off openstack infra20:27
devanandajroll, JoshNang: i dont recall if we talked about this explicitly20:28
*** jgrimm has quit IRC20:28
*** bensonmiller has quit IRC20:28
devanandabut I am going to (and should have a while ago) -2'd the IPA patch series20:28
JayFuntil the spec is approved?20:29
jrollyep20:29
devanandaas a way to indicate which ones are ready for merging (like the ones at the start, which arent dependent on the spec20:29
jrolliirc20:29
devanandaright20:29
JayFI assumed that it already was tbh :)20:29
devanandacool20:29
devanandai've been lax about actually doing that20:29
devanandawell, busy20:29
jrollrloo: has been on it20:29
jrollbut missed that one, I guess20:29
rloojroll: which one did I miss? you guys have so many!20:30
JayFThat uploaded ipa image looks golden20:30
devanandarloo: all of them :)20:30
rloolet me take a look. I thought I -2'd the 'base' patch. but maybe I missed or you snuck some new ones in ;)20:30
jrollrloo: :)20:30
jrollrloo: 10102020:31
devanandaright. i'm really running out of fuel now ... gotta find food20:31
rloowe should just approve the spec soon. (although i just added a bunch of comments to it.)20:31
devanandabbiah20:31
devanandarloo: or that :)20:31
jroll^^20:31
* devananda is really away now20:31
JoshNangrloo: yeah i saw those. on my todo list for today to fix up the spec20:31
JoshNangand thanks :)20:31
*** bandicot has joined #openstack-ironic20:31
rlooso 101020 depends on 81391 which I blocked. but 81391 is a diff blueprint so yeah, I should block 101020. thx for letting me know jroll!20:32
jrollno problem :)20:32
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add get_host_uptime to nova driver  https://review.openstack.org/10468020:34
jrollShrews: uhhh, how do we define 'host' in ironic? :)20:35
Shrewsjroll: hrm, good question. i was assuming compute host, but that could be node i suppose. It's WIP anyway20:36
jrollyeah20:36
jrolljust a thoughut20:37
Shrewsjroll: i pretty much copied libvirt driver on it20:37
jrollShrews: right... I get the idea. it's just a weird abstraction for ironic20:37
jrollalso, shouldn't ever care how long an ironic compute node has been running20:38
jrollbecause it doesn't necessarily affect an ironic node20:38
Shrews"uptime of the hypervisor" ....20:38
jrollbut like20:38
jrollthere is no hypervisor20:39
jroll(maybe that's what you're trying to say)20:39
Shrewsjroll: yeah, it's weird. i'll dig deeper later. i just wanted to get the code saved somewhere before i forget about it for the long weekend20:39
jrollok :)20:39
jrollso, this doesn't work at all, as far I can tell: http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#deploying-ironic-with-devstack20:40
jrollboot fails20:40
jrolldelete fails20:40
jrolleverything is awesome20:40
JoshNangjroll: gah that rebase messed up the topics for the patches.20:40
Shrewsjroll: sure it does20:40
jrollJoshNang: I've decided to stop caring about topics20:40
Shrewsjroll: what os?20:40
jrollShrews: not in my recent experience20:40
JoshNangha k20:40
jroll14.0420:40
Shrewsjroll: fail20:40
jrollwhich is what infra runs now20:40
jrollso it should work20:40
Shrewsjroll: no problems on 12.0420:41
jrollif this doesn't work on 14.04 I'm going to flip things20:41
jrolloh, wait.20:41
ShrewsYES!20:41
jrollI just realized20:41
jrollha20:41
jrollignore me20:41
jrollthis is debian testing20:41
adam_gjroll, IRONIC_VM_SPECS_RAM=512  thats outdated, set to 102420:41
jrollI switched boxes without thinking20:41
jrolladam_g: yeah, grabbed some things from dwalleck's patch20:41
Shrews/ignore jroll20:41
jrollincluding that20:41
JayFIronic devs: http://tarballs.openstack.org/ironic-python-agent/coreos/ipa-coreos.tar.gz -- Infra is setup now to build a new tarball of the IPA CoreOS ramdisk on each commit. When conbined with http://git.openstack.org/cgit/openstack/ironic-python-agent/tree/imagebuild/coreos/README.rst that should get us closer to not-horribly-difficult testing of the agent and agent patches :)20:41
jrollthanks Shrews20:42
JayFjroll: ^ for great benefit devstack testing20:42
jrollJayF: (which I'm trying to do)20:42
jrollright20:42
openstackgerritA change was merged to openstack/ironic: Allow noauth for Neutron  https://review.openstack.org/9892220:42
Shrewsok, after a 2+ hour IRC meeting, i'm going to officially start my long weekend by cracking open beers now.20:44
Shrewslater gators20:44
jrollnice, have a good one Shrews20:46
*** jdob has quit IRC20:55
ellenhrloo: hi, are you around/available?21:00
*** jbjohnso_ has quit IRC21:00
rloohi ellenh21:01
ellenhrloo: so, for the info logging patch, I just tried again to use the nicer import statement for _LI21:01
ellenhrloo: and it still breaks21:01
rlooellenh: i wonder why it works for the non-ironic 'core' code but not for your changes.21:02
rlooellenh: give me 15 minutes, and I'll look into it.21:02
ellenhrloo: cool, thanks.21:04
*** foexle has quit IRC21:06
jrollellenh, rloo: see the last line of tox.ini21:12
jrollyou could add an exception for ironic.openstack.common.gettextutils._LI, etc21:12
jrollthe existing code works because it will skip the entire line, and the existing line is: from ironic.openstack.common.gettextutils import _, _LI21:13
rloojroll: ahh. so ellenh just needs to add '_, ' to it too.21:13
jrollI mean, that's not the "right" way to do it, but that's the easy way :)21:14
jrolleasy way being not adding exceptions for each _L*21:14
ellenhjroll: rloo that gives a “one import per line” complaint though21:14
jrollgah21:14
jrollI might be wrong then21:14
jrollidk21:14
jrollyou could add to import_exceptions in tox.ini21:14
rlooah, tox.ini excludes some directories like *openstack/common*, *ironic/nova*. that's why they worked there.21:16
ellenhjroll: rloo hmm.  so, would it be better to block both info log patches and depend on a “change tox.ini” patch, change tox.ini in both patches, or just leave both as-is and deal with it later?21:17
rlooellenh: it'd be better NOT to use the _L* stuff but I've been overruled here ;)21:18
jrollellenh: I'd say the first21:18
rlooellenh: I think updating tox.ini first would be better.21:18
jrollrloo: problem with making _LI happen in the log.info() calls would be that you can't translate it until you log it21:19
jrollwhich is not always ideal21:19
jrollbecause you want to translate before formatting21:19
jrolllike, _LI('Node %(node)s blablabla')21:19
jrolldoes that make sense? I had this realization the other day21:20
rloojroll: it just seems like we should be able to do better, maybe not as well but better. putting _LI, then if the log changes to warn, having to remember to change _LI etc.21:20
jrollrloo: sure, and I've always agreed with you until I noticed this21:20
rloojroll: it just makes me sad. oh well.21:21
jrollindeed21:22
ellenhat least there’s a clear ruling, though21:23
NobodyCambbiafm21:26
jrollwow, this totally works on ubuntu21:28
jroll:)21:28
*** Haomeng|2 has joined #openstack-ironic21:41
*** Haomeng has quit IRC21:42
openstackgerritEllen Hui proposed a change to openstack/ironic: Add gettextutils._L* to import_exceptions  https://review.openstack.org/10469621:48
*** mrda-away is now known as mrda21:53
mrdaMorning Ironic!21:54
*** ccrouch has quit IRC21:56
NobodyCammorning mrda21:58
mrda\o21:59
jrollheya mrda22:00
mrdao/22:03
*** ccrouch has joined #openstack-ironic22:07
NobodyCamdevananda: when you get back could you ping me22:07
*** hemna_ has quit IRC22:08
NobodyCamany one played with barbican???22:09
JayFNobodyCam: I haven't personally, but Jarret has demoed some of the bits to us internally before22:10
JayFNobodyCam: but honestly IDK how much of that is 'we want it to do this' vs 'it does this today' :)22:10
NobodyCam:)22:11
NobodyCamI was hopping we could off load "secure storage for IPMI creds"22:12
JayF++++22:13
JayFabsolutely, and Barbican would be great for that22:13
mrdaNobodyCam: Always a trade-off - adding a new dependency vs writing it yourself.22:13
JayFIDK if it's ready, but Jarret is a great guy and would probably like being asked at least22:13
* mrda hasn't played with Barbican22:13
jrollI think it's ready like ironic is ready22:15
jrollso, usable22:15
mrdaso, awesome! :)22:15
NobodyCamwe have a BP for it just need a spec..22:15
NobodyCamoh and ofc code to do it:-p22:18
NobodyCams/do it/use it/22:19
openstackgerritEllen Hui proposed a change to openstack/ironic: Add more INFO logging to ironic/common  https://review.openstack.org/9949522:21
* devananda is back22:27
devanandaNobodyCam: and someone to lead that code, the testing of it, write docs, etc...22:27
NobodyCamlol :)22:28
NobodyCamoh i'll brb22:28
*** eghobo has quit IRC22:29
*** radsy has joined #openstack-ironic22:37
*** romcheg1 has quit IRC22:37
*** romcheg has joined #openstack-ironic22:39
*** Mikhail_D_ltp has quit IRC22:40
jrolldevananda: when putting up devstack patches for this stuff... should I add to the existing 3 nodes or take one of them for agent driver?22:47
devanandajroll: toggle22:47
jrollthanks22:47
jrollI might have something today22:47
jrolllikely monday22:47
devanandajroll: so in the context of testing within devstack-gate, the direction we need to go is supporting the full set of tempest/api/compute tests22:48
jrollright, that's my eventual goal. first up is "random ironic dev can test"22:48
devanandajroll: which means having available min 3 (for serial) or 6 (for parallel) VMs for testing22:48
jrollwhich right now, is looking like 1 file changed, 11 insertions(+), 8 deletions(-) (and adding a couple of configs)22:49
jrollnod22:49
devanandajroll: we could have a jenkins job run on ironic's pipe that changes a flag (IRONIC_DRIVER=ipa_ssh) which triggers a bunch of other changes to set up the env for that22:49
devanandajroll: and then eg. pass that in from infra/config for that job22:50
jrollright22:50
jrolldevananda: but I'd also like parallel testing22:50
devanandajroll: are you guys going to run this as third-party CI? or try to get it in d-g?22:50
jrolljust d-g cause why not :)22:50
devanandajroll: ah, test both drivers at the same time?22:50
jrollboth in parallel is the eventual goal, yes22:51
jrollbecause that means users can upgrade22:51
jrolletc22:51
devanandajroll: like enroll some nodes with each, then let nova pick them at random22:51
jroll(also, heterogeneous environments)22:51
devanandaright22:51
devanandaactually, that's neat22:51
jrollwell, I'd say run each test twice in parallel22:51
devanandano22:51
jrollor... something22:51
devanandathat's less neat22:51
jrollha22:51
jrollrandom is fine22:51
devanandabecause it means twice as long to test, and doesn't really gain us much22:52
jrollwell, I'd kind of like each driver gets each test at least once22:52
devanandaso the tests aren't driver specific22:52
jrollto make sure e.g. we don't break agent tear_down and miss it22:52
devanandaor even specific to Ironic22:52
devanandait's all the nova tests22:52
jrollright...22:52
devanandaIOW, one run of tempest is ~ 70 deploys22:52
jrollwow22:53
*** romcheg has quit IRC23:06
openstackgerritEllen Hui proposed a change to openstack/ironic: Add gettextutils._L* to import_exceptions  https://review.openstack.org/10469623:07
openstackgerritEllen Hui proposed a change to openstack/ironic: Add more INFO logging to ironic/common  https://review.openstack.org/9949523:07
*** matty_dubs is now known as matty_dubs|gone23:08
adam_gdevananda, would hard-coding the root=ramfs in the template possibly complicate things for other ramdisks? IPA?23:09
jrolladam_g: which template?23:09
adam_gactually, i guess its specific to the pxe driver23:10
adam_gjroll, https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/pxe_config.template#L1123:10
jrollah yeah, we made our own template23:10
devanandathat's what I thought23:10
devanandawtf? gertty is suddenly saying it has 750 changes to sync23:12
openstackgerritEllen Hui proposed a change to openstack/ironic: Add more INFO logging to ironic/conductor  https://review.openstack.org/9974423:13
*** LiveOne has joined #openstack-ironic23:18
NobodyCamdevananda: you saw that https://review.openstack.org/#/c/104322 has two +2 but was not do it is location (our nova tree)23:19
NobodyCamgah23:19
NobodyCams/was not do/was not landed do/23:20
devanandaNobodyCam: oh, hadn't seen. thanks23:22
*** ccrouch has quit IRC23:25
openstackgerritMichael Davies proposed a change to openstack/ironic: Ironic nova driver to cache ironic client calls  https://review.openstack.org/10269523:25
*** max_lobur has quit IRC23:32
openstackgerritDevananda van der Veen proposed a change to openstack/python-ironicclient: Make a few minor updates to node shell help strings  https://review.openstack.org/10231223:47
devanandarloo: if by chance you're still around, this spec https://review.openstack.org/#/c/102435/ now has 2 +2 and several +1, pending you taking another look at it, it could be approved23:48
*** eghobo has joined #openstack-ironic23:52
*** derekh_ has joined #openstack-ironic23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!