Tuesday, 2016-06-07

*** fragatina has quit IRC00:01
*** caldw__ has joined #openstack-ironic00:01
*** wajdi has quit IRC00:01
*** fragatin_ has quit IRC00:02
*** rbudden has joined #openstack-ironic00:03
*** bsilverman has joined #openstack-ironic00:04
*** mbound has quit IRC00:04
*** blakec1 has quit IRC00:04
*** fragatina has joined #openstack-ironic00:05
*** sdake_ has quit IRC00:05
*** caldw__ has quit IRC00:06
*** fragatina has quit IRC00:07
*** fragatin_ has joined #openstack-ironic00:07
*** rama_y has quit IRC00:08
*** fragatin_ has quit IRC00:08
*** garthb has quit IRC00:15
*** amotoki has joined #openstack-ironic00:18
*** jaybeale has quit IRC00:33
*** mtanino has quit IRC00:34
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Drivers  https://review.openstack.org/31370800:34
*** sdake has joined #openstack-ironic00:39
*** r-daneel has quit IRC00:39
*** Sukhdev has quit IRC00:42
*** hoangcx has joined #openstack-ironic00:43
*** ChrisAusten has joined #openstack-ironic00:52
*** penick has joined #openstack-ironic00:58
*** sacharya has joined #openstack-ironic00:59
*** Sukhdev has joined #openstack-ironic01:03
*** Sukhdev has quit IRC01:04
*** sacharya has quit IRC01:04
*** harlowja has joined #openstack-ironic01:08
*** amotoki has quit IRC01:08
*** anteaya has quit IRC01:10
*** Goneri has joined #openstack-ironic01:14
*** piet has quit IRC01:15
*** jasondotstar has quit IRC01:18
*** Goneri has quit IRC01:20
*** vishwanathj has joined #openstack-ironic01:31
*** rloo has quit IRC01:35
*** amotoki has joined #openstack-ironic01:35
*** sdake has quit IRC01:50
*** divya_ has quit IRC01:57
*** sacharya has joined #openstack-ironic01:59
*** rbudden has quit IRC01:59
*** PollyZ has joined #openstack-ironic01:59
*** amotoki has quit IRC02:00
openstackgerritYibo Cai proposed openstack/ironic: Support multi arch deployment  https://review.openstack.org/32263202:10
*** penick has quit IRC02:10
*** sdake has joined #openstack-ironic02:11
*** amotoki has joined #openstack-ironic02:12
*** rm_work has quit IRC02:16
*** sdake_ has joined #openstack-ironic02:18
*** sdake has quit IRC02:20
*** PollyZ has quit IRC02:20
*** causten_ has joined #openstack-ironic02:21
*** ChrisAusten has quit IRC02:24
*** rbudden has joined #openstack-ironic02:25
*** _vsaienko has joined #openstack-ironic02:45
*** vsaienko has joined #openstack-ironic02:45
*** mbound has joined #openstack-ironic02:47
*** phuongnh has joined #openstack-ironic02:49
*** yuanying_ has joined #openstack-ironic02:51
*** yuanying has quit IRC02:51
*** mbound has quit IRC02:51
*** rm_work has joined #openstack-ironic02:56
*** rbudden has quit IRC03:02
*** rm_work has quit IRC03:07
*** fragatina has joined #openstack-ironic03:10
*** fragatin_ has joined #openstack-ironic03:11
*** rm_work has joined #openstack-ironic03:11
*** phuongnh has quit IRC03:13
*** rama_y has joined #openstack-ironic03:14
*** fragatina has quit IRC03:15
*** wajdi_ has quit IRC03:19
*** Nisha has joined #openstack-ironic03:21
openstackgerritVasyl Saienko proposed openstack/ironic: WIP: Patch to test Grenade.  https://review.openstack.org/31110903:21
*** kalpase has joined #openstack-ironic03:31
*** sdake_ has quit IRC03:38
*** links has joined #openstack-ironic03:56
*** Nisha_away has joined #openstack-ironic04:00
*** Nisha has quit IRC04:03
*** ijw has quit IRC04:06
*** Nisha_away has quit IRC04:06
*** Nisha_brb has joined #openstack-ironic04:06
*** pcaruana has joined #openstack-ironic04:11
*** yuanying_ has quit IRC04:18
*** yuanying has joined #openstack-ironic04:21
*** pcaruana has quit IRC04:24
* jlvillal is unsure when vsaienko1 sleeps04:25
*** rama_y has quit IRC04:29
*** rama_y has joined #openstack-ironic04:33
*** pcaruana has joined #openstack-ironic04:34
*** sacharya has quit IRC04:36
*** sacharya has joined #openstack-ironic04:37
*** pcaruana has quit IRC04:40
*** sacharya has quit IRC04:42
*** YiboCai has quit IRC04:53
*** rbartal has joined #openstack-ironic05:00
*** Nisha_away has joined #openstack-ironic05:00
*** Nisha_brb has quit IRC05:04
*** moshele has joined #openstack-ironic05:07
*** _vs_ has joined #openstack-ironic05:10
*** _vsaienko has quit IRC05:11
*** vsaienko has quit IRC05:11
*** _vs_ has quit IRC05:11
*** Nisha_away has quit IRC05:21
*** Sukhdev has joined #openstack-ironic05:21
*** ChubYann has quit IRC05:24
*** rcernin has joined #openstack-ironic05:36
*** sacharya has joined #openstack-ironic05:38
*** Sukhdev has quit IRC05:41
*** Sukhdev has joined #openstack-ironic05:41
*** rama_y has quit IRC05:42
*** sacharya has quit IRC05:43
*** mjura has joined #openstack-ironic05:52
*** mjura has quit IRC05:59
*** mjura has joined #openstack-ironic06:00
openstackgerritMerged openstack/proliantutils: Modify minimum disk for RAID 0 in hpssa  https://review.openstack.org/31171406:02
*** vsaienko1 has left #openstack-ironic06:03
*** vsaienko1 has joined #openstack-ironic06:03
*** mjura has quit IRC06:22
openstackgerritShivanand Tendulker proposed openstack/proliantutils: Add RIS support for power operations  https://review.openstack.org/28980006:22
*** milan has joined #openstack-ironic06:23
*** ijw has joined #openstack-ironic06:23
*** vsaienko1 has quit IRC06:29
*** vsaienko has joined #openstack-ironic06:30
vsaienkoMorning Ironic'ers06:30
*** moshele has quit IRC06:31
*** fragatin_ has quit IRC06:34
*** jtomasek has joined #openstack-ironic06:48
*** Sukhdev has quit IRC07:06
*** mjura_ has joined #openstack-ironic07:08
*** mjura has joined #openstack-ironic07:08
*** mjura has quit IRC07:09
*** vishwanathj has quit IRC07:11
*** e0ne has joined #openstack-ironic07:24
*** athomas has joined #openstack-ironic07:26
vsaienkoproject-config gates are broken by dib ;) http://logs.openstack.org/45/315545/6/check/gate-project-config-dib/95b1cbf/console.html#_2016-06-07_07_11_00_41794707:27
*** dtantsur|afk is now known as dtantsur07:28
dtantsurMorning Ironic, morning vsaienko07:28
dtantsuroh that DIB07:28
vsaienkomorning dtantsur07:28
vsaienkodtantsur what do you think about project-config-dib job for DIB?07:29
dtantsurvsaienko, I'm totally fine with it, but I'm not in charge in DIB :)07:29
dtantsuryou can chat with folks on #tripleo about it07:30
vsaienkodtabntsur can you please have look at http://logs.openstack.org/29/299229/14/check/gate-ironic-staging-drivers-dsvm-nv/5647b80/logs/screen-ir-cond.txt.gz#_2016-06-06_12_44_18_14107:31
vsaienkosorry, dtantsur: the error comes from https://review.openstack.org/#/c/299229/ amt driver, it requires pywsman07:32
dtantsurvsaienko, correct, I don't see pywsman installing in the devstack log07:33
vsaienkodtantsur: yeah, we can't just install pywsman, we need to compile/install openwsman first https://openwsman.github.io/07:34
dtantsurvsaienko, can't you install it from package?07:35
dtantsurmy F24 has openwsman-python package, for example07:35
*** ijw has quit IRC07:43
vsaienkodtantsur, it works if install openwsman and python-openwsman  packages07:44
*** ifarkas has joined #openstack-ironic07:47
*** ohamada has joined #openstack-ironic07:47
vsaienkodtantsur: I will prepare patchset07:48
*** e0ne has quit IRC07:48
dtantsurthnx07:51
dtantsurvsaienko, see also about s/bash/source/ there07:52
vsaienkodtantsur, yeah I just thought about it07:56
*** SiRiuS has joined #openstack-ironic07:57
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-ironic08:00
*** e0ne has joined #openstack-ironic08:02
*** e0ne has quit IRC08:02
lucasagomesmorning all08:04
*** mjura_ has quit IRC08:06
vsaienkomorning lucasabomes08:07
dtantsurmorning lucasagomes08:09
openstackgerritDavanum Srinivas (dims) proposed openstack/ironic: [WIP] Testing latest u-c  https://review.openstack.org/31844008:11
openstackgerritDavanum Srinivas (dims) proposed openstack/ironic: [WIP] Testing latest u-c  https://review.openstack.org/31844008:11
*** hoangcx has quit IRC08:20
*** bcornec1 has joined #openstack-ironic08:20
bcornec1Good morning evrybody08:20
bcornec1I'm trying to solve issues with my stadalone Ironic install, and would like some help for that.08:21
bcornec1Thanks to TheJulia, I've been able to have a working Bifrost and using the agent_ilo, beig able to enroll and activate my ProLiant server (380 Gen9 in UEFI mode)08:22
bcornec1Now, I'm facing issues with debugging of the install itself.08:22
bcornec1Besides issues with dib, I'm trying to understand how I can build a correct image to install on my node. Especially when it fails (which is all the time now :-)) I have no way to avoid that the node reboots. It would help if I could keep control of the node (through the Virtual console) and look at logs to understand what isn't working fine.08:24
bcornec1Any idea of what I should do for that ?08:24
bcornec1Also I'd like to understand which formats can be used to deploy the system, and what is the mechanism behind the scene used by IPA to realize that.08:26
*** rbartal has quit IRC08:26
*** mjura has joined #openstack-ironic08:28
*** MattMan has quit IRC08:37
*** MattMan has joined #openstack-ironic08:37
*** derekh has joined #openstack-ironic08:46
*** hoangcx has joined #openstack-ironic08:46
*** tesseract has joined #openstack-ironic08:50
*** moshele has joined #openstack-ironic08:51
lucasagomesbcornec, hi there, what is failing specifically (or that's what you are trying to figure out?) ?08:52
lucasagomesthe image gets deployed, so the node reboots and it fails to boot the new OS ?08:52
lucasagomesbcornec, for the serial, I believe you can use the ipmi console http://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-node-web-console08:53
lucasagomesfor the console*08:54
bcornec1Yes, it doesn't seem the install worked as there is nothing on the disk. I guesses it was due to lack of Smart Array support, so I started to add a Raid JSON file, but all that is trial and error from the outside, would be much better to get logs and console shell :-(08:54
bcornec1I don't use IPMI. I'm on the iLO support. Want to have agent_ilo working fine, in UEFI, so I can start hacking the Redfish support from that once it's working.08:55
bcornec1I'll need to change room as I'm with a customer for a PoC, on this !!! Connecting back in 10 minutes or so.08:56
bcornec1The node reboots, then goes to PXE. Using UEFI I can see that the disk has not been prepared. I also guess that the image may not be UEFI compliant, but again no easy way to see it ...08:56
*** milan has quit IRC08:59
*** bcornec1 has quit IRC09:01
*** electrofelix has joined #openstack-ironic09:03
*** dmk0202 has joined #openstack-ironic09:05
*** moshele has quit IRC09:06
*** dtantsur is now known as dtantsur|bbl09:10
*** vmud213 has joined #openstack-ironic09:17
openstackgerritSzymon Borkowski proposed openstack/ironic-python-agent: Add PCI devices list into the inventory  https://review.openstack.org/32631809:17
lucasagomesbcornec, right, so the ilo console is the same as the ipmi (https://github.com/openstack/ironic/blob/8e81b964a5fc815a7d42be749331571f4bb56be1/ironic/drivers/modules/ilo/console.py#L24)09:18
lucasagomesso you can use it09:18
openstackgerritAparna proposed openstack/proliantutils: Add support to sort values of raid_config for 'share_physical_disks'  https://review.openstack.org/32631909:19
lucasagomesbcornec, you will have more logs on the console, but maybe make the environment more constraint for testing first and then gradually add stuff? Like disable UEFI, check if it works09:20
lucasagomesto try to isolate where's the problem09:20
sambetts|afkMorning all09:23
*** sambetts|afk is now known as sambetts09:23
* sambetts is sad because the neutron fwaas has broken his CI 09:24
*** mgoddard_ has quit IRC09:26
*** e0ne has joined #openstack-ironic09:28
*** vdrok has joined #openstack-ironic09:28
*** bcornec1 has joined #openstack-ironic09:31
sambettslooks like there is already a fix merging :)09:31
openstackgerritMerged openstack/python-ironicclient: Add sanity tests for testing actions with Port  https://review.openstack.org/24094409:31
*** moshele has joined #openstack-ironic09:38
*** mbound has joined #openstack-ironic09:39
*** sacharya has joined #openstack-ironic09:40
*** causten_ has quit IRC09:43
openstackgerritMerged openstack/ironic: Update resources subnet CIDR  https://review.openstack.org/31708209:43
*** causten_ has joined #openstack-ironic09:43
openstackgerritSam Betts proposed openstack/ironic-specs: Add spec for interface attach and detach API  https://review.openstack.org/31763609:44
*** sacharya has quit IRC09:45
lucasagomessambetts, morning09:46
sambettso/ lucasagomes09:47
openstackgerritAparna proposed openstack/proliantutils: Sort values of raid_config for 'share_physical_disks'  https://review.openstack.org/32631909:48
*** vishwanathj has joined #openstack-ironic09:54
openstackgerritBharath kumar proposed openstack/ironic-specs: Correct RAID approved spec JSON  https://review.openstack.org/32634509:56
vdrokmorning ironic09:57
*** vishwanathj has quit IRC09:57
*** vishwanathj has joined #openstack-ironic09:58
lucasagomessambetts, quick q: where do I find the packages/applications available for tiny core linux?10:03
sambettslucasagomes: http://distro.ibiblio.org/tinycorelinux/7.x/x86_64/tcz/10:03
lucasagomessambetts, cheers10:03
*** vishwanathj has quit IRC10:04
openstackgerritDavanum Srinivas (dims) proposed openstack/ironic: [WIP] Testing latest u-c  https://review.openstack.org/31844010:10
*** milan has joined #openstack-ironic10:16
milanmorning ironic10:17
sambettso/ milan10:17
milansambetts, o/ :)10:19
openstackgerritVasyl Saienko proposed openstack/ironic: WIP: Patch to test Grenade.  https://review.openstack.org/31110910:21
openstackgerritMerged openstack/ironic-inspector: Remove deprecated alias "root_device_hint" for "raid_device" hook  https://review.openstack.org/32535110:22
*** athomas has quit IRC10:24
*** hparekh has quit IRC10:31
*** dtantsur|bbl is now known as dtantsur10:32
dtantsurMorning sambetts, milan10:32
openstackgerritShivanand Tendulker proposed openstack/ironic-lib: Support configdrive in iscsi deploy for whole disk images  https://review.openstack.org/23092410:37
*** hoangcx has quit IRC10:38
*** rbartal has joined #openstack-ironic10:41
sambettsHi dtantsur10:43
vdrokmorning milan sambetts and dtantsur10:43
sambettso/ vdrok10:43
dtantsurmorning vdrok10:44
openstackgerritAparna proposed openstack/proliantutils: Passing 'process_input' to the running hpssacli process  https://review.openstack.org/32573010:44
*** hparekh has joined #openstack-ironic10:44
*** itamarl has joined #openstack-ironic10:46
openstackgerritLucas Alvares Gomes proposed openstack/ironic-specs: Collect system logs from IPA upon a deploy failure  https://review.openstack.org/32351110:50
dtantsursambetts, when you have some time take a look at spec https://review.openstack.org/315413 please. looks promising, but I'm afraid I can miss something10:55
openstackgerritShivanand Tendulker proposed openstack/ironic-lib: Support configdrive in iscsi deploy for whole disk images  https://review.openstack.org/23092410:55
sambettsdtantsur: sure :) \10:55
*** amotoki has quit IRC11:00
*** wajdi has joined #openstack-ironic11:08
*** wajdi has quit IRC11:15
milanmorning vdrok, dtantsur11:20
*** amotoki has joined #openstack-ironic11:20
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Add CLI tests to check JSON response body  https://review.openstack.org/29123911:25
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Add test for 'port-show --field' command  https://review.openstack.org/30400811:26
*** mgoddard has joined #openstack-ironic11:28
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Negative tests for port-related commands  https://review.openstack.org/25935411:28
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing port-create command  https://review.openstack.org/29180211:30
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing port-create command  https://review.openstack.org/29180211:30
openstackgerritBharath kumar proposed openstack/ironic: adding back the inspect_ports as an optional parameter  https://review.openstack.org/32523011:31
*** joprovost has joined #openstack-ironic11:34
*** chihhsin has quit IRC11:35
*** lucasagomes is now known as lucas-hungry11:40
*** sacharya has joined #openstack-ironic11:41
*** joprovost has quit IRC11:43
*** sacharya has quit IRC11:46
sambettsdtantsur: just reading through the pci spec, does this spec need reworking to populate this information in the hardware manager/inventory and process it from the inventory on the inspector side?11:46
dtantsursambetts, that's a hard question. I'm afraid that such change would pollute the inventory with not-so-useful for everyone stuff11:47
dtantsuralso I'd prefer it to be disableable11:47
dtantsurjroll, you opinion re ^^^? spec https://review.openstack.org/31541311:48
openstackgerritMerged openstack/ironic: Keep original error message when cleaning tear down fails  https://review.openstack.org/32599111:49
sambettsdtantsur: I thought we were moving away from having inspector specific collectors though right, using the inventory instead? Thats why I implemented the lldp stuff in the generic hardware manager11:50
openstackgerritAline Bousquet proposed openstack/ironic-lib: Add support for metrics  https://review.openstack.org/30152611:51
dtantsursambetts, I've never heard about moving away from inspector collectors. I rather thought of it as of something as weird it probably should not go to inventory.11:52
dtantsursambetts, e.g. the extra_hardware collector gets back HUGE amount of data and requires plenty of dependencies on the ramdisk11:52
dtantsurso I was thinking this PCI case might be the same.. or maybe not11:52
* dtantsur looks how many PCI devices he has and tries to imagine11:53
*** kalpase has left #openstack-ironic11:53
dtantsurso, my dell tower has 77 PCI devices. it will probably increase the inventory substantially11:54
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing node-create command  https://review.openstack.org/26205511:54
*** thrash|g0ne is now known as thrash11:56
openstackgerritVladyslav Drok proposed openstack/ironic: Update the api-ref documentation for Drivers  https://review.openstack.org/31370811:58
*** dprince has joined #openstack-ironic11:59
*** amotoki has quit IRC12:00
*** baoli has joined #openstack-ironic12:01
sambettsdtantsur: hmmm, interesting, I guess I don't know the full extent of the inventory's uses, I guess I have the attitude of if its useful for inspector its probably useful for someone else, but I don't know who that someone else is12:01
*** baoli_ has joined #openstack-ironic12:02
*** baoli has quit IRC12:05
*** amotoki has joined #openstack-ironic12:09
dtantsursambetts, "I don't know the full extent of the inventory's uses" neither do I, that's the problem :)12:12
sambettsJayF, maybe you can help enlighten us ^12:13
openstackgerritVladyslav Drok proposed openstack/ironic: Gracefully degrade start_iscsi_target for Mitaka ramdisk  https://review.openstack.org/31918312:15
*** mgould|afk is now known as mgould12:17
sambettso/ mgould12:17
openstackgerritDmitry Tantsur proposed openstack/ironic-specs: Tenant network separation for in-band inspection  https://review.openstack.org/32000312:17
dtantsurhey-hey mgould, back finally?12:17
mgouldyep12:18
mgouldsystem upgrades: just say no12:18
sambettshehe12:18
dtantsurlol12:19
dtantsur(says man using F24 beta because of hardware upgrade)12:19
sambettsdtantsur: did you manage to get most of your issues sorted?12:19
openstackgerritBrad P. Crochet proposed openstack/python-ironicclient: Implementation of baremetal power state commands  https://review.openstack.org/17251712:20
openstackgerritBrad P. Crochet proposed openstack/python-ironicclient: Add maintenance mode commands  https://review.openstack.org/21672712:20
openstackgerritBrad P. Crochet proposed openstack/python-ironicclient: Add provision state commands  https://review.openstack.org/20611912:20
mgoulddtantsur: I considered installing F24, but decided I'd already exceeded my YOLO ration12:21
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Fix quotation mark in docstring  https://review.openstack.org/32641612:21
dtantsurhehe12:21
*** trown|outtypewww is now known as trown12:22
*** piet has joined #openstack-ironic12:22
*** piet has quit IRC12:26
*** jjohnson2 has joined #openstack-ironic12:29
*** jcoufal has joined #openstack-ironic12:30
TheJuliagood morning everyone12:33
sambettsHi TheJulia12:33
mgouldmorning TheJulia12:36
jrolldtantsur: my opinion on what, specifically?12:36
jrollalso how do you have 77 pci devices O_o12:37
jrollmorning everyone12:37
* jroll $ lspci | wc -l12:38
jroll1512:38
vdrokmorning TheJulia and jroll12:38
jrollsambetts: also, we don't use inspector so I'm not sure if jay can enlighten you :P12:39
dtantsurjroll, dunno, I used the same command :) on which hardware do you use it?12:40
*** lucas-hungry is now known as lucasagomes12:40
sambettsjroll: I'm hoping JayF might have an idea of what the IPA inventory is used for other than inspector12:40
dtantsurjroll, opinion on putting heavy stuff to inventory versus creating an inspection collector12:40
jrolldtantsur: a tower I built12:40
vdroksambetts, there is a return inside the loop instead of break12:41
jrollsambetts: oh, originally we had hoped to store it all in ironic but it never happened because, cmdb12:41
dtantsurjroll, "I built" may be the key :)12:41
jrollheh, yeah12:41
jrollit has a gpu and stuff, dunno12:41
jrolldtantsur: anyway, now I see what you mean, that does seem heavy for the inventory posted to ironic12:41
sambettsvdrok: oooh yeah sorry, my eyes were playing silly with the indentation, I thought the return was outside the loop12:42
*** rbudden has joined #openstack-ironic12:42
lucasagomesTheJulia, jroll vdrok morning12:42
vdroksambetts, yep, gerrit is not good at indentation highlight in this case :)12:42
vdrokmorning lucasagomes12:42
dtantsuralso g'morning jroll, TheJulia12:43
jroll:)12:43
dtantsurjroll, could you please evaluate RFE https://bugs.launchpad.net/ironic/+bug/1572529 as soon as you get some coffee?12:44
openstackLaunchpad bug 1572529 in Ironic "[RFE] DRAC RAID configuration" [Wishlist,In progress] - Assigned to Imre Farkas (ifarkas)12:44
dtantsursambetts, so, are you fine with keeping the PCI listing in a collector?12:45
jrolldtantsur: sounds like generic raid but do it through the bmc, seems fine to me12:46
dtantsurgood. rfe-approved?12:46
openstackgerritJulia Kreger proposed openstack/ironic-specs: Update active node creation spec based on feedback  https://review.openstack.org/32606612:47
sambettsdtantsur: not sure, probably initally, I guess the alternative is to extend the hardware manager, and have a kernal parameter to toggle whether to include it in the inventory or not12:47
dtantsursambetts, well, I don't quite like having inventory depend on kernel parameters too much (lldp being an exception). especially if nobody except for inspector is going to benefit from it12:48
dtantsur(in this case list of collectors is a kernel parameter you're looking for)12:48
openstackgerritJulia Kreger proposed openstack/ironic-specs: Update active node creation spec based on feedback  https://review.openstack.org/32606612:49
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing node-create command  https://review.openstack.org/26205512:49
sambettsso the thing thats bugging me is that I don't why we bother with the hardware manager inventory if 1 Ironic ignores it, and 2 inspector is going to circumvent it12:50
*** links has quit IRC12:51
*** ccarmack has joined #openstack-ironic12:51
*** links has joined #openstack-ironic12:52
dtantsursambetts, well, not really circumvent, that's not true. we're still extending inventory (just landed CPU flags and boot mode).12:52
*** thiagop has joined #openstack-ironic12:53
dtantsursambetts, the problem is that sometimes we need to get A LOT of information. e.g. the extra_hardware collector collects a JSON with hundreds to thousands elements12:53
dtantsurwe don't want THAT to be posted to ironic :)12:53
thiagopGood morning Ironic12:53
dtantsurmorning thiagop12:53
jrolldtantsur: yeah, I'll approve it12:53
dtantsurthnx12:54
jrolldtantsur: sambetts: I'd love to actually store the hw mgr inventory in ironic one day, I think we could do some interesting things with it12:55
dtantsurjroll, we could, yeah. so would you prefer the PCI listing to be there?12:55
jrolldtantsur: probably not yet :D12:56
jrolldtantsur: if we do it, let's figure out the db structure and such first (well, and if it's useful at all), and then maybe promote some collectors as needed12:57
dtantsurmakes sense12:57
*** bcornec1 has quit IRC12:57
dtantsursambetts, ^^?12:57
sambettsdtantsur: right, but we're extending it for ourselves only, ideally I guess we should all be using the hardware managers, and I guess we should have a list of hardware managers to run as a kernal parameter, like we have for collectors now12:57
jrollanother alternative is to always use the hw mgrs, but be very specific (in code, not config) about what's sent to ironic12:58
*** rloo has joined #openstack-ironic13:00
*** oberezovskyi has quit IRC13:01
*** vmud213 has quit IRC13:03
sambettshmmm, /me suddenly had the thought that we should be not posting any inventory etc at lookup/inspection, and if inspector/ironic wants specific info it should be able to request it via the API :/13:04
sambetts(obviously we'd have to post the mac address/ipmi address because lookup)13:05
dtantsursambetts, I have a patch for not posting it on lookup, but for inspection.. it brings us close to the idea of SSH ramdisk, doesn't it? ;)13:05
jrollsambetts: yeah, not a bad idea13:06
*** Goneri has joined #openstack-ironic13:08
dtantsuryeah, but a bit unrelated to this particular patch :)13:08
jrollgiving someone else a chance to approve this: https://review.openstack.org/#/c/319183/2013:08
dtantsurs/unrelated/out of scope/13:08
jroll1/2 of the patches left to get grenade working13:08
sambettsdtantsur: a refactor I guess13:08
dtantsursambetts, yeah. I just don't want to block this patch until we make up our mind and do the refactor13:09
dtantsurjroll, looking13:09
jrollwell, hopefully 1/2, it depends on a -2 patch :|13:09
sambettsdtantsur: sure, ok, I'll re-read that spec with that in mind :)13:10
dtantsurjroll, usually I would -1 it for missing logging13:10
jrolldtantsur: oh, good catch,t hanks13:10
dtantsurvdrok, could you please follow up with a logging if we approve https://review.openstack.org/#/c/319183/ now?13:10
jrolltoo late, I -1'd13:11
dtantsurok :)13:11
*** r-daneel has joined #openstack-ironic13:11
*** amotoki has quit IRC13:13
openstackgerritJulia Kreger proposed openstack/ironic: Active Node Creation via adopt state  https://review.openstack.org/27576613:13
NobodyCamGood Morning Ironicers, says the person who feels much less like a zombie then yesterday.13:14
sambettsHi NobodyCam13:14
NobodyCamhey sambetts :)13:14
*** nico-afk is now known as nicodemos13:15
TheJuliaNobodyCam: Have you seen the movie Maggie?13:15
*** amotoki has joined #openstack-ironic13:15
jrollmorning NobodyCam :)13:16
NobodyCamMorning TheJulia jroll13:17
NobodyCam:)13:17
NobodyCamTheJulia: I don't think I've seen that one yet13:18
rbartallucasagomes, dtantsur, Hi guys, I try to disable the erase_device clean step by setting its priority to 0 (erase_devices_priority=0) and got this error "Failed to tear down from cleaning for node"13:21
nicodemosmorning ironic!13:22
NobodyCammorning nicodemos13:22
thiagopmorning NobodyCam13:22
*** chihhsin has joined #openstack-ironic13:22
NobodyCammorning thiagop :)13:22
dtantsurmorning NobodyCam13:22
dtantsurand nicodemos13:22
NobodyCamhey hey dtantsur :)13:23
NobodyCamgood morning :)13:23
dtantsurjroll, rbartal sees a strange keystone error (HTTP 404) when tearing down cleaning. have you seen anything like that?13:23
jrolldtantsur: nope O_o13:23
*** r-daneel has quit IRC13:24
jrolldtantsur: at which piece, I'm surprised we talk to keystone on teardown at all, maybe to talk to neutron though?13:24
dtantsurjroll, we talk neutronclient, which talks to keystone first13:24
dtantsurto authenticate the request13:24
jrolldtantsur: rbartal: is it reproducible every time or sporadic?13:24
jrollalso, master or?13:24
lucasagomesyeah, this happens trhough neutronclient when it tries to list the ports13:25
dtantsurmitaka. dunno about reproduceability13:25
NobodyCamgood morning lucasagomes13:26
jrollwe let neutronclient auth for us, so could be a bug there or just bad config13:26
jrolle.g. if conf.neutron.auth_url is wrong13:26
lucasagomesNobodyCam, hi there13:26
dtantsurwell, but deployment seems to work13:26
NobodyCam:)13:26
dtantsurjroll, I don't think we have conf.neutron.auth_url, it uses keystone_authtoken section (which is wrong, but still)13:26
jrolldtantsur: yeah, I'd guess some sort of bug in the client then13:26
jrollmitaka certainly has neutron.auth_url13:27
jrollI'm staring at it :)13:27
dtantsurjroll, link handy?13:27
jrolloh no, you're right13:27
* jroll is dumb13:27
jrollyou're sure the 404 is keystone?13:27
*** sdake has joined #openstack-ironic13:28
dtantsurrbartal, mind pasting the backtrace to a public location?13:28
rbartaldtantsur, sure, 1 sec13:28
*** frontrunner has joined #openstack-ironic13:29
*** PollyZ has joined #openstack-ironic13:29
*** maurosr has quit IRC13:30
rbartaldtantsur, jroll http://paste.openstack.org/show/508645/13:32
*** _milan_ has joined #openstack-ironic13:32
* lucasagomes finds it odd that unauthorized returns 40413:32
dtantsurI think they have a generic exception there13:32
rbartallucasagomes, Bluejeans link please13:33
*** milan has quit IRC13:33
jrollso, File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 335, in retry_request13:34
jrollmakes me think the token went invalid somewhere here13:34
jrollwhich is why I ask how reproducible this is :)13:34
*** sdake_ has joined #openstack-ironic13:34
openstackgerritMerged openstack/ironic-inspector-specs: Generic PCI Resource spec  https://review.openstack.org/31541313:34
* sambetts thinks I've seen this before 13:34
*** sdake has quit IRC13:36
sambettsjroll: I think I merged a fix for this which later got reverted13:36
*** r-daneel has joined #openstack-ironic13:36
jrollsambetts: :|13:37
*** maurosr has joined #openstack-ironic13:40
*** causten_ has quit IRC13:42
sambettsjroll: So my at least part of my change is still there I think, I belive I saw this issue when the token in the task context is reused and passed into neutron client13:44
*** mgoddard_ has joined #openstack-ironic13:45
sambettsjroll: in my out of tree driver I think I fixed it by just simply not passing a token into neutron _build_client13:45
sambettsjroll: but there was a problem were we didn't pass the username and password into neutron client in _build_client, so in retry_request in neutron client it fell over because it couldn't reauthenticate13:46
sambettsjroll: and thats what I fixed in Ironic13:46
jrollsambetts: oh, interesting13:46
*** mgoddard has quit IRC13:48
*** amotoki has quit IRC13:48
sambettsjroll: https://bugs.launchpad.net/ironic/+bug/152477913:50
openstackLaunchpad bug 1524779 in Ironic "Neutron client will fail to re-authenticate if token expires because no credentials are provided" [Undecided,Fix released] - Assigned to Sam Betts (sambetts)13:50
jrollsambetts: so part of that was reverted? why?13:51
*** ametts has joined #openstack-ironic13:53
sambettsjroll: I'm sure I remember it being reverted, but git blaming it, it seems like some of it is there still, there was some talk that Keystone v3/auth/sessions would fix this I think :/13:53
jrollsambetts: hrm, ok13:54
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing node-create command  https://review.openstack.org/26205513:55
sambettsjroll: looking at the code, I would guess that one of these         params['username'] = CONF.keystone_authtoken.admin_user13:57
sambetts        params['tenant_name'] = CONF.keystone_authtoken.admin_tenant_name13:57
sambetts        params['password'] = CONF.keystone_authtoken.admin_password13:57
sambettsis misconfigured for rbartals error13:57
jrollsambetts: well, then it would have a hard time auth'ing incoming requests no?13:58
jrollit's also used for the authtoken middleware13:58
dtantsurcorrect13:58
sambettshmmm, good point :/13:58
* jroll takes super early lunch, bbl13:59
sambettso/13:59
*** Goneri has quit IRC14:01
*** rloo has quit IRC14:03
*** rloo has joined #openstack-ironic14:03
sambettsdtantsur, rbartal: Do you know if your running ks v2 or v3 ?14:05
dtantsurjudging by configuration, v3. not sure what actually.14:06
*** bcornec1 has joined #openstack-ironic14:06
rbartaldtantsur, How can I check?14:08
openstackgerritHugo Nicodemos proposed openstack/python-oneviewclient: Adding dynamic allocation operations  https://review.openstack.org/25869914:10
*** ChrisAusten has joined #openstack-ironic14:10
dtantsurrbartal, `openstack endpoint show identity` shows v2.0 for me on liberty, for example14:10
dtantsurthough my ironic configuration also used v3. wtf??14:10
devanandamorning, all14:13
sambettsrbartal: you don't happen to have the keystone logs for the other end of this request do you? Because the 404 error in the ironic logs doesn't give us much insight into which part of the request caused the 40414:14
devanandahmm. not awake yet, but I see ya'll talking about keystone v3 ?14:14
thiagopdtantsur: I think both works, but the rc file by default sets 2.0 on liberty14:14
sambettsdevananda: yeah, rbartal is seeing a weird keystone 404 from neutron client on cleaning, not sure if related to https://bugs.launchpad.net/ironic/+bug/1524779 which I fixed a little while ago14:14
openstackLaunchpad bug 1524779 in Ironic "Neutron client will fail to re-authenticate if token expires because no credentials are provided" [Undecided,Fix released] - Assigned to Sam Betts (sambetts)14:14
dtantsurmorning devananda. early for you, isn't it?14:15
thiagopdtantsur: that was supposed to change on mitaka, don't know if it landed though14:15
devanandadtantsur: a bit, yes14:15
rbartalsambetts, I can get the keystone log, just need some time14:15
devanandaI have been looking at keystone v2/v3 things recently as I've been adding policy support14:15
*** chlong has quit IRC14:16
rbartaldtantsur, http://paste.openstack.org/show/508667/14:16
*** Goneri has joined #openstack-ironic14:17
devanandarbartal: what's the error you're seeing in ironic's logs?14:17
sambettsdevananda: http://paste.openstack.org/show/508645/14:18
sambettsironic logs ^14:18
thiagoprbartal: do you see anything on neutron logs?14:18
lucasagomesfolks, wondering what you guys think about the proposed solution here: https://bugs.launchpad.net/ironic/+bug/158890114:19
openstackLaunchpad bug 1588901 in Ironic "Nodes stuck on power state transitions" [High,Confirmed] - Assigned to Lucas Alvares Gomes (lucasagomes)14:19
rbartalthiagop, I'm going to check14:20
rbartaldtantsur, sambetts my keystone is v214:20
mgouldmorning devananda14:21
thiagoplucasagomes: wonder about the workload of a periodic like that, but seem like a good solution14:22
devanandalucasagomes: minor edit to your proposal there:14:22
openstackgerritJohn L. Villalovos proposed openstack/ironic: Update comment from NOTE to TODO  https://review.openstack.org/32650414:22
lucasagomesthiagop, devananda thanks, will take a look14:22
devanandalucasagomes: http://paste.openstack.org/show/508668/14:23
*** rama_y has joined #openstack-ironic14:23
*** Goneri has quit IRC14:23
thiagopdevananda: question: do we have to know which conductor is handling a node? I never deployed Ironic with multiple conductors...14:24
lucasagomesdevananda, a-ha ++ yes, we should only look at the conductor that are marked as "offline" inded14:24
thiagophow to know*14:24
devanandathiagop: yes14:24
thiagopvery cool14:24
lucasagomesthiagop, if you look at the "reservation" field of the node, that's the name of the last conductor that touched that node14:25
lucasagomeslast/current*14:25
jlvillalvsaienko, vdrok mgould: We are getting very close on Grenade :)  2 patches remaining, I think.14:26
sambetts\o/14:26
dtantsur\o/14:26
jlvillal:)14:26
* jlvillal needs to get ready and into the office for an 8am meeting14:27
vdrokmorning jlvillal14:29
vdrokdtantsur, jroll fixing now14:29
lucasagomesjlvillal, w00t14:31
*** _milan_ has quit IRC14:32
*** chlong has joined #openstack-ironic14:32
NobodyCammorning vdrok jlvillal :)14:33
vdrokmorning NobodyCam14:33
*** frontrunner has quit IRC14:33
thiagoplucasagomes: I thought that field would be used for something on the claims thing :)14:33
NobodyCam:)14:33
*** mgoddard_ has quit IRC14:33
lucasagomesthiagop, well, basically the claim would reserve the node for a given conductor to be deployed on14:34
lucasagomesso yeah, I guess it would be used yes14:34
lucasagomesif we have a claims api, last proposed changes on scheduling in nova may not need that14:34
rbartalsambetts, dtantsur devananda keystone error: "2016-06-06 17:26:28.883 1531 DEBUG keystone.middleware.auth [req-df8007ad-0837-4ede-b8d3-b68ef5701a22 - - - - -] There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set. _build_auth_context /usr/lib/python2.7/site-packages/keystone/middleware/auth.py:7114:35
rbartal2016-06-06 17:26:28.883 1531 INFO keystone.common.wsgi [req-df8007ad-0837-4ede-b8d3-b68ef5701a22 - - - - -] GET http://172.16.1.1:5000/v2.0/14:35
thiagoplucasagomes: I heard of it, I need to keep up on it today14:35
lucasagomesthiagop, https://review.openstack.org/#/c/312696/14:35
thiagoplucasagomes: thanks14:35
*** sdake_ has quit IRC14:36
thiagoprbartal: I would use paste.openstack since logs have a bad reading on irc14:36
thiagop:)14:37
*** Goneri has joined #openstack-ironic14:37
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing chassis-create command  https://review.openstack.org/29363414:37
*** blakec has joined #openstack-ironic14:38
vdrokdtantsur, jroll are you good with the following? "The ironic python agent in the ramdisk on node %(node)s failed to start the iSCSI target because it doesn't support wipe_disk_metadata parameter, retrying without passing it. If you need to have node's root disk wiped before exposing it via iSCSI, please update the ramdisk to use version >= 1.3 (Newton, or higher) of ironic python agent."14:39
vdrokas a warning14:39
rbartalthiagop, right, http://paste.openstack.org/show/508678/14:42
jrollvdrok: sounds fine to me14:42
sambettsdtantsur, rbartal, devananda: Looking at the neutronclient code, _authenticate_keystone, which is where the error is occuring from, passes the user id or username and password and tenantName or id to keystone to generate a token14:42
*** mgoddard has joined #openstack-ironic14:43
openstackgerritVladyslav Drok proposed openstack/ironic: Gracefully degrade start_iscsi_target for Mitaka ramdisk  https://review.openstack.org/31918314:44
sambettsdtantsur, rbartal: from that keystone log it appears you are running both v2 and v3 APIs for keystone14:45
* sambetts is listening to the thunderstorm spinning up outside and is wondering if he's suddenly going to go AFK in a minute 14:46
rbartalsambetts, dtantsur http://paste.openstack.org/show/508683/14:46
* thiagop likes thunderstorms *_*14:49
sambettsthiagop: me too!14:49
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing chassis-create command  https://review.openstack.org/29363414:51
openstackgerritVladyslav Drok proposed openstack/ironic: WIP: Patch to test Grenade.  https://review.openstack.org/31110914:53
*** amotoki has joined #openstack-ironic14:53
*** frontrunner has joined #openstack-ironic14:53
JayFalineb: https://review.openstack.org/#/c/321153/ got merged so your patch should be unblocked now14:54
NobodyCammorning JayF14:54
sambettso/ JayF14:54
JayFo/14:54
JayFalthough brb voting so don't wave too hard yet :)14:54
dtantsurmorning JayF14:54
alinebo/ JayF, I've updated the patch according to the spec changes14:56
lucasagomesdtantsur, https://review.openstack.org/#/c/232571/ (capabilities as dict, apparent openstackgerrit is dead)14:59
jlvillalGood morning vdrok NobodyCam alineb dtantsur JayF sambetts rbartal jroll thiagop lucasagomes devananda rloo and everyone else :)14:59
sambettsMorning jlvillal14:59
dtantsurlucasagomes, fantastic!14:59
dtantsurmorning jlvillal14:59
dtantsurlucasagomes, 2 moments though: 1. will Nova understand it? 2. what if we make one more step and move capabilities to their own node attribute?15:00
thiagopgood morning jlvillal15:00
lucasagomesdtantsur, good points. So nova needs to be updated to support it15:02
*** absubram has quit IRC15:02
dtantsuryeah..15:02
lucasagomesdtantsur, we can move it to another field, biggest problem is the data migration15:02
rloogood morning jlvillal, sambetts, dtantsur, alineb, lucasagomes, thiagop, JayF, NobodyCam15:03
dtantsurmorning rloo15:03
dtantsurlucasagomes, it's a problem anyway...15:03
openstackgerritVladyslav Drok proposed openstack/ironic: Update the api-ref documentation for Drivers  https://review.openstack.org/31370815:03
dtantsurlike what do we do about clients which are not updated yet15:03
alinebmorning jlvillal, rloo15:03
lucasagomesdtantsur, yeah, depends how we want to do it, exactly, we probably need to create a proxy from properties/capabilities to the new field15:04
lucasagomesit gets complicated15:04
dtantsurlucasagomes, it's required even in your patch, but with a separate field it might be easier to mess with API versions15:04
lucasagomesdtantsur, if the idea is just make it easy to manipulate I think we should just do as the patch do, and have it as a dict (and string for backward compat)15:04
dtantsurlucasagomes, no, it's the same problem: how to make old clients work15:04
dtantsurlucasagomes, what I suggest is: if API version < 1.x, read and write properties/capabilities, else read and write node.capabilities15:05
lucasagomesdtantsur, interesting, yeah seems a way around that. Would we need to translate node.capabilities to properties/capabilities on older versions?15:06
*** vishwanathj has joined #openstack-ironic15:06
dtantsurlucasagomes, I'm afraid, yes15:06
lucasagomesdtantsur, maybe a spec would be good for it15:06
lucasagomescause there are corner cases15:06
dtantsurprobably15:07
*** yuriyz_ has joined #openstack-ironic15:08
*** aarefiev_ has joined #openstack-ironic15:08
*** blakec has quit IRC15:09
*** vdrok_ has joined #openstack-ironic15:09
*** e0ne_ has joined #openstack-ironic15:09
*** vsaienko1 has joined #openstack-ironic15:10
*** aarefiev has quit IRC15:12
*** yuriyz has quit IRC15:12
*** e0ne has quit IRC15:12
*** vsaienko has quit IRC15:12
*** Romanenko_K has quit IRC15:12
*** vdrok has quit IRC15:12
*** kromanenko_ has quit IRC15:12
*** kromanenko_ has joined #openstack-ironic15:12
*** kromanenko__ has quit IRC15:13
*** Romanenko_K has joined #openstack-ironic15:13
*** garthb has joined #openstack-ironic15:13
*** kromanenko__ has joined #openstack-ironic15:13
*** vdrok_ has quit IRC15:15
*** vdrok-afk is now known as vdrok15:16
*** rcernin has quit IRC15:17
*** e0ne_ has quit IRC15:18
*** e0ne has joined #openstack-ironic15:18
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Functional tests for portgroups in ironicclient  https://review.openstack.org/29772715:19
*** links has quit IRC15:26
*** itamarl has quit IRC15:28
openstackgerritSofiia Andriichenko proposed openstack/python-ironicclient: Add test for chassis-node-list  https://review.openstack.org/29948215:28
*** sdake has joined #openstack-ironic15:28
*** rama_y has quit IRC15:31
openstackgerritSofiia Andriichenko proposed openstack/python-ironicclient: Add test for chassis-node-list  https://review.openstack.org/29948215:31
jrolllucasagomes: dtantsur: if we're going to move capabilities around, can we get rid of the json field there anyway? :)15:33
dtantsurmaybe? :)15:34
jrollmoving to node.capabilities is 90% of that pain anyway15:34
lucasagomesjroll, heh that's the question. We probably need a spec for covering all the cases15:34
lucasagomesjroll, but yeah, if we do move let's make it indexable15:35
*** mjura has quit IRC15:35
jlvillalOne delicious comment line change patch up for review :)  https://review.openstack.org/#/c/326504/15:37
*** lifeless has quit IRC15:37
jrolllucasagomes: ++15:37
jlvillalsambetts: I'm curious about the current state of the Cisco CI. How's it working at the moment?15:40
lucasagomesdtantsur, btw, not today but when you get some time I've updated the logs spec with ur suggestions (local storage for standalone and so on) https://review.openstack.org/#/c/323511/15:40
dtantsurcool15:40
sambettsjlvillal: it had a sad moment last night, because tempest broke the neutron-fwaas tempest plugin15:41
sambettsjlvillal: the fix mergered this morning and its passing tests again now15:41
jlvillalsambetts: I just saw a failure on a patch I submitted an hour ago. So curious.15:41
jlvillalsambetts: Okay great :)15:41
sambettsjlvillal: oh link?15:41
jlvillalsambetts: https://review.openstack.org/#/c/326504/15:42
jlvillalsambetts: Okay slightly more than an hour ago.15:42
*** sacharya has joined #openstack-ironic15:42
dtantsurFYI I'm adding a topic to https://etherpad.openstack.org/p/ironic-newton-midcycle to discuss this situation around building RAID at deploy-time15:43
jlvillalJayF: Not sure if you saw my message about adding your name to the list of people working on CI improvements. I think you have been doing work in that area.15:43
sambettsdtantsur: awesome :D15:43
jlvillalhttps://etherpad.openstack.org/p/IronicWhiteBoard  Line 9615:43
JayFjlvillal: there's a list of people?15:44
JayFjlvillal: yeah, I've mainly been plugging all the new bits into project-config15:44
jlvillalPeople to blam^H^H^H^H give credit to.15:44
*** mtanino has joined #openstack-ironic15:44
JayFjlvillal: did a little bit of science as well but all that ended up with is knowing that ipxe is faster, and we shouldn't change our nic in qemu15:44
sambettsjlvillal: somethings gone weird in the devstack run for that test, its failing to configure a neutron router... but I have neutron l3 disabled in local.conf... soooo ... /me digs15:44
jlvillalJayF: I added you. If you object, feel free to delete your name :)15:45
jlvillalsambetts: Thanks!15:45
*** rbartal has quit IRC15:45
NobodyCamMorning rloo :-)15:45
jlvillalsambetts: There was a patch doing things to the neutron router, that that patch is updating the comment for.15:45
* jlvillal goes to find link.15:45
jlvillalsambetts: https://review.openstack.org/31708215:46
jlvillalsambetts: Not sure if that would blow your tests up or not.15:46
JayFjlvillal: I'm just curious where said list is :)15:46
sambettsjlvillal: shouldn't do, as far as I know that shouldn't run in a normal devstack flow :/15:46
jlvillalJayF: https://etherpad.openstack.org/p/IronicWhiteBoard  Line 9615:47
jlvillalsambetts: Oh yeah. True.15:47
jrolljlvillal: JayF: the subteam list of people is the people noted as contacts in the priorities doc15:47
jrollso, I removed JayF again :{P15:47
jrollwhoa, I grew a mustache15:47
sambettshahaha15:47
jlvillalheh15:47
JayFyeah please to not put me as a contact15:47
JayFlol15:47
* jlvillal wonders if jroll invented a new emoticon15:47
*** sacharya has quit IRC15:47
jrollJayF: I read your mind, don't worry :)15:48
jlvillalsambetts: I'm not sure then why the Cisco CI is breaking then :(15:49
*** lifeless has joined #openstack-ironic15:49
sambettsno, I'm trying to work out if something merged in the last 4 hours that might have broken something, my last passed test was @ 12:1615:50
*** harlowja_at_home has joined #openstack-ironic15:52
sambettsjlvillal: I think this might be the cause https://github.com/openstack-dev/devstack/commit/a45b40a630d4615455725bcfd8774c2afb05418915:53
jlvillalsambetts: Those son of a guns! ;)15:54
sambettsjlvillal: its the only thing thats merged in the last 4 hours that might have affected it, so now I've got to work out why...15:54
*** harlowja_at_home has quit IRC15:54
sambettsjlvillal: it seems like they have removed an is_service_enabled q-l315:55
sambettsjlvillal: which is causing it to go into code that it didn't go before15:55
*** absubram has joined #openstack-ironic15:55
*** sacharya has joined #openstack-ironic16:03
*** Jabadia has joined #openstack-ironic16:05
openstackgerritLucas Alvares Gomes proposed openstack/ironic-specs: Collect system logs from IPA upon a deploy failure  https://review.openstack.org/32351116:06
dtantsurlucasagomes, oops, also on one of previous revisions I've asked to have an option to *always* store deploy logs, even on success16:07
dtantsurcould you please add it too?16:07
lucasagomesdtantsur, yeah I saw that. I think I've asked to extend it later... lemme think on it a bit more16:09
*** frontrunner has quit IRC16:09
lucasagomesdtantsur, perhaps the option to enable/disable it should support values like "always", "deploy failure" or something like that?16:09
dtantsurI'm not going to block due to that, but based on my inspector experience that's an absolute must16:09
dtantsuryeah16:09
dtantsurenough times it just does the wrong thing without failing... >_<16:09
lucasagomesyeah, it can finish the deployment and the image does boot or things like that16:10
*** mgoddard has quit IRC16:10
openstackgerritJim Rollenhagen proposed openstack/ironic: Register all tempest options if not registered  https://review.openstack.org/32660716:10
openstackgerritJim Rollenhagen proposed openstack/ironic: Register tempest options unconditionally  https://review.openstack.org/32660816:10
lucasagomeslemme just see how it would look like to with that feature16:10
openstackgerritKyrylo Romanenko proposed openstack/python-ironicclient: Tests for testing node-create command  https://review.openstack.org/26205516:12
*** frontrunner has joined #openstack-ironic16:12
*** Jabadia has quit IRC16:13
*** tesseract has quit IRC16:13
*** Jabadia has joined #openstack-ironic16:13
lucasagomesJayF, you have any thoughts/objects on ^ ?16:14
lucasagomesI think it's useful, but will need to change the spec16:14
openstackgerritJim Rollenhagen proposed openstack/ironic: Register all tempest options if not registered  https://review.openstack.org/32660716:15
lucasagomesdtantsur, maybe I can leave open grounds to it? Make the config option open to accept other values (not only true/false) and do it as a separated work?16:15
lucasagomesI'm afraid of making the spec too big16:15
jrolloops, messed that one up16:15
lucasagomesppl may want log from cleaning and other tasks, I don't wanna keep putting too much onto that work16:15
lucasagomesbut I can make it simple to extend16:15
dtantsurlucasagomes, this is a tiny change, I'm not sure what's the problem with it16:17
*** dprince has quit IRC16:17
dtantsurlike: if deploy_failed or CONF.agent.store_deploy_logs == 'always'16:17
openstackgerritJim Rollenhagen proposed openstack/ironic: Register tempest options unconditionally  https://review.openstack.org/32660816:17
openstackgerritJim Rollenhagen proposed openstack/ironic: Register all tempest options if not registered  https://review.openstack.org/32660716:17
jrollthere we go16:17
*** r-daneel has quit IRC16:17
lucasagomesdtantsur, right, yeah, I guess most of the changes are just how we word the spec (even the title is about "deploy failure")16:19
lucasagomessame for problem description and so on16:19
lucasagomesdtantsur, I will mark as WIP and rewrite the bits tomorrow16:19
dtantsurlucasagomes, just a wider definition of "failure", I guess :)16:19
dtantsurlucasagomes, well, ok, we can add it as an RFE later16:19
lucasagomesdtantsur, it's cool, marking as WIP. Tomorrow I will update it16:20
jlvillalrloo: Great catch on https://review.openstack.org/#/c/319183/  :)16:20
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Add an option to not fail when root device is not supplied  https://review.openstack.org/32662016:21
rloojlvillal: which one. cuz the one i thought was wrong, was actually right.16:21
jlvillalrloo: Though you did do a +1.  /me would have thought -116:21
jlvillalrloo: Ahh. I didn't see your follow-on comment16:21
*** sabeen has joined #openstack-ironic16:21
rloojlvillal: oh wait. i think it is wrong. not sure.16:22
JayFlucasagomes: no objection here at all. I might would even advocate for our production config to enable it.16:22
jrolljlvillal: rloo: it's right16:22
lucasagomesJayF, cool, thanks. Marked the spec as WIP and will update it tomorrow16:22
rloojlvillal: depends on whether the return result line is indented under the while, or at the same level of the while. i can't tell by eyeballing.16:22
vdrokjlvillal: rloo it won't loop forever16:23
* jlvillal will look more closely16:23
rloojroll, vdrok: i think i would have liked a break instead of a continue. but that may just be me, cuz i tripped on the existing code.16:24
vdroksee the previous patchset comments16:24
jrollthe test verifies that it's correct, btw16:24
*** amotoki has quit IRC16:24
jrollrloo: sure, that's fair, just saying it's right :)16:24
*** Jabadia has quit IRC16:24
rloovdrok, jroll: yes, i realize it is right. just wondering whether we can/should redo it so that it is more obvious.16:24
*** ifarkas has quit IRC16:25
rloovdrok, jroll: or add more comments. eg, in this case, we don't retry cuz ...16:25
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Add an option to not fail when root device is not supplied  https://review.openstack.org/32662016:25
vdrokrloo: OK will do, just going to let the grenade run to finish16:26
*** r-daneel has joined #openstack-ironic16:26
rloovdrok: just a suggestion :)16:26
rloovdrok: why do we need to make a copy of param?16:26
*** jabadia has joined #openstack-ironic16:27
*** jaybeale has joined #openstack-ironic16:27
vdrokrloo: it's for the unittest unfortunately16:29
*** Romanenko_K has quit IRC16:29
*** jabadia has quit IRC16:29
*** kromanenko__ has quit IRC16:29
vdrokif not copy it, mock.call has the reference to a params dictionary16:29
*** Jabadia has joined #openstack-ironic16:29
vdrokso if we just do the del, it will be the same in both mock calls16:29
*** kromanenko_ has quit IRC16:29
*** dmk0202 has quit IRC16:30
rloovdrok: should at least put a comment then. although i really don't like it; i'd almost rather we didn't test for that.16:30
vdrokand instead of being shown as called with 3 params first time and 2 params second time, it will report that it was called twice with 2 params16:30
vdrokrloo: I can try to do it with mock.call_args[0][1] or something like that, but it will be uglier16:32
dtantsursee you tomorrow, g'night16:32
rloovdrok: so my preference would be no copy, and put a comment in the actual test. but that is me. i think if you want to do the copy, put a comment in the code.16:32
rloonight dtantsur16:32
vdrokgood night dtantsur16:32
rloovdrok: i think we want to get the code in, so i am fine with a comment. someone could try to improve on it later.16:33
NobodyCamnight dtantsur16:33
*** dtantsur is now known as dtantsur|afk16:33
*** fragatina has joined #openstack-ironic16:34
*** e0ne has quit IRC16:35
*** fragatina has quit IRC16:35
*** fragatina has joined #openstack-ironic16:35
*** e0ne has joined #openstack-ironic16:35
*** e0ne has quit IRC16:40
*** rama_y has joined #openstack-ironic16:40
*** derekh has quit IRC16:40
*** e0ne has joined #openstack-ironic16:41
jlvillalThe Ironic email thread has been interesting for me to read. FYI, if people haven't seen it: http://lists.openstack.org/pipermail/openstack-dev/2016-June/096715.html16:45
*** e0ne has quit IRC16:49
lucasagomescalling it a day, have a great evening all16:51
lucasagomestalk to you all tomorrow16:51
*** bharath has joined #openstack-ironic16:52
*** lucasagomes is now known as lucas-dinner16:52
vdroknight lucas-dinner16:52
NobodyCamnight lucas-dinner16:52
bharathcan someone review https://review.openstack.org/#/c/326345/ ?16:53
*** dprince has joined #openstack-ironic16:53
jlvillallucas-dinner: Ciao!16:55
jrollbharath: approved16:56
jrollthanks for that16:56
*** ijw has joined #openstack-ironic16:59
*** bharath has quit IRC16:59
*** vishwanathj has quit IRC17:00
*** r-daneel has quit IRC17:02
*** trown is now known as trown|lunch17:02
*** bcornec1 has quit IRC17:05
openstackgerritBrad P. Crochet proposed openstack/python-ironicclient: Implementation of baremetal power state commands  https://review.openstack.org/17251717:06
openstackgerritBrad P. Crochet proposed openstack/python-ironicclient: Add maintenance mode commands  https://review.openstack.org/21672717:06
openstackgerritBrad P. Crochet proposed openstack/python-ironicclient: Add provision state commands  https://review.openstack.org/20611917:06
vdrokrloo: so the only way I found to get rid of .copy in the function is to override _mock_call private method of mock.Mock to do the deepcopy of the arguments17:09
rloovdrok: don't do that. I don't think it is worth it.17:10
vdrok++17:10
rloovdrok: but thx for looking. just add a comment. the comment could also have a TODO(vdrok) get rid of this one day :)17:10
vdrokand I think we want to do some unittesting of what was called, so I'll add a comment17:11
vdrokyep17:11
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Update devstack plugin with new auth options  https://review.openstack.org/29731217:11
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Migrate to using keystoneauth Sessions  https://review.openstack.org/23698217:11
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Warn on legacy auth options on conductor start  https://review.openstack.org/32666017:11
*** ohamada has quit IRC17:13
*** moshele has quit IRC17:14
*** r-daneel has joined #openstack-ironic17:15
*** PollyZ has quit IRC17:20
*** amotoki has joined #openstack-ironic17:24
openstackgerritMerged openstack/ironic-specs: Correct RAID approved spec JSON  https://review.openstack.org/32634517:28
*** amotoki has quit IRC17:29
sambettsjlvillal: my CI got broke because that commit made the neutron tempest code suddenly start caring if I hadn't configured a IPv6 subnet CIDR in my local.conf17:31
jlvillalsambetts: Thanks for figuring it out! :)17:31
sambettsjlvillal: I've pushed an update to my tests local.conf to force v4 so it should me happy again now I hope17:31
*** sabeen has quit IRC17:37
*** PollyZ has joined #openstack-ironic17:37
sambettsAnyone else seeing the gerrit sign in broken?!17:38
*** sabeen has joined #openstack-ironic17:39
*** keekz_ is now known as nicholas17:40
*** nicholas is now known as keekz17:40
*** keekz is now known as nicholas17:40
*** PollyZ has quit IRC17:42
sambettsits late and I can't sign into gerrit, so /me -> afk17:44
*** nicholas is now known as keekz17:44
*** sambetts is now known as sambetts|afk17:44
*** Sukhdev has joined #openstack-ironic17:46
sambetts|afkNight all17:46
vdroknight sambetts|afk17:48
*** Sukhdev has quit IRC17:48
*** PollyZ has joined #openstack-ironic17:51
*** priteau has joined #openstack-ironic17:52
*** garthb_ has joined #openstack-ironic17:53
NobodyCamnight sambetts|afk17:54
*** garthb has quit IRC17:55
*** PollyZ has quit IRC17:55
*** fragatina has quit IRC17:56
*** e0ne has joined #openstack-ironic17:59
devanandaJayF: around?18:02
JayFaye18:02
*** electrofelix has quit IRC18:02
JayFWas out sick yesterday but roaring to go today18:02
*** e0ne has quit IRC18:02
devanandaJayF: ugh. well, glad you're feeling better!18:03
devanandaJayF: I've got the devstack changes up too, now, though I haven't finished the base changes to ironic18:03
JayFYeah; I saw the patchset exists but haven't looked at it in-depth yet18:03
JayFjust gone done w/morning meetings and email catchup and was going to start into 'real work' :)18:04
devanandaJayF: even though the service actually works, unit tests are blowing up with 500 errors, which is where I stopped debugging last night18:04
JayF500 errors! That's a lot of errors! :P18:05
JayFI can take a look at the tests if that's what you're asking? See what's tossing the 500 error18:06
devanandahah18:06
*** rbudden has quit IRC18:06
devanandaJayF: up to you. just sharing where I'm at. when you have spec things for me to review / add to, LMK18:07
JayFI'll add you to the review when I get it up18:07
openstackgerritVladyslav Drok proposed openstack/ironic: Gracefully degrade start_iscsi_target for Mitaka ramdisk  https://review.openstack.org/31918318:07
JayFlikely today maybe tomorrow18:07
devanandacool18:07
JayFhonestly depends on if I update rescue spec first or write that one first18:07
*** mgould is now known as mgould|afk18:07
JayFgiven the policy spec seems like ... paperwork (the main decisions are obvious/been made) I might do the rescue first18:07
*** moshele has joined #openstack-ironic18:08
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Warn on legacy auth options on conductor start  https://review.openstack.org/32666018:08
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Update devstack plugin with new auth options  https://review.openstack.org/29731218:08
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Migrate to using keystoneauth Sessions  https://review.openstack.org/23698218:08
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic: Update ironic.conf.sample  https://review.openstack.org/32668218:08
*** trown|lunch is now known as trown18:10
openstackgerritVladyslav Drok proposed openstack/ironic: Gracefully degrade start_iscsi_target for Mitaka ramdisk  https://review.openstack.org/31918318:12
*** piet_ has joined #openstack-ironic18:17
openstackgerritPavlo Shchelokovskyy proposed openstack/python-ironicclient: Move OpsnStack client to test-requirements  https://review.openstack.org/29773018:17
devanandajroll, JayF: do we really need a spec for policy?18:20
JayFI was just wondering that myself.18:20
devanandawe *definitely* need docs on it18:20
JayFbut usually when I think that, it means writing the spec is easy anyway18:21
jrollI think I'd like a spec listing the default policy and the endpoints, and maybe a bit about the implementation18:21
devanandaI'm pretty sure I have implemented it so that there's no immediate effect on upgrade (but I want grenade to convince me of that)18:21
devanandabut there will need to be operator actions to make the changes go into effect18:21
jrollwell, the immediate effect will almost certainly depend on the user people are using :)18:21
jrollyeah18:22
JayFtoday you have no option but to use an admin user18:22
JayFif admin user keeps having full perms, then what's the immediate effect jroll ?18:22
jrollJayF: yeah, I guess you're right18:23
jrolland there's no change until the new policy.json is dropped in18:23
JayFeven with the proposed policy.json, there's no change18:23
devanandajroll: assume that deployers may have changed policy.json18:24
devananda*do we ..18:24
devananda(I do)18:24
JayFme too18:24
JayFit's a config file, not code18:24
devanandaright18:24
JayFwhich is the whole reason for the policy-in-code thing18:24
devanandaand this can very definitely be changed today: https://github.com/openstack/ironic/blob/master/etc/ironic/policy.json#L218:24
JayFand if we do ours using policy-in-code (I hope we do?) even then, the default settings to be implemented would work with an existing, configured user aiui18:25
jroll+1, make that assumption18:25
devanandainside ironic, we enforce the "admin_api" policy18:25
devanandabut an operator could change what that means, as far as keystone roles18:25
*** amotoki has joined #openstack-ironic18:26
devanandathis is hte main check: https://github.com/openstack/ironic/blob/master/ironic/api/hooks.py#L12118:26
devanandabtw, this one is, as far as I can tell, actually a no-op: https://github.com/openstack/ironic/blob/master/ironic/api/hooks.py#L8218:26
*** ametts has quit IRC18:27
devanandamy point is, today, an operator actually could change what keystone roles pass our "is_admin" check18:27
JayFany way you slice it this should be upgrade-safe18:27
jrollright, agree with that18:27
devanandasuch an operator would need to modify the new policy.json to match their existing roles -- and they'd know that18:27
JayF1) if you edited your policy.json, it stays in place and we don't break you18:28
JayF2) If you didn't edit the policy.json, our new policies would mean the user you're using (admin role) would keep working18:28
devanandaJayF: correct18:28
JayFwe only have to be backwards compat with configs we shipped18:28
JayFnot with arbitrary user configs, because they get to keep those18:28
devanandaI will need to change https://review.openstack.org/#/c/325599/3/etc/ironic/policy.json a bit, actually18:29
devanandato retain compat, is_admin shouldn't check is_member18:29
devanandawell - it will pass devstack18:30
devanandabecause in devstack, the admin user is a member of hte demo project18:30
*** amotoki has quit IRC18:30
devanandaI feel like "but it works in devstack" is never a good explanation for something18:30
*** e0ne has joined #openstack-ironic18:32
*** PollyZ has joined #openstack-ironic18:32
JayFlol, that's a good philosophy18:32
*** baoli_ has quit IRC18:32
*** PollyZ has quit IRC18:37
*** rbudden has joined #openstack-ironic18:40
*** ametts has joined #openstack-ironic18:40
*** Sukhdev has joined #openstack-ironic18:47
*** PollyZ has joined #openstack-ironic18:47
*** mbound has quit IRC18:51
*** fragatina has joined #openstack-ironic18:58
*** frontrunner has quit IRC18:58
*** ChubYann has joined #openstack-ironic19:01
*** piet_ has quit IRC19:02
*** moshele has quit IRC19:03
*** SiRiuS has quit IRC19:09
devanandajroll, JayF: as we start enforcing policy in a lot more places, wdyt of changes in errors that an unauth'd user will get back?19:10
devanandaI can try to preserve them, or I can change them19:10
JayFthat's really interesting19:10
JayFbecause you can't really "hide" some of it behind a microversion19:10
NobodyCammorning devananda :)19:11
JayFb/c obviously you'd have to 404 any resource they weren't allowed to know if it existed (i.e. if you don't have node list, we have to return 404 for *any* node action, even if node is valid)19:11
devanandaJayF: right19:11
JayFI can't think of any case other than that where we'd have to 404 instead of 403 a resource, so maybe that's just hte most complex edge case19:11
devanandaNobodyCam: g'morning!19:12
NobodyCam:)19:12
JayFIs there any case where it makes sense to have a role that wouldn't have node-list?19:12
devanandaJayF: right now, I've put the policy checks as the very first call in every location19:12
JayFIf no; then it gets a hell of a lot simpler19:12
JayFbecause we don't have any other types of resources we'd have to fake a 404 for (instead of a 403)19:12
devanandaso, eg, it's happening before API version checks, API routing checks, validation of state machine, etc19:12
JayFand I'm all for changing errors to 403s19:12
JayFooooooh19:12
JayFthat's gnarly19:12
devanandawhich makes sense to me -- before ironic does any work, make sure the user is ALLOWED to request that work19:12
JayFbecause 403 could also be pushed by something upstream19:12
JayFlike if I run ironic wsgi in Apache, I upgrade, start getting 403s19:13
devanandadon't tell them "your request is invalid while node is in state X" if they aren't even authorized to see the list of Nodes19:13
JayFmy client can't know if it's some stuff setup in apache19:13
JayFor if Ironic is rejecting based on permissions19:13
devanandaright19:13
devanandaso yea, things get tricky19:13
JayF*if* we eliminate the security case (i.e. where you need to 404 a resource to deny info leakage)19:14
JayFand we could move the check behind a microversion, I think that's the golden solution19:14
devanandauh19:14
devanandawhy would we microversion policy support when our API technocally already supports it?19:14
devanandait's just becoming more granular19:15
JayFThat's valid too.19:15
JayFI think you change the errors, and it's fine19:15
JayFbut I know I'm highly biased toward forward motion over backwards compat19:15
*** frontrunner has joined #openstack-ironic19:15
JayFI'd be interested to hear what someone with a different use case woul think19:15
devanandafor instance, if an unauthenticated user requests GET /v1/chassis/CCCC/nodes/NNNN they will currently hit this code19:16
devananda        if self.from_chassis:19:16
devananda            raise exception.OperationNotPermitted()19:16
devanandaerm, edit19:16
devanandaif an *AUTHENTICATED* user ...19:16
devanandaan unauth'd user will get rejected by the TrustedCallHook much higher up the stack19:16
devanandaso to preserve that, I'm pretty sure I need to be checking policy before that if statement19:17
JayFauthenticated but unauthorized user19:17
devanandaright19:17
JayFthat still returns a 40319:18
JayFooooh19:18
JayFbut maybe a 404 is the 'better' thing in some cases19:18
JayFI think the key question is: Is there anything "secret" about acknowleding a  resource exists by returning 403 vs 40419:19
JayFIn Ironic, I suspect that answer is no, but I could be wrong19:19
JayFI think 404'ing resources you aren't authorized to know exists is a significant enough api change to outweigh the security risk19:20
devanandaJayF: yes19:20
JayFand almost completely undiscoverable by an unsophisticated operator19:20
devanandaJayF: with my security hat on, I would say we do not disclose whether a thing exists inthe response to a request, if the requestor has no permissions to act upon the thing19:21
devanandalike a bad login attempt -- you don't disclose the validity of the email address19:21
JayFYeah, but security is always a slider, right? between UX and security? In this case, I think the risk is small and the harm to UX would be significant19:22
devanandaI think we may be mincing hairs at this point19:22
JayFyep19:22
JayFI could easily go either way19:22
devanandaI think I can implement 403 DENIED at a level where it would happen regardless of whether the resource exists ir not19:22
JayFthat has no downsides then, does it? That sounds like the golden solution19:23
devanandalet's see if I'm right :)19:25
rloodevananda, JayF: what do the other openstack projects do? we should be consistent?19:27
devanandarloo: I believe the api-wg is trying to create consistency,but there isn't that today19:27
*** baoli has joined #openstack-ironic19:28
*** MattMan has quit IRC19:29
* JayF brb, off to lunch break19:30
*** MattMan has joined #openstack-ironic19:30
*** e0ne has quit IRC19:40
*** SiRiuS__ has joined #openstack-ironic19:43
*** morgabra_ is now known as morgabra19:52
*** morgabra has joined #openstack-ironic19:52
*** mbound has joined #openstack-ironic19:52
*** jaybeale has quit IRC19:54
*** trown is now known as trown|brb19:55
*** Goneri has quit IRC19:56
*** mbound has quit IRC19:57
* jlvillal wonders if gate is broken or if he is just unlucky. Goes to read the log files19:58
* jlvillal is unlucky19:58
*** trown|brb is now known as trown19:59
*** Sukhdev has quit IRC20:02
*** clenimar has joined #openstack-ironic20:04
jlvillalHas anyone else seen gate failures where Apache fails to restart? I've seen two different jobs experience it over two test runs. A job failed in one run and a different job failed in the other run. But both with Apache failing to restart.20:06
TheJuliajlvillal: as part of horizon setup?20:07
jlvillalTheJulia: Yep20:08
jlvillalSample at very end of: http://logs.openstack.org/04/326504/1/gate/gate-tempest-dsvm-ironic-pxe_ipa/3680916/logs/devstacklog.txt.gz20:08
TheJuliajlvillal: I had two jobs that failed yesterday due to that, I noticed it, searched around, saw no known bugs so kind of figured it might have been a transient fluke, and rechecked to see if it appeared in more jobs or if everything cleared up.  Truthfully I bet the socket is being held open by the kernel and thus cannot be rebound until it times out20:10
jlvillalTheJulia: Thanks. I forget the magic needed to search the failed job messages.20:10
jlvillalI know there is some master log stash. And you can search it. I just forget how. Not sure I really have learned how. The UI was a bit confusing the last time I remember using it.20:11
*** baoli has quit IRC20:12
TheJuliajlvillal: http://logstash.openstack.org/#/dashboard/file/logstash.json20:12
jroll++20:12
TheJuliaI've used it once in a blue moon, but yeah, you have to be very careful with what you submit to the ES cluster20:12
jrollthis is a perfect thing to file a bug for, and add to elastic-recheck20:12
TheJulia++20:12
jroll(and e-r is a good source of example queries)20:12
*** baoli has joined #openstack-ironic20:12
TheJulia++20:13
jlvillalTheJulia: That's the UI. Maybe it is better, but last time I failed miserably trying to make it work :)20:13
jlvillaljroll: Thanks20:13
TheJuliajlvillal: custom queries, just remember to keep a time constraint on the queries, the longer you go out, the more on disk it has to look at, thus gets slower :)20:14
jlvillalTheJulia: Noted. Going to clone elastic recheck and file a bug.20:14
jlvillalhttps://bugs.launchpad.net/ironic/+bug/159013920:17
openstackLaunchpad bug 1590139 in Ironic "Gate jobs failing during Horizon setup" [Undecided,New]20:17
*** jjohnson2 has quit IRC20:17
openstackgerritMerged openstack/python-oneviewclient: Adding dynamic allocation operations  https://review.openstack.org/25869920:17
*** jjohnson2 has joined #openstack-ironic20:17
jrolljlvillal: for e-r purposes, I'd re-name the bug to be the actual failure, rather than "this makes our jobs fail"20:18
jlvillaljroll: Okay20:18
openstackgerritAndre Aranha proposed openstack/python-oneviewclient: Added functions to translate between uuid and uri  https://review.openstack.org/30187720:21
jlvillal#openstack-meeting is now talking about the golang proposal20:22
* jlvillal grabs his soda and watches...20:22
TheJuliaWell, make popcorn too?20:28
jlvillalTheJulia: ;)20:30
*** jaybeale has joined #openstack-ironic20:37
*** ayoung has quit IRC20:44
jrolljlvillal: did you get a logstash query going?20:47
* jroll curious how hard this is hitting20:47
jlvillaljroll: Not yet. Got distracted by golang meeting20:47
jlvillaljroll: And I have to go pick my wife up from school20:47
* jlvillal is already late :(20:47
* jlvillal and possibly going to be in trouble...20:48
TheJuliaeek20:48
jrolloops, gogogo20:48
TheJulia++20:48
jlvillaljroll: Thanks. I will do it when I get back, unless you or someone else does it first.20:48
jrolljlvillal: sure,20:48
jlvillaljroll: I liked your question about swift leaving. I didn't really see a response though :(20:49
*** frontrunner has quit IRC20:49
jrollnop20:49
JayFIs there at least a bug filed?20:49
JayFoh yep above20:49
*** nicodemos is now known as nico-afk20:51
thiagopjlvillal: or you if you don't survive...20:51
thiagop:)20:51
*** baoli has quit IRC20:52
*** baoli has joined #openstack-ironic20:52
TheJulialucas-dinner: Tomorrow, if you have time, please take a look at https://bugs.launchpad.net/ironic/+bug/1590146 which I've filed. Since you put in the cleaning timeout, I'm curious to know if there was an intent behind the current behavior, and if it is intended or not, before I go trying to submit a fix. :)20:53
openstackLaunchpad bug 1590146 in Ironic "A timed out manual cleaning cannot be retried successfuly" [Undecided,New]20:53
*** dprince has quit IRC20:55
*** baoli has quit IRC20:56
*** baoli has joined #openstack-ironic20:56
*** priteau has quit IRC20:59
*** baoli has quit IRC21:01
*** baoli has joined #openstack-ironic21:01
*** clenimar has quit IRC21:02
*** ayoung has joined #openstack-ironic21:04
*** jjohnson2 has quit IRC21:05
*** trown is now known as trown|outtypewww21:05
*** Goneri has joined #openstack-ironic21:07
*** r-daneel has quit IRC21:09
*** spandhe has joined #openstack-ironic21:12
openstackgerritVladyslav Drok proposed openstack/ironic: Gracefully degrade start_iscsi_target for Mitaka ramdisk  https://review.openstack.org/31918321:12
*** SiRiuS__ has quit IRC21:19
*** frontrunner has joined #openstack-ironic21:19
*** Jabadia has quit IRC21:23
*** priteau has joined #openstack-ironic21:25
*** amotoki has joined #openstack-ironic21:27
*** r-daneel has joined #openstack-ironic21:30
*** ijw has quit IRC21:32
*** ijw has joined #openstack-ironic21:32
*** amotoki has quit IRC21:33
*** ijw has quit IRC21:33
*** thiagop has quit IRC21:33
*** jcoufal has quit IRC21:34
*** r-daneel has quit IRC21:35
*** priteau has quit IRC21:40
*** r-daneel has joined #openstack-ironic21:48
JayFjroll: where do you plug in queries that are candidates for e-r?21:49
JayFjroll: I know you said there's an es somewhere, I've not used it21:49
* JayF thinks he figured it out, but apparently is doing SOMETHING wrong21:52
jrollJayF: logstash.o.o21:53
JayFyeah I found it, and figured out a thing or two I was doing wrong21:53
*** ayoung has quit IRC21:54
*** Sukhdev has joined #openstack-ironic21:54
jrollcool21:54
jrollthanks for doing that, I realized after TC meeting I should stop working21:54
JayFit's either not working or I don't know how to use it21:54
*** frontrunner has quit IRC21:54
JayF(message:"AH00072: make_sock: could not bind to address [::]:8088" AND message:"The apache2 instance did not start within 20 seconds. Please read the log files to discover problems")21:54
JayFspecific enough for an e-r you think?21:54
jrollprobably21:55
jrollreviewers can tell you if not, I guess21:55
mrdaMorning Ironic21:55
jrollhey mrda :)21:55
mrdao/21:55
*** Sukhdev has quit IRC21:57
TheJuliagood morning mrda21:57
jlvillalJayF: Glad I am not the only one who has been confused by logstash :)21:57
JayFhonestly I've been poking at it for all of 15 minutes now21:58
JayFafter I asked jim I looked, the e-r repo has a great readme21:58
JayFand I used an existing query as a template21:58
JayFabout to push up the patchset to e-r now21:58
jrollnice21:58
JayFhttps://review.openstack.org/326739 Add e-r query for 128287621:59
*** ccarmack has quit IRC21:59
JayFjlvillal: the trickiest thing about es, is it defaults to a very short timeline, so you have to zoom out to catch most of anything22:01
devanandaJayF: no, not quite22:01
devanandaJayF: you need to specify a file name or a "tag"22:02
JayFhmm. Do you have an example?22:02
JayFhttp://logs.openstack.org/83/319183/23/check/gate-tempest-dsvm-ironic-pxe_ipa/4e3ae7a/logs/devstacklog.txt.gz#_2016-06-07_19_45_09_71422:02
JayFis an example of what I'm trying to catch22:02
*** sdake_ has joined #openstack-ironic22:03
devanandaJayF: https://github.com/openstack-infra/elastic-recheck/blob/master/queries/1393099.yaml22:03
jrollanother example: https://github.com/openstack-infra/elastic-recheck/blob/master/queries/1353962.yaml22:04
jroll(TIL)22:04
devanandaJayF: http://docs.openstack.org/infra/elastic-recheck/readme.html#queries22:04
JayFdevananda: updated, thanks for the tip22:04
devanandanp22:05
devanandaheh. in between other meetings and emails, I've made progress on policy22:05
*** Jabadia has joined #openstack-ironic22:05
openstackgerritChris Krelle proposed openstack/ironic-python-agent: Add basic troubleshooting doc  https://review.openstack.org/32674322:05
*** Sukhdev has joined #openstack-ironic22:05
devanandaI'm now getting 283 "403" errors, rather than 500 errors22:05
*** sdake has quit IRC22:06
JayFthat's like, 217 less errors!22:06
JayF/s22:06
JayFupdated again22:06
JayFapparently you're supposed to use tag:blah.txt instead of filename:blah.txt so you catch the files in grenade jobs too22:06
*** sdake has joined #openstack-ironic22:06
*** ayoung has joined #openstack-ironic22:08
*** sdake_ has quit IRC22:10
*** Sukhdev has quit IRC22:12
*** Sukhdev has joined #openstack-ironic22:12
*** jaybeale_ has joined #openstack-ironic22:14
*** jaybeale has quit IRC22:14
openstackgerritYosef Hoffman proposed openstack/ironic-python-agent: [WIP] Add configuration options for DISK_WAIT  https://review.openstack.org/32490922:15
*** Sukhdev has quit IRC22:16
*** Sukhdev has joined #openstack-ironic22:16
*** r-daneel has quit IRC22:17
JayFboth of the api-ref docs items outstanding - https://review.openstack.org/#/c/313187/ and https://review.openstack.org/#/c/313708/ are 1x+2 and has some +1s (from me for sure)22:18
JayFwould appreciate some core reviews on them, as they are in great shape and way better than what we have now22:18
*** ijw has joined #openstack-ironic22:19
*** PollyZ has quit IRC22:20
*** ijw has quit IRC22:21
*** ijw_ has joined #openstack-ironic22:23
devanandaJayF: thinking out loud here for a minute ...22:25
devanandawe have two different config flags that pretty much do almost the same thing22:25
openstackgerritChris Krelle proposed openstack/ironic-python-agent: Add basic troubleshooting doc  https://review.openstack.org/32674322:25
devanandaCONF.auth_strategy && ironic/api/config.py "enable_acl"22:25
*** PollyZ has joined #openstack-ironic22:26
openstackgerritMerged openstack/ironic: Update comment from NOTE to TODO  https://review.openstack.org/32650422:26
JayFI didn't know api/config.py was operator editable?22:27
devanandait's not -- and yet22:27
JayFI always read that code as that being pecan's config file written out in ironic code22:27
devanandawe have a bunch of code that keys off of whether it is True or False22:27
devanandaso clearly, someone sometime expected it to be mutable22:27
*** ijw_ has quit IRC22:27
JayFI don't think it's safe to assume that was explicit, but sure :)22:28
JayFI would imagine we should change all those checks to be if CONF.auth_strategy is none (or noauth, or whatever the secret string is), right?22:28
*** ijw has joined #openstack-ironic22:29
devanandathat's my thinking22:29
devanandathanks for confirming :)22:29
*** amotoki has joined #openstack-ironic22:30
JayFand even if someone has jiggled an option in that file22:30
*** rbudden has quit IRC22:30
JayFIMO we have no responsiblity whatever to support them22:30
JayFanymore than we would anyone who patched Ironic code to act differently22:30
devanandacool22:30
*** ametts has quit IRC22:30
*** PollyZ has quit IRC22:31
devanandaWOOT! down to 5 errors22:31
*** amotoki has quit IRC22:34
*** PollyZ has joined #openstack-ironic22:40
*** sabeen has quit IRC22:41
*** sdake has quit IRC22:43
JayFAnyone have any experience with fixtures.MonkeyPatch?22:45
*** PollyZ has quit IRC22:45
JayFI'm well down the rabbithole. e-r gate is broken and I'm trying to fix it22:45
JayFand I'm feeling very like http://vignette1.wikia.nocookie.net/someordinarygamers/images/a/ab/I-have-no-idea-what-im-doing-science-dog.jpg22:45
jlvillalJayF: Did you get a working logstash query?22:46
JayFYep.22:46
JayFand I have it up for review, linked it in here earlier22:46
JayFbut unit tests for it are failing because e-r is broken22:46
JayFgiven I can't barely get a review from infra because they are busy, I was trying to fix unit tests22:46
jlvillalI mean one that I can see on logstash.o.o?22:46
JayFand am now stuck, squarely at the bottom of the rabbithole22:46
JayFI mean, you can look at my review and input it?22:47
openstackgerritDevananda van der Veen proposed openstack/ironic: [WIP] Add keystone policy support to Ironic  https://review.openstack.org/32559922:47
jlvillalOkay. I tried and I got zero results.22:47
JayFI didn't make a direct link, not sure you can... well, more accurately, idk how I'd do it22:47
JayFhaha22:47
jlvillalWell I did two of the three22:47
JayFyou have to zoom out22:47
JayFand be very, very patient22:47
jlvillalmessage:"AH00072: make_sock: could not bind to address [::]:8088" AND tag:"devstacklog.txt.gz"22:47
JayFand paste the *whole thing in* as one filter22:47
jlvillalJayF: Trying: message:"AH00072: make_sock: could not bind to address [::]:8088" AND tag:"devstacklog.txt.gz" AND message:"The apache2 instance did not start within 20 seconds. Please read the log files to discover problems"22:48
jlvillalAnd have it set for 1week, I believe22:48
jlvillalAnd it seemed to stop with zero results.22:48
jlvillalSo /me is confused22:48
devanandaJayF: that ^ is passing unit tests locally :-D22:49
devanandaJayF: so I succeeded in not breaking the error codes for unauth'd users22:49
JayF\o/22:49
JayFjlvillal: I can't reproduce it now either22:51
* JayF flips table22:51
jlvillalwhew!22:51
JayFscrew this bug, I'm going to go do something else22:51
* jlvillal was feeling like an idiot over here22:51
* devananda feels accomplished with the policy work and goes afk for a bit22:52
* jlvillal also remembers why he thought the logstash UI sucked before and continues to think it is non-intuitive. Apologies to logstash developers reading this ;)22:52
devanandashameless self promotion for API documentation - https://review.openstack.org/#/c/313187/9 needs another +222:53
*** PollyZ has joined #openstack-ironic22:53
devanandaditto for https://review.openstack.org/#/c/313708/722:53
*** frontrunner has joined #openstack-ironic22:55
JayFI already shilled them too, I +1'd 'em earlier22:55
jlvillaldevananda: I'm thinking dumb question. But I click the generated docs link. And I see API. And assume I click the only thing which says API, would be the content. But it doesn't appear to be. So now I wonder, where is it buried in the docs?22:57
* jlvillal feels dumb again...22:57
*** PollyZ has quit IRC22:58
jlvillalLooking at: http://docs-draft.openstack.org/87/313187/9/check/gate-ironic-docs/724c5a2//doc/build/html/22:59
*** Goneri has quit IRC23:02
TheJuliadevananda: they are on my screen :)23:02
*** absubram has quit IRC23:04
*** Sukhdev has quit IRC23:05
*** Sukhdev has joined #openstack-ironic23:05
*** baoli has quit IRC23:07
*** Sukhdev has quit IRC23:09
*** sdake has joined #openstack-ironic23:12
TheJuliadevananda: is the removal of error response codes intended?23:14
*** sacharya_ has joined #openstack-ironic23:15
*** sacharya has quit IRC23:15
*** Sukhdev has joined #openstack-ironic23:19
*** PollyZ has joined #openstack-ironic23:20
*** ijw has quit IRC23:21
devanandajlvillal: look at the api-ref job output23:21
devanandajlvillal: that renders the api-ref pages, which is where these changes end up, as opposed to the docs.os.o/dev/ site23:21
jlvillaldevananda: Ah, see I knew I was doing something dumb.23:22
devanandaTheJulia: yea. because they were bogus and I haven't vetted what the real ones are23:22
TheJulia10-423:22
jlvillaldevananda: Thanks!23:22
devanandajlvillal: TY!23:22
*** ijw has joined #openstack-ironic23:22
devanandaI've been trying to land these doc updates for a month now :p23:22
jlvillalOh pretty :)23:22
devanandajlvillal: isn't it?!23:22
jlvillal:)23:22
TheJuliavery pretty23:22
*** PollyZ has quit IRC23:25
*** sacharya_ has quit IRC23:25
*** Sukhdev has quit IRC23:30
*** amotoki has joined #openstack-ironic23:30
*** Sukhdev has joined #openstack-ironic23:32
*** ppiela has joined #openstack-ironic23:33
*** PollyZ has joined #openstack-ironic23:34
*** amotoki has quit IRC23:35
*** Jabadia has quit IRC23:35
*** rbrud has joined #openstack-ironic23:36
*** PollyZ has quit IRC23:38
*** Sukhdev has quit IRC23:40
*** Sukhdev has joined #openstack-ironic23:40
*** garthb_ has quit IRC23:44
*** sdake has quit IRC23:46
*** ChrisAusten has quit IRC23:46
*** PollyZ has joined #openstack-ironic23:48
*** clenimar has joined #openstack-ironic23:49
*** PollyZ has quit IRC23:52

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!