Thursday, 2016-05-05

*** rloo has joined #openstack-ironic00:04
*** rama_y has quit IRC00:10
*** keedya has joined #openstack-ironic00:10
*** piet has quit IRC00:12
openstackgerritJohn L. Villalovos proposed openstack/ironic-specs: Add ability to reverse the toctree and use it  https://review.openstack.org/31279400:16
jlvillaljroll: ^^^ :)00:16
cineramajlvillal: https://github.com/sphinx-doc/sphinx/issues/971 hmmm....00:21
jlvillalcinerama: Yeah it never got done00:21
cineramai saw that :/00:21
*** garthb_ has joined #openstack-ironic00:23
cineramai was just wondering if we should try to get this further upstream as well00:24
cineramalike we could at least add it to cookiecutter if it's more generally wanted across openstack00:25
jlvillalcinerama: I was going to submit a patch, but openstack is way behind on the version of Sphinx it uses.00:25
cineramaalso wouldn't hurt to have it upstream anyway :)00:25
*** garthb_ has quit IRC00:27
*** garthb has quit IRC00:27
*** wajdi_ has joined #openstack-ironic00:28
jlvillalcinerama: On the devstack-gate-test patch you submitted. Well that still work with Ansible 1.5.4?00:28
jlvillals/Well that/Will that/00:28
* jlvillal uses Ansible 1.5.4 on his Ubuntu 14.04 box that then creates the VM00:29
cineramajlvillal oh heh00:30
jlvillalcinerama: I want it to work for people who are running Ubuntu 14.04 and did "apt-get install ansible" :)00:31
cineramajlvillal, nope, 1.9 and greater00:31
jlvillalOh :(00:31
cineramaso...i feel like people should just pip install ansible00:31
cineramabut you're welcome to disagree on that point00:32
jlvillalI can see that point. But I don't want to make it hard on people who are using 14.04.00:32
jlvillalI'm torn...00:32
jlvillalI'll ponder it!00:33
cineramasounds good00:33
jlvillalcinerama: But thanks a lot for the patch.00:33
cineramacall it a day, sleep on it, etc00:33
jlvillalReally looking forward to your results on where grenade fails for you.00:33
cineramai would have to retest the get_url stuff etc with old ones00:33
cineramaold ansibles that is00:33
*** sacharya has joined #openstack-ironic00:34
cineramahave you tested what we have on the distro ansible?00:34
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Root and Nodes  https://review.openstack.org/31279500:34
devanandashiney new API docs ^ :)00:34
cineramai've applied the suggested changes from our etherpad from this AM00:34
cineramaso i feel like i am going to get further00:34
*** Sukhdev has quit IRC00:35
jlvillalcinerama: Thanks. And....  https://github.com/sphinx-doc/sphinx/pull/252700:35
* jlvillal is calling it a night00:35
jlvillalTime to go play with my home network. Moving wires around :)00:36
cineramaah a busman's holiday00:37
cineramajlvillal: awesome sauce00:37
*** baoli has quit IRC00:37
*** keedya has quit IRC00:39
*** sacharya has quit IRC00:39
*** amit213 has quit IRC00:42
*** hoangcx has joined #openstack-ironic00:43
openstackgerritMerged openstack/bifrost: Add Ubuntu 16.04 defaults for ironic role  https://review.openstack.org/31230300:45
*** absubram has joined #openstack-ironic00:57
*** ijw has quit IRC00:58
*** baoli has joined #openstack-ironic01:01
*** thrash is now known as thrash|g0ne01:04
*** piet has joined #openstack-ironic01:04
*** mtanino has quit IRC01:05
*** baoli has quit IRC01:09
*** rloo has quit IRC01:10
*** baoli has joined #openstack-ironic01:12
*** absubram has quit IRC01:12
*** amit213 has joined #openstack-ironic01:13
*** amit213 has quit IRC01:16
*** baoli has quit IRC01:25
*** dims has quit IRC01:30
*** snizar has joined #openstack-ironic01:33
*** dims has joined #openstack-ironic01:35
*** thiagop has joined #openstack-ironic01:42
*** snizar has quit IRC01:50
openstackgerritZhenguo Niu proposed openstack/ironic: Clear target_power_state on conductor startup  https://review.openstack.org/31009001:50
*** piet has quit IRC01:56
*** anteaya has quit IRC02:05
openstackgerritZhenguo Niu proposed openstack/ironic-specs: Check duplicate nodes enrollment/update  https://review.openstack.org/29635702:08
*** wajdi_ has quit IRC02:17
*** wajdi has joined #openstack-ironic02:18
*** thiagop has quit IRC02:20
*** hoangcx has quit IRC02:20
*** hoangcx has joined #openstack-ironic02:21
*** sacharya has joined #openstack-ironic02:39
*** praneshp has quit IRC02:51
*** fragatina has quit IRC03:17
*** garthb has joined #openstack-ironic03:20
*** praneshp has joined #openstack-ironic03:22
*** wajdi has quit IRC03:46
*** piet has joined #openstack-ironic03:51
*** pleia2 has quit IRC03:51
*** daemontool has quit IRC03:57
*** daemontool has joined #openstack-ironic03:58
*** fragatina has joined #openstack-ironic04:39
*** piet has quit IRC04:39
*** vmud213 has joined #openstack-ironic05:03
*** fragatin_ has joined #openstack-ironic05:04
*** fragatina has quit IRC05:08
*** harlowja_at_home has joined #openstack-ironic05:20
*** ifarkas has quit IRC05:22
*** harlowja_at_home has quit IRC05:26
*** yolanda has joined #openstack-ironic05:26
*** rama_y has joined #openstack-ironic05:39
*** openstackgerrit has quit IRC06:03
*** openstackgerrit has joined #openstack-ironic06:03
*** fragatin_ has quit IRC06:07
*** garthb has quit IRC06:12
*** itamarl has joined #openstack-ironic06:17
*** rama_y has quit IRC06:25
*** ankkumar5 has joined #openstack-ironic06:26
*** e0ne has joined #openstack-ironic06:32
*** stendulker has joined #openstack-ironic06:33
*** e0ne has quit IRC06:43
*** tesseract has joined #openstack-ironic06:44
*** tesseract is now known as Guest2128806:45
openstackgerritShivanand Tendulker proposed openstack/ironic-lib: Support configdrive in iscsi deploy for whole disk images  https://review.openstack.org/23092406:58
*** praneshp has quit IRC06:59
*** praneshp has joined #openstack-ironic07:01
*** praneshp has quit IRC07:02
*** praneshp has joined #openstack-ironic07:05
*** sivaramakrishna has joined #openstack-ironic07:10
*** dtantsur|afk is now known as dtantsur07:15
dtantsurMorning Ironic07:15
*** sacharya has quit IRC07:20
*** athomas has joined #openstack-ironic07:21
*** oberezovskyi has quit IRC07:27
*** yolanda has quit IRC07:36
*** irf has joined #openstack-ironic07:36
*** ifarkas has joined #openstack-ironic07:40
*** yolanda has joined #openstack-ironic07:44
*** dmk0202 has joined #openstack-ironic07:57
*** praneshp has quit IRC07:57
*** zzzeek has quit IRC08:00
*** dmk0202 has quit IRC08:03
*** zzzeek has joined #openstack-ironic08:04
*** e0ne has joined #openstack-ironic08:06
*** yolanda has quit IRC08:08
*** mbound has joined #openstack-ironic08:10
*** dmk0202 has joined #openstack-ironic08:11
*** yolanda has joined #openstack-ironic08:13
*** lucas-afk is now known as lucasagomes08:19
lucasagomesmorning all08:19
*** sacharya has joined #openstack-ironic08:20
*** derekh has joined #openstack-ironic08:23
*** appu103_ has joined #openstack-ironic08:24
*** sacharya has quit IRC08:26
openstackgerritTan Lin proposed openstack/ironic-specs: Support rolling upgrades  https://review.openstack.org/29924508:27
lucasagomeshi all, can I get some eyes on https://review.openstack.org/#/c/312575/ ? The project-config change depending on this change already has a +208:31
dtantsurlucasagomes, morning. will take a look once I get to the office08:32
lucasagomesdtantsur, no rush08:32
vmud213morning lucasagomes,dtantsur08:32
lucasagomesthanks08:32
*** dtantsur is now known as dtantsur|brb08:33
*** ohamada has joined #openstack-ironic08:33
*** jistr has joined #openstack-ironic08:34
*** ankkumar5 has quit IRC08:34
*** ankkumar5 has joined #openstack-ironic08:35
*** ankkumar5 has quit IRC08:37
*** ankkumar5 has joined #openstack-ironic08:37
*** ankkumar5 has quit IRC08:42
*** ankkumar5 has joined #openstack-ironic08:43
*** ankkumar_ has joined #openstack-ironic08:44
*** ankkumar5 has quit IRC08:44
*** ankkumar5 has joined #openstack-ironic08:46
*** e0ne has quit IRC08:46
openstackgerritZhenguo Niu proposed openstack/ironic-inspector: Place db and cmd files to seperate directories  https://review.openstack.org/31286208:47
*** e0ne has joined #openstack-ironic08:49
*** ankkumar_ has quit IRC08:49
*** electrofelix has joined #openstack-ironic08:56
*** mkovacik has joined #openstack-ironic09:08
mkovacikmorning all :)09:08
*** appu103_ is now known as aparnav09:14
*** sacharya has joined #openstack-ironic09:22
*** sacharya has quit IRC09:27
sambetts|afkMorning all09:40
*** irf has quit IRC09:40
*** sambetts|afk is now known as sambetts09:40
*** irf has joined #openstack-ironic09:46
*** e0ne has quit IRC09:48
*** e0ne has joined #openstack-ironic09:49
*** dtantsur|brb is now known as dtantsur09:51
dtantsurmorning mkovacik, sambetts09:54
sambettso/ dtantsur09:54
vmud213morning mkovacik,sambetts09:54
mkovacikmorning dtantsur, sambetts vmud213 :)09:54
openstackgerritZhenguo Niu proposed openstack/ironic-inspector: Place db and cmd files to seperate directories  https://review.openstack.org/31286209:55
dtantsursambetts, ifarkas, aarefiev, I'm close to -2ing ^^^, but maybe you guys see more sense in it, dunno09:57
vmud213dtantsur: Regarding https://review.openstack.org/#/c/303503 i have seen your review comments only after i uploaded the patch.So i could not address them.09:57
dtantsurvmud213, no worries, please make sure to address them if you make another revision09:57
vmud213dtantsur:Sure, i will09:58
*** stendulker_ has joined #openstack-ironic09:59
sambettsdtantsur: I personally think if we are going to do a refactor of the tree like that then it needs to be done properly, I personally hate things like "import ironic_inspector.db.db" -_-09:59
sambettsdtantsur: if done properly it should be broken down e.g. ironic_inspector.db.models etc10:00
dtantsursambetts, yep, and I just as well hate ironic_inspector.db.dbsync...10:00
dtantsuralthough I don't see a big need for refactoring right now10:00
sambettstbh dbsync should be in cmd based on the way this patch is dividing things10:01
dtantsur++10:02
*** stendulker has quit IRC10:02
irfmorning all10:04
sambettsMorning irf10:04
*** e0ne has quit IRC10:05
*** stendulker_ has quit IRC10:05
*** mgould has joined #openstack-ironic10:06
mgouldmorning Ironic!10:08
sambettso/ mgould10:08
dtantsurmorning mgould10:09
*** yolanda has quit IRC10:09
mgouldmorning sambetts dtantsur10:10
irfregarding yesterday issue .. i refered the doc : http://docs.openstack.org/developer/ironic/deploy/troubleshooting.html#nova-returns-no-valid-host-was-found-error .. actually, not able to understand the 2 point ...10:10
irfmorning mgould10:10
*** amotoki has quit IRC10:10
mgouldmorning irf10:10
*** amotoki has joined #openstack-ironic10:11
sambettsirf: 2. refers to making sure that your nova flavor matches your BM exactly10:12
irfmy question is : what is the meaning of , Regarding the extra specs in flavor, you should make sure they map to node.properties['capabilities'].10:12
irfok...10:12
sambettsirf: As an example: http://docs.openstack.org/developer/ironic/deploy/install-guide.html#enabling-local-boot-with-compute-service10:13
irfbut my BM is just desktop box as of now ... which has some high end hardware box ....10:13
irfok let me chk this10:13
sambettsyou'll see if you set a capability on the ironic node, then you need to add it to the falvor10:13
mgouldirf: as far as nova is concerned, it's just a VM with certain properties10:13
*** yolanda has joined #openstack-ironic10:13
mgoulduser says "please give me a $flavor node"10:14
mgouldnova looks up what $flavor says, and says "hey, Ironic, can you give me a node with these properties?"10:14
irfok10:14
irflet me chk10:14
mgouldIronic says "sure thing, here you go" or "no, sorry, I haven't got one of those" depending on whether the flavor accurately describes your hardware10:15
mgouldor rather, depending on whether the flavour accurately matches *Ironic's beliefs about your hardware*10:15
* dtantsur continues downstream conversation10:18
dtantsurmgould, I clearly remember the devstack gate accepts a variable to append to local.conf10:18
mgoulddtantsur, sounds plausible10:19
sambettsdtantsur, mgould: it does, we use it in the gate jobs10:19
mgouldthe problem was really that it was changing timeouts in a way we didn't expect10:20
sambetts++10:20
dtantsurmgould, DEVSTACK_LOCAL_CONFIG :)10:20
mgoulddtantsur, thanks10:20
sambettsIMO we should be setting ironic specific things using that variable (which we already use) in the jobs10:21
sambettshttps://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L63610:21
dtantsuror https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ironic.yaml#L110-L12810:21
dtantsuryeah10:21
mgouldyeah, keep local info local rather than spreading it around the place10:21
sambetts++10:21
vmud213morning mgould,irf10:21
mgouldvmud213, mornign10:22
mgould*morning10:22
irfmorning10:22
sambettsdtantsur, mgould: TheJulia mentioned something to do with the devstack-gate versioning that was why things were spread everywhere10:22
sambettsbut not sure :/10:22
mgouldyeah, I think she said that devstack-gate wasn't versioned, but that can't be right10:23
mgouldnot least because I'm looking at a git repo containing it right now10:23
dtantsurmaybe she meant stable branches? devstack-gate is definitely branchless10:23
mgouldoh, that might be it10:23
*** hoangcx has quit IRC10:24
mgouldI think that's an argument for keeping devstack-gate small and setting project-relevant info elsewhere, though10:24
mgould*project-specific10:24
sambettsyeah, being stable branchless makes the gate act weird, but I don't think that should be a problem, we already compensate for that in the job, https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L70710:24
mgouldyay10:24
sambettswell its not compensating for devstack-gate specifically, but compensating for the fact that the jobs are tied to a particular stable/*10:26
* mgould nods10:26
dtantsurdevstack-gate has to be branchless, because it defines branches for jobs. kind of chicken-and-egg :)10:27
sambetts++10:27
mgouldhmm, I guess so10:28
sambettswe should just have everything thats in that if virt_driver = ironic block in devstack moved into the ironic job builder10:31
sambettsinto the DEVSTACK_LOCAL_CONFIG10:32
sambettsvariable10:32
mgouldsambetts, +110:32
irffor BM node , do we need to create nova flavor rite ? lets say call it as "baremetal"10:32
mgouldirf, yes10:32
irfok10:32
* mgould isn't totally sure what marks that flavor as being for Ironic's attention - can anyone help?10:33
vdrokgood morning/afternoon!10:34
sambettsliterally nothing, the flavors go through the nova schedular just like any other flavor does, and the ironic nova virt driver keeps makes nova aware of the nodes in ironic for schedulaing10:34
mgouldvdrok, morning10:34
mgouldsambetts, thanks!10:34
jrollmornin10:35
sambettsHey hey jroll10:35
sambettsthanks for merging my tinyipa optimisation10:35
jrolldid I?10:35
sambettsI think JayF and you +2d it10:36
jrollah10:36
jrollyeah np10:36
sambetts:D just spinning up a fresh devstack using JayFs research into different qemu nic driver and my tinyipa optimisation to see how fast it's gotten10:37
dtantsurmorning jroll10:38
*** MattMan has quit IRC10:39
*** MattMan has joined #openstack-ironic10:40
dtantsurjroll, a not-so-obvious question for you (and other folks too). We agreed to promote BMC reset to core API, right? Now, in which states do we allow that? Will we even require a lock for such action?10:41
dtantsurlucasagomes, ^^^10:41
irfanother doubt is, the nova floavor which is created for "baremetal" should exatcly match the configuration as that of the BM node ? i mean , memory , hard disk size and cpus ??10:42
dtantsurirf, it depends on which filters nova is using. the safest bet is a strict match10:43
* mgould -> relocate10:43
*** mgould has quit IRC10:43
jrolldtantsur: don't make me think this early10:43
dtantsurlol10:43
dtantsursorry man, it just occurred to me, that for this operation to be useful we need to allow it in as many situations as possible10:44
mkovacikvdrok, jroll morning!10:44
irfok actually , my BM is nothing but just one desktop machine so i was just thinking that , is it s good idea to first check the configuration of that box and then cresate the nova flavor for that BM node ?10:44
vdrokmorning mkovacik jroll and sambetts !10:45
vdrokand morning dtantsur10:45
dtantsurirf, nova flavor should match whatever is in ironic node properties field. the actual hardware specs are not that important (obviously, you don't want a complete mismatch there)10:45
dtantsurmorning vdrok10:45
dtantsurifarkas, morning! could you please review https://review.openstack.org/#/c/300011/ and https://review.openstack.org/310031 ? O10:46
dtantsurI'd like to make a release after these 2 land10:46
ifarkasdtantsur, sure10:46
dtantsur(also https://review.openstack.org/#/c/276331/ if you have any time left)10:47
* lucasagomes reads10:50
lucasagomesdtantsur, currently it's supported on any states10:50
lucasagomesdtantsur, I believe that is still the case ? I mean, BMCs can jam at any time10:51
*** mgould has joined #openstack-ironic10:51
dtantsurlucasagomes, yep. I'm not sure BMC reset during non-interruptable cleaning is a great thing to do though...10:51
*** e0ne has joined #openstack-ironic10:51
dtantsurlucasagomes, on the other hand, right now it requires an exclusive lock, which greatly reduces potential usefulness10:51
lucasagomesdtantsur, right, we won't necessarily call it during cleaning automatically right?10:51
lucasagomesdtantsur, yeah10:51
dtantsurwe never call BMC reset :) that's kind of a last resort for an operator before calling the node dead10:52
lucasagomesI'm not sure if a warm reset would actually reset credentials or things like that10:52
lucasagomesI hope not, but this method should be used with precautions by operators10:52
irfok.. thanks10:52
irflet me chk10:52
lucasagomesdtantsur, exactly, it's a last resort thingy10:52
dtantsurlucasagomes, or do we maybe want a "force" flag to bypass locking and state checking?10:53
dtantsuri.e. without "force", it will only act in non-transition states and with a lock10:53
dtantsurbut with "force=True" it will Just Do It (tm)10:53
dtantsurjroll, sorry, one more attempts to make you think ^^^10:53
lucasagomesdtantsur, not sure that we need it? Say the BMC is jammed within deployment or cleaning etc... The conductor will probably move the node to a fail state if it can't reboot/power control it10:54
lucasagomesand then the operator can go and run a bmc_reset via the API10:54
dtantsurlucasagomes, well, first it can get stuck for half an hour10:54
dtantsurwith a lock held10:54
dtantsur(not such a big issue since liberty, but still)10:54
lucasagomeswhy is that? The number of retries on the power control commands?10:55
* lucasagomes is using power as an example, but could be othe rthings like changing boot device etc10:55
dtantsurlucasagomes, a dead BMC can respond infinitely long. my record is 10 minutes per request10:55
lucasagomesright, this is a ipmitool problem right?10:56
dtantsurhopefully only it, but hey! that's our main production driver :)10:56
lucasagomesheh yeah, but at least in the first implementation I would vote to not mix such problems10:57
lucasagomesif it's an ipmitool problem we can overcome it in python10:57
lucasagomeskilling the sub-process running it with a resonable timeout (if it doesn't return in time) etc10:57
irfok .. i have created the baremetal nova flavor which matches the configuration of the BM node ...10:57
lucasagomesmaybe need some small investigation on it10:57
lucasagomeswdyt?10:57
dtantsurlucasagomes, so in which states would you suggest to allow BMC reset?10:58
lucasagomesIMHO, any state (counting that we will block it when the nodes are locked)10:58
lucasagomesany state that the node is not holding a private lock10:59
lucasagomesJayF, ^ you might have a better experience than I do with such situations11:00
*** hshiina has joined #openstack-ironic11:01
*** sivaramakrishna has quit IRC11:02
*** links has joined #openstack-ironic11:05
TheJuliagood morning everyone11:06
jrolllucasagomes: dtantsur: +1 for any time the node is not locked11:07
dtantsurok, good11:07
dtantsurmorning TheJulia11:07
*** hamzy has quit IRC11:13
*** ri0 has joined #openstack-ironic11:15
*** ri0 has left #openstack-ironic11:15
lucasagomescool, morning :-)11:20
vdrokmorning lucasagomes and TheJulia !11:23
lucasagomesvdrok, morning11:29
lucasagomesfolks, if you have time mind taking a look at this patch for devstack/ironic https://review.openstack.org/#/c/312575/ ? The project-config patch that adds a job in the pyghmi's gate depends on it already has 2+2 :-)11:30
lucasagomescheerios11:30
lucasagomes(project-config patch for reference: https://review.openstack.org/#/c/312607/)11:30
*** yolanda has quit IRC11:32
*** thrash|g0ne is now known as thrash11:36
xavierrdef __morning__: ...11:37
jroll  _fetch_coffee()11:37
jrollhi xavierr, TheJulia :)11:38
*** yolanda has joined #openstack-ironic11:38
jrollhey vdrok \o11:38
xavierrgood morning jroll vdrok TheJulia lucasagomes dtantsur o/11:41
dtantsurmorning xavierr :)11:42
lucasagomesxavierr, morning11:43
*** baoli has joined #openstack-ironic11:46
*** hamzy has joined #openstack-ironic11:47
*** Goneri has quit IRC11:47
*** baoli_ has joined #openstack-ironic11:47
*** yhvh- is now known as yhvh11:48
*** baoli has quit IRC11:51
*** irf has quit IRC11:52
vmud213is there any changes in the latest code to lock the node when it is in "deploying" state?11:52
*** irf has joined #openstack-ironic11:52
vmud213I just tried to provision the node and then tried to delete it.11:52
vmud213it's in 'deploying' state. Now i don't have a way to get it to available state?11:53
*** links has quit IRC11:53
vmud213any idea?11:53
vmud213or should i wait till the deploy timeout is reached and the state is transitioned to "deploy_failed"11:55
lucasagomesvmud213, no changes there, yeah you have to wait it to timeout11:56
lucasagomesor, if just a test, restarting the conductor will take you out of that state11:57
vmud213thanks lucasagomes, earlier i remember was able to do it.11:58
lucasagomesvmud213, from deploying state? I'm sure if it was ever the case11:58
lucasagomesit works from deploywait state tho (wait for call-back)11:59
vmud213ok.may be i am wrong then.12:00
*** ekarlso has quit IRC12:06
*** ekarlso has joined #openstack-ironic12:06
jrolldtantsur: making this pyghmi LP... apparently it isn't an official project (i.e. in our governance tree)12:12
openstackgerritMerged openstack/ironic-inspector: Store ramdisk logs on all processing failures, not only reported by the ramdisk  https://review.openstack.org/30001112:12
openstackgerritMerged openstack/ironic-inspector: is-empty conditions should accept missing values  https://review.openstack.org/31003112:13
jrolldtantsur: is that something we want to do, or just want a bug tracker?12:13
TheJuliaWell, made it nice and far with the smoke test, actually got the vm up, test logged in... only in cleanup things exploded :(12:13
TheJuliahttp://paste.openstack.org/show/m2ddvQ15oukWiQWrdhDK/12:13
dtantsurjroll, dunno? we have dracclient as an official project, but it's up to a maintainer12:13
jrolldtantsur: yeah, I'll just make the LP for now12:13
*** cdearborn has joined #openstack-ironic12:14
dtantsurTheJulia, I remember seeing that in gate. is it persistent?12:14
TheJuliadtantsur: it was run when I was on the call last night, I'm going to destroy/recreate and rerun now once I look at the nova log12:14
TheJuliaerr, call yesterday12:15
*** cdearborn has quit IRC12:15
*** dprince has joined #openstack-ironic12:15
dtantsurack12:15
jrolldtantsur: done https://launchpad.net/pyghmi12:15
* jroll moves things12:15
dtantsur\o/12:16
jroll"No results for search pyghmi"12:16
jrollhm12:16
dtantsurjroll, by default it searches only for open bugs, I think...12:17
jrollah, did you close them?12:17
dtantsursome were fixed, some were closed12:17
TheJuliadtantsur: I think nova-compute actually crashed....12:18
dtantsuroops12:18
lucasagomes+1 for pyghmi's own bug tracker12:19
* lucasagomes files a bug about pyhton 312:19
jrollI moved the one that wasn't "fix released"12:19
*** krtaylor has quit IRC12:20
*** cdearborn has joined #openstack-ironic12:21
*** ankkumar5 has quit IRC12:27
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Support Ironic node names in our API  https://review.openstack.org/27633112:35
*** aparnav has quit IRC12:36
*** appu103_ has joined #openstack-ironic12:38
*** yolanda has quit IRC12:38
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector: Support Ironic node names in our API  https://review.openstack.org/27633112:43
*** appu103_ has quit IRC12:43
*** yolanda has joined #openstack-ironic12:44
*** jcoufal has joined #openstack-ironic12:46
*** cdearborn has quit IRC12:46
openstackgerritMerged openstack/ironic-webclient: Added selection hover state to node list  https://review.openstack.org/30168112:52
*** bapalm has joined #openstack-ironic12:54
*** vmud213 has quit IRC12:55
*** trown|outtypewww is now known as trown12:58
*** ChrisAusten has quit IRC12:59
*** mbound has quit IRC12:59
*** yolanda has quit IRC13:00
*** piet has joined #openstack-ironic13:05
*** yolanda has joined #openstack-ironic13:05
*** jjohnson2 has joined #openstack-ironic13:09
*** Goneri has joined #openstack-ironic13:13
*** caiobo has joined #openstack-ironic13:15
*** rbudden has joined #openstack-ironic13:19
*** irf has quit IRC13:19
*** sacharya has joined #openstack-ironic13:23
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: driver base: add methods for ConsoleInterface  https://review.openstack.org/30171313:25
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: SSH: add SSHIcsConsole class  https://review.openstack.org/30718113:25
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: IPMItool: add IPMIIcsConsole class  https://review.openstack.org/30171513:25
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: console utils: add methods for ironic console server  https://review.openstack.org/30171413:25
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: conductor manager: apply console-log related methods  https://review.openstack.org/30171713:25
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: REST API: add an API controller for console log  https://review.openstack.org/30171813:25
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: add ironic-console-server  https://review.openstack.org/30229113:25
*** itamarl has quit IRC13:26
*** ohamada has quit IRC13:28
*** mbound has joined #openstack-ironic13:28
*** sacharya has quit IRC13:28
*** ChrisAusten has joined #openstack-ironic13:28
*** piet has quit IRC13:30
*** krtaylor has joined #openstack-ironic13:30
*** wajdi has joined #openstack-ironic13:33
*** daemontool_ has joined #openstack-ironic13:35
*** daemontool has quit IRC13:37
NobodyCamGood Morning Ironicers13:37
sambettsMorning morning NobodyCam13:38
NobodyCammorning sambetts :)13:38
wajdigood morning all!13:40
*** raildo has left #openstack-ironic13:40
NobodyCammorning wajdi13:40
wajdimorning! :)13:40
*** mtanino has joined #openstack-ironic13:41
*** joprovost has joined #openstack-ironic13:41
lucasagomesNobodyCam, sambetts morning13:43
NobodyCammorning lucasagomes :)13:44
dtantsurmorning NobodyCam, wajdi13:44
*** fragatina has joined #openstack-ironic13:45
NobodyCammorning dtantsur :)13:46
vdrokmorning NobodyCam and wajdi !13:47
NobodyCamhey hey vdrok good morning :)13:47
*** openstackgerrit has quit IRC13:47
*** openstackgerrit has joined #openstack-ironic13:48
*** appu103_ has joined #openstack-ironic13:52
*** athomas has quit IRC13:53
*** ametts has joined #openstack-ironic13:55
wajdilucasagomes - I had a question w.r.t this doc: https://etherpad.openstack.org/p/ironic-grenade-tests . It seems like the image of trusty I am using was missing backports and when running stack.sh for devstack, it failed due to missing liberasurecode-dev. Is this simply an isolated issue due to my image, and the assumption is that this should not be a pre-requisite? Just wondering if I should add a note to the doc about this?13:56
*** rloo has joined #openstack-ironic13:58
lucasagomeswajdi, hi there, first thanks for testing it out! Yes, please add this information to the etherpad13:58
lucasagomesmore people may hit this problem so it would be good to have at least a note there saying "make sure librasurecode-dev is installed"13:59
wajdiawesome. thanks lucasagomes . I'll add that in. If I experience any other oddities I'll add some notes then as well.13:59
*** garthb has joined #openstack-ironic14:00
lucasagomescool, thanks a lot!14:00
wajdinp :)14:00
*** athomas has joined #openstack-ironic14:00
NobodyCamI hit the same issue jlvillal listed on his paste14:00
lucasagomesNobodyCam, I'm re-installing ubuntu (I didn't snapshot, shame on me)14:01
lucasagomeswill re-run it soonish14:01
*** joprovost1 has joined #openstack-ironic14:02
NobodyCam:) ahh ... I built a master and cloned it for testing14:02
*** joprovost has quit IRC14:03
*** joprovost1 is now known as joprovost14:03
jlvillalNobodyCam, Glad to hear you got the same as me. Now we have to figure out the problem :)14:05
jlvillalmgould, Have you had a chance to try the Grenade stuff yet?14:06
*** links has joined #openstack-ironic14:07
*** snizar has joined #openstack-ironic14:08
openstackgerritZhenguo Niu proposed openstack/ironic: Clear target_power_state on conductor startup  https://review.openstack.org/31009014:09
mgouldjlvillal, created a new VM this morning, haven't run the tests yet14:09
* mgould will get some lunch and then do just that :-)14:09
*** mgould is now known as mgould|afk14:10
jrollmorning NobodyCam, wajdi, rloo14:10
jrollrloo: we're missing yuriyz and haomeng, do you want to wait for them here? https://review.openstack.org/#/c/311530/14:10
rloomorning jroll, NobodyCam, wajdi, lucasagomes14:11
TheJuliagood morning rloo14:11
lucasagomesrloo, hello there, good morning14:11
wajdimorning jroll14:11
jlvillaljroll, Thanks for crushing my dreams of a fancy sphinx plugin. Though your idea is much simpler and easier I will still grumble ;)14:11
NobodyCammorning jroll, rloo :)14:11
*** joprovost has quit IRC14:11
rloojroll: i guess it is too late to ping yuriyz and haomeng?14:12
rloojroll: OH, I think someone said it is a russian holiday or something this week14:12
jrolljlvillal: if you want to work on sphinx plugins, sdague would probably love help on the rest_parameters thing that builds api-ref14:12
jrollrloo: aha, you're right, I did ping yuriy yesterday, haomeng isn't online14:12
wajdimorning rloo :)14:13
jrollrloo: I'm happy to wait but would like to get this off our plates and published14:13
jrolls/like/prefer/14:13
dtantsurmorning rloo, jlvillal14:13
rloojroll: yeah, guess it is ok to go ahead and merge it then. although i feel a bit bad, maybe we should have given more advance notice.14:13
rloojroll: on the other hand, i don't think they will disagree.14:13
jrollrloo: yeah, on both counts14:14
*** joprovost has joined #openstack-ironic14:14
rloojroll: so go for it14:14
jrollrloo: ok, thanks :)14:14
rloojroll: just make a note in the review that we're going ahead w/o their feedback.14:14
jrollrloo: done14:15
rloojroll: thx. Now I know what I should be doing. Ha ha.14:15
jrollhehe14:15
jrollI'll be writing some summit wrap-up stuff and then starting on a dashboard thing14:15
*** thiagop has joined #openstack-ironic14:16
*** links has quit IRC14:16
thiagopGood morning, Ironic!14:16
NobodyCamjlvillal: quick question are you still hitting the issue in the paste on the ether pad?14:17
*** joprovost1 has joined #openstack-ironic14:17
NobodyCammorning thiagop14:17
thiagophiya NobodyCam14:17
NobodyCam:)14:17
*** joprovost has quit IRC14:18
*** joprovost1 is now known as joprovost14:18
*** absubram has joined #openstack-ironic14:18
jlvillalNobodyCam, Yes14:19
NobodyCamack :)14:19
openstackgerritMerged openstack/ironic-specs: Add newton priorities doc  https://review.openstack.org/31153014:20
thiagop^^ wooow14:21
TheJuliadtantsur: re-ran, didn't get the same error, oddness rooted with neutron upon delete, so given my variable performance, I'm guessing it is power throttling is not helping things :(14:22
BadCubmorning Ironic!14:22
sambettsHi BadCub!14:22
thiagophello BadCub14:23
dtantsurmorning thiagop, BadCub14:23
thiagopdtantsur: o/14:23
dtantsurTheJulia, ouch..14:23
BadCubhiya sambetts thiagop dtantsur14:23
NobodyCammorning BadCub :)14:24
BadCubhiya NobodyCam TheJulia14:24
wajdijlvillal: I added it to http://paste.openstack.org/show/496100/ . But I managed to so far replicate one of the failures you encountered as well.14:24
*** jaypipes has joined #openstack-ironic14:25
jlvillalwajdi, Thanks14:26
*** krotscheck has quit IRC14:27
*** jistr has quit IRC14:29
* TheJulia turns off the throttling and smiles14:30
*** krotscheck has joined #openstack-ironic14:33
*** piet has joined #openstack-ironic14:37
sambetts:( the imports when first starting IPA in the ramdisk take >20seconds14:42
TheJuliaso basically 2 minutes?14:45
TheJuliafrom power on to heartbeat?14:45
*** mgould|afk is now known as mgould14:46
sambettsI've got it down to 1m 9secs from power on to heartbeat now14:46
TheJulianice14:46
jrollsambetts: stevedore is slow I guess?14:46
thiagopsambetts TheJulia : you guys are sorcerers...14:47
TheJuliathiagop: I'm just a sorcerer at corrilation and understanding obscure things14:47
thiagopdark magic14:47
thiagopkkkkkk14:47
thiagopkkk == lol14:48
TheJuliatruthfully, 1 minute 9 seconds is far far better than what we encounter with coreos today14:48
sambettstrying to debug it now, I've identified that the from oslo_cfg import cfg and from ipa import agent are to 2 slowest imports14:48
sambettsthe 2 slowest imports14:48
mgouldsambetts, so that's ~10s to transfer the ramdisk over the network, 20s to import Python libs, ~3s for the actual IPA runtime, ~45s kernel boot and init.d?14:50
*** absubram has quit IRC14:51
*** jistr has joined #openstack-ironic14:51
jrollsambetts: huh, surprised the ipa.agent import is that slow, rather than ipa.agent.IPA() instantiation, given that does stevedore magic14:53
NobodyCammorning mgould :)14:53
*** garthb has quit IRC14:54
*** garthb has joined #openstack-ironic14:54
sambettsmgould: power on -> first init.d log is 31 seconds now14:54
*** caiobo has quit IRC14:55
mgouldmorning NobodyCam TheJulia thiagop jroll wajdi and everyone else I missed since this morning my time :-)14:55
jrollohai mgould :)14:55
wajdi:) morning mgould14:55
mgouldsambetts, cool!14:55
sambettsinit.d start to IPA start is 16 seconds14:55
thiagopmgould: mornin' yo14:55
mgouldsambetts, cool14:56
mgouldso that's approximately (31 - 10) = 21 seconds to boot the kernel?14:57
sambettsso 31 + 16 + 20 seconds for IPA to actually start running + 2 seconds to start and receive the heartbeat == our 1m 9seconds14:58
sambettsmgould: where are you getting the 10s from?14:59
*** ayoung has quit IRC14:59
*** hshiina has left #openstack-ironic14:59
*** derekh has quit IRC14:59
sambettsI'm so deep in numbers I'm starting to drown haha14:59
openstackgerritJarrod Johnson proposed openstack/pyghmi: Implement Python3 compatibility  https://review.openstack.org/31301214:59
wajdi^^oh cool.15:00
mgouldsambetts, didn't we stopwatch the "deploying........" stage recently?15:00
mgouldprobably worth timing it more accurately, though!15:00
sambettsyeah thats a good idea, not sure how to though15:00
mgouldhmm, not sure15:01
sambettsI wonder if there is a way to hook into the tftp server side and work out how long the transfer took15:01
mgouldtcpdump port tftp?15:01
* mgould only started using tcpdump recently, and it is his new Favourite Thing15:01
sambetts;) its great isn't it15:01
jrollI think inbound tftp requests log in syslog as well15:01
*** mbound has quit IRC15:03
mgouldjroll, excellent15:03
mgouldI know dnsmasq can log tftp requests if you ask, but I don't know if you can get transfer time out of that15:04
*** absubram has joined #openstack-ironic15:05
TheJuliadepends on the daemon and logging level15:05
TheJuliawhatever devstack puts in place you can determine the start and end of transfer in the logs I believe15:05
mgouldTheJulia, awesome15:06
TheJuliain the syslog file I believe15:07
mgouldTheJulia, thanks!15:07
*** mbound has joined #openstack-ironic15:09
jrollFYI, I'm going to portland for nova midcycle, let me know if you have things you want me to bring up or send me items to throw at people15:10
dtantsurjroll, I still wonder about passthuing RAID configuration, but I guess it's not the biggest priority now..15:13
dtantsurbut if you have chances to figure out - would be awesome15:13
jrolldtantsur: I think I got one "hell no" in austin, but I might chat with some people15:14
dtantsursigh...15:14
jrollI suspect the future of capabilities may solve that15:14
jrollthere's other things we want to do that way15:14
jrollI'll try to noodle on those things before the midcycle15:15
dtantsurcool!15:15
jrolldtantsur: to be clear, the 'no' was for specifying raid via api, not about flavors or whatever15:15
*** pleia2 has joined #openstack-ironic15:15
jroll(and I think the way the resource pools thing will work, we can hack it in)15:15
*** ppiela has quit IRC15:19
*** ppiela has joined #openstack-ironic15:20
*** Nisha has joined #openstack-ironic15:20
JayFlucasagomes: agreed re: can happy in any state15:23
lucasagomesJayF, cool, thanks15:23
lucasagomesdtantsur, ^15:23
*** ifarkas has quit IRC15:23
* jroll 5215:23
*** appu103_ has quit IRC15:23
jroller.15:23
* jroll brb15:24
*** sacharya has joined #openstack-ironic15:24
*** thrash is now known as thrash|mtg15:25
*** appu103_ has joined #openstack-ironic15:25
*** ohamada has joined #openstack-ironic15:25
*** absubram has quit IRC15:26
*** absubram has joined #openstack-ironic15:27
*** sacharya has quit IRC15:29
*** ayoung has joined #openstack-ironic15:32
*** garthb has quit IRC15:33
*** dmk0202 has quit IRC15:38
openstackgerritJarrod Johnson proposed openstack/pyghmi: Implement Python3 compatibility  https://review.openstack.org/31301215:39
* lucasagomes is restructing virtualbmc to be OpenStack-ish at https://review.openstack.org/#/c/313025 (would be good to have some eyes on it)15:41
*** appu103_ has quit IRC15:41
*** appu103_ has joined #openstack-ironic15:42
*** rloo has quit IRC15:44
*** rloo has joined #openstack-ironic15:44
jlvillallucasagomes: Is that a cookiecutter thing?15:44
lucasagomesjlvillal, yeah15:45
jlvillalcool15:45
dtantsurnote the cookiecutter is usually pretty old :)15:45
mat128lucasagomes: isn't AUTHORS and ChangeLog auto-generated by pbrT?15:45
mat128s/isn't/aren't/15:46
dtantsurmeh, started writing the same :(15:46
TheJuliadtantsur: jlvillal: So yeah, I'm now in resource creation on grenade, killing the cpu throttling seems to have helped15:46
dtantsuryep, they are15:46
jlvillalTheJulia: cpu throttling?15:46
lucasagomesmat128, they are yes15:46
lucasagomesI still have to change a lot of those files15:46
lucasagomesand write tests etc15:46
* jlvillal wonders if he should be doing something with CPU throttling15:46
*** ayoung has quit IRC15:46
*** Guest21288 has quit IRC15:46
mat128will help :) let's get the cookie cutter thing out the door first15:46
TheJuliajlvillal: yeah, I've got a high end desktop with two xeons, seems like the throttling was making things very racey15:47
TheJuliaat least, that is my perception15:47
lucasagomesthanks a lot mat128 yeah, I'm just trying to get the tests to run now (even if there's no real tests there)15:47
jlvillalTheJulia: Where did you change that?15:48
mat128lucasagomes: option --slowest not recognized, probably missing a plugin15:48
*** appu103_ has quit IRC15:48
lucasagomesyeah it's pretty old15:48
lucasagomesI kinda got it now I guess15:48
lucasagomes  py27: commands succeeded15:48
lucasagomes  congratulations :)15:48
lucasagomeswill update the patch soon-ish15:49
mat128good job :), poke me when you do15:49
TheJuliajlvillal: /sys/devices/system/cpu/ but it depends on your hardware15:50
NobodyCamlucasagomes: ahh so you saw the error: option --slowest not recognized15:51
mat128NobodyCam: he fixed it, new revision coming soon15:51
NobodyCam:)15:52
jlvillalTheJulia: Thanks15:52
* jlvillal goes and peeks at a machine of his15:52
mgouldjlvillal, looks like my tests failed during `resource phase create`15:55
openstackgerritJay Faulkner proposed openstack/ironic: WIP: Science the gate with IPXE enabled  https://review.openstack.org/31303515:55
jlvillalmgould: Woo hoo! :)15:55
TheJuliahopefully we have consistency, mine is still running15:55
mgouldhas anyone tried running it with s/coreos/tinyipa yet?15:55
jaypipesJayF: lol. "Science the gate". nice.15:55
TheJuliamgould: I'm running with tinyipa15:55
mgouldTheJulia, cool - what results are you having?15:56
TheJuliaWell, this is the furthest I've gotten thus far15:56
* mgould and sambetts are currently trying to cycle-shave library-import time15:56
jlvillalmgould: I have not, but that is on my list for today15:56
JayFjaypipes: I have some evidence showing ipxe jobs run ~10m faster than non-ipxe jobs. Wanted to get a gate run in with the default set to ipxe to see if I saw the speed of the other jobs improve too15:57
* jlvillal just realizes jaypipes lurks in here. Is aghast!15:57
TheJuliajlvillal: but... he has been lurking here for a while ;)15:57
NobodyCamjlvillal: lol15:57
jlvillal:)15:57
jaypipesJayF: "Science!" <-- said like Adam from Mythbusters15:57
NobodyCamTheJulia: ++15:57
dtantsurJayF, I definitely see more random failures to boot with PXE rather than iPXE15:58
* dtantsur thinks of switching all inspector gates to iPXE15:58
lucasagomesmat128, NobodyCam tests should be passing on the next patch-set :-)15:58
jjohnson2lucasagomes, arg... python3....15:58
* lucasagomes needs to add virtualbmc to the openstackgerrit bot15:58
jjohnson2lucasagomes, that is more than I thought....15:58
JayFdtantsur: I'm going to post something to project-config changing all of them to ipxe if that's faster, with maybe a -nv job for regular-pxe?15:58
TheJuliajlvillal: how long does resource creation run for you?15:58
lucasagomesjjohnson2, heh it's a PITA isn't it ?15:58
JayFdtantsur: and probably switch it to tinyipa while I'm at it :)15:58
jjohnson2lucasagomes, trying to figure out the painless way of difference in bytes treatment15:59
jaypipesjlvillal, TheJulia: I don't lurk. I get notified whenever "Jay" or "jay" is said on a channel I'm in ;)15:59
jlvillalTheJulia: I'm not exactly sure. Maybe 5-10 minutes...15:59
thiagopJayF: Are you testing [i]PXE against VirtualMedia too?15:59
lucasagomesjjohnson2, yeah, there's a lot to change in pyghmi to support it, we can do it partially don't need to put all in one commit15:59
dtantsurJayF, dunno about nv though.. does it mean we'll ignore it if it fails on something not too important?15:59
jlvillaljaypipes: :)15:59
jjohnson2yeah, it doesn't work at all in python 3 for simplest case right now15:59
JayFthiagop: the patch I pushed just changes the default in devstack to ipxe enabled, os it'll run on all jobs15:59
* NobodyCam now has Oingo Boingo's Weird Science running thru his head15:59
JayFdtantsur: That discussion is what merge requests are for :) we can't keep demanding the gate be quicker without being willing to make tradeoffs for it though15:59
jjohnson2lucasagomes, a = b'\x00' in python 3, b[0] is 0, in python 2, b[0] is '\x00' trying to think on a way to deal...15:59
lucasagomesjjohnson2, urgh indeed, bytes vs unicode is very different in py{2,3}15:59
JayFdtantsur: this is one of those tradeoffs that are pretty sane imo15:59
* jlvillal re-installs the snapshot devstack-gate image to try a new run...16:00
* jlvillal re-installs the snapshot devstack-gate image to try a new run...16:00
dtantsurJayF, I'm fine as long as people will actually start ignoring it when it's needed and not 1. ignore it always, 2. recheck it always16:00
dtantsurjudging by tripleo experience, people tend to just to #116:01
* sambetts wonders if jlvillal intentionally sent that message twice ;)16:01
JayFdtantsur: recheck no bug16:01
JayFdtantsur: seeing if I re-run tests if you'll agree with me then ;)16:01
jaypipesNobodyCam: you're welcome :)16:01
jlvillalsambetts: Nope. I sent it.  Accidentally hit the up arrow. Then thought I hadn't sent it and hit enter again...16:01
jjohnson2lucasagomes, trying to think how to cope without incurring too much trickery in performance sensitive byte wrangling sections..16:02
*** thrash|mtg is now known as thrash16:02
dtantsurJayF, :)16:02
jlvillalmgould: sambetts TheJulia Firing off a run with 'tinyipa' as the ramdisk type16:04
dtantsurleaving for a day, see you tomorrow16:04
*** dtantsur is now known as dtantsur|afk16:04
jlvillaldtantsur|afk: Dos ve dos!16:04
lucasagomesjjohnson2, true, I wonder if we can use some lib to actually do the byte manipulation parts... I'm not super familiar at how pyghmi does that16:04
* lucasagomes looks16:04
jjohnson2lucasagomes, ok, I'll be away for a bit16:05
*** davideagnello has joined #openstack-ironic16:05
lucasagomesjjohnson2, if you do bytearray(b'\x00')16:05
*** derekh has joined #openstack-ironic16:06
lucasagomesjjohnson2, and use the object returned in the manipulation it should be consistent in both py2 and 316:06
* lucasagomes tests16:06
lucasagomesjjohnson2, yeah, that's correct, both will return 016:06
lucasagomes>>> a = bytearray(b'\x00')16:06
lucasagomes>>> a[0]16:06
lucasagomes016:06
*** garthb has joined #openstack-ironic16:08
jlvillalTheJulia: JayF sambetts mgould and anyone else I'm missing. Would there be value to doing another audio conference next week on grenade? Was it useful this week?16:08
jlvillalrloo: ^^^16:09
JayFI mean, honestly at this point, I'd love to just know exactly what we're doing and where we're at16:09
JayFthere's a lot of talking but not a lot of divvying up of work16:09
JayFso I don't feel like there's a good way to contribute to the effort16:09
rlooyeah, i think it might be worth having quick status every day, or people update the etherpad with where they are at.16:09
rlooweekly is not frequent enough if we want to get this done soon.16:10
wajdiI've been updating the etherpad on whatever I've discovered. But I'm also touching grenade for the first time. So I'm trying to understand what's going on to know how to easily replicate the failures I'm seeing.16:10
rloothe 'update the etherpad' or something is for folks that are in different time zones16:10
jlvillalJayF: Yeah. Some of this stuff isn't easily broken up. At the moment we in parallel the 'tempest smoke' stuff with vdrok and jroll.  And then grenade stuff with jlvillal mgould TheJulia and some others.16:11
JayFI think jroll has a patch up that makes tempest smoke pass16:11
jlvillalOh great. I could update the devstack-gate-test stuff with it.16:11
jjohnson2lucasagomes, ok, thanks, but for now hungry16:12
lucasagomesjjohnson2, heh yeah no rush on it16:12
jjohnson2I'd used bytearray before, I think I forgot because it doesn't survive the behavior through a buffer()16:12
jjohnson2or whatever it was...16:12
jlvillalAt the moment it seems like most people for Grenade are getting to the failure in resource create. Now hopefully someone will make the break through to move to the next step. I'm trying tinyipa right now. And then more debugging.16:13
* lucasagomes checks his grenade test16:13
jlvillalrloo: Agree with frequent status updates. Weekly is kind of slow.16:13
TheJuliajlvillal: value would be awesome, I could possibly jump on one tomorrow or later today (Traveling tonight and moderately unavailable after noon central tomorrow for medical reasons)16:14
TheJuliaactually, make that very unavailable16:14
jlvillalTheJulia: Thanks. And good luck!16:15
lucasagomesjlvillal, apparently my "2016-05-05 16:10:13.174 | Running resource phase create" have been running for more than a hour now16:16
*** snizar has quit IRC16:16
mgouldjlvillal, I think another audio conference would be useful16:16
jlvillallucasagomes: Maybe good sign?? Or bad...16:16
mgouldbut updating the etherpad frequently is a Good Thing too16:16
wajdiIs there an easy way to try to run a failing piece or some subset of the grenade testing to quickly replicate the resource failure?16:16
lucasagomesyeah heh it will soon start timing out heh16:16
lucasagomesI guess16:16
jlvillalwajdi: Not that I know of. But it would be great if there is a way.16:17
wajdiThanks jlvillal . That confirms my sanity a bit. I was having a hard time trying to run some individual pieces. I'll make note of that. Do you think it would be worth (down the road) to actually try to support something like that?16:18
lucasagomesjlvillal, I will have to run and catch the train back home16:19
lucasagomesbut I let you know later if that worked or not16:19
jlvillallucasagomes: Ciao  Thanks16:20
lucasagomeshave a great evening all16:20
mgouldgoodnight lucasagomes16:20
lucasagomessee you all tomorrow16:20
lucasagomesmgould, night :-)16:20
*** lucasagomes is now known as lucas-afk16:20
jlvillalwajdi: It would be great if we could support that.  sdague would likely know if it is possible. Not sure how much work has been done for people to debug Grenade.16:21
wajdithanks16:21
*** derekh has quit IRC16:22
*** Nisha has quit IRC16:22
mgouldjlvillal, how are you setting the ramdisk type in devstack-gate-test?16:22
sambettsin the jenkins job we pass it in through DEVSTACK_LOCAL_CONF or whatever that variables called16:23
jlvillalmgould: I edited 'ironic-grenade.sh'16:23
mgouldjlvillal, sambetts thanks16:24
*** yolanda has quit IRC16:25
*** trown is now known as trown|lunch16:25
*** snizar has joined #openstack-ironic16:25
TheJuliajlvillal: I hit the same nova error \o/16:29
*** yolanda has joined #openstack-ironic16:30
jlvillalTheJulia: Woot! :)16:30
jlvillalHow long did the resource phase step take you think?16:30
TheJuliaa very long time16:30
TheJuliabasic_baremetal_ops completed in 195 seconds!16:30
sambetts:O 195 seconds!16:30
mgouldjlvillal, do you still need to `vagrant destroy && vagrant up` between ironic-grenade.sh runs?16:31
sambettswere you runnning that in a nested virt env?16:31
mgouldTheJulia, woohoo!16:31
jlvillalmgould: I do.  So I package the initial devstack-gate-test run up into another directory.16:31
jlvillalAnd then use that box and bring it up and down (destroy)16:32
TheJuliaOh... it issued the server create at 2016-05-05 15:42:39.929 and it timed out at 2016-05-05 16:16:01.64516:32
jlvillalThat way it only takes 2 minutes to have a new VM ready.16:32
mgouldjlvillal, not sure I follow16:32
TheJuliasambetts: good question, I think it is nested virt16:32
TheJuliachecking16:32
mgouldyou're creating a vagrant box which is already prepared, and you have another Vagrantfile which just imports it?16:33
TheJuliasambetts: previously, deploys were taking ~380 seconds16:33
jlvillalmgould: cd ~/devstack-gate-test; time vagrant destroy -f && time vagrant up && rm package.box && time vagrant package && time vagrant box add --force --name  devstack-gate package.box && vagrant destroy -f16:33
sambettsTheJulia: did you change over to tinyIPA?16:33
TheJuliasambetts: yes, nested with tinyipa16:33
sambettssweet!16:33
sambettsmajor speedups :D16:34
jlvillalmgould: cd ~/devstack-gate-test-PACKAGED;  time vagrant destroy -f && time vagrant up && vagrant ssh16:34
jrollrloo: jlvillal: at very minimum what I was to see is the etherpad always being up to date. as of now there's been barely anything added to the pad.16:34
jlvillalmgould: But, need to use the other Vagrantfile16:34
jlvillalmgould: There is a Vagrantfile.packaged that would get copied to ~/devstack-gate-test-packaged/Vagrantfile16:35
jlvillalmgould: That is what I do.16:35
jlvillaljroll: Agreed. I will keep it up to date.  This etherpad: https://etherpad.openstack.org/p/ironic-newton-grenade-whiteboard16:36
jrollrloo: jlvillal: we need a single source of truth as to what needs to be done, what is being done, etc16:36
jrolloh, new etherpad. fun16:36
jrollI want someone to be able to join in on this work and have a single pointer to let them discover how to help16:36
mgouldjlvillal, thanks!16:36
rloojroll: ++16:37
jlvillaljroll: rloo I updated the Ironic Whiteboard with a link to the Grenade whiteboard16:37
devanandamorning, all16:37
NobodyCammorning devananda16:37
rloothx jlvillal16:37
rloomorning devananda16:38
jrolljlvillal: cool, thanks. I put my notes on the new pad16:38
* rloo wondered why devananda was so late, but he isn't :)16:38
NobodyCamis this replacing https://etherpad.openstack.org/p/ironic-grenade-tests ?16:38
mgouldmorning devananda16:38
NobodyCamthis == new etherpad16:38
jlvillalNobodyCam: That was lucas-afk work in progress doc16:38
jlvillalNobodyCam: So yes16:38
devanandajroll: it's almost like you want a board to track the stories we're all working on ...16:38
jrolldevananda: :|16:38
jlvillalNobodyCam: https://etherpad.openstack.org/p/ironic-newton-grenade-whiteboard16:39
devanandarloo: I'm a little late today, yea16:39
*** sacharya has joined #openstack-ironic16:39
devanandagot behind on things like laundry, so breakfast kinda became also doing some other things16:39
jrolldevananda: even if we had that, if people aren't making work items and tracking progress on them, it isn't terribly helpful :) I can't do that all myself16:40
devanandajroll: absolutely16:40
rloodevananda: 9:30ish isn't that late. just that there seems to have been a lot of action so it seems to already have been a long day :)16:40
devanandarloo: I see. perhaps I should read backscroll ...16:40
rloodevananda: mostly about grenade. folks are working...16:41
devanandaawesome16:41
sambettsits pretty cool to all be working on a single goal16:41
NobodyCamjlvillal: added a note to lucas-afk's pad pointing to the new one16:41
rloojlvillal: is the stuff in lucas' ironic-grenade-tests still useful, or is that all covered in the new etherpad?16:41
jlvillalI think most of it is covered. I updated my project with a script that does those steps. And added some documentation about it.16:42
rloojlvillal: ok. so folks should be using your project/scripts then?16:43
jlvillalrloo: I would like them to. I think they work.16:43
mgouldTheJulia, are you using hardware-accelerated nested virt?16:44
rloojlvillal: i will add notes to that effect then. lucas-afk, you OK with that? ^^ Or maybe lucas' etherpad could mention that the instructions are captured in your scripts.16:45
TheJuliamgould: whatever jlvillal's vagrantfile sets up, so I believe so yes, so for real gate runs, I guess we should expect ~20% slower16:45
jlvillalrloo: Line 13 in that etherpad says that the script should work.16:45
TheJuliamgould: then again, I also killed the cpu throttling on my machine, so that may have chnaged some things for me which makes comparisons difficult16:45
jlvillalmgould: TheJulia If using VirtualBox then no nested virt.16:46
jrolldevstack turns off nested virt as well, afaik16:46
jlvillalThat too...16:46
TheJuliaso cool16:46
TheJuliathen we should be seeing relatively realistic times I think16:46
rloojlvillal: lucas' etherpad mentions in line 33, to clone our project. https://etherpad.openstack.org/p/ironic-grenade-tests16:47
rloojlvillal: s/our/your/16:47
rloojlvillal: so for someone new, should they look at that etherpad and follow those instructions?16:47
*** dprince has quit IRC16:48
jlvillalRight. But if clone my project first and then do the 'vm-setup.sh' then it will work.16:48
jlvillalrloo: https://github.com/JohnVillalovos/devstack-gate-test  Has instructions for running on a VM without Vagrant16:48
jlvillalWhich is what lucas-afk etherpad is about16:48
rloojlvillal: ok, i see, line 15 in the etherpad mentions that. it is just a matter of combining the info in these two etherpads so it is clear. cuz it isn't that clear now.16:49
jlvillalagreed.16:49
rloojlvillal: there is also line 7 in lucas' etherpad about 'current failure'. is that still current?16:50
jlvillalrloo: Yes, also mentioned in other etherpad16:50
rloojlvillal: ok, will update...16:50
jlvillalThanks!16:50
JayFhttps://review.openstack.org/#/c/313035/ look at how fast those jobs ran! Cut like 10m off any of the jobs not using ipxe today16:52
jlvillalJayF: Nice :)16:53
JayFnow I'm slightly tempted to push another change switching to tinyipa16:54
JayFand seeing if that speeds it up further16:54
jroll+116:54
jlvillalI thought we had already planned to switch to tinyipa.16:54
jrollwe have16:54
sambettsJayF: iPXE http?16:54
jrollso jfdi! :D16:54
JayFI'm tired of planning16:54
JayFmore doing16:55
jlvillalLike it was in the stage of 'let's do it"16:55
JayFless planning16:55
jrollJayF: yeah, don't test, just do ti16:55
JayFjroll: going to test it since I have this here and I can :)16:55
JayFjroll: data > nodata16:55
JayFsambetts: yep, our ipxe support is smart enough to use the ipxe rom that the vms boot with too16:55
JayFsambetts: so no extra step, and uses http instead of tftp16:55
jrollJayF: sure, suppose so16:55
mgouldJayF, awesome16:56
sambettsJayF: nice!16:56
*** igordcar1 has quit IRC16:56
openstackgerritJay Faulkner proposed openstack/ironic: WIP: Science the gate with IPXE + tinyipa  https://review.openstack.org/31303516:56
*** igordcard has joined #openstack-ironic16:57
*** run_dfc has joined #openstack-ironic16:57
mgouldhurrah for SCIENCE!16:58
jaypipesSCIENCE!16:58
JayFRealistically, I think that patch might be the most sane way to change the gate to tinyipa+ipxe anyway16:58
TheJuliaScience++16:59
JayFif that's how the gate works, why wouldn't we want those to be a devstack default?16:59
NobodyCamjroll: IRONIC_VM_COUNT=64  # because I can <- that's just evil16:59
jlvillalNobodyCam: Not sure if you saw: https://github.com/JohnVillalovos/devstack-gate-test/blob/master/vm-setup.sh16:59
jlvillalNobodyCam: Basically what lucas-afk etherpad talks about17:00
jrollJayF: good point :)17:01
*** mbound has quit IRC17:01
*** mbound has joined #openstack-ironic17:02
mgouldJayF, also, why would we want local devstacks to be unnecessarily slow?17:05
NobodyCamjlvillal: I have a base vm I keep cloning that was taken just at the point that script finishes.17:06
jlvillalNobodyCam: Great17:06
NobodyCam:)17:06
JayFmgould: exactly17:06
*** yolanda has quit IRC17:07
*** fragatina has quit IRC17:07
rloofolks, I updated lucas' etherpad to have the steps to set up VM with devstack-gate-like environment: https://etherpad.openstack.org/p/ironic-grenade-tests17:07
*** yolanda has joined #openstack-ironic17:07
jlvillalThanks rloo !17:07
rlooi hope i got it right and that lucas-afk doesn't kill me.17:07
*** fragatina has joined #openstack-ironic17:07
sambettsJayF: with your iPXE patch chaing the default in the devstack lib, I think it'll be overridden by the configuration being set in the local.conf by the job17:11
JayFsambetts: I checked; those configs are only set for nondefaults17:11
JayFsambetts: so the ipxe + tinyipa changes will take effect for most jobs17:12
sambettsI thought that tinyipa/coreos was set for every run explictly :/17:12
JayFI think tinyipa is set explicitly but not coreos17:13
JayFi can check when that job is done17:14
sambetts:) awesome17:14
jrollhttps://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L67817:14
jrollthey are both set explicitly, ctrl-f ramdisk-type17:14
jlvillalsambetts: :(17:15
JayFd'oh17:15
jlvillalsambetts: git clone git://git.qemu-project.org/qemu.git /opt/stack/old/ironic-python-agent/imagebuild/tinyipa/tinyipabuild/tmp/qemu --depth=1 --branch v2.5.017:15
jlvillalWhy git:// ?!?!?!?!?!?!!?17:15
* jroll -> sandwich17:15
jrollwhat's wrong with git://17:15
jlvillalsambetts: Do you have no sympathy in your cold dark heart for those of us behind proxy servers?17:15
wajdirloo: jlvillal Do you think it would be a good idea to add my mention about the backports to the vm-setup.sh script? Or is it best left as a possible edge-case mentioned in the doc?17:15
jlvillal;)17:15
jrolloh, fake internet :|17:15
sambettsjlvillal: they don't have a https repo, I have been planning to update the script to use the github mirror of qemu17:16
* jroll points at a bunch of public clouds17:16
jlvillalwajdi: I'm not quite sure what you mean.  Backports?17:16
*** praneshp has joined #openstack-ironic17:16
jlvillalsambetts: +1 to that idea :)17:16
wajdijlvillal:  sorry yeah...I'll add context. When trying to set up the grenade environment per the doc: https://etherpad.openstack.org/p/ironic-grenade-tests I ran in to an issue bringing up devstack that failed on unable to locate liberasurecode-dev17:17
*** Nisha has joined #openstack-ironic17:17
wajdithe solution was to add ubuntu backports: https://help.ubuntu.com/community/UbuntuBackports to my sources.list17:17
wajdiand apt-get update17:17
*** jistr has quit IRC17:17
jlvillalwajdi: Hmmm. I did not see that issue on any VM I have run it on.17:18
*** fragatina has quit IRC17:18
jlvillalwajdi: Be that Vagrant, RackSpace, and DigitalOcean17:18
wajdijlvillal:  that is what I was concerned about. Whether it was a specific edge case with the images I'm running off of. I mentioned it to lucas this morning, he thought it would be a good idea to put it in the pad. But if this is really isolated and just me, then I'm fine keeping that note to myself for future testing17:18
jlvillalwajdi: If you want to add it to the docs, that would be fine. As a note.17:19
wajdijlvillal: yep...that was done already. So I'll leave that. But I won't go ahead and play with the script. thanks.17:19
jlvillalwajdi: Or not add it.17:19
*** Sukhdev has joined #openstack-ironic17:20
jlvillalUp to you :)17:20
wajdiI had already added it earlier today. But I don't want to throw people off doing something that they might not need to do. Yeah. I'll remove it and keep it in my personal note-bank.17:20
jlvillalwajdi: Thanks.17:20
jlvillalsambetts: If you do propose a patch for that. Please let me know.17:21
jlvillalFor my tinyipa testing I need it.17:21
run_dfcare there any publicly available IPA HardwareManagers that implement RAID configuration other than proliantutils?17:22
*** causten_ has joined #openstack-ironic17:22
*** ChrisAusten has quit IRC17:22
run_dfcspecifically, i'm looking for something that supports LSI RAID controllers17:22
JayFWe don't have one publically, I have one downstream that does it with RSTe / iMSM raid controllers17:22
JayFwhich I would gladly share17:23
*** athomas has quit IRC17:23
rloowajdi: sorry, just catching up. i think it is good for you to leave that note there in case someone else encounters the same issue.17:23
wajdihehe :) OK. I can revert again.17:23
rloothx wajdi :)17:24
wajdinp17:24
*** kuthurium has quit IRC17:24
run_dfcJayF: i would appreciate that as another example to reference if we end up implementing LSI support ourselves17:24
*** trown|lunch is now known as trown17:25
thiagopwajdi: I came into the same problem with liberasurecode-dev some months ago. It's related to using swift or not in the setup, for what I can remember17:26
*** mbound has quit IRC17:27
wajdithiagop: interesting. I actually first encountered this problem trying to bring up devstack, using the local.conf sample that is in the ironic devstack doc: http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#deploying-ironic-with-devstack17:27
NobodyCamquestion, so to use tinyIPA just change 'export IRONIC_RAMDISK_TYPE=coreos' like ' s/coreos/tinyipa/ '17:28
*** mbound has joined #openstack-ironic17:28
*** mbound has quit IRC17:28
*** aginwala has joined #openstack-ironic17:28
jrollNobodyCam: yep17:28
NobodyCamsweet... kicking off another run with tiny17:29
wajdisame17:31
JayFrun_dfc: full disclosure: we don't implement the raidinterface, we just lay a raid down as a clean step17:32
JayFrun_dfc: I'll try to get that split out for you in the next week17:32
run_dfcJayF: thanks!17:32
*** ayoung has joined #openstack-ironic17:33
thiagopwajdi jlvillal are you guys still having that problem with openvswitch/db.sock? I'm seeing it on a pure devstack build too...17:34
wajdithiagop: I have a fresh devstack running right now. I can run whatever you ran to see if I can replicate17:35
wajdior is yours happening in deploy?17:35
* devananda gets back to API docs17:36
thiagopwajdi: stack.sh17:36
wajdithiagop: does not look like I was able to replicate that then. Because I successfully brought it up late last night17:37
thiagopwajdi: it looks the same as here: http://paste.openstack.org/show/496100/17:39
thiagopwajdi: don't know if it was really you that pasted that, though :)17:39
wajdiLet me re-phrase just to make sure we both are on the same page: running devstack on its own, I was able to run stack.sh successfully. Running the test steps inside grenade this morning, I got that exact failure you just pasted.17:41
thiagopweird...17:42
thiagopwajdi: could be related to USE_SSL17:42
thiagopwajdi: I'm running this instance to test ironicclient with ssl, and apparently devstack-gate runs over ssl by default17:43
TheJuliathiagop: checkout https://review.openstack.org/#/c/311055/17:45
thiagopTheJulia: thanks!17:46
thiagopawn devstack, Y U so bad to me?!17:47
*** dprince has joined #openstack-ironic17:48
*** vishwanathj has joined #openstack-ironic17:48
openstackgerritMerged openstack/ironic: Fix VirtualBox cannot set boot device when powered on  https://review.openstack.org/29095117:52
*** ohamada has quit IRC17:53
*** bodepd has joined #openstack-ironic17:54
*** Sukhdev has quit IRC17:57
*** electrofelix has quit IRC17:59
*** Sukhdev has joined #openstack-ironic17:59
wajdiTheJulia: is that something that will help with the error in grenade that is happening here: http://paste.openstack.org/show/496100/ ?18:00
TheJuliawajdi: it has to do with the error executing worlddump18:01
TheJuliaso secondary error18:01
TheJuliathe root error seems to be related to the timeout18:01
wajdiOK. So there are different errors happening here. They aren't necessarily related.18:02
TheJuliawell, I was able to produce the error in 496100, but my version just lacks the error at line 124 "_18:03
bodepdwhat are the creds to log into the default IPA image that bifrost configures?18:03
TheJuliabodepd: There are none18:03
TheJuliayou can pass a ssh key in18:04
TheJuliaI believe the instructions are in troubleshooting.rst18:04
* TheJulia suddenly realizes bifrost should also go to tinyipa for the dhcp tests18:04
cineramasounds good18:05
wajdithanks TheJulia! Makes sense to me.18:05
mgouldTheJulia, tinyipa all the things!18:06
*** aginwala has quit IRC18:07
*** aginwala has joined #openstack-ironic18:07
TheJuliamgould: my only concern is now I need to learn all the things about it in order to update documentation :)18:07
mgouldheh, good point18:08
sambettsTheJulia: if you want I can give a run down about it and its build process etc at some point18:08
TheJuliasambetts: maybe in a week or two18:08
sambettsYeah :)18:08
* mgould has only watched sambetts build it, but the process seems very straightforward18:09
mgouldthere's a Makefile that runs `pip install` in a chroot, and then tars up the result18:09
sambetts;) pretty much, theres a bit more complexity to do with system packages etc but thats the essence of it18:10
* NobodyCam thinks his vm's clock is slightly off 2016-05-05 18:05:42.093 | Running base stack.sh18:11
mgouldNobodyCam, looks about 6min slow to me18:11
sambetts-5/6mins18:11
*** aginwala has quit IRC18:11
NobodyCamlol but its only 11:11 am here :p18:11
*** Sukhdev has quit IRC18:12
*** fragatina has joined #openstack-ironic18:12
sambettshehe its in UTC18:12
*** Sukhdev has joined #openstack-ironic18:12
wajdihas anyone run grenade with tinyipa yet?18:15
mgouldwajdi, not successfully :-(18:15
*** yolanda has quit IRC18:15
* mgould sees that his latest attempt just failed with a 502 error18:16
wajdimgould: same here. However, I am no longer seeing this error: No nw_info cache associated with instance18:16
wajdionly the ovs-vsctl error18:16
wajdiprogress?18:16
mgouldwajdi, review.openstack.org seems to be unstable at the moment18:16
mgouldwajdi, definitely progress :-)18:16
mgouldwajdi, did you try applying the patch TheJulia mentioned about the ovs-vsctl error?18:17
mgouldanyway, /me -> voting -> home. Good night, everybody!18:17
wajdimgould: I was about to actually. But wanted to check my logs before I ran it again. Glad I did. Because if I'm not crazy. That tinyipa might have resolved an issue18:17
sambettscya tomorrow mgould18:17
wajdimgould: take care18:17
mgouldhurrah for democracy, or something ;-)18:17
mgouldsambetts, good night!18:17
mgouldwajdi, tinyipa should help with "deploy timed out" errors, I don't think it will help with the ovs error18:18
mgouldgood luck!18:18
*** mgould has quit IRC18:18
wajdimgould: yes. My understanding as well. bye!18:18
*** Nakato has quit IRC18:19
*** Nakato has joined #openstack-ironic18:22
mat128JayF, run_dfc: any chance this (and future work) could be part of the ironic tree or at least ironic-staging-drivers?18:28
JayFmat128: Most of our downstream hardware managers at a minimum require NDA-required vendor tools or proprietary tools18:29
*** snizar has quit IRC18:29
JayFmat128: we're going to try and split the stuff we can open from the stuff we can't18:29
JayFmat128: but it's like, on a list somewhere :) you know how that goes18:30
mat128yeah, figured :)18:30
openstackgerritSam Betts proposed openstack/ironic-python-agent: TinyIPA: Precompile python code for faster load  https://review.openstack.org/31310318:30
sambettsMore optimisations ^18:31
sambetts:d18:31
* sambetts -> home 18:31
*** sambetts is now known as sambetts|afk18:31
*** e0ne has quit IRC18:32
NobodyCamnight sambetts|afk18:34
openstackgerritSam Betts proposed openstack/ironic: Pass environment through to create-node.sh  https://review.openstack.org/31310618:34
*** Sukhdev has quit IRC18:34
sambetts|afkJayF, jroll ^ please make this happen18:34
sambetts|afknight all18:35
*** ijw has joined #openstack-ironic18:37
run_dfcmat128: no guarantees, but i definitely hope to publish any code we write; the only thing is it would likely require the proprietary MegaCLI tool as JayF pointed out18:43
JayF+1 I'd be glad to help review that18:43
NobodyCamand boom: [WARNING]: provided hosts list is empty, only localhost is available18:48
*** aginwala has joined #openstack-ironic18:55
*** aginwala has quit IRC18:59
openstackgerritJohn L. Villalovos proposed openstack/ironic-python-agent: Stop using git:// and be nice to people behind proxy servers  https://review.openstack.org/31311518:59
*** piet has quit IRC19:00
openstackgerritJohn L. Villalovos proposed openstack/ironic-python-agent: Stop using git:// and be nice to people behind proxy servers  https://review.openstack.org/31311519:00
jlvillalNobodyCam: What's that?19:01
*** piet has joined #openstack-ironic19:01
*** aginwala has joined #openstack-ironic19:02
*** thiagop is now known as thiagop-afk19:02
NobodyCamjlvillal: that's the error I hit running ironic-grenade.sh with tinyIPA19:02
jlvillalNobodyCam: Hmmm. Does it stop there?19:03
mat128run_dfc no problem on MegaCLI, that might be useful to have :)19:03
NobodyCamyep exit 119:03
jlvillalI see messages about only localhost is available on normal runs19:03
*** thiagop has joined #openstack-ironic19:04
cineramayeah that's a known thing i guess19:04
jlvillalNobodyCam: Okay, that message is a red herring. The actual error is farther up19:04
jlvillalI believe19:04
NobodyCamI've lost that screen19:04
wajdiNobodyCam: what's the error you see in /opt/stack/logs/grenade.sh.txt.gz19:04
NobodyCamlet me re-run19:05
NobodyCamwill take a few19:05
NobodyCamw19:05
NobodyCamwajdi: one sec let me get back in there19:06
*** rloo has quit IRC19:06
*** rloo_ has joined #openstack-ironic19:06
jlvillalDoes anyone know what 'short_source' is?19:07
jlvillalAs in: https://bugs.launchpad.net/ironic-python-agent/+bug/157877819:07
openstackLaunchpad bug 1578778 in ironic-python-agent "Unable to build tinyipa behind a proxy server" [Undecided,In progress] - Assigned to John L. Villalovos (happycamp)19:08
*** Nisha has quit IRC19:08
*** aginwala has quit IRC19:08
jlvillalFound it, a devstack thingy19:09
NobodyCamjlvillal: wajdi: http://paste.openstack.org/show/6SDJ3jeVOFP7LyJJdp4L/19:11
jlvillalNobodyCam: Interesting...19:12
*** piet has quit IRC19:12
wajdivery. I haven't gotten that far. But I haven't been able to apply that patch yet either. Did you apply any patches to your run?19:12
openstackgerritJarrod Johnson proposed openstack/pyghmi: Implement Python3 compatibility  https://review.openstack.org/31301219:12
jjohnson2lucas-afk, well, it worked for python 2.7 and 3.4 in a very simple test19:13
NobodyCamwajdi: no patches just switch to tinyIPA19:13
wajdihmm...definitely don't have that and I have the tinyIPA addition.19:14
jlvillalTheJulia: Do you recommend I add:     (cd /opt/stack/new/devstack; git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/55/311055/1 && git cherry-pick FETCH_HEAD)19:15
*** mkovacik has quit IRC19:15
jlvillalTo the update-projects.sh19:15
jlvillal?19:15
wajdijlvillal:  FWIW I was about to do that as well to see if it helped my permission issue.19:16
wajdifor science sake19:16
jlvillalwajdi: I don't think it is the cause of the errors. Since it happens in worlddump. Which I think means you have already failed. I could be wrong though.19:17
wajdimaybe we are talking about a different patch then. The one I was referring to is an update in worlddump: https://review.openstack.org/gitweb?p=openstack-dev/devstack.git;a=commitdiff;h=68ea8ab0b1d6f798b53fd71fece221cd810c193f#patch119:18
-openstackstatus- NOTICE: Gerrit is restarting to address performance issues related to a suspected memory leak19:21
jlvillalwajdi: The patch I meant: https://review.openstack.org/#/c/311055/19:22
jlvillalwajdi: Sounds like the same one. I thought worlddump.py is when an error has occurred.19:23
wajdijlvillal:  yeah that is true.....it says it right there at the top of the script: """Dump the state of the world for post mortem.""" :)19:25
wajdisorry...I'm learning grenade for the first time and am on day 1.519:25
*** trown is now known as trown|outtypewww19:26
wajdiwill keep digging19:26
*** e0ne has joined #openstack-ironic19:27
*** trown|outtypewww is now known as trown19:29
jlvillalwajdi: :)19:30
*** trown is now known as trown|outtypewww19:32
TheJuliajlvillal: I would say optional but world dumps may be useful :)19:38
jlvillalTheJulia: I've added it and doing a test run now. Thanks.19:39
TheJuliaawesome19:41
TheJuliadigging into my machine where the last failure was to try and figure out what exactly happened19:42
*** absubram has quit IRC19:47
NobodyCamjlvillal: wajdi: hum also found this in the log: http://paste.openstack.org/show/LWyKarUOITBcSD4NqEGU/19:51
*** absubram has joined #openstack-ironic19:52
*** dprince has quit IRC19:54
*** e0ne has quit IRC19:55
jlvillalNobodyCam: That is a very long line ;)19:57
*** piet has joined #openstack-ironic19:57
*** piet has quit IRC19:57
*** thiagop has quit IRC19:58
wajdiNobodyCam: I just ran again applying the patch that jlvillal mentioned above and I think it helped clear out an error blocking some other errors from appearing19:58
wajdiI seem to be having swift and neutron errors19:58
wajdibut I just opened the log and went through any error & fail mentions19:58
wajdithe most recent error I do see in the grenade log file is "./stack.sh: line 480: kill: (28819) - No such process"19:59
* jlvillal tries to wait patiently for his test run to finish...20:00
wajdi^^I if you match one of our test results, we are on a good path. If we each have different results.....that would not be fun :P20:00
*** daemontool_ has quit IRC20:01
wajdiafk for a bit. Meeting. I20:01
*** wajdi is now known as wajdi_afk20:01
*** wajdi_af_ has joined #openstack-ironic20:02
*** wajdi_a__ has joined #openstack-ironic20:03
*** wajdi____ has joined #openstack-ironic20:05
*** wajdi____ is now known as wajdi20:05
NobodyCamand this... http://paste.openstack.org/show/blT4Bvt5LAMRNqtrQV4r/20:06
*** wajdi_afk has quit IRC20:06
*** Sukhdev has joined #openstack-ironic20:06
*** wajdi_af_ has quit IRC20:06
*** wajdi_a__ has quit IRC20:08
TheJuliaso, my last attempt failed because apparently dhcping failed20:08
*** ayoung has quit IRC20:10
*** hamzy has quit IRC20:10
TheJuliahttp://paste.openstack.org/show/j02z1C90DVq1M6pmQzyL/20:13
*** piet has joined #openstack-ironic20:13
lucas-afkrloo_, jlvillal no problem killing that etherpad if needed to centralize that information, it was started just so I could save my progress on trying to run grenade w/o vagrant20:15
jlvillallucas-afk: Thanks20:16
rloo_lucas-afk: didn't kill it, i think it is still useful. i just modified it.20:16
lucas-afkright on :-)20:16
rloo_lucas-afk: hopefully my mods were ok. that's the page that contains instructions on how to set up vm blah blah.20:17
lucas-afkjjohnson2, cool! btw, I think you may want to look at "six" for the things like range vs xrage stuff20:17
lucas-afkand httplib etc20:17
wajdiIt helped me. Following that doc I had nearly zero issues getting set-up and running successfully. Therefore, doc was helpful. :)20:17
lucas-afkjjohnson2, I will comment on the patch20:17
lucas-afkrloo_, sure thing! Actually I have to say thanks for updating it20:17
rloo_wajdi: it helped before I updated it, right? I think I updated it correctly but I didn't test it :)20:18
*** alex_xu has quit IRC20:18
*** aginwala has joined #openstack-ironic20:19
jjohnson2lucas-afk, so much more fun to do it this way ;)20:21
*** alex_xu has joined #openstack-ironic20:22
lucas-afkjjohnson2, heh it's not a good practice to overwrite the built-in functions20:23
lucas-afkit can trigger some funny things20:24
lucas-afkjjohnson2, commented20:26
jjohnson2"Service Unavailable"20:27
jjohnson2that's what it thinks of your comment20:27
lucas-afkactually I didn't, gerrit is giving me a service unavaible20:27
lucas-afkhaha20:27
lucas-afkdamn20:27
jjohnson2see, that's what you get20:27
lucas-afklol20:27
jlvillalSeems like a bad day for Gerrit...20:27
NobodyCamyep20:27
NobodyCam I have a review 313025 but can't post it20:28
* lucas-afk steps away for a bit while gerrit is off20:28
lucas-afkok it's back20:28
NobodyCamcomments posted20:28
NobodyCam:p20:28
-openstackstatus- NOTICE: Gerrit is restarting to revert incorrect changes to test result displays20:29
NobodyCamlol20:29
lucas-afkjjohnson2, done :-)20:29
* lucas-afk steps away for a bit20:29
*** dmk0202 has joined #openstack-ironic20:29
jjohnson2hmm, not relishing pulling in six since it's not built in and not in some of the enterprise distros, means I have to pull in another thing to our build server20:30
*** daemontool_ has joined #openstack-ironic20:35
*** jaybeale has joined #openstack-ironic20:38
cineramajlvillal: what'd you do about "short_source"?20:39
jlvillalcinerama: Submited a patch :)20:39
jlvillalcinerama: https://review.openstack.org/31313220:40
cineramajlvillal: nice. is that in upstream update-projects.sh now?20:40
jlvillalcinerama: The current error doesn't break anything functionally. Just spews messages.20:40
cineramaah so just a red herrirng20:40
jlvillalcinerama: Uh, no...20:40
jlvillalBut it has one +2 :)20:40
devanandaJayF: would it be easy for you to grab a sample of the HTTP request for manual cleaning? I'm trying to confirm what the BODY of the request should look like when there's >1 clean step requested20:43
*** piet has quit IRC20:46
*** jaybeale has quit IRC20:47
*** aginwala has quit IRC20:48
*** amit213 has joined #openstack-ironic20:50
*** aginwala has joined #openstack-ironic20:51
mrdaMorning Ironic20:55
devanandamornin!20:56
NobodyCamgood morning mrda20:56
mrdao/20:56
*** amotoki has quit IRC20:58
*** piet has joined #openstack-ironic20:59
*** mbound has joined #openstack-ironic21:01
*** rloo has joined #openstack-ironic21:01
*** aginwala has quit IRC21:03
*** rcernin has joined #openstack-ironic21:05
*** rloo_ has quit IRC21:05
NobodyCamcan someone help point me to the actual ironic logs in this grenade world.. I thought they would be in log/screen but nope :p21:06
jlvillalNobodyCam: /opt/stack/logs/21:07
NobodyCamhum didn't see them21:08
NobodyCamstarted another run21:08
jlvillalNobodyCam: TheJulia cinerama sambetts|afk and everyone else. Updated https://github.com/JohnVillalovos/devstack-gate-test  to use tinyipa and pulled in a couple new cherry-pick patches21:08
NobodyCamwill check in a few21:08
jlvillalBasically updates to 'ironic-grenade.sh' and 'update-projects.sh'21:09
NobodyCamoh sure after I already started21:09
NobodyCamlol21:09
jlvillalNobodyCam: So /opt/stack/ is where all the work is done.  We are primarily concerned with /opt/stack/old/ at this point.21:09
jlvillalLogs are in /opt/stack/logs/ and /opt/stack/old/  I think they get copied from old to the logs directory.21:10
jlvillalNobodyCam: As a note there is a program.  uncompress-gz-files.py21:11
jlvillalIf you change into /opt/stack/logs/ and then run uncompress-gz-files.py21:11
jlvillalIt will uncompress all the log files21:11
jrollor just zcat mylog.gz21:12
jrollzgrep, etc21:13
*** aginwala has joined #openstack-ironic21:14
jlvillaljroll: True. That too.21:14
*** aginwala has quit IRC21:16
mrdaCan someone point me to the repo where our tempest test using virtualbmc is defined?21:19
* mrda doesn't really know about how the tests are organised21:20
jrollmrda: openstack-infra/project-config/jenkins/jobs/devstack-gate.yaml defines the jobs21:20
jrollthe driver is defined there21:20
mrdathanks jroll!21:20
jrollif it's an ipmitool driver and not hardware, it uses vbmc21:20
jlvillalDoes IBM give out the title "Master Inventor"?21:22
*** mbound has quit IRC21:24
cineramajlvillal: thanks for those updates, now i don't have to carry local debt21:28
jlvillal:)21:28
*** jjohnson2 has quit IRC21:32
*** praneshp has quit IRC21:33
*** ametts has quit IRC21:34
*** causten_ has quit IRC21:37
*** causten__ has joined #openstack-ironic21:37
*** aginwala has joined #openstack-ironic21:37
jrollnight y'all21:38
NobodyCamnight jroll21:39
*** Goneri has quit IRC21:39
mrdao/21:40
*** aginwala has quit IRC21:44
*** aginwala has joined #openstack-ironic21:45
*** aginwala_ has joined #openstack-ironic21:47
*** aginwala has quit IRC21:50
*** piet has quit IRC21:50
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Root and Nodes  https://review.openstack.org/31279521:53
*** thrash is now known as thrash|g0ne21:56
*** aginwala has joined #openstack-ironic21:56
*** ametts has joined #openstack-ironic21:56
*** aginwala_ has quit IRC21:59
*** ayoung has joined #openstack-ironic22:03
*** aginwala has quit IRC22:07
*** Sukhdev has quit IRC22:08
*** rloo has quit IRC22:11
*** Sukhdev has joined #openstack-ironic22:11
*** rloo has joined #openstack-ironic22:12
*** aginwala has joined #openstack-ironic22:13
*** aginwala has quit IRC22:17
*** rloo has quit IRC22:17
*** wajdi has quit IRC22:17
*** absubram has quit IRC22:22
*** Sukhdev has quit IRC22:24
*** jcoufal has quit IRC22:26
*** praneshp has joined #openstack-ironic22:26
*** rcernin has quit IRC22:29
*** Sukhdev has joined #openstack-ironic22:33
thiagop-afkg'night folks22:33
*** thiagop-afk has quit IRC22:33
*** ametts has quit IRC22:35
*** fragatin_ has joined #openstack-ironic22:37
*** aginwala has joined #openstack-ironic22:38
*** aginwala has quit IRC22:40
*** fragatina has quit IRC22:41
*** aginwala has joined #openstack-ironic22:42
*** sacharya has quit IRC22:45
*** garthb has quit IRC22:47
*** jcoufal has joined #openstack-ironic22:47
*** garthb_ has joined #openstack-ironic22:47
harlowjaqq for folks, probably controversial, but what are the ways to deploy ironic that aren't nova, i see bifrost, but is there anything else?22:48
harlowjalike something that extracts https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py and puts it behind a tiny api22:48
*** klindgren has joined #openstack-ironic22:48
harlowjaaddons to ironic perhaps :-P22:49
*** Sukhdev has quit IRC22:49
*** jcoufal has quit IRC22:49
*** david-lyle has quit IRC22:51
*** david-lyle has joined #openstack-ironic22:53
*** aginwala has quit IRC22:57
*** david-lyle has quit IRC22:58
devanandaharlowja: right now, the only one I'm actively aware of is bifrost22:58
*** aginwala has joined #openstack-ironic22:59
harlowjadevananda kk, nobody tried to shift driver.py into a small rest api or something23:00
harlowjanot saying such a thing should be done, just wondering if anyone has tried :-P23:00
*** davideag_ has joined #openstack-ironic23:04
devanandaharlowja: not afaik23:04
devanandabut now you're just tempting me to do it, aren't you :p23:04
harlowjalol23:04
harlowjanah i'd never tempt such a thing, ha23:05
harlowjai'm just a poor coal-miner in the mines23:05
harlowjalol23:05
harlowjawith a tiny brain23:05
harlowjalol23:06
*** ayoung has quit IRC23:07
*** davideagnello has quit IRC23:07
*** jaypipes has quit IRC23:16
*** rloo has joined #openstack-ironic23:18
*** rloo has quit IRC23:23
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Root and Nodes  https://review.openstack.org/31279523:25
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Ports  https://review.openstack.org/31318723:25
* devananda steps away23:26
*** rbudden has quit IRC23:26
*** mtanino has quit IRC23:30
*** mtanino has joined #openstack-ironic23:30
*** krotscheck has quit IRC23:31
*** krotscheck has joined #openstack-ironic23:31
*** causten__ has quit IRC23:34
*** dmk0202 has quit IRC23:42
*** davideag_ has quit IRC23:45
*** sacharya has joined #openstack-ironic23:45
*** ijw has quit IRC23:45
*** ijw has joined #openstack-ironic23:46
*** dtantsur|afk has quit IRC23:47
*** rloo has joined #openstack-ironic23:47
*** rloo has quit IRC23:48
*** rloo has joined #openstack-ironic23:49
*** sacharya has quit IRC23:50
*** ijw has quit IRC23:51
*** ayoung has joined #openstack-ironic23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!