| Nisha_Agarwal | TheJulia, re comments on SUM patch, i will let owner do it. its already 5.35 am here. she should be there in office by 8.30 am or so. | 00:08 |
|---|---|---|
| openstackgerrit | Richard G. Pioso proposed openstack/ironic master: Add 'hardware type' for Dell EMC iDRACs https://review.openstack.org/491263 | 00:08 |
| TheJulia | Nisha_Agarwal: Okay, get some sleep :) | 00:10 |
| TheJulia | rpioso: ^^^ good? | 00:10 |
| Nisha_Agarwal | TheJulia, :) actually i dont have those nitty details so i cant update the patch | 00:11 |
| Nisha_Agarwal | TheJulia, thanks for the reviews :) now logging off. will login my day time in few hrs from now | 00:13 |
| *** Nisha_Agarwal has quit IRC | 00:17 | |
| rpioso | TheJulia: Ack re: Redfish vendor extensions. | 00:27 |
| rpioso | TheJulia: Thank you for the review. I've left the Workflow -1 on it. I'm waiting on a 3d party CI test result. | 00:28 |
| TheJulia | Understood | 00:28 |
| rpioso | TheJulia: As I commented, it'll be removed by midday tomorrow. That would be US EDT. | 00:29 |
| rpioso | That's all for me. Good day ironicers! | 00:30 |
| *** rpioso has quit IRC | 00:35 | |
| *** stendulker has joined #openstack-ironic | 00:52 | |
| *** rama_y has joined #openstack-ironic | 00:54 | |
| *** hw_wutianwei has joined #openstack-ironic | 00:58 | |
| *** litao__ has joined #openstack-ironic | 00:58 | |
| *** phuongnh has joined #openstack-ironic | 01:00 | |
| *** libin_ has quit IRC | 01:04 | |
| *** rloo has quit IRC | 01:13 | |
| *** rama_y has quit IRC | 01:17 | |
| *** gcb has joined #openstack-ironic | 01:32 | |
| *** tuanluong has joined #openstack-ironic | 01:44 | |
| *** crushil has quit IRC | 01:46 | |
| *** crushil has joined #openstack-ironic | 01:53 | |
| *** stendulker has quit IRC | 02:07 | |
| *** cdearborn has quit IRC | 02:09 | |
| *** r-daneel has joined #openstack-ironic | 02:09 | |
| *** crushil has quit IRC | 02:12 | |
| *** crushil has joined #openstack-ironic | 02:12 | |
| *** rama_y has joined #openstack-ironic | 02:35 | |
| *** krtaylor has joined #openstack-ironic | 02:42 | |
| *** baoli has joined #openstack-ironic | 02:47 | |
| *** baoli has quit IRC | 02:48 | |
| *** mat128 has quit IRC | 02:48 | |
| *** baoli has joined #openstack-ironic | 02:48 | |
| *** baoli has quit IRC | 02:49 | |
| *** zhangfei has joined #openstack-ironic | 02:53 | |
| *** rama_y has quit IRC | 03:10 | |
| *** vishwana_ has joined #openstack-ironic | 03:18 | |
| *** vishwanathj has quit IRC | 03:21 | |
| *** stendulker has joined #openstack-ironic | 03:28 | |
| *** yangyapeng has quit IRC | 03:28 | |
| *** yangyapeng has joined #openstack-ironic | 03:32 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Documetation for 'snmp' hardware type https://review.openstack.org/493468 | 03:32 |
| *** finix has joined #openstack-ironic | 03:34 | |
| finix | Hi folks. Many links in ocata installation guide is broken due to doc | 03:41 |
| finix | migration. This has been fixed here: https://review.openstack.org/#/c/491966/ Could somebody help to review this bug fix for making it go on? Thanks. | 03:41 |
| patchbot | patch 491966 - ironic (stable/ocata) - Fix broken links to ocata installation guide | 03:41 |
| *** udesale has joined #openstack-ironic | 03:44 | |
| *** links has joined #openstack-ironic | 03:47 | |
| *** makowals has quit IRC | 03:47 | |
| *** makowals has joined #openstack-ironic | 03:57 | |
| *** finix has quit IRC | 04:04 | |
| *** finix has joined #openstack-ironic | 04:04 | |
| *** Nisha_Agarwal has joined #openstack-ironic | 04:10 | |
| *** yangyapeng has quit IRC | 04:15 | |
| *** rama_y has joined #openstack-ironic | 04:20 | |
| *** rama_y has quit IRC | 04:28 | |
| *** deray has joined #openstack-ironic | 04:48 | |
| deray | Good Morning ! | 04:49 |
| *** r-daneel has quit IRC | 04:50 | |
| *** r-daneel has joined #openstack-ironic | 04:54 | |
| *** aparnav has joined #openstack-ironic | 05:08 | |
| *** Nisha_Agarwal has quit IRC | 05:11 | |
| *** Nisha_Agarwal has joined #openstack-ironic | 05:18 | |
| *** wong has quit IRC | 05:19 | |
| *** wong has joined #openstack-ironic | 05:19 | |
| *** mvpnitesh has joined #openstack-ironic | 05:21 | |
| openstackgerrit | Aparna proposed openstack/ironic master: Support SUM based firmware update as clean step for iLO drivers https://review.openstack.org/422572 | 05:26 |
| *** zhangfei has quit IRC | 05:31 | |
| *** sasha21 has quit IRC | 05:32 | |
| openstackgerrit | Aparna proposed openstack/ironic master: Support SUM based firmware update as clean step for iLO drivers https://review.openstack.org/422572 | 05:32 |
| *** udesale__ has joined #openstack-ironic | 05:35 | |
| *** udesale has quit IRC | 05:35 | |
| *** mjura has joined #openstack-ironic | 05:38 | |
| *** rama_y has joined #openstack-ironic | 05:38 | |
| *** pcaruana has joined #openstack-ironic | 05:45 | |
| *** zhangfei has joined #openstack-ironic | 05:45 | |
| *** aparnav has quit IRC | 05:48 | |
| *** finix has quit IRC | 05:58 | |
| *** marios has joined #openstack-ironic | 06:00 | |
| openstackgerrit | Nisha Agarwal proposed openstack/ironic master: Add the new capabilities to the iLO InspectInterface https://review.openstack.org/422902 | 06:03 |
| *** namnh has joined #openstack-ironic | 06:05 | |
| *** hoonetorg has quit IRC | 06:08 | |
| *** ChubYann has quit IRC | 06:14 | |
| *** Nisha_Agarwal has quit IRC | 06:19 | |
| *** adisky__ has joined #openstack-ironic | 06:19 | |
| *** Nisha_Agarwal has joined #openstack-ironic | 06:20 | |
| *** hoonetorg has joined #openstack-ironic | 06:21 | |
| *** udesale has joined #openstack-ironic | 06:23 | |
| *** udesale__ has quit IRC | 06:23 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Follow-up to ``ilo`` hardware type documentation https://review.openstack.org/494084 | 06:29 |
| openstackgerrit | Hironori Shiina proposed openstack/ironic master: iRMC: Add documentation for remote volume boot https://review.openstack.org/489620 | 06:34 |
| *** udesale has quit IRC | 06:35 | |
| *** udesale has joined #openstack-ironic | 06:36 | |
| *** aparnav has joined #openstack-ironic | 06:38 | |
| *** jaganathan has joined #openstack-ironic | 06:42 | |
| *** stendulker has quit IRC | 06:44 | |
| *** rcernin has joined #openstack-ironic | 06:57 | |
| *** masuberu has joined #openstack-ironic | 07:14 | |
| *** rcernin has quit IRC | 07:16 | |
| *** masber has quit IRC | 07:16 | |
| *** mgoddard has joined #openstack-ironic | 07:17 | |
| *** masuberu has quit IRC | 07:18 | |
| *** ccamacho has joined #openstack-ironic | 07:18 | |
| *** rcernin has joined #openstack-ironic | 07:19 | |
| *** jaypipes has joined #openstack-ironic | 07:25 | |
| *** masuberu has joined #openstack-ironic | 07:28 | |
| *** hw_wutianwei has quit IRC | 07:29 | |
| *** hw_wutianwei has joined #openstack-ironic | 07:31 | |
| *** masuberu has quit IRC | 07:31 | |
| *** rama_y has quit IRC | 07:44 | |
| *** rbartal_ has joined #openstack-ironic | 07:49 | |
| *** karimb has joined #openstack-ironic | 07:51 | |
| *** e0ne has joined #openstack-ironic | 08:01 | |
| *** ralonsoh has joined #openstack-ironic | 08:04 | |
| *** lucas-afk is now known as lucasagomes | 08:06 | |
| *** andymccr_ is now known as andymccr | 08:08 | |
| *** derekh has joined #openstack-ironic | 08:10 | |
| *** ralonsoh has quit IRC | 08:12 | |
| *** ralonsoh has joined #openstack-ironic | 08:12 | |
| wong | hi,who knows about drac-driver? | 08:15 |
| wong | my node error is like this:Timeout reached while cleaning the node. Please check if the ramdisk responsible for the cleaning is running on the node | 08:15 |
| wong | When I make the node status to provide, I can see the deploy image boot in the node,and the deploy system seems OK in the DELL server node | 08:16 |
| wong | then it’s motionless.After for a long time, the deploy system shutdown and ironic show me the node clean failed. The reason is time out. I don’t know why | 08:17 |
| *** mgoddard has quit IRC | 08:17 | |
| *** karimb has quit IRC | 08:19 | |
| *** kromanenko has joined #openstack-ironic | 08:21 | |
| vdrok | good morning ironic | 08:26 |
| vsaienk0 | morning vdrok | 08:28 |
| openstackgerrit | Merged openstack/bifrost master: [releasenotes] update https://review.openstack.org/493870 | 08:28 |
| vdrok | morning vsaienk0 | 08:28 |
| vsaienk0 | vdrok: dtantsur|afk could you please review https://review.openstack.org/#/q/I0874fe3b3628cb3e662ee01f24c4599247fdc82d,n,z increase host_subset_size for ironic case | 08:28 |
| vdrok | done, this does work, can be seen at http://logs.openstack.org/12/493812/5/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/85cab0f/logs/old/screen-n-sch.txt.gz | 08:30 |
| deray | vdrok, g'morning | 08:32 |
| deray | vsaienk0, o/ | 08:32 |
| deray | vdrok, could you approve this: https://review.openstack.org/#/c/478741/ | 08:33 |
| patchbot | patch 478741 - ironic - Update proliantutils version for Pike release | 08:33 |
| tuanluong | morning ironic. vdrok vsaienk0 wong deray | 08:33 |
| vdrok | I will take a look some time today | 08:33 |
| vdrok | morning tuanluong | 08:33 |
| deray | tuanluong, good morning o/ | 08:34 |
| deray | vdrok, np.. tyt | 08:34 |
| etingof | morning everyone ;) | 08:37 |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic master: Remove ensure_logs_exist check during upgrade https://review.openstack.org/494118 | 08:38 |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic master: Remove ensure_logs_exist check during upgrade https://review.openstack.org/494118 | 08:39 |
| vdrok | morning etingof | 08:39 |
| deray | tuanluong, thx for looking :) | 08:40 |
| deray | etingof, g'morning | 08:40 |
| openstackgerrit | Vasyl Saienko proposed openstack/ironic master: WIP: wait for resources after destroy https://review.openstack.org/493767 | 08:41 |
| *** mvpnitesh has quit IRC | 08:47 | |
| *** mvpnitesh has joined #openstack-ironic | 08:48 | |
| openstackgerrit | OpenStack Release Bot proposed openstack/ironic-python-agent stable/pike: Update .gitreview for stable/pike https://review.openstack.org/494128 | 08:53 |
| openstackgerrit | OpenStack Release Bot proposed openstack/ironic-python-agent stable/pike: Update UPPER_CONSTRAINTS_FILE for stable/pike https://review.openstack.org/494129 | 08:53 |
| openstackgerrit | OpenStack Release Bot proposed openstack/ironic-python-agent master: Update reno for stable/pike https://review.openstack.org/494130 | 08:53 |
| *** sambetts|afk is now known as sambetts | 08:58 | |
| sambetts | Morning all | 08:58 |
| Nisha_Agarwal | vdrok, | 09:00 |
| Nisha_Agarwal | morning | 09:00 |
| *** amarao has joined #openstack-ironic | 09:00 | |
| Nisha_Agarwal | morning ironic | 09:00 |
| vdrok | morning sambetts and Nisha_Agarwal | 09:00 |
| Nisha_Agarwal | o/ | 09:01 |
| Nisha_Agarwal | vdrok, please review https://review.openstack.org/422902 at your leisure. Julia gave +2 and then there was a minor comment which i updated in the morning, hence no +2 votes now on it | 09:02 |
| patchbot | patch 422902 - ironic - Add the new capabilities to the iLO InspectInterface | 09:02 |
| deray | sambetts, Nisha_Agarwal good morning | 09:05 |
| Nisha_Agarwal | deray, o/ | 09:05 |
| *** stendulker has joined #openstack-ironic | 09:05 | |
| *** hoonetorg has quit IRC | 09:07 | |
| *** lucasagomes is now known as lucas-brb | 09:08 | |
| *** hoonetorg has joined #openstack-ironic | 09:10 | |
| *** milan has joined #openstack-ironic | 09:13 | |
| openstackgerrit | M V P Nitesh proposed openstack/python-ironicclient master: Fixes multiple chassis delete https://review.openstack.org/455662 | 09:17 |
| *** zhangfei has quit IRC | 09:22 | |
| *** yufei has joined #openstack-ironic | 09:27 | |
| openstackgerrit | Merged openstack/python-ironic-inspector-client master: Update Inspector docs links in README https://review.openstack.org/493835 | 09:29 |
| *** OctopusZhang has joined #openstack-ironic | 09:30 | |
| *** yufei has quit IRC | 09:30 | |
| *** OctopusZhang has quit IRC | 09:30 | |
| *** zhangfei has joined #openstack-ironic | 09:35 | |
| *** deadnull has joined #openstack-ironic | 09:36 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Follow-up to ``ilo`` hardware type documentation https://review.openstack.org/494084 | 09:37 |
| *** deadnull has quit IRC | 09:41 | |
| *** hw_wutianwei has quit IRC | 09:47 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Documetation for 'snmp' hardware type https://review.openstack.org/493468 | 09:50 |
| milan | morning Ironic! :) | 09:51 |
| stendulker | Morning Milan :) | 09:58 |
| milan | morning stendulker! :) | 09:58 |
| vdrok | morning milan stendulker | 09:58 |
| milan | mornign vdrok! :) | 09:59 |
| stendulker | Good morning vdrok :) | 09:59 |
| deray | milan, stendulker morning o/ | 10:01 |
| milan | morning deray! :) | 10:02 |
| stendulker | morning deray :) | 10:03 |
| *** namnh has quit IRC | 10:04 | |
| *** dtantsur|afk is now known as dtantsur | 10:06 | |
| dtantsur | morning ironic | 10:06 |
| vdrok | o/ dtantsur | 10:06 |
| deray | dtantsur, good morning | 10:07 |
| *** karimb has joined #openstack-ironic | 10:08 | |
| aparnav | hey dtantsur, vdrok, Could you review this patch https://review.openstack.org/#/c/422572/ ? | 10:12 |
| patchbot | patch 422572 - ironic - Support SUM based firmware update as clean step fo... | 10:12 |
| dtantsur | depending on how things look wrt the release.. it feels like the CI is broken, so we may not be able to merge stuff today | 10:13 |
| vdrok | dtantsur: it's not just the feeling :) i've updated the whiteboard | 10:14 |
| dtantsur | aparnav: please avoid rechecking until it's fixed | 10:14 |
| aparnav | dtantsur, sure. | 10:15 |
| *** lucas-brb is now known as lucasagomes | 10:18 | |
| *** Nisha_Agarwal has quit IRC | 10:26 | |
| *** Nisha_Agarwal has joined #openstack-ironic | 10:27 | |
| *** udesale has quit IRC | 10:29 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic-specs master: Enumerate Inspector errors https://review.openstack.org/494157 | 10:37 |
| *** mgoddard has joined #openstack-ironic | 10:38 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-inspector master: Remove ensure_logs_exist check during upgrade https://review.openstack.org/494159 | 10:39 |
| dtantsur | vdrok: is ^^ blocking ironic-inspector release? | 10:39 |
| vdrok | dtantsur: if there is no pike branch for inspector, then I believe so | 10:40 |
| dtantsur | vdrok: then please review :) | 10:41 |
| dtantsur | sambetts: ^^ | 10:42 |
| vdrok | easy :) | 10:42 |
| *** ralonsoh has quit IRC | 10:44 | |
| *** ralonsoh_ has joined #openstack-ironic | 10:44 | |
| *** priteau has joined #openstack-ironic | 10:44 | |
| vdrok | wtf, rechecks don't work | 10:45 |
| vdrok | i love release time :D | 10:46 |
| *** stendulker has quit IRC | 10:49 | |
| dtantsur | yep | 10:50 |
| dtantsur | vdrok: FYI we need to re-release networking-baremetal after adding the missing publish-to-pypi job. I'll handle it. | 10:50 |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic master: Remove ensure_logs_exist check during upgrade https://review.openstack.org/494118 | 10:51 |
| vdrok | dtantsur: sorry. want to retrigger the jobs faster | 10:51 |
| dtantsur | no problem | 10:52 |
| *** zhangfei has quit IRC | 10:52 | |
| Nisha_Agarwal | dtantsur, so ironic can not be released today? | 10:53 |
| Nisha_Agarwal | i was just reading the scroll up | 10:53 |
| dtantsur | Nisha_Agarwal: I'm afraid so, but we'll see | 10:53 |
| Nisha_Agarwal | dtantsur, :( | 10:53 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/ironic-ui master: Imported Translations from Zanata https://review.openstack.org/494165 | 10:54 |
| *** jkilpatr has quit IRC | 10:54 | |
| *** aparnav has quit IRC | 10:57 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/ironic-ui stable/pike: Imported Translations from Zanata https://review.openstack.org/493839 | 10:57 |
| *** crushil has quit IRC | 10:58 | |
| *** crushil has joined #openstack-ironic | 10:59 | |
| *** tuanluong has quit IRC | 11:04 | |
| openstackgerrit | Merged openstack/ironic-python-agent master: Update reno for stable/pike https://review.openstack.org/494130 | 11:07 |
| *** lucasagomes is now known as lucas-hungry | 11:10 | |
| Nisha_Agarwal | dtantsur, can u do favor for https://review.openstack.org/422902 as well :) | 11:11 |
| patchbot | patch 422902 - ironic - Add the new capabilities to the iLO InspectInterface | 11:11 |
| *** deray has quit IRC | 11:15 | |
| *** aparnav has joined #openstack-ironic | 11:16 | |
| *** phuongnh has quit IRC | 11:18 | |
| *** fellypefca has joined #openstack-ironic | 11:19 | |
| *** gcb has quit IRC | 11:20 | |
| TheJulia | Good morning | 11:22 |
| dtantsur | it's on the list, but I need to get some food and do some errands. I'll try to get to it, if nobody ends up being faster | 11:22 |
| dtantsur | morning TheJulia | 11:22 |
| openstackgerrit | M V P Nitesh proposed openstack/ironic master: Validate ipmi_address format on ipmitool power interface validation https://review.openstack.org/436945 | 11:22 |
| dtantsur | TheJulia: the CI went away for a walk, it does not feel like we're releasing this morning :( | 11:22 |
| dtantsur | TheJulia: you may have to release tomorrow instead of me (please keep in mind that both ironic and ironic-inspector require a major version bump) | 11:23 |
| *** Nisha_Agarwal has quit IRC | 11:23 | |
| *** Nisha_Agarwal has joined #openstack-ironic | 11:24 | |
| * TheJulia sighs | 11:29 | |
| TheJulia | dtantsur: so ironic to 9 instead of 8, 8.0.0 was early pike | 11:29 |
| dtantsur | right | 11:29 |
| dtantsur | IIRC we follow SemVer, not named-releases-based versioning | 11:30 |
| *** jkilpatr has joined #openstack-ironic | 11:30 | |
| *** Nisha_Agarwal has quit IRC | 11:31 | |
| TheJulia | I think the resource class changes does kind of warrant 9.0.0 | 11:31 |
| TheJulia | since "if you don't do this, your going to break your installation" things exist | 11:31 |
| *** Nisha_Agarwal has joined #openstack-ironic | 11:32 | |
| TheJulia | dtantsur: how badly did CI go for a walk overnight? | 11:32 |
| *** Nisha_Agarwal has joined #openstack-ironic | 11:32 | |
| TheJulia | did it actually board an airplane and go on vacation? | 11:32 |
| dtantsur | TheJulia: rather to the nearby river. vdrok has a patch for the hard breaking grenade problem, and I think the multinode grenade is still unstable. | 11:33 |
| *** aparnav has quit IRC | 11:34 | |
| *** aparnav has joined #openstack-ironic | 11:34 | |
| TheJulia | That is not too bad. | 11:34 |
| *** mat128 has joined #openstack-ironic | 11:44 | |
| *** rpioso has joined #openstack-ironic | 11:46 | |
| rpioso | Good morning! | 11:46 |
| rpioso | https://review.openstack.org/#/c/491263/ needs Workflow+1 tender loving care :) | 11:48 |
| patchbot | patch 491263 - ironic - Add 'hardware type' for Dell EMC iDRACs | 11:48 |
| dtantsur | rpioso: morning, done | 11:50 |
| dtantsur | rpioso: it will not merge though, until the CI is fixed | 11:50 |
| vdrok | Heh | 11:51 |
| dtantsur | see, you've got even twice workflow+1 :D | 11:51 |
| vdrok | Morning rpioso | 11:51 |
| * TheJulia wonders if she should make that three :) | 11:52 | |
| * TheJulia holds off in case we need to trick gerrit later | 11:53 | |
| *** litao__ has quit IRC | 11:54 | |
| rpioso | Good morning, vdrok :) | 11:55 |
| rpioso | dtantsur: Thank you. CI :( | 11:55 |
| rpioso | 2 Workflow+1 > CI, though | 11:56 |
| rpioso | Good morning, TheJulia :) | 11:57 |
| TheJulia | good morning rpioso | 11:57 |
| *** lucas-hungry is now known as lucasagomes | 11:59 | |
| *** trown|outtypewww is now known as trown | 12:04 | |
| *** jcoufal has joined #openstack-ironic | 12:10 | |
| *** dprince has joined #openstack-ironic | 12:15 | |
| *** rpioso is now known as rpioso|eat | 12:17 | |
| *** aparnav has quit IRC | 12:17 | |
| *** aparnav has joined #openstack-ironic | 12:18 | |
| fellypefca | Good morning everyone :) | 12:20 |
| milan | morning fellypefca, rpioso|eat , TheJulia and all! :) | 12:28 |
| Nisha_Agarwal | TheJulia, morning | 12:31 |
| Nisha_Agarwal | TheJulia, need the favor for https://review.openstack.org/#/c/422902 again | 12:32 |
| patchbot | patch 422902 - ironic - Add the new capabilities to the iLO InspectInterface | 12:32 |
| *** rpioso|eat has quit IRC | 12:34 | |
| *** sasha21 has joined #openstack-ironic | 12:39 | |
| TheJulia | Nisha_Agarwal: Once the gate fix merges | 12:39 |
| dtantsur | vdrok: https://review.openstack.org/#/c/494118/ is going to fail multinode grenade :( | 12:39 |
| patchbot | patch 494118 - ironic - Remove ensure_logs_exist check during upgrade | 12:39 |
| openstackgerrit | Merged openstack/ironic master: [reno] update https://review.openstack.org/493947 | 12:40 |
| TheJulia | speaking of whichhttp://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/console.html#_2017-08-16_12_30_37_729776 | 12:41 |
| TheJulia | err, http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/console.html#_2017-08-16_12_30_37_729776 | 12:41 |
| dtantsur | ffs, what is happening? :( | 12:43 |
| *** dyasny_ has quit IRC | 12:44 | |
| *** hw_wutianwei has joined #openstack-ironic | 12:46 | |
| TheJulia | looks like cleaning alone took an extra 4 minutes | 12:47 |
| *** jlabarre has joined #openstack-ironic | 12:47 | |
| *** lucasagomes is now known as lucas-brb | 12:48 | |
| *** jaganathan has quit IRC | 12:49 | |
| TheJulia | approached 1.5 hours and didn't even get through the first half it looks like :( | 12:51 |
| vdrok | dtantsur: yeah, the usual resources thing | 12:53 |
| *** rloo has joined #openstack-ironic | 12:55 | |
| sambetts | why is nova attempting to do anything to a node thats in a transition "*ing" state?? | 12:55 |
| *** osdev has quit IRC | 12:55 | |
| *** mvpnitesh has quit IRC | 12:57 | |
| *** osdev has joined #openstack-ironic | 12:57 | |
| sambetts | and there are no logs of the error in the nova conductor is seeing in the n-cpu ... | 12:57 |
| sambetts | something does not look happy with rabbit in the ir-api logs... http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/old/screen-ir-api.txt.gz#_2017-08-16_12_01_36_066 | 13:01 |
| vdrok | sambetts: no, it's fine. we were filtering such messages before | 13:02 |
| vdrok | seems like we need to add one more filter | 13:02 |
| *** karimb has quit IRC | 13:04 | |
| openstackgerrit | Milan Kováčik proposed openstack/ironic-inspector master: Refactoring the firewall https://review.openstack.org/471831 | 13:04 |
| openstackgerrit | Milan Kováčik proposed openstack/ironic-inspector master: Allow periodics to terminate inspector https://review.openstack.org/487909 | 13:04 |
| openstackgerrit | Milan Kováčik proposed openstack/ironic-inspector master: Introducing a dnsmasq PXE filter driver https://review.openstack.org/466448 | 13:04 |
| openstackgerrit | Milan Kováčik proposed openstack/ironic-inspector master: TEST check dnsmasq PXE filter in devstack https://review.openstack.org/468712 | 13:04 |
| sambetts | vdrok: ok ... so the request that 409d was a DELETE vifs request... I expect that it is related to https://review.openstack.org/#/c/422678/[3~ | 13:05 |
| patchbot | patch 422678 - nova - Change order of _cleanup_deploy and _unprovision i... (ABANDONED) | 13:05 |
| sambetts | bascally if it takes too long to move from cleaning ---> clean wait then nova will try to do stuff to it but the node is locked | 13:05 |
| vdrok | yup, it seems so | 13:06 |
| pas-ha | vdrok: dtantsur: i've tried to improve the scheduling in nova today too - https://review.openstack.org/#/c/494136/, take a look, should kind of help with rescheduling chances, especially when using resource classes | 13:07 |
| patchbot | patch 494136 - nova - Shuffle best hosts from weighed hosts | 13:07 |
| sambetts | vdrok: I think we need to move unplug vifs to before we issue delete on the node | 13:08 |
| fellypefca | TheJulia, dtantsur Hey, just a little doubt, do you think I need to open a bug/rfe for this patch? https://review.openstack.org/#/c/493983/ | 13:08 |
| patchbot | patch 493983 - ironic - [WIP/DNM] Updating the minimum version of python-o... | 13:08 |
| TheJulia | Not to take away from the current discussion, but did anyone look at the subside logs? | 13:08 |
| sambetts | vdrok: other ways I've fixed it when I've seen that error downstream is to increase the nova-->ironic timeout | 13:09 |
| dtantsur | fellypefca: usually we do not bump versions for bug fixes, unless everything is broken. I'd like to see a bug | 13:09 |
| TheJulia | fellypefca: what dtantsur said. We don't understand the issue that actually caused it. | 13:10 |
| fellypefca | dtantsur TheJulia Ok! thanks. | 13:10 |
| openstackgerrit | Ruby Loo proposed openstack/ironic master: [releasenotes] update irmc's boot-from-volume support https://review.openstack.org/494007 | 13:11 |
| dtantsur | rloo: thanks, I was thinking about ^^^ too | 13:11 |
| rloo | dtantsur: :) | 13:13 |
| d0ugal | Is there something special about introspecting more than 15 nodes at the same time? | 13:13 |
| rloo | good morning all, dtantsur, fellypefca, TheJulia, sambetts, vdrok, pas-ha | 13:13 |
| pas-ha | morning rloo :) | 13:13 |
| rloo | this is a dumb question, but can we turn off cleaning in the gate, temporarily? | 13:14 |
| vdrok | morning rloo | 13:14 |
| dtantsur | d0ugal: PXE, iPXE, DHCP :) all of these can be special | 13:14 |
| rloo | if the cleaning is the issue right now? | 13:14 |
| d0ugal | A strange question, I realise. However, I am testing introspection with tripleo and I have 18 nodes - every time 3 nodes time out. I have rebuilt this env a number of times. | 13:14 |
| vdrok | sambetts: it seems like that exception happens on the server build, not tear down | 13:14 |
| dtantsur | rloo: this may be an unfortunate answer to our problems.. | 13:14 |
| dtantsur | a;sp ,prmomg | 13:14 |
| dtantsur | wow, what was that? | 13:15 |
| sambetts | vdrok: its on a rescheduled build, so it tries to tear down the old instance first | 13:15 |
| dtantsur | it was supposed to be "also, morning" :D | 13:15 |
| rloo | dtantsur: i was waiting for you to tell us :) | 13:15 |
| sambetts | vdrok: at least that is what it seems like | 13:15 |
| dtantsur | rloo: blind typing lets me down when I'm tired :) | 13:15 |
| rloo | dtantsur: :( Hang in there, only a few more hours for you! | 13:16 |
| *** cdearborn_ has joined #openstack-ironic | 13:16 | |
| dtantsur | right, but I won't feel safe and relaxed, knowing that our release is hanging in the air.. | 13:16 |
| vdrok | sambetts: hrm http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz#_2017-08-16_12_26_35_478 | 13:16 |
| vdrok | this seems to be the first try to deploy | 13:17 |
| TheJulia | rloo: I have one issue with that reno revision, note added | 13:17 |
| TheJulia | dtantsur: or perhaps cleaning for multimode ? | 13:17 |
| vdrok | at least I don't see anything related to server d1e316bf-5395-45de-b011-4052e9eb23d9 before that | 13:17 |
| vdrok | checked the primary too | 13:17 |
| fellypefca | Good morning, rloo :) | 13:17 |
| vdrok | morning fellypefca | 13:17 |
| rloo | TheJulia: just saw it. I removed the reference cuz in the 'property' description itself, it doesn't mention subnet mask. | 13:17 |
| fellypefca | Morning vdrok o/ | 13:18 |
| TheJulia | rloo: it should :( | 13:18 |
| sambetts | vdrok: why would nova schedule an instance on to a transition state node? | 13:18 |
| openstackgerrit | Hironori Shiina proposed openstack/ironic master: iRMC: Add documentation for remote volume boot https://review.openstack.org/489620 | 13:18 |
| rloo | TheJulia: https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/irmc/boot.py#L70 | 13:18 |
| sambetts | vdrok: the call thats failing there is actually the add_instance_info_to_node | 13:18 |
| rloo | TheJulia: also, most of that shouldn't even be in the reno, it is way too long. we should really put a link to something. | 13:18 |
| vdrok | sambetts: no idea. yeah, I see that | 13:18 |
| TheJulia | rloo: perhaps the patch just updated? | 13:19 |
| sambetts | vdrok: which means that the node must be in some transition state before nova even triggers on it | 13:19 |
| rloo | TheJulia: huh? | 13:19 |
| TheJulia | https://review.openstack.org/489620 | 13:19 |
| patchbot | patch 489620 - ironic - iRMC: Add documentation for remote volume boot | 13:19 |
| sambetts | vdrok: is there a race somewhere, and we're changing something behind nova's back?? | 13:19 |
| rloo | TheJulia: oh, yeah, maybe that. what state is it in? I'll look. We have a bit more time now before release, right? | 13:20 |
| sambetts | vdrok: or nova is scheduling onto an already scehduled node :/ | 13:20 |
| sambetts | because of multinode races | 13:20 |
| vdrok | sambetts: well, this should have failed then http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz#_2017-08-16_12_26_33_596 | 13:20 |
| TheJulia | rloo: it was just updated, I'll have to take a look. | 13:20 |
| vdrok | *should* :) | 13:20 |
| dtantsur | rloo: yeah, we cannot (IMO) release with a broken CI and a lot of patches approved with a minus from Jenkins.. | 13:21 |
| vdrok | it says that the node is ready for use, all the resources have used=0 | 13:21 |
| vdrok | dtantsur: sambetts: oh, huh http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz#_2017-08-16_12_26_33_886 | 13:23 |
| vdrok | resource provider does not exist | 13:24 |
| vdrok | then another request grabs the node, yeah | 13:24 |
| vdrok | but then this http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz#_2017-08-16_12_26_34_054 | 13:25 |
| TheJulia | rloo: the proposed doc reads a little awkwardly, and since it is doc, it can land quite easily. | 13:25 |
| TheJulia | rloo: that being said, it gets the point across, at least to me. | 13:25 |
| rloo | TheJulia: i'm commenting/editing it now. Although after I comment, we could land and there could be a follow up. | 13:25 |
| TheJulia | rloo: that could work, that way the reno just becomes mostly a link | 13:26 |
| rloo | TheJulia: err. I am only at the first paragraph though. Which means it could take me awhile to comment. If you think it is ready to land, maybe we should land. then I'll do a followup edit? | 13:26 |
| sambetts | vdrok: aren't the resource provider UUID meant to match the ironic node uuids?? | 13:26 |
| vdrok | sambetts: nope. name matches the uuid of the ironic node | 13:27 |
| vdrok | which is a bit unfortunate | 13:27 |
| TheJulia | rloo: if your wanting to correct the awkwardness, yes, I think that is best. I have only one nit that I stumble over but that is easily followed up | 13:27 |
| rloo | TheJulia: we have a plan! | 13:27 |
| TheJulia | \o/ | 13:27 |
| rloo | TheJulia: did you note that here too, there is no mention of subnet mask? | 13:28 |
| TheJulia | it says network size, which is the same thing, just different way of stating it | 13:28 |
| TheJulia | the example shows a CIDR example, which works | 13:28 |
| sambetts | vdrok: ah... ok ... errr... I'm very confused... this is talking to the placement API but there are no placement API logs in that test output | 13:29 |
| rloo | TheJulia: ah yes. I thought I had that too in the reno, but i think i copied from something that wasn't that clear :-(. | 13:29 |
| vdrok | sambetts: wut. they should be in apache | 13:29 |
| sambetts | ah right... | 13:29 |
| sambetts | le sigh... | 13:29 |
| * vdrok looks too | 13:29 | |
| *** chlong_ has joined #openstack-ironic | 13:31 | |
| vdrok | sambetts: soo, node was in clean wait at that moment http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-ir-cond.txt.gz#_2017-08-16_12_25_45_845 | 13:31 |
| * dtantsur wonders which release we're upgrading to which | 13:31 | |
| dtantsur | like, nova already has stable/pike, but we don't.. | 13:31 |
| openstackgerrit | Hironori Shiina proposed openstack/ironic master: iRMC: Add documentation for remote volume boot https://review.openstack.org/489620 | 13:31 |
| vdrok | dtantsur: we still do ocata to master | 13:32 |
| sambetts | vdrok: then nova shouldn't have even attempted to schdule onto it ... | 13:32 |
| vdrok | which means that if some breaking change happens before we cut pike, we are screwed :) | 13:32 |
| TheJulia | rloo: looks like hshiina edited the patch | 13:33 |
| rloo | TheJulia: yeah, just saw that. Like a slow-motion movie :) | 13:33 |
| *** Nisha_Agarwal has quit IRC | 13:33 | |
| vdrok | dtantsur: wrt the issue we're talking about with sambetts , it seems like nova tries to schedule on a node in cleanwait | 13:34 |
| dtantsur | also https://github.com/openstack-dev/devstack/commit/6b354a909535a9dfef164c3e844544989d17514a broke inspector in one more way omg... | 13:34 |
| sambetts | cleanwait/cleaning | 13:34 |
| *** Nisha_Agarwal has joined #openstack-ironic | 13:34 | |
| *** osdev has quit IRC | 13:34 | |
| TheJulia | rloo: a little better, +2'ing | 13:35 |
| rloo | TheJulia: I +2'd again :) | 13:35 |
| *** jlabarre has quit IRC | 13:35 | |
| sambetts | vdrok: by the error it seems like cleaning rather than clean wait, because of "transition state" part of the error | 13:35 |
| TheJulia | rloo: +2+A'ed | 13:36 |
| rloo | TheJulia: :) | 13:36 |
| vdrok | sambetts: in conductor logs it says clean wait at that moment | 13:36 |
| dtantsur | TheJulia, rloo, folks, I think we'll have to release ironic-inspector :( it's CI is broken too deeply by https://github.com/openstack-dev/devstack/commit/6b354a909535a9dfef164c3e844544989d17514a | 13:36 |
| sambetts | vdrok: clean wait isn't a locked state | 13:36 |
| dtantsur | s/it | 13:36 |
| vdrok | I think we consider any state transition if the target_provision_state is set | 13:36 |
| dtantsur | s/it's/its/ | 13:37 |
| sambetts | vdrok: oh :/ | 13:37 |
| vdrok | hrm | 13:37 |
| sambetts | dtantsur: blah... | 13:37 |
| TheJulia | dtantsur: have to release now with broken ci? | 13:37 |
| sambetts | dtantsur: so frustrating... | 13:37 |
| rloo | dtantsur: if we know why the CI is broken and it isn't due to inspector, i am fine with releasing it | 13:38 |
| dtantsur | TheJulia: hopefully, stable/pike will work. we have 2 breakages due to grenade already defaulting to upgrade from Pike | 13:38 |
| *** jlabarre has joined #openstack-ironic | 13:38 | |
| *** crushil has quit IRC | 13:38 | |
| TheJulia | ugh | 13:38 |
| *** mat128 has quit IRC | 13:38 | |
| TheJulia | dtantsur: ugh * infinity | 13:39 |
| rloo | if we release, we can at least see if stable/pike will work. this is a candidate, not necessarily *final*. | 13:39 |
| TheJulia | Yeah, I guess we could back port a couple changes to the branch from there and cut a final release after branching. | 13:40 |
| *** karimb has joined #openstack-ironic | 13:40 | |
| dtantsur | rloo: ++ and anyway the code is working, we just cannot land new changes | 13:41 |
| rloo | dtantsur: right. So go for it! | 13:41 |
| sambetts | vdrok: yup... clean wait it was ... my bad... | 13:42 |
| sambetts | vdrok: I guess the question is why is nova attempting to schedule onto a clean wait node... | 13:43 |
| vdrok | yup. I'm sure this has something to do with all the hacks we do while reporting resources | 13:43 |
| vdrok | in the virt driver | 13:44 |
| sambetts | this isn't even port-upgrade though... this is like ... stable/ocata code right?/ | 13:44 |
| sambetts | post-upgrade* | 13:44 |
| vdrok | sambetts: no on master | 13:45 |
| *** Nisha_Agarwal has quit IRC | 13:45 | |
| vdrok | the ocata tests passed fine | 13:45 |
| sambetts | all these logs are in the /old folder though | 13:46 |
| openstackgerrit | Fellype Cavalcante proposed openstack/ironic master: Updating the minimum version of python-oneviewclient https://review.openstack.org/493983 | 13:46 |
| fellypefca | dtantsur, TheJulia ^ I think it's more 'understandable' now. :) | 13:46 |
| *** Nisha_Agarwal has joined #openstack-ironic | 13:46 | |
| TheJulia | dtantsur: I wonder if we could cut the release, move the branch to be rebased off a commit from master.... >.> | 13:47 |
| TheJulia | since, it is really just a tag..... | 13:47 |
| dtantsur | TheJulia: I think the release team won't like it | 13:48 |
| dtantsur | anyway, chances are high that ironic-inspector stable/pike will work after branches | 13:48 |
| dtantsur | we have problems now because we essentially upgrade ocata->queens | 13:48 |
| milan | dtantsur, +1 for me | 13:49 |
| TheJulia | dtantsur: I believe it has been done before, either way we have options | 13:50 |
| vdrok | sambetts: yes, because we don't upgrade nova on multinode | 13:51 |
| *** crushil has joined #openstack-ironic | 13:51 | |
| dtantsur | meanwhile, our nova patch was approved | 13:52 |
| vdrok | dtantsur: ++, it seems like for inspector cutting pike should be ok | 13:52 |
| *** mjturek has joined #openstack-ironic | 13:52 | |
| dtantsur | vdrok: could you please propose a backport of https://review.openstack.org/#/c/492964/ to pike once it merges? | 13:53 |
| patchbot | patch 492964 - nova - Fix reporting inventory for provisioned nodes in t... | 13:53 |
| dtantsur | and please recheck if it does not :) | 13:53 |
| vdrok | dtantsur: okie | 13:53 |
| *** lucas-brb is now known as lucasagomes | 13:53 | |
| rloo | yay for approved nova patch! | 13:53 |
| dtantsur | yeah.. it took Vlad and myself 4 days since last Thursday to figure out | 13:54 |
| sambetts | vdrok: so basically something is going weird in stable/ocata nova's resourcing when we kill and restart/upgrade ironic | 13:56 |
| vdrok | sambetts: it's a periodic task executing each minute, and the failure happens like 12 minutes after upgrade, so it seems to be master's issue | 13:57 |
| sambetts | vdrok: stable/ocata nova against master ironic | 13:58 |
| vdrok | ah, right | 13:58 |
| sambetts | the nova logic shouldn't be changing... so it must be getting confused about something when we restart/upgrade ironic | 13:59 |
| *** hw_wutianwei has quit IRC | 13:59 | |
| *** chlong_ has quit IRC | 13:59 | |
| sambetts | errr is this expected? http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz?level=TRACE#_2017-08-16_11_51_21_206 | 14:00 |
| *** rama_y has joined #openstack-ironic | 14:00 | |
| *** udesale has joined #openstack-ironic | 14:01 | |
| sambetts | basically something is going wrong with the placement api | 14:01 |
| TheJulia | sambetts: no, but I think that is before ironic is up and running for the first time | 14:01 |
| vdrok | yup, this is. dtantsur 's patch works around this | 14:01 |
| vdrok | sambetts: | 14:01 |
| vdrok | sambetts: https://review.openstack.org/#/c/494216/1 this is the backport | 14:02 |
| patchbot | patch 494216 - nova (stable/pike) - Fix reporting inventory for provisioned nodes in t... | 14:02 |
| vdrok | to pike though | 14:02 |
| sambetts | seems like it was a major issue in ocata too... | 14:03 |
| sambetts | and I'm wondering if its related to what we're seeing | 14:03 |
| sambetts | because if the inventory isn't zero'd out in placement then aren't some of the nodes still "avaible" to be schduled onto? | 14:03 |
| *** yangyapeng has joined #openstack-ironic | 14:03 | |
| sambetts | I'm seeing a whole ton of http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz?level=TRACE#_2017-08-16_12_15_22_791 | 14:04 |
| sambetts | vdrok: there is a lot of http://logs.openstack.org/18/494118/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/dff3d51/logs/subnode-2/old/screen-n-cpu.txt.gz?#_2017-08-16_12_15_22_749 | 14:09 |
| TheJulia | grenade log shows networking-generic-switch getting upgraded at the same time | 14:09 |
| * dtantsur relocates home, back in an hour | 14:09 | |
| vdrok | sambetts: yup, I see that. | 14:10 |
| *** dtantsur is now known as dtantsur|bbl | 14:10 | |
| vdrok | sorry, /me has to switch to another thing for a bit | 14:10 |
| *** jaypipes has quit IRC | 14:14 | |
| *** ralonsoh_ has quit IRC | 14:16 | |
| *** mat128 has joined #openstack-ironic | 14:20 | |
| *** radez has joined #openstack-ironic | 14:23 | |
| radez | dtantsur: ping, hey I had talked you about uefi in the past, I've got some baremetal nodes that are provisioning now, but when they reboot the drop to the grub prompt | 14:25 |
| radez | if I give it a configfile (hd0,gpt3)/boot/grub2/grub.cfg it boots fine | 14:25 |
| radez | there are separate partitions for efi and boot | 14:26 |
| radez | any thoughts that could help? | 14:26 |
| *** rama_y has quit IRC | 14:26 | |
| *** jaypipes has joined #openstack-ironic | 14:27 | |
| *** rbartal_ has quit IRC | 14:27 | |
| *** aparnav has quit IRC | 14:30 | |
| openstackgerrit | Michael Turek proposed openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 14:31 |
| *** links has quit IRC | 14:32 | |
| *** aparnav has joined #openstack-ironic | 14:36 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 14:42 |
| openstackgerrit | Michael Turek proposed openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 14:42 |
| TheJulia | mjturek: you have whitespace ^^^ | 14:43 |
| *** srobert has joined #openstack-ironic | 14:45 | |
| mjturek | whoops! | 14:46 |
| TheJulia | :) | 14:46 |
| TheJulia | I tried to fix it real quick, but then you edited it | 14:46 |
| mjturek | yeah just saw, I'll update again | 14:46 |
| mjturek | weird, it passed edocs on my machine | 14:47 |
| mjturek | thought that'd be checked | 14:47 |
| TheJulia | pep8 checks it | 14:47 |
| TheJulia | it seems weird | 14:47 |
| *** makowals has quit IRC | 14:48 | |
| *** makowals has joined #openstack-ironic | 14:49 | |
| *** fellypefca has quit IRC | 14:52 | |
| openstackgerrit | Michael Turek proposed openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 14:53 |
| *** dtantsur|bbl is now known as dtantsur | 14:58 | |
| * dtantsur is back but on a meeting | 14:59 | |
| dtantsur | radez: no instant idea, unfortunately. I don't have much experience with grub+uefi | 14:59 |
| *** crushil has quit IRC | 15:00 | |
| *** crushil has joined #openstack-ironic | 15:01 | |
| *** namnh has joined #openstack-ironic | 15:01 | |
| *** _milan_ has joined #openstack-ironic | 15:01 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Follow-up to ``ilo`` hardware type documentation https://review.openstack.org/494084 | 15:04 |
| *** milan has quit IRC | 15:04 | |
| openstackgerrit | Akira Yoshiyama proposed openstack/virtualbmc master: Add Serial-over-LAN (SOL) support https://review.openstack.org/482853 | 15:06 |
| *** racedo has quit IRC | 15:07 | |
| *** rcernin has quit IRC | 15:07 | |
| openstackgerrit | Stephane Robert proposed openstack/ironic master: Baytech MRP27 PDU support https://review.openstack.org/387718 | 15:07 |
| *** racedo has joined #openstack-ironic | 15:09 | |
| *** karimb has quit IRC | 15:12 | |
| *** mjura has quit IRC | 15:12 | |
| TheJulia | rloo: https://review.openstack.org/#/c/494016/ since docs can still land, in theory | 15:12 |
| patchbot | patch 494016 - ironic - Add admin guide for boot from volume | 15:12 |
| rloo | TheJulia: yes they should, am looking at it now. | 15:13 |
| TheJulia | 10-4 | 15:13 |
| dtantsur | mjturek, TheJulia, sorry, a few things to fix | 15:15 |
| dtantsur | mainly, the storage interface is missing from the hardware interfaces documentation in the install guide | 15:15 |
| TheJulia | dtantsur: I think that can be followed up on, the first step goal was to just get a page up so we could link to it in the prelude | 15:16 |
| dtantsur | dunno, it's like 5 mins to fix... | 15:16 |
| * TheJulia shrugs | 15:16 | |
| *** rpioso has joined #openstack-ironic | 15:18 | |
| rloo | dtantsur: I *just* want a link so I can update the reno for pike :) | 15:20 |
| dtantsur | sorry :) | 15:21 |
| *** aparna has joined #openstack-ironic | 15:21 | |
| rloo | dtantsur: no worries. just that gate is so slow, even for doc-only patches :-( | 15:22 |
| dtantsur | oh yeah | 15:22 |
| *** Sukhdev has joined #openstack-ironic | 15:22 | |
| anupn | Good Morning Ironic'ers :) | 15:22 |
| dtantsur | well, we have enough time. we still cannot merge non-docs patches.. | 15:22 |
| dtantsur | morning anupn | 15:23 |
| dtantsur | the nova patch merged \o/ | 15:23 |
| *** aparnav has quit IRC | 15:23 | |
| anupn | dtantsur, morning, TG hump day ;) | 15:24 |
| rloo | dtantsur: yay! | 15:24 |
| vdrok | morning anupn | 15:24 |
| *** MattMan has quit IRC | 15:24 | |
| *** MattMan has joined #openstack-ironic | 15:24 | |
| anupn | vdrok, morning :) | 15:24 |
| *** appu103 has joined #openstack-ironic | 15:25 | |
| *** udesale has quit IRC | 15:25 | |
| mjturek | dtantsur: updating now, thanks! | 15:26 |
| anupn | Hi sambetts: while building tinyipa, i see weird warning "warning: no previously-included files matching '*.pyc' found anywhere in distribution | 15:26 |
| anupn | " and the building stops on this | 15:26 |
| anupn | sambetts, have you seen at this anytime? | 15:26 |
| *** aparna has quit IRC | 15:28 | |
| sambetts | anupn: hmm, if you do "export PYOPTIMIZE_TINYIPA=false" and then re-run the build using "make clean && make all" does it still happen | 15:29 |
| anupn | sambetts, i haven't set PYOPTIMIZE_TINYIPA=false, but we don't usually set that and want .pyo instead correct? | 15:32 |
| anupn | sambetts, but yes, let me try that | 15:32 |
| sambetts | pyoptimize is true by default | 15:32 |
| sambetts | for tinyipa | 15:32 |
| anupn | sambetts, ohh yeah | 15:33 |
| *** links has joined #openstack-ironic | 15:33 | |
| openstackgerrit | Merged openstack/ironic master: iRMC: Add documentation for remote volume boot https://review.openstack.org/489620 | 15:34 |
| anupn | so hence right now it removes all .py and .pyc | 15:34 |
| sambetts | exactly | 15:34 |
| anupn | yo, let me check that | 15:37 |
| *** aparnav has joined #openstack-ironic | 15:38 | |
| *** appu103 has quit IRC | 15:40 | |
| mjturek | dtantsur: other guides don't seem to further indent second lines (ie: https://docs.openstack.org/ironic/latest/admin/portgroups.html) Did you want me to align the bottom args with the top ones? | 15:41 |
| dtantsur | mjturek: I'd say it looks nicer, but I don't insist | 15:42 |
| mjturek | np, I'll add the indent! | 15:42 |
| *** yolanda has quit IRC | 15:44 | |
| dtantsur | thanks! | 15:44 |
| *** yolanda has joined #openstack-ironic | 15:44 | |
| *** ccamacho has quit IRC | 15:44 | |
| *** ccamacho has joined #openstack-ironic | 15:46 | |
| *** links has quit IRC | 15:49 | |
| openstackgerrit | Michael Turek proposed openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 15:50 |
| * dtantsur waits for Jenkins on ^^^ | 15:50 | |
| * rloo waits for dtantsur who waits for ... :) | 15:51 | |
| dtantsur | lol | 15:51 |
| TheJulia | heh | 15:51 |
| mjturek | :) | 15:51 |
| *** marios has quit IRC | 15:51 | |
| *** Sukhdev_ has joined #openstack-ironic | 15:53 | |
| *** rama_y has joined #openstack-ironic | 15:55 | |
| *** pcaruana has quit IRC | 15:57 | |
| *** jaypipes has quit IRC | 15:58 | |
| *** crushil has quit IRC | 16:03 | |
| vdrok | dtantsur: do you know someone from devstack core? I have trouble finding the person to hit approve | 16:05 |
| dtantsur | vdrok: sdague? | 16:05 |
| dtantsur | you can get the list from gerrit | 16:05 |
| vdrok | dtantsur: he's on vacation I think | 16:05 |
| vdrok | ok, then I'll just ping everyone from that list closer to the end of my day :) | 16:06 |
| rama_y | Hi vdrok, sambetts, Trying to deploy baremetal node and I am getting this error "No VIFs found...". Cleaning works. Any suggestions? | 16:11 |
| rama_y | Here is the error in logs: http://paste.openstack.org/show/618440/ | 16:11 |
| sambetts | rama_y: how are you deploying? via nova? | 16:11 |
| rama_y | Tried via Nova; getting "no valid host found" | 16:12 |
| rama_y | Fails immediately | 16:12 |
| Nisha_Agarwal | rloo, thanks for the review on https://review.openstack.org/#/c/422902/21 | 16:12 |
| patchbot | patch 422902 - ironic - Add the new capabilities to the iLO InspectInterface | 16:12 |
| rama_y | Tried "ironic node-set-provision-state node-0 active" and I get the above error | 16:12 |
| dtantsur | rama_y: do you have ports (ironic) attached to nodes? | 16:12 |
| rama_y | Yes | 16:13 |
| sambetts | thats a scheduling issue, https://docs.openstack.org/ironic/latest/admin/troubleshooting.html#nova-returns-no-valid-host-was-found-error | 16:13 |
| rama_y | Cleaning works! | 16:13 |
| Nisha_Agarwal | rloo, i just now replied to your queries, please have a look and let me know if the explanations look ok | 16:13 |
| rama_y | I am able to bring the node to available state | 16:13 |
| sambetts | rama_y: if you deploy using ironic directly you have to manually attach tenent network ports to the ironic node via ironic vif attach | 16:13 |
| *** trown is now known as trown|lunch | 16:13 | |
| dtantsur | FYI the pike backport of the nova fix was also approved | 16:13 |
| rloo | Nisha_Agarwal: ok, will look in a few min | 16:13 |
| Nisha_Agarwal | rloo, thanks | 16:13 |
| sambetts | rama_y: cleaning vifs are internal to ironic so Ironic creates them, tenant vifs are either created by nova or by the user | 16:14 |
| rama_y | sambetts: do I need to create a neutron port? | 16:14 |
| Nisha_Agarwal | dtantsur, yay | 16:14 |
| sambetts | rama_y: if you are deploying via ironic manually... but if you are in an OpenStack environment with nova you should really deploy via nova and it'll create and manage those ports for you | 16:14 |
| sambetts | rama_y: if you are getting no valid host found in nova that indicates there is a bigger problem in your environment than just missing vif on the node | 16:15 |
| sambetts | rama_y: most likely missing scheduling information on the node, or incorrectly set up flavors | 16:15 |
| rama_y | sambetts: ok | 16:15 |
| rama_y | Here is my node show: http://paste.openstack.org/show/618555/ | 16:16 |
| Nisha_Agarwal | dtantsur, whats the progress on gates? and whats the plan on release? | 16:16 |
| *** crushil has joined #openstack-ironic | 16:16 | |
| rama_y | sambetts: properties are set on the node and I am trying to boot flavor 'baremetal' | 16:17 |
| sambetts | rama_y: that looks good at first glance, does your flavor match your node's properties exactly ? | 16:17 |
| dtantsur | Nisha_Agarwal: not much so far :( we hope that TheJulia will release it tomorrow | 16:17 |
| rama_y | sambetts: no it doesn't match exactly | 16:18 |
| Nisha_Agarwal | :) | 16:18 |
| dtantsur | Nisha_Agarwal: vdrok is looking for someone from devstack to approve our fix | 16:18 |
| dtantsur | we'll then probably need a backport to pike and ocata | 16:18 |
| Nisha_Agarwal | ok, | 16:18 |
| rama_y | sambetts: i will give it a try with a new flavor to match my node; is there anything else to look for, other than the flavor, for nova to pick up the node? | 16:18 |
| * TheJulia sees her name mentioned, but she is in a meeting at the moment | 16:19 | |
| dtantsur | it's ok, nothing critical | 16:19 |
| Nisha_Agarwal | :) | 16:19 |
| sambetts | rama_y: I expect that is your issue, flavors for baremetal have to exactly match the resources your baremetal node offers for it to work, you can check if nova sees the baremetal node by looking at "nova hypervisor-stats" and seeing if the ram and cpus are as expected | 16:20 |
| sambetts | if they are then just fixing the flavor *should* work # | 16:20 |
| rama_y | sambetts: I checked that; nova doesn't see the baremetal node | 16:21 |
| sambetts | rama_y: is the baremetal node in the avaiable state? | 16:21 |
| sambetts | rama_y: it has to be in that state and not in maintenance to be useable | 16:21 |
| rama_y | sambetts: ok, I will bring the node to available again and check. It is deploy failed at the moment. | 16:22 |
| openstackgerrit | Stephane Robert proposed openstack/ironic master: Adds snmp driver auto discovery for APC and baytech https://review.openstack.org/389736 | 16:25 |
| *** stendulker has joined #openstack-ironic | 16:26 | |
| rloo | Nisha_Agarwal: I replied. Of those three, is is possible that a node will have BootMode capability? | 16:26 |
| Nisha_Agarwal | no | 16:27 |
| rloo | Nisha_Agarwal: also, wrt max_raid_level. not sure if i understand that, was it actually set in a node, or do you mean 'raid_level" was set? | 16:27 |
| rloo | Nisha_Agarwal: I don't understand how we let capability keys into the code, that were never used. | 16:28 |
| rloo | Nisha_Agarwal: given that they were there and your patch is removing them, there needs to be an explanation in the patch. | 16:29 |
| rama_y | hi sambetts: nova sees the properties now: here is the paste: http://paste.openstack.org/show/618557/ | 16:29 |
| Nisha_Agarwal | rloo, they were added as part of spec approval in kilo release | 16:30 |
| *** aparnav has quit IRC | 16:30 | |
| Nisha_Agarwal | rloo, if you want I can retain them but actually prolaintutelis doesnt discover tehm | 16:31 |
| Nisha_Agarwal | them | 16:31 |
| radez | dtantsur: anyone you could suggest to ask about it? | 16:31 |
| *** kromanenko has quit IRC | 16:32 | |
| dtantsur | radez: mmmm, maybe ilo folks, Nisha_Agarwal or stendulker, they may have some good experience with UEFI | 16:32 |
| *** ccamacho has quit IRC | 16:32 | |
| radez | dtantsur: I think the solution is just writing an extra grub.conf file to the EFI partition that points to the boot partition | 16:32 |
| radez | doesn't ironic handle creating those partitions? | 16:32 |
| dtantsur | radez: it does | 16:33 |
| dtantsur | radez: I don't quite get why "boot partition" is different from "EFI partition" though | 16:33 |
| *** e0ne has quit IRC | 16:33 | |
| rloo | Nisha_Agarwal: it doesn't make sense to retain them if they aren't used. it doesn't make sense to me as to why they were there in the first place. So they should be removed, but I want it documented as to what is going on. I commented in the patch. | 16:33 |
| radez | dtantsur: agreed, I'd think it would be simpler to just through the efi stuff on the boot part with grub | 16:33 |
| rloo | Nisha_Agarwal: you can choose to remove that change since it is outside the scope of that patch. up to you. | 16:34 |
| Nisha_Agarwal | rloo, ok can i do follow up of this? | 16:34 |
| Nisha_Agarwal | i will place the followup in few min | 16:34 |
| rloo | Nisha_Agarwal: no, you can't do a followup. because commit msgs can't be edited afterwards. | 16:34 |
| rloo | Nisha_Agarwal: that patch needs to be updated. as far as i'm concerned. | 16:35 |
| Nisha_Agarwal | ohk | 16:35 |
| Nisha_Agarwal | ok. i will do it. i will address other comments as well | 16:35 |
| radez | Nisha_Agarwal: stendulker: ping, I'm installing UEFI machine with ironic, when the machine boots it falls to a grub prompt. if I execute configfile (hd0,gpt3)/boot/grub2/grub.conf the machine boots fine. Any ideas what may be amiss? | 16:35 |
| sambetts | vdrok: going through a second set of logs for fail multinode and for the second time nova has tried to schedule onto a node in clean wait | 16:35 |
| dtantsur | folks, seems like IPA pike CI does not feel too well.. https://review.openstack.org/#/c/494128/ | 16:36 |
| patchbot | patch 494128 - ironic-python-agent (stable/pike) - Update .gitreview for stable/pike | 16:36 |
| vdrok | dtantsur: hrm, it's just the timeout | 16:36 |
| vdrok | sambetts: ugh. | 16:36 |
| dtantsur | ah, ok | 16:36 |
| dtantsur | we have too many jobs there, we need to urgently start reducing the count | 16:37 |
| * dtantsur wants a standalone job with tinyipa there | 16:37 | |
| sambetts | vdrok: and again this happens in the nova logs just before it schdules onto a node that isn't in a avaiable state http://logs.openstack.org/88/491288/4/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/65aebf7/logs/new/screen-n-cpu.txt.gz#_2017-08-16_01_40_10_021 | 16:37 |
| stendulker | radez: You may have to run the steps in this method manually and see if they go through successfully. https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/extensions/image.py#L79 | 16:38 |
| stendulker | radez: This method takes care of installing grub2 bootloader | 16:39 |
| stendulker | radez: What kind of image are you using? | 16:39 |
| vdrok | sambetts: I think I remember a patch doing deletion of resource provider if it's not available. but, that needs investigation anyway | 16:39 |
| vdrok | s/it's/node | 16:39 |
| radez | stendulker: one built with dib | 16:40 |
| radez | https://images.rdoproject.org/aarch64/ocata/delorean/current-tripleo/stable/overcloud-full.tar | 16:40 |
| stendulker | is it partition image or whole disk? | 16:40 |
| dtantsur | we need to remove jlvillal-bot until we have the CI fixed :) | 16:40 |
| dtantsur | stendulker: I think tripleo uses partition images | 16:40 |
| stendulker | ok | 16:41 |
| dtantsur | built out of centos in this case | 16:41 |
| radez | stendulker: it's a whole disk I thought? | 16:41 |
| radez | dtantsur: ^ | 16:41 |
| stendulker | radez: Are you setting the boot_option to local? | 16:41 |
| radez | yup | 16:41 |
| dtantsur | radez: I'm pretty sure it's a partition image by default, unless images for aarch are built differently | 16:41 |
| stendulker | radez: If its whole disk then Ironic does not install bootloader. It will try to use one in the image | 16:42 |
| radez | ohoh, you're right dtantsur I thought that backwards in my mind | 16:42 |
| radez | stendulker: dtantsur is right it's a partition image | 16:42 |
| stendulker | Whole disk image should have all necessary EFI partitions to make it boot | 16:42 |
| stendulker | For partition image, set 'boot_option' to 'local' in node's 'instance_info' so that Ironic installs bootloader during provisioning | 16:43 |
| radez | profile:compute,boot_option:local,boot_mode:uefi | 16:43 |
| radez | that's my capabilities on all my nodes | 16:44 |
| *** yangyapeng has quit IRC | 16:44 | |
| stendulker | and same thing in node->instance_info as well? | 16:44 |
| radez | instance info? /me checks | 16:44 |
| dtantsur | note that instance_info is set by nova, not by an admin | 16:45 |
| radez | u'capabilities': u'{"profile": "compute", "boot_option": "local"}', | 16:45 |
| stendulker | yes, if its nova boot. | 16:45 |
| radez | so it's missing uefi there | 16:45 |
| *** crushil has quit IRC | 16:45 | |
| radez | I think the bootloader is installed | 16:45 |
| stendulker | then you need to set the boot_mode to uefi in your nova flavor | 16:45 |
| radez | but efi is on a different part than boot, so I think grub.efi is loading but can't find the kernel and such | 16:46 |
| stendulker | bootloader will get installed in bios boot mode as well | 16:46 |
| dtantsur | I think the flavor does not have to be updated for UEFI, but I may be wrong in it | 16:46 |
| * dtantsur never deployed with UEFI, unfortunately | 16:46 | |
| radez | dtantsur: not for profile matching, but maybe to set the bootloader up corectly | 16:46 |
| *** _milan_ has quit IRC | 16:46 | |
| dtantsur | if you see grub.efi booting, then you already have UEFI, so dunno. you can try though | 16:47 |
| dtantsur | it won't hurt trying, right? | 16:47 |
| radez | stendulker: what will boot_mode efi in the flavor do for me? | 16:47 |
| *** yangyapeng has joined #openstack-ironic | 16:47 | |
| radez | I expect it won't hurt... question is if it will help :) | 16:48 |
| *** rcernin has joined #openstack-ironic | 16:49 | |
| stendulker | radez: ok. it would use one in node's capabilities. instance_info not reqd | 16:49 |
| radez | ack | 16:50 |
| stendulker | then we would have to debug the IPA code to see if those steps are giving any warning | 16:50 |
| stendulker | Also, some of the baremetals needs it to be in UEFI mode to install grub2 bootloader for UEFI | 16:50 |
| stendulker | These are the steps followed by IPA to install grub2 in UEFI https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/extensions/image.py#L79 | 16:51 |
| radez | looking through that now | 16:51 |
| *** yangyapeng has quit IRC | 16:52 | |
| radez | stendulker: so it is correct that the efi and boot partitions are different right? so shouldnt there be a grub.conf file written to the efi part so it can find boot? | 16:54 |
| *** lucasagomes is now known as lucas-afk | 16:54 | |
| radez | I only see where it writes grub.cfg to boot | 16:54 |
| stendulker | wont grub-install take care of that? | 16:54 |
| radez | I'm not sure | 16:54 |
| radez | I would guess not if you give it the boot partition to install it on | 16:55 |
| radez | but I'm not sure how grub.efi makes in on the efi part | 16:55 |
| stendulker | grub2-install should have taken care of that. Better to try these commands manually and see if we are missing any step here | 16:57 |
| *** dprince has quit IRC | 16:58 | |
| radez | stendulker: kk, I'll see if I can fix it manually using the grub-install commands and let you know what I dig up, thx for your help | 16:58 |
| stendulker | radez: np | 16:59 |
| *** leyal has quit IRC | 17:00 | |
| *** strigazi_OFF is now known as strigazi | 17:00 | |
| * TheJulia exits meeting | 17:00 | |
| *** leyal has joined #openstack-ironic | 17:00 | |
| jlvillal | :q | 17:00 |
| * jlvillal grumbles | 17:01 | |
| TheJulia | I _think_ grub install mainly just makes sure that things are in-place file wise and then does the nvram change for "hey, boot this file" when it is setting efi boot. | 17:01 |
| sambetts | vdrok: I think I have an idea of what might be going on... we seem to have two instances of nova-compute running on the primary node after upgrade | 17:01 |
| TheJulia | but, it has been like a year and a half since I mucked with that by hand. | 17:02 |
| sambetts | vdrok: and they are racing | 17:02 |
| *** aparnav has joined #openstack-ironic | 17:02 | |
| vdrok | wut | 17:02 |
| TheJulia | sambetts: that is amazingly awesome | 17:02 |
| TheJulia | and makes a lot of sense :( | 17:02 |
| *** mgoddard has quit IRC | 17:02 | |
| sambetts | vdrok: yeah ... basically I've just realised there are two sets of n-cpu logs on the primary with the same timestamps but different information | 17:02 |
| sambetts | TheJulia: yeah... :/ | 17:03 |
| sambetts | vdrok: http://logs.openstack.org/88/491288/4/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/65aebf7/logs/new/screen-n-cpu.txt.gz and http://logs.openstack.org/88/491288/4/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/65aebf7/logs/old/screen-n-cpu.txt.gz | 17:03 |
| sambetts | *mind splode* | 17:03 |
| * TheJulia thinks tequila | 17:04 | |
| vdrok | sambetts: hrm, they are different | 17:04 |
| sambetts | lol | 17:04 |
| vdrok | old one log finishes at 1:26 | 17:04 |
| *** crushil has joined #openstack-ironic | 17:04 | |
| vdrok | new one starts at 1:27 | 17:04 |
| dtantsur | Oo | 17:04 |
| vdrok | I doubt nova would even allow to register two computes with the same name | 17:05 |
| sambetts | ah nuts... I must have been misreading the timestamps.. | 17:05 |
| * sambetts was excited then...# | 17:06 | |
| vdrok | :) | 17:06 |
| vdrok | phew | 17:06 |
| dtantsur | jlvillal: could you please avoid rechecking stuff until the CI is back to us? | 17:06 |
| *** derekh has quit IRC | 17:06 | |
| vdrok | if that was true, we'd have a REALLY hard release :D | 17:06 |
| *** deray has joined #openstack-ironic | 17:07 | |
| *** Sukhdev_ has quit IRC | 17:09 | |
| dtantsur | if it was not hard enough... | 17:09 |
| *** karthiks has quit IRC | 17:10 | |
| *** ElCoyote_ has joined #openstack-ironic | 17:11 | |
| NobodyCam | Good Morning Ironic'ers | 17:11 |
| anupn | Good Morning NobodyCam | 17:12 |
| openstackgerrit | Nisha Agarwal proposed openstack/ironic master: Add the new capabilities to the iLO InspectInterface https://review.openstack.org/422902 | 17:13 |
| *** amarao has quit IRC | 17:13 | |
| NobodyCam | Morning anupn :) | 17:13 |
| Nisha_Agarwal | rloo, Addressed the comments in above patch | 17:13 |
| Nisha_Agarwal | ^^^ | 17:13 |
| deray | NobodyCam, anupn morning o/ | 17:13 |
| NobodyCam | Morning deray :) | 17:14 |
| dtantsur | morning NobodyCam, deray | 17:14 |
| NobodyCam | hey hey Morning dtantsur :) | 17:14 |
| TheJulia | good morning NobodyCam | 17:15 |
| jlvillal | dtantsur, Sure. I just stopped my watch script | 17:15 |
| dtantsur | cool :) | 17:15 |
| deray | TheJulia, dtantsur, good morning | 17:15 |
| NobodyCam | Good Morning TheJulia :) | 17:15 |
| * TheJulia watches the tempest tests run on the other screen | 17:15 | |
| deray | dtantsur, https://review.openstack.org/#/c/494118/ should make the gate jobs pass, it seems.. | 17:16 |
| patchbot | patch 494118 - ironic - Remove ensure_logs_exist check during upgrade | 17:16 |
| vdrok | morning NobodyCam jlvillal | 17:16 |
| dtantsur | TheJulia: I'll have to run soon, I guess I leave the release in your capable hands :) | 17:16 |
| NobodyCam | Good Morning vdrok :) | 17:16 |
| TheJulia | deray: As I understand it, that is what we are waiting for | 17:16 |
| TheJulia | dtantsur: Have a wonderful evening! | 17:16 |
| jlvillal | Hi vdrok and everyone else :) | 17:17 |
| dtantsur | we could wait till Monday, but ideally we should not wait too much, or something else will break us :) | 17:17 |
| deray | TheJulia, okay.. | 17:17 |
| dtantsur | though, at this rate, we may not push everything through the gate even tomorrow.... | 17:17 |
| vdrok | dtantsur: I've pinged all the cores that seem to be in irc and not in japan, but no luck yet | 17:17 |
| deray | jlvillal, o/ | 17:17 |
| dtantsur | anyway, I'll leave it up to you folks | 17:17 |
| jlvillal | :) | 17:17 |
| dtantsur | thanks vdrok.. also please add them to the review | 17:17 |
| dtantsur | they may see it later | 17:18 |
| vdrok | yup | 17:18 |
| dtantsur | have a great weekend everyone, see you on Monday | 17:18 |
| vdrok | good night dtantsur | 17:18 |
| dtantsur | ping me on hangouts/telegram if you need anything (I can't promise a quick response) | 17:18 |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| Nisha_Agarwal | rloo, if any other comment plz leave it on the patch. I will update it asap | 17:20 |
| *** e0ne has joined #openstack-ironic | 17:21 | |
| rloo | Nisha_Agarwal: looking... | 17:21 |
| Nisha_Agarwal | vdrok, when u find time plz see this again https://review.openstack.org/#/c/422902/ | 17:21 |
| patchbot | patch 422902 - ironic - Add the new capabilities to the iLO InspectInterface | 17:21 |
| Nisha_Agarwal | rloo, thanks | 17:21 |
| vdrok | Nisha_Agarwal: I don't think there is a need to rush when we have a broken gate. I doubt it gets better today. | 17:22 |
| vdrok | and it's time to leave for /me | 17:22 |
| vdrok | good night everyone! | 17:22 |
| sambetts | same... and my brain is fried | 17:22 |
| openstackgerrit | Stephane Robert proposed openstack/ironic master: Caching of PDU autodiscovery https://review.openstack.org/399234 | 17:22 |
| sambetts | night vdrok | 17:23 |
| *** sambetts is now known as sambetts|afk | 17:23 | |
| sambetts|afk | night all o/ | 17:23 |
| vdrok | sambetts|afk: good night | 17:23 |
| NobodyCam | night sambetts|afk | 17:23 |
| sambetts|afk | cya NobodyCam | 17:23 |
| NobodyCam | night vdrok | 17:23 |
| NobodyCam | :0 | 17:23 |
| NobodyCam | :) | 17:23 |
| anupn | good night sambetts|afk | 17:23 |
| rloo | good night sambetts|afk and vdrok. Thx for looking into the gate! | 17:24 |
| rloo | good morning NobodyCam :) | 17:24 |
| NobodyCam | Good Morning rloo :) | 17:25 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 17:25 |
| sambetts|afk | rloo: I wish I could have got further debugging it tbh ... but its a real swamp of logs | 17:25 |
| Nisha_Agarwal | vdrok, :) ok | 17:25 |
| rloo | sambetts|afk: at least you were debugging! | 17:25 |
| TheJulia | rloo: ^^^ if you want to +2+A, i fixed the nit | 17:26 |
| sambetts|afk | rloo: heh yeah | 17:26 |
| sambetts|afk | night! | 17:26 |
| *** dprince has joined #openstack-ironic | 17:26 | |
| rloo | TheJulia: thx. | 17:26 |
| anupn | deray, morning :) | 17:26 |
| rloo | TheJulia, mjturek: who wants to address dtantsur|afk's comment? I'm giving you first dibs on it :) | 17:30 |
| TheJulia | I'm working on it | 17:30 |
| *** namnh has quit IRC | 17:30 | |
| rloo | Thx TheJulia! | 17:31 |
| *** Sukhdev has quit IRC | 17:32 | |
| anupn | Hi rloo, TheJulia: anyone tried to clean tinyipa sometime? I am unable to clean it because it can't remove proc/ under directory tinyipafinal/ | 17:34 |
| *** e0ne has quit IRC | 17:34 | |
| rloo | anupn: dunno, not me. i hate cleaning ;) | 17:34 |
| anupn | rloo, :) | 17:35 |
| *** dsneddon is now known as dsneddon_pto | 17:36 | |
| mjturek | d'oh. Sorry dtantsur|afk, I missed that one | 17:36 |
| TheJulia | yeah, not a five minute update either :( | 17:39 |
| rloo | TheJulia: i was wondering about that, so thx for volunteering! | 17:39 |
| *** jtomasek is now known as jtomasek|afk | 17:41 | |
| *** mjturek has quit IRC | 17:43 | |
| *** mjturek has joined #openstack-ironic | 17:44 | |
| TheJulia | building now | 17:47 |
| *** stendulker has quit IRC | 17:50 | |
| *** trown|lunch is now known as trown | 17:52 | |
| *** milan has joined #openstack-ironic | 17:53 | |
| *** strigazi is now known as strigazi_OFF | 17:55 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Add storage interface to enabling-drivers doc https://review.openstack.org/494283 | 17:56 |
| TheJulia | rloo: ^^^^ | 17:56 |
| TheJulia | any other cores ^^^^ | 17:56 |
| rloo | TheJulia: thx | 17:57 |
| *** e0ne has joined #openstack-ironic | 17:57 | |
| TheJulia | I guess in theory, redfish hardware type will work too | 17:58 |
| TheJulia | but in theory only :) | 17:59 |
| *** mgoddard has joined #openstack-ironic | 18:01 | |
| TheJulia | anupn: sorry, I've only built tiny ipa by hand once and it was with the scripts | 18:06 |
| *** ChubYann has joined #openstack-ironic | 18:07 | |
| *** e0ne has quit IRC | 18:07 | |
| anupn | TheJulia, np, i cloned new repo, and i figured i was having trouble because my instance was active with the tinyipa image and i was trying to delete those ramdisk/kernel | 18:08 |
| *** e0ne has joined #openstack-ironic | 18:08 | |
| *** e0ne has quit IRC | 18:08 | |
| TheJulia | And 494118 is in the gate | 18:09 |
| *** pcaruana has joined #openstack-ironic | 18:09 | |
| TheJulia | rloo: how about s/life/lifecycle/? | 18:12 |
| TheJulia | err | 18:12 |
| TheJulia | no, lifetime works better | 18:12 |
| TheJulia | because it refers to the instance | 18:12 |
| rloo | yeah. no cycle there. | 18:12 |
| rloo | unless you mean the big cycle of life... :) | 18:13 |
| TheJulia | if we were talking about the node it's self, then it would be lifecycle | 18:13 |
| rloo | TheJulia: right | 18:13 |
| rloo | TheJulia: but we're talking about fleas on cows | 18:13 |
| rloo | or cattle or something like that, just not dogs. | 18:13 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Add storage interface to enabling-drivers doc https://review.openstack.org/494283 | 18:13 |
| TheJulia | well, the fleas get on both the pets and the cattle | 18:14 |
| TheJulia | :) | 18:14 |
| TheJulia | rloo: I took the bit out of the note. We don't want to draw too much attention but the information should be there... | 18:15 |
| rloo | yay, devstack patch was approved: https://review.openstack.org/#/c/493989/ | 18:16 |
| patchbot | patch 493989 - openstack-dev/devstack - Increase host_subset_size for ironic | 18:16 |
| rloo | TheJulia: thx! | 18:19 |
| *** makowals_ has joined #openstack-ironic | 18:21 | |
| *** sasha21 has quit IRC | 18:23 | |
| *** yuriyz has quit IRC | 18:28 | |
| *** sasha21 has joined #openstack-ironic | 18:29 | |
| *** mgoddard has quit IRC | 18:41 | |
| *** ElCoyote_ has quit IRC | 18:42 | |
| *** Sukhdev has joined #openstack-ironic | 18:52 | |
| openstackgerrit | Aparna proposed openstack/ironic master: Support SUM based firmware update as clean step for iLO drivers https://review.openstack.org/422572 | 18:52 |
| *** e0ne has joined #openstack-ironic | 18:54 | |
| *** _milan_ has joined #openstack-ironic | 18:55 | |
| *** aparnav has quit IRC | 18:56 | |
| *** milan has quit IRC | 18:57 | |
| *** _milan_ has quit IRC | 18:57 | |
| *** ElCoyote_ has joined #openstack-ironic | 18:58 | |
| *** _milan_ has joined #openstack-ironic | 18:59 | |
| *** deray has quit IRC | 19:00 | |
| openstackgerrit | Merged openstack/ironic-ui master: Imported Translations from Zanata https://review.openstack.org/494165 | 19:03 |
| *** sasha21 has quit IRC | 19:03 | |
| TheJulia | hopefully ~20-30 minutes on 494118 | 19:06 |
| *** lihi has quit IRC | 19:13 | |
| TheJulia | One oneview folks around? | 19:15 |
| TheJulia | https://review.openstack.org/#/c/493983/ | 19:16 |
| patchbot | patch 493983 - ironic - Updating the minimum version of python-oneviewclient | 19:16 |
| TheJulia | tl;dr I feel like the release note is way too verbose. | 19:16 |
| *** lihi has joined #openstack-ironic | 19:16 | |
| TheJulia | and actually, that should be upgrade | 19:17 |
| *** mjturek has quit IRC | 19:22 | |
| *** mjturek has joined #openstack-ironic | 19:22 | |
| TheJulia | one more test.... | 19:23 |
| *** adisky__ has quit IRC | 19:23 | |
| openstackgerrit | Merged openstack/ironic master: Remove ensure_logs_exist check during upgrade https://review.openstack.org/494118 | 19:29 |
| *** trown is now known as trown|brb | 19:29 | |
| openstackgerrit | Merged openstack/ironic master: Add admin guide for boot from volume https://review.openstack.org/494016 | 19:29 |
| openstackgerrit | Merged openstack/ironic master: Add storage interface to enabling-drivers doc https://review.openstack.org/494283 | 19:29 |
| rloo | yay, gate is unblocked! https://review.openstack.org/#/c/494118/ | 19:30 |
| patchbot | patch 494118 - ironic - Remove ensure_logs_exist check during upgrade (MERGED) | 19:30 |
| TheJulia | yup, just rechecked the idrac hw type | 19:30 |
| TheJulia | looks like we're going to have to wait for 493421 to finish before we can recheck it | 19:31 |
| TheJulia | rloo: do you want to revise the prelude and we can just +A it once the idrac hw type lands? | 19:35 |
| rloo | TheJulia: yup, now that it looks like things are working again :) | 19:35 |
| TheJulia | :) | 19:35 |
| TheJulia | Yeah, there ism a lot churning on the gate, so it is going to be at least... six to eight hours :(\ | 19:36 |
| rloo | TheJulia: that's fine with me :) | 19:36 |
| rloo | TheJulia: better late than never! | 19:36 |
| TheJulia | :) | 19:37 |
| *** trown|brb is now known as trown | 19:39 | |
| openstackgerrit | Ruby Loo proposed openstack/ironic master: [reno] Add prelude for Pike release https://review.openstack.org/493906 | 19:41 |
| TheJulia | I'll check back in on the gate in ~2 hours, since it looks like there are plenty of resources in the gate at the moment | 19:41 |
| rloo | TheJulia: thx | 19:42 |
| *** Sukhdev has quit IRC | 19:43 | |
| *** yangyapeng has joined #openstack-ironic | 19:44 | |
| *** yangyapeng has quit IRC | 19:49 | |
| *** rcernin has quit IRC | 19:55 | |
| openstackgerrit | Ruby Loo proposed openstack/ironic master: [releasenotes] update irmc's boot-from-volume support https://review.openstack.org/494007 | 19:59 |
| *** _milan_ has quit IRC | 20:01 | |
| *** milan has joined #openstack-ironic | 20:01 | |
| *** Sukhdev has joined #openstack-ironic | 20:13 | |
| *** srobert has quit IRC | 20:13 | |
| *** mat128 has quit IRC | 20:17 | |
| *** dprince has quit IRC | 20:18 | |
| *** trown is now known as trown|outtypewww | 20:28 | |
| *** fellypefca has joined #openstack-ironic | 20:33 | |
| *** makowals_ has quit IRC | 20:36 | |
| *** e0ne has quit IRC | 20:36 | |
| *** jcoufal_ has joined #openstack-ironic | 20:36 | |
| *** fellypefca has quit IRC | 20:36 | |
| *** jcoufal has quit IRC | 20:39 | |
| *** pcaruana has quit IRC | 20:41 | |
| jlvillal | rloo, TheJulia: Do we think the gate is working again? | 20:44 |
| rloo | jlvillal: yes, i hope so anyway. one patch merged and the devstack patch merged. | 20:44 |
| rloo | jlvillal: am waiting to see what happens with the patches that were recheck'd... | 20:45 |
| TheJulia | The grenade tests passed 4x for the in tree patch to land, so I have hope :) | 20:45 |
| rloo | what I don't know, is whether it is urgent for the devstack backports to land: https://review.openstack.org/#/q/I0874fe3b3628cb3e662ee01f24c4599247fdc82d,n,z | 20:47 |
| jlvillal | rloo, Thanks | 20:47 |
| *** jlabarre has quit IRC | 20:47 | |
| rloo | jlvillal: yw. i suppose we should update the etherpad but i'm in the middle of two doc changes... | 20:47 |
| *** jlabarre has joined #openstack-ironic | 20:47 | |
| *** rpioso has quit IRC | 20:48 | |
| *** rpioso has joined #openstack-ironic | 20:48 | |
| *** rpioso has quit IRC | 20:48 | |
| TheJulia | I'll take a look at the couple failures I see in a little bit and update the etherpad | 20:49 |
| *** jcoufal_ has quit IRC | 20:52 | |
| *** ElCoyote_ has quit IRC | 20:54 | |
| radez | slagle: ping, are you familiar with either of these messages: | 20:54 |
| radez | os-collect-config[2023]: No hosts in Heat, nothing written. | 20:54 |
| radez | crap, I lost the other one... sec | 20:55 |
| TheJulia | Wow, os-collect-config, that brings back some memories | 20:56 |
| radez | yea sry, just realized this is more triple-o related than ironic | 20:57 |
| TheJulia | no worries | 20:57 |
| TheJulia | one of the failures, specifically on the recheck of the idrac hw type is a job timeout. It took devstack 40 minutes to stand-up :( | 20:59 |
| *** vishwana_ has quit IRC | 21:03 | |
| *** _milan_ has joined #openstack-ironic | 21:03 | |
| *** milan has quit IRC | 21:03 | |
| *** vishwanathj has joined #openstack-ironic | 21:04 | |
| *** crushil has quit IRC | 21:07 | |
| *** ElCoyote_ has joined #openstack-ironic | 21:11 | |
| *** rama_y has quit IRC | 21:18 | |
| TheJulia | jlvillal: I'm guessing some of those early rechecks didn't actually start with the patch merged :\ | 21:19 |
| TheJulia | since there is the main review server, and ?9? git servers things are replicated down to... might have taken a little while :( | 21:20 |
| jlvillal | TheJulia, Okay. The patch merged at 12:29 for the grenade issue | 21:20 |
| TheJulia | Yeah, I'm saying that even minutes after that, some git servers that code was pulled from might not have had an up to date master branch | 21:21 |
| jlvillal | TheJulia, I think it should propogate very fast. Can look in the logfile to see what it used as the base of Ironic | 21:21 |
| jlvillal | TheJulia, Do you have a pointer to something specific? A failed build? | 21:21 |
| TheJulia | http://logs.openstack.org/75/490675/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/417c978/ | 21:22 |
| jlvillal | Looking | 21:22 |
| jlvillal | TheJulia, That failed before it even got to the new portion of grenade I think. | 21:23 |
| TheJulia | oh...... | 21:24 |
| TheJulia | the log fix | 21:24 |
| jlvillal | Hmmm Failed during upgrade to ironic | 21:24 |
| TheJulia | it needs to be backported | 21:24 |
| jlvillal | I'm not sure | 21:24 |
| jlvillal | TheJulia, Well I doubt it, if the fix merged.. | 21:24 |
| TheJulia | grenade log looks like it couldn't find ir-cond.log | 21:25 |
| TheJulia | one ironic rev now in the gate | 21:25 |
| TheJulia | gah, one of the grenade smoke tests just failed | 21:28 |
| jlvillal | TheJulia, You are correct. That patch did not take affect for the logs you posted. | 21:30 |
| jlvillal | http://logs.openstack.org/75/490675/3/check/gate-grenade-dsvm-ironic-multinode-multitenant-ubuntu-xenial/417c978/logs/grenade.sh.txt.gz#_2017-08-16_18_31_34_286 | 21:31 |
| jlvillal | TheJulia, The job started the recheck at 12:34 and the fix merged at 12:29. So you are correct in the lag. | 21:32 |
| TheJulia | woot! | 21:32 |
| jlvillal | TheJulia, Things I Learned. I thought it was almost instantaneous. | 21:32 |
| *** jlabarre has quit IRC | 21:33 | |
| *** rama_y has joined #openstack-ironic | 21:33 | |
| jlvillal | TheJulia, Oh wait! | 21:34 |
| jlvillal | TheJulia, Your recheck was at 12:34, but there was already a running jenkins job | 21:34 |
| jlvillal | TheJulia, That job finished at 12:37... | 21:34 |
| jlvillal | TheJulia, It had probably started several hours before. | 21:34 |
| TheJulia | ugh | 21:34 |
| jlvillal | TheJulia, A recheck is in progress for it now | 21:35 |
| TheJulia | looks like we are still getting sporadic grenade failures, but at least it passes sometimes now | 21:35 |
| openstackgerrit | Ruby Loo proposed openstack/ironic master: [docs] update irmc boot-from-volume https://review.openstack.org/494329 | 21:41 |
| openstackgerrit | Merged openstack/ironic-python-agent stable/pike: Update .gitreview for stable/pike https://review.openstack.org/494128 | 21:43 |
| openstackgerrit | Merged openstack/ironic-python-agent stable/pike: Update UPPER_CONSTRAINTS_FILE for stable/pike https://review.openstack.org/494129 | 21:43 |
| TheJulia | it looks like random grenade failures and sporadic timeouts :( | 21:49 |
| rloo | :( random is better than 100% failure... | 21:50 |
| jlvillal | +0.5 ;) | 21:52 |
| *** vishwana_ has joined #openstack-ironic | 21:54 | |
| TheJulia | my gut feeling is that 491263 will just timeout due to some reason deep inside the standalone job | 21:55 |
| TheJulia | although I have high hopes for https://review.openstack.org/#/c/493421/ :) | 21:56 |
| patchbot | patch 493421 - ironic - Fix DRAC classic driver double manage/provide | 21:56 |
| TheJulia | I updated the etherpad with a new status. Time to wait some more | 21:57 |
| *** vishwanathj has quit IRC | 21:58 | |
| *** priteau has quit IRC | 21:59 | |
| rloo | thx TheJulia | 21:59 |
| TheJulia | at least... one job is in the gate queue.... | 21:59 |
| *** cdearborn_ has quit IRC | 22:07 | |
| *** _milan_ has quit IRC | 22:07 | |
| *** _milan_ has joined #openstack-ironic | 22:08 | |
| *** jkilpatr has quit IRC | 22:09 | |
| * TheJulia gets a cursed sign out | 22:19 | |
| TheJulia | http://logs.openstack.org/17/473717/21/gate/gate-ironic-python35/2a4c759/console.html | 22:19 |
| openstackgerrit | Ramamani Yeleswarapu proposed openstack/ironic master: [TEST] [DO NOT MERGE] Testing tls-proxy enabling https://review.openstack.org/492664 | 22:21 |
| openstackgerrit | Ruby Loo proposed openstack/ironic master: [admin docs] ironic -> openstack baremetal CLI https://review.openstack.org/494335 | 22:38 |
| rloo | TheJulia: what's with the timeouts? :-( | 22:40 |
| rloo | TheJulia: we think the gate is random, so I think jlvillal should resurrect (again) his script and recheck 493421 | 22:41 |
| rloo | TheJulia: i didn't actually look at 493421, is it really a dependency for iDRAC hw type? | 22:41 |
| TheJulia | We're basically hitting variable performance. I think we need to consider revising out job timeouts | 22:42 |
| jlvillal | rloo, Started up the watcher script again on 493421 | 22:43 |
| TheJulia | rloo: not looking at review at the moment, but if it is the double provide fix, I believe so | 22:43 |
| rloo | TheJulia: yup, sorry should have mentioned it. ok, too bad. | 22:43 |
| rloo | thx jlvillal! | 22:43 |
| TheJulia | I'm waiting for dinner to appear with the family, will check back in a little bit | 22:44 |
| rloo | TheJulia: I have to take off too. | 22:44 |
| rloo | jlvillal: am hoping this will work but maybe add this to your recheck, i think it is the only non-doc patch that is blocking the release: https://review.openstack.org/#/c/473717/ | 22:45 |
| patchbot | patch 473717 - ironic - Add functional API tests for volume connector and ... | 22:45 |
| jlvillal | rloo, done | 22:45 |
| rloo | thx jlvillal! | 22:46 |
| rloo | ciao, might be back later... | 22:46 |
| *** rloo is now known as rloo_afk | 22:47 | |
| *** jkilpatr has joined #openstack-ironic | 22:49 | |
| openstackgerrit | Akira Yoshiyama proposed openstack/virtualbmc master: Add Serial-over-LAN (SOL) support https://review.openstack.org/482853 | 22:51 |
| *** makowals has quit IRC | 23:06 | |
| *** _milan_ has quit IRC | 23:10 | |
| *** _milan_ has joined #openstack-ironic | 23:11 | |
| *** makowals has joined #openstack-ironic | 23:11 | |
| *** Nisha_Agarwal has quit IRC | 23:15 | |
| *** patchbot has quit IRC | 23:17 | |
| jlvillal | TheJulia, rloo:( Job in the gate failed on the Python 3 unit tests. Seriously??? And multi-node grenade. | 23:20 |
| jlvillal | https://review.openstack.org/#/c/473717/ | 23:22 |
| *** patchbot has joined #openstack-ironic | 23:23 | |
| * jlvillal updates his local ironic, cherry-picks the patch, and tries the python 3 tests locally. | 23:24 | |
| jlvillal | And it passed | 23:24 |
| TheJulia | Ugh | 23:27 |
| *** rama_y has quit IRC | 23:31 | |
| *** mjturek has quit IRC | 23:44 | |
| *** yangyapeng has joined #openstack-ironic | 23:45 | |
| *** yangyapeng has quit IRC | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!