| openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Rework make_partitions logic when preserve_ephemeral is set https://review.openstack.org/97590 | 00:13 |
|---|---|---|
| JayF | devananda: I missed your request for help because I disconnected, but the answer is probably that I'm busy right now :( my main opinion is that it stinks to have to workaround older software when newer software exists that already solves the problem | 00:16 |
| *** takadayuiko has joined #openstack-ironic | 00:16 | |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Use processutils properly https://review.openstack.org/91202 | 00:17 |
| *** rloo_ has quit IRC | 00:22 | |
| *** matsuhashi has joined #openstack-ironic | 00:23 | |
| *** kevinbenton has joined #openstack-ironic | 00:45 | |
| openstackgerrit | A change was merged to openstack/ironic-python-agent: Flow extension uses extension manager from agent https://review.openstack.org/92080 | 00:53 |
| *** ellenh has quit IRC | 01:03 | |
| *** Haomeng has quit IRC | 01:31 | |
| *** nosnos has joined #openstack-ironic | 01:38 | |
| *** Haomeng has joined #openstack-ironic | 02:08 | |
| *** takadayuiko has quit IRC | 02:38 | |
| *** harlowja is now known as harlowja_away | 02:42 | |
| *** rwsu has quit IRC | 03:10 | |
| *** k4n0 has joined #openstack-ironic | 03:16 | |
| *** nosnos has quit IRC | 03:29 | |
| *** matsuhashi has quit IRC | 03:31 | |
| *** k4n0 has quit IRC | 03:35 | |
| *** JoshNang has quit IRC | 03:43 | |
| *** k4n0 has joined #openstack-ironic | 03:48 | |
| *** JoshNang has joined #openstack-ironic | 03:59 | |
| *** Jatin360 has joined #openstack-ironic | 04:02 | |
| *** rameshg87 has joined #openstack-ironic | 04:03 | |
| *** sabah has joined #openstack-ironic | 04:25 | |
| *** matsuhashi has joined #openstack-ironic | 04:25 | |
| *** nosnos has joined #openstack-ironic | 04:26 | |
| *** JoshNang has quit IRC | 04:38 | |
| *** JoshNang has joined #openstack-ironic | 04:38 | |
| *** vinbs has joined #openstack-ironic | 04:41 | |
| *** pcrews has quit IRC | 04:44 | |
| vinbs | Hello Ironic! | 04:44 |
| *** sabah has quit IRC | 04:44 | |
| vinbs | I'm trying to launch instances on a baremetal server using Ironic | 04:44 |
| vinbs | Do I need to configure any options in neutron so that dhcp works with baremetal? | 04:45 |
| *** Poornima has joined #openstack-ironic | 04:46 | |
| vinbs | dhcp and pxe boot goes through with a dnsmasq service started manually | 04:47 |
| *** Faizan has joined #openstack-ironic | 04:48 | |
| *** Faizan has quit IRC | 04:49 | |
| vinbs | but how can I configure neutron so that it starts the dnsmasq with the right settings for the dhcp and pxe boot to happen | 04:49 |
| *** shakamunyi has joined #openstack-ironic | 05:04 | |
| *** rakesh_hs has joined #openstack-ironic | 05:15 | |
| *** coolsvap|afk is now known as coolsvap | 05:30 | |
| *** Jatin360_ has joined #openstack-ironic | 05:37 | |
| *** Jatin360 has quit IRC | 05:39 | |
| *** Jatin360_ is now known as Jatin360 | 05:39 | |
| *** pcrews has joined #openstack-ironic | 05:44 | |
| *** lazy_prince has joined #openstack-ironic | 05:49 | |
| *** coolsvap is now known as coolsvap|afk | 05:52 | |
| *** shausy has joined #openstack-ironic | 05:52 | |
| *** pcrews has quit IRC | 05:56 | |
| *** coolsvap|afk is now known as coolsvap | 05:58 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/ironic: Imported Translations from Transifex https://review.openstack.org/96063 | 06:02 |
| *** takadayuiko has joined #openstack-ironic | 06:02 | |
| *** Jatin360_ has joined #openstack-ironic | 06:07 | |
| *** Jatin360 has quit IRC | 06:09 | |
| *** Jatin360_ is now known as Jatin360 | 06:09 | |
| *** Poornima has quit IRC | 06:17 | |
| *** pcrews has joined #openstack-ironic | 06:21 | |
| Haomeng | vinbs: welcome | 06:21 |
| vinbs | Haomeng: Hi :) | 06:22 |
| Haomeng | vinbs: by default, neutron dhcp agent is enabled | 06:22 |
| Haomeng | vinbs: so if you create new network which enable dhcp, it should work for our ironic | 06:22 |
| Haomeng | vinbs: make sure the layer 2(L2) is fine between neutron dhcp agent and our baremetal | 06:23 |
| Haomeng | vinbs: I understand your question here is that your neutron dhcp does not work for our baremetal, righ? | 06:24 |
| Haomeng | right? | 06:24 |
| vinbs | Haomeng: I see that the dnsmasq is running on a tap interface, but it's not forwarding these packets to the actual physical interface to which the baremetal is connected | 06:24 |
| Haomeng | vinbs: ok, did you enable vlan or not? | 06:25 |
| Haomeng | vinbs: what your network type is? | 06:25 |
| Haomeng | Haomeng: did you capture the dhcp package from the physical and tap interface? | 06:25 |
| Haomeng | you can run " neutron subnet-show $subnet-id" to check if enable_dhcp is "True" first | 06:27 |
| vinbs | Haomeng: I don't know much neutron networking | 06:28 |
| Haomeng | vinbs: then, go ahead to check where the dhcp RA package is sent to | 06:28 |
| Haomeng | vinbs: dont worry | 06:28 |
| vinbs | Haomeng: yes it's enabled | 06:28 |
| Haomeng | let me guide you | 06:28 |
| Haomeng | vinbs: ok, that is required, so net step | 06:28 |
| vinbs | I see in the ml2_conf.ini that type_driver is gre | 06:28 |
| vinbs | should it be vlan? | 06:28 |
| Haomeng | vinbs: run dhcp client from our baremetal site to send dhcp RA package to L2 | 06:28 |
| vinbs | Haomeng: Do I need to start a dnsmasq service manually? | 06:30 |
| openstackgerrit | A change was merged to openstack/ironic-python-agent: Allow configdrive partition to be precreated https://review.openstack.org/95969 | 06:31 |
| Haomeng | vinbs: no, it should be existing dnsmasq process which serves your request | 06:31 |
| *** coolsvap is now known as coolsvap|afk | 06:31 | |
| vinbs | So how do I make this existing dnsmasq service to send dhcp packets to the physical interface? | 06:32 |
| Haomeng | just run "ps -ef | grep dnsmasq" to check the existing process | 06:33 |
| Haomeng | to check such "/var/lib/neutron/dhcp/40d74101-3ca9-439d-846a-3ba5746ed722/addn_hosts" arguments | 06:34 |
| Haomeng | and cat the content to see if your physical baremetal mac is in the file | 06:34 |
| Haomeng | such arguments "--dhcp-hostsfile=/var/lib/neutron/dhcp/40d74101-3ca9-439d-846a-3ba5746ed722/host" | 06:34 |
| *** Mikhail_D_ltp has joined #openstack-ironic | 06:35 | |
| Haomeng | or run this command "find /var/lib/neutron/dhcp |xargs grep $YOUR_BM_MAC" to check directly | 06:36 |
| Mikhail_D_ltp | Good morning Ironic! :) | 06:37 |
| Haomeng | Mikhail_D_ltp: morning:) | 06:37 |
| mrda | hi Mikhail_D_ltp (and Haomeng :) | 06:37 |
| Haomeng | mrda: morning:) | 06:37 |
| openstackgerrit | Michael Davies proposed a change to openstack/ironic: Set power-state to POWER_OFF when registering node https://review.openstack.org/97693 | 06:38 |
| Mikhail_D_ltp | mrda, Haomeng hi! :) | 06:38 |
| Haomeng | Mikhail_D_ltp: :) | 06:38 |
| vinbs | Haomeng: I screwed up my openstack setup somehow.. going back to a snapshot | 06:39 |
| *** ccit has joined #openstack-ironic | 06:39 | |
| vinbs | Haomeng: Will come back to you once I get it running again | 06:39 |
| vinbs | thanks a bunch! | 06:40 |
| *** lsmola has quit IRC | 06:40 | |
| Haomeng | vinbs: ok, welcome | 06:41 |
| Haomeng | vinbs: :) | 06:41 |
| *** coolsvap|afk is now known as coolsvap | 06:47 | |
| *** foexle has joined #openstack-ironic | 06:50 | |
| *** eghobo has joined #openstack-ironic | 07:08 | |
| *** radsy has quit IRC | 07:09 | |
| *** matsuhashi has quit IRC | 07:17 | |
| *** matsuhashi has joined #openstack-ironic | 07:18 | |
| *** coolsvap is now known as coolsvap|afk | 07:21 | |
| *** eghobo has quit IRC | 07:24 | |
| *** Jatin360 has quit IRC | 07:25 | |
| *** ccit has quit IRC | 07:26 | |
| *** coolsvap|afk is now known as coolsvap | 07:29 | |
| *** Jatin360 has joined #openstack-ironic | 07:31 | |
| *** max_lobur has joined #openstack-ironic | 07:32 | |
| dtantsur | Morning, Ironic | 07:36 |
| mrda | dtantsur: \o | 07:37 |
| dtantsur | mrda, good evening :) | 07:37 |
| mrda | almost finish time ;) | 07:38 |
| dtantsur | heh | 07:39 |
| takadayuiko | dtantsur: Hi :D | 07:42 |
| dtantsur | takadayuiko, hi :) | 07:43 |
| takadayuiko | dtantsur: Thank you for reviewing my Tempest patch testing Ironic API! | 07:43 |
| dtantsur | np) | 07:43 |
| takadayuiko | https://review.openstack.org/#/c/90282/ | 07:43 |
| dtantsur | I wonder if it's possible to merge _anything_ due to Jenkins failing in 90% cases >_< | 07:44 |
| *** Poornima has joined #openstack-ironic | 07:47 | |
| mrda | :( | 07:49 |
| dtantsur | funny thing: we have a 0-line patch, that was approved on Monday morning. It's still unable to merge :( | 07:50 |
| dtantsur | After they fixed https://bugs.launchpad.net/openstack-ci/+bug/1325689 I at least hope to get valuable logs | 07:50 |
| *** mrda is now known as mrda-away | 07:51 | |
| mrda-away | night ironic! | 07:52 |
| Mikhail_D_ltp | dtantsur: good morning :) | 07:52 |
| dtantsur | morning Mikhail_D_ltp, g'night mrda-away :) | 07:54 |
| openstackgerrit | A change was merged to openstack/ironic-python-agent: Readme should point to Openstack, not travis-ci https://review.openstack.org/97611 | 07:57 |
| *** ndipanov has joined #openstack-ironic | 07:57 | |
| *** derekh_ has joined #openstack-ironic | 07:59 | |
| *** ifarkas has joined #openstack-ironic | 08:00 | |
| dtantsur | Lucky IPA folks, it takes so little to merge their changes ^^^ | 08:00 |
| dtantsur | rameshg87, hi! Could you give some update on https://bugs.launchpad.net/ironic/+bug/1314954 ? | 08:09 |
| *** pradipta_away is now known as pradipta | 08:10 | |
| *** romcheg has joined #openstack-ironic | 08:10 | |
| *** shakamunyi has quit IRC | 08:10 | |
| *** jistr has joined #openstack-ironic | 08:10 | |
| *** athomas has joined #openstack-ironic | 08:12 | |
| *** pbrooko has joined #openstack-ironic | 08:12 | |
| *** radsy has joined #openstack-ironic | 08:22 | |
| *** lucasagomes has joined #openstack-ironic | 08:26 | |
| *** pelix has joined #openstack-ironic | 08:27 | |
| *** Alexei_987 has joined #openstack-ironic | 08:33 | |
| lucasagomes | devananda, ah nice! cheers for that | 08:35 |
| lucasagomes | and morning Ironic :) | 08:35 |
| romcheg | Morning lucasagomes and everyone else! | 08:36 |
| takadayuiko | Good evening, romcheg, lucasagomes! | 08:36 |
| *** igordcard has joined #openstack-ironic | 08:37 | |
| lucasagomes | takadayuiko, konbanwa :) | 08:37 |
| romcheg | Hi takadayuiko! | 08:40 |
| takadayuiko | lucasagomes, Wao! Why do you know Japanese?? Konbanwa & Ohayo(= Good morning) in your country :D | 08:40 |
| Mikhail_D_ltp | romcheg: morning :) | 08:41 |
| lucasagomes | takadayuiko, Ah thanks! I know only very few words, I will keep in mind the ohayo, ohayo gozaimasu :) | 08:42 |
| *** matsuhashi has quit IRC | 08:42 | |
| lucasagomes | takadayuiko, btw, in são paulo we have this "japantown", so couple of my friends when I was back in brazil were decedent of japanese | 08:43 |
| takadayuiko | lucasagomes, Ahaha, very good ;) | 08:43 |
| lucasagomes | takadayuiko, http://en.wikipedia.org/wiki/Liberdade_%28district_of_S%C3%A3o_Paulo%29 | 08:43 |
| rameshg87 | hello dtantsur | 08:45 |
| takadayuiko | lucasagomes, I didn't know such a town! I saw wikipedia, Torii :-o Do you live in Brazil? | 08:46 |
| rameshg87 | dtantsur, i had spent some time working on it. there is no option in ipmitool, niether there is a way to determine if the node is not reachable. i think it is ultimately dependent on the network to reach the timeout on packets and figure out that the ip address is not reachable. | 08:46 |
| rameshg87 | dtantsur, given that i found this bug can fix this: https://bugs.launchpad.net/ironic/+bug/1314961 | 08:47 |
| lucasagomes | takadayuiko, :D torii the gate? no I don't I live in Ireland, but I was born in brazil | 08:47 |
| lucasagomes | takadayuiko, my background is a mix, I'm half brazilian (part of mother) and half spanish | 08:48 |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** vinbs has quit IRC | 08:49 | |
| takadayuiko | lucasagomes, Yes, the gate of Jinja... Do you know Jinja? Similar to temple or church... | 08:49 |
| *** vinbs has joined #openstack-ironic | 08:50 | |
| lucasagomes | takadayuiko, don't know, will google it :D | 08:50 |
| *** matsuhashi has joined #openstack-ironic | 08:50 | |
| * lucasagomes jinja reminds me the template engine only :) | 08:50 | |
| *** openstackgerrit has joined #openstack-ironic | 08:50 | |
| takadayuiko | lucasagomes, Brazil and Spain? Coool!! | 08:51 |
| openstackgerrit | Martin Geisler proposed a change to openstack/ironic: Add UTF-8 coding lines to all Python files https://review.openstack.org/97027 | 08:51 |
| lucasagomes | :) | 08:51 |
| takadayuiko | I'm 100% Japanese... :( | 08:52 |
| lucasagomes | and that's pretty awesome :D | 08:54 |
| takadayuiko | Yes, yes, the name of template engine. I can't understand why named so :-o | 08:54 |
| *** lokesh184 has joined #openstack-ironic | 08:54 | |
| lucasagomes | heh yeah dunno as well, we use jinja2 to build the templates for the pxe driver AFAIR | 08:55 |
| takadayuiko | Lucasgomes, Before I forget again, I wanna ask to you! | 08:55 |
| *** radsy has quit IRC | 08:55 | |
| *** pbrooko has quit IRC | 08:56 | |
| dtantsur | morning lucasagomes, romcheg! | 08:57 |
| lucasagomes | dtantsur, morning | 08:57 |
| *** shausy has quit IRC | 08:57 | |
| romcheg | Morning dtantsur | 08:57 |
| *** shausy has joined #openstack-ironic | 08:58 | |
| takadayuiko | lucasagomes, I found the IRC log in which you and matty_dubs were talking, and I have a question about it. http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2014-01-16.log | 08:58 |
| dtantsur | rameshg87, are you working on it now or intend to in the near future? We're decided on not keeping bugs assigned w/o patch attached for > 7 days | 08:59 |
| lucasagomes | takadayuiko, sure | 08:59 |
| takadayuiko | lucasagomes, Thank you, you ware saying that you used cirros-0.3.1-x86_64-uec as image file. Did you use cirros for "REAL" baremetal machine? | 09:00 |
| lucasagomes | takadayuiko, that's just to make the guideline simpler, and for tests cirros might be ok. I've an old fedora image built with DIB (diskimage-builder) that I use in my internal tests in order to have a full fedora stack | 09:03 |
| lucasagomes | tl;dr I mostly use the fedora image | 09:03 |
| *** max_lobur has quit IRC | 09:03 | |
| dtantsur | heh, I just use cirros :) | 09:04 |
| *** Jatin360 has quit IRC | 09:04 | |
| * lucasagomes needs to rebuild that image, it's a f18 image :P | 09:04 | |
| takadayuiko | lucasagomes, I also want to use Ironic as a trial, it is no problem, but... I used cirros and the procedures of Ironic itself succeed, but after rebooting, "Boot From AHCI CD-ROM" message was shown, so I think it's a failure, right? | 09:06 |
| takadayuiko | dtantsur, Thank you, I am reassured :) | 09:08 |
| *** SpamapS has quit IRC | 09:08 | |
| lucasagomes | takadayuiko, seems so, I haven't hit that tho | 09:09 |
| lucasagomes | takadayuiko, after the reboot it tries to get an IP via DHCP and all? or just show that error straight away? | 09:10 |
| takadayuiko | lucasagomes, Mmm... | 09:10 |
| *** SpamapS has joined #openstack-ironic | 09:11 | |
| takadayuiko | lucasagomes, Sorry for lack of explanation. All of process(Getting IP address from DHCP server, copying image files with "dd" command, ...) finished without problems. | 09:11 |
| lucasagomes | takadayuiko, yeah... so I didn't hit that problem... but seems that in the second reboot (to boot in the image you deployed) it's not PXE booting and trying to do a local boot | 09:13 |
| lucasagomes | like if it wasn't configured to boot from the network or something | 09:13 |
| takadayuiko | lucasagomes, Right, second reboot. | 09:14 |
| rameshg87 | dtantsur, i am not working on it right now. i think i will unassign myself from it | 09:16 |
| lucasagomes | takadayuiko, yeah, so even the second boot will do a network booting via PXE (we have plans to support localboot but it's not done yet) | 09:17 |
| takadayuiko | lucasagomes, I checked the BIOS setting, but "IDE HDD" was the 1st in the boot-priority. So my machine didn't try to PXE boot and there are some problem about image file, I suggest... | 09:18 |
| lucasagomes | takadayuiko, can you check whether the machine is configured to boot from network? | 09:18 |
| lucasagomes | takadayuiko, yeah | 09:18 |
| takadayuiko | lucasgomes, (we have plans to support localboot but it's not done yet)-> What?? | 09:18 |
| lucasagomes | it might be a bug against us because we should be setting the boot device | 09:18 |
| lucasagomes | takadayuiko, https://review.openstack.org/#/c/73054/ | 09:19 |
| takadayuiko | lucasagomes. Ah, no wonder I couldn't any codes about reboot both in Ironic and diskimage-builder. | 09:19 |
| lucasagomes | takadayuiko, we want to add support for booting directly from the local disk after the image is deployed, without having to pxe boot it again | 09:19 |
| takadayuiko | lucasagomes, Thank you, now I can see. | 09:21 |
| lucasagomes | takadayuiko, np :) | 09:22 |
| *** Jatin360 has joined #openstack-ironic | 09:22 | |
| *** lsmola has joined #openstack-ironic | 09:25 | |
| lazy_prince | since local boot is being discussed, could you guys please review https://review.openstack.org/#/c/79289/ (Adds local storage Boot support on PXE failures). seems related.. | 09:25 |
| takadayuiko | lucasagomes, Sorry I've become unable to understand again. So...Should my machine try to PXE boot 2 times and shouldn't try to do a local boot for the second time? | 09:26 |
| openstackgerrit | Yongli He proposed a change to openstack/ironic: Rewrite ironic policy to use the new changes of common policy https://review.openstack.org/97731 | 09:27 |
| lucasagomes | takadayuiko, yeah the current approach is that the machine will always pxe boot, even after deployed (it will load the image from the local disk, but the kernel and ramdisk will be booted from the network) | 09:30 |
| lucasagomes | takadayuiko, the reason is to guarantee the order that the machines boots, it's just a way to assuring that the tentants won't start before the hosts | 09:32 |
| dtantsur | rameshg87, thanks! | 09:34 |
| rameshg87 | dtantsur: :-) | 09:35 |
| lucasagomes | takadayuiko, and, we want to make it optional, we want to have it to be configurable whether the tenant after the deploy will boot from the local disk directly or network | 09:36 |
| takadayuiko | lucasagomes, But switch_pxe_config() in ironic/drivers/modules/deploy_utils.py, pxe config file is modified(rewritten?) as "default boot", right? | 09:36 |
| lucasagomes | we also want to support deploying partition images (like today) or whole disk images (like #73054 is implementing) | 09:37 |
| lucasagomes | takadayuiko, yes! | 09:37 |
| lucasagomes | takadayuiko, but that default is still booting via network, the kernel and ramdisk of the tenant image is in ur tftp directory | 09:39 |
| takadayuiko | So, I think that the machine will boot not as PXE but default after copying image files finished. It's strange doing pxe boot again and again. | 09:40 |
| lucasagomes | takadayuiko, yeah it's a bit odd indeed. But it will pxe boot again | 09:41 |
| takadayuiko | lazy_prince, Sorry I missed your comment. I will check the patch, but I am not sure if I can understand the codes... | 09:41 |
| lucasagomes | takadayuiko, imagine if all the machines are off, and you power it all on again... we don't want the tenant machines to boot before the hosts and start running their services. We need the hosts first and then we can boot the rest | 09:42 |
| dtantsur | JayF, russell_h, jroll https://review.openstack.org/#/c/95969/ see my comment there | 09:42 |
| lucasagomes | that's why it always pxe boot by default, so we can guarantee that order | 09:42 |
| takadayuiko | lazy_prince, Thank you for giving useful information :) | 09:42 |
| lucasagomes | lazy_prince, nice, cheers | 09:43 |
| * dtantsur thinks of using -2 next time >_< | 09:43 | |
| openstackgerrit | Hans Lindgren proposed a change to openstack/ironic: Fix host manager node detection logic https://review.openstack.org/97447 | 09:44 |
| takadayuiko | lucasagomes, yeah that' right. | 09:46 |
| lucasagomes | :) | 09:46 |
| takadayuiko | I learned a lot :D | 09:47 |
| *** martyntaylor has joined #openstack-ironic | 09:50 | |
| dtantsur | Epic success, folks! Seems like we got our gates broken by Nova change | 09:55 |
| *** max_lobur has joined #openstack-ironic | 09:56 | |
| lucasagomes | dtantsur, heh yeah some changes in the host manager | 09:58 |
| lucasagomes | dtantsur, https://review.openstack.org/#/c/97447/ is fixing it, I'm testing it now | 09:58 |
| dtantsur | lucasagomes, yeah, me too... Suggest to merge it asap | 09:59 |
| lucasagomes | dtantsur, +1000! I'm testing it locally first | 09:59 |
| *** jcoufal has joined #openstack-ironic | 10:01 | |
| dtantsur | lucasagomes, what a crazy day! Broken code got merged to IPA, Nova change broke our tests... Is it Monday today? :) | 10:01 |
| lucasagomes | dtantsur, I hear ya! | 10:02 |
| romcheg | testing it as well | 10:02 |
| lucasagomes | I'm also a bit upset with the amount of random failures we are getting in the gate | 10:02 |
| lucasagomes | many of the approved patches is still hanging in the queue waiting for jenkins | 10:03 |
| dtantsur | lucasagomes, that's just awful... | 10:04 |
| takadayuiko | OMG :( | 10:04 |
| *** Jatin360 has quit IRC | 10:04 | |
| dtantsur | lucasagomes, romcheg, test do not pass for me locally :( what about you? | 10:05 |
| romcheg | dtantsur: I'm not yet at that point | 10:05 |
| *** Jatin360 has joined #openstack-ironic | 10:07 | |
| lucasagomes | dtantsur, no dice "TypeError: expected string or buffer" | 10:07 |
| dtantsur | the same for me :( | 10:08 |
| *** nosnos has quit IRC | 10:08 | |
| romcheg | +! | 10:08 |
| romcheg | +1 | 10:08 |
| dtantsur | we don't have Hans here, right? | 10:09 |
| lucasagomes | dunno his IRC name :/ | 10:09 |
| dtantsur | ouch... | 10:10 |
| dtantsur | ok, will relocate to the office, as nothing works anyway... :) | 10:10 |
| romcheg | hanlind is his launchpad id | 10:11 |
| romcheg | I suppose it might be his IRC name as well | 10:11 |
| *** dtantsur is now known as dtantsur|brb | 10:11 | |
| dtantsur|brb | we may ask on nova channel | 10:11 |
| *** athomas has quit IRC | 10:12 | |
| lucasagomes | dtantsur|brb, the fix is small | 10:12 |
| *** yuriyz has quit IRC | 10:13 | |
| dtantsur|brb | lucasagomes, it's not the best practice, but maybe just upload a new version of patch with fix? | 10:13 |
| lucasagomes | dtantsur|brb, if I can't find him I will, I will add a git diff with the fix in the patch | 10:13 |
| lucasagomes | dtantsur|brb, http://paste.openstack.org/show/82752/ | 10:14 |
| romcheg | We have to re-write Ironic from scratch in Swift programming language https://pbs.twimg.com/media/BpRzuI7CIAAKZEj.jpg:large | 10:19 |
| *** shausy has quit IRC | 10:19 | |
| *** shausy has joined #openstack-ironic | 10:20 | |
| *** athomas has joined #openstack-ironic | 10:21 | |
| lucasagomes | romcheg, heh | 10:21 |
| *** matsuhashi has quit IRC | 10:24 | |
| *** yuriyz has joined #openstack-ironic | 10:24 | |
| openstackgerrit | Ramakrishnan G proposed a change to openstack/ironic-specs: iLO Virtual Media Deploy Driver https://review.openstack.org/97744 | 10:32 |
| openstackgerrit | Ramakrishnan G proposed a change to openstack/ironic-specs: iLO Virtual Media Deploy Driver https://review.openstack.org/97744 | 10:35 |
| *** shakamunyi has joined #openstack-ironic | 10:37 | |
| *** takadayuiko has quit IRC | 10:39 | |
| *** Poornima has quit IRC | 10:40 | |
| *** shakamunyi has quit IRC | 10:41 | |
| *** pradipta is now known as pradipta_away | 10:42 | |
| *** romcheg has left #openstack-ironic | 10:43 | |
| *** romcheg has joined #openstack-ironic | 10:43 | |
| *** matsuhashi has joined #openstack-ironic | 10:47 | |
| *** SpamapS has quit IRC | 10:50 | |
| *** SpamapS has joined #openstack-ironic | 10:52 | |
| *** lokesh184 has quit IRC | 11:13 | |
| openstackgerrit | Hans Lindgren proposed a change to openstack/ironic: Fix host manager node detection logic https://review.openstack.org/97447 | 11:14 |
| *** loki184 has joined #openstack-ironic | 11:16 | |
| *** rameshg87 has left #openstack-ironic | 11:17 | |
| *** rameshg87 has joined #openstack-ironic | 11:23 | |
| *** rameshg87 has left #openstack-ironic | 11:23 | |
| *** coolsvap is now known as coolsvap|afk | 11:23 | |
| dtantsur|brb | romcheg, that will be ironic enough :D | 11:23 |
| *** dtantsur|brb is now known as dtantsur | 11:23 | |
| *** matsuhashi has quit IRC | 11:25 | |
| *** matsuhashi has joined #openstack-ironic | 11:26 | |
| *** rameshg87 has joined #openstack-ironic | 11:26 | |
| *** matsuhashi has quit IRC | 11:29 | |
| *** matsuhashi has joined #openstack-ironic | 11:29 | |
| *** r0j4z0 has joined #openstack-ironic | 11:30 | |
| *** loki184 has quit IRC | 11:42 | |
| *** Alexei_987 has quit IRC | 11:42 | |
| *** Alexei_987 has joined #openstack-ironic | 11:42 | |
| *** lucasagomes is now known as lucas-hungry | 11:47 | |
| *** Jatin360_ has joined #openstack-ironic | 11:48 | |
| dtantsur | btw, lucas-hungry, could you give updates on bugs https://bugs.launchpad.net/ironic/+bug/1308680 and https://bugs.launchpad.net/ironic/+bug/1313779 ? | 11:49 |
| *** Jatin360 has quit IRC | 11:50 | |
| *** Jatin360_ is now known as Jatin360 | 11:50 | |
| *** coolsvap|afk is now known as coolsvap | 11:58 | |
| openstackgerrit | Hans Lindgren proposed a change to openstack/ironic: Fix host manager node detection logic https://review.openstack.org/97447 | 12:00 |
| *** coolsvap is now known as coolsvap|afk | 12:04 | |
| *** vinbs has quit IRC | 12:07 | |
| dtantsur | romcheg, could you give some update on https://bugs.launchpad.net/ironic/+bug/1286244 or unassign yourself please? | 12:08 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic-specs: Add instance_info field to Node model https://review.openstack.org/95701 | 12:08 |
| romcheg | dtantsur: I merged all pre-requisites | 12:09 |
| romcheg | Now working on a different bug | 12:09 |
| romcheg | Unassigned myself | 12:09 |
| dtantsur | thanks! | 12:09 |
| romcheg | It's a low-hanginh fruit for someone | 12:09 |
| *** rameshg87 has left #openstack-ironic | 12:13 | |
| boris-42 | romcheg are you going to finish ironic patch? | 12:15 |
| romcheg | boris-42: Now working on a fix to Ironic | 12:15 |
| romcheg | After that will take a look at the Ironic patch | 12:15 |
| boris-42 | romcheg okay nice | 12:17 |
| *** linggao has joined #openstack-ironic | 12:17 | |
| dtantsur | romcheg, lucas-hungry please review https://review.openstack.org/#/c/97447/ again, we need it badly | 12:17 |
| *** jistr is now known as jistr|eng | 12:18 | |
| *** Jatin360 has quit IRC | 12:20 | |
| *** jdob has joined #openstack-ironic | 12:22 | |
| lucas-hungry | dtantsur, ack, unittests passed, will try to deploy a vm with the change | 12:23 |
| lucas-hungry | dtantsur, ok worked here approving | 12:29 |
| dtantsur | lucas-hungry, \o/ | 12:30 |
| lucas-hungry | dtantsur, now we pray for jenkins to not fail in gate :) | 12:31 |
| dtantsur | lucas-hungry, oh, yes... | 12:31 |
| * lucas-hungry brb | 12:31 | |
| *** lazy_prince has quit IRC | 12:38 | |
| openstackgerrit | Sean Dague proposed a change to openstack/ironic: Set instance default_ephemeral_device https://review.openstack.org/96498 | 12:43 |
| openstackgerrit | Sean Dague proposed a change to openstack/ironic: Remove 'node' parameter from the Console and Rescue interfaces https://review.openstack.org/91097 | 12:43 |
| openstackgerrit | Sean Dague proposed a change to openstack/ironic: Work around iscsiadm delete failures https://review.openstack.org/96577 | 12:45 |
| openstackgerrit | Sean Dague proposed a change to openstack/ironic: Remove 'fake' and 'ssh' drivers from default enabled list https://review.openstack.org/88711 | 12:46 |
| *** rloo has joined #openstack-ironic | 12:49 | |
| dtantsur | Interesting way of dealing with problems ^^^ | 12:53 |
| linggao | morning dtantsur | 12:53 |
| dtantsur | morning, linggao! | 12:53 |
| *** rloo has quit IRC | 12:54 | |
| *** rloo has joined #openstack-ironic | 12:54 | |
| linggao | I have a question for you, I updated the pyghmi version in global-requirements.txt, but jekins failed. I have no clue what went wrong. | 12:55 |
| linggao | https://review.openstack.org/#/c/97508/ | 12:55 |
| linggao | dtantsur, could help? | 12:56 |
| dtantsur | lemme see | 12:56 |
| dtantsur | linggao, I don't see anything related to us in a failure. Maybe just recheck? | 12:57 |
| linggao | should I do anything? | 12:58 |
| dtantsur | linggao, recheck bug 1324793 as suggested, I guess | 12:59 |
| linggao | what does that mean? I have never done it before. and this patch is not related to that bug. | 13:00 |
| dtantsur | linggao, that bug is supposed to be the cause of transient failure. Recheck command asks to run tests one more, bug number is provided for statistics | 13:01 |
| dtantsur | linggao, for details see: https://wiki.openstack.org/wiki/GerritJenkinsGit#Test_Failures | 13:01 |
| *** lucas-hungry is now known as lucasagomes | 13:02 | |
| linggao | thanks dtantsur, I'll follow the instructions from that doc. | 13:04 |
| NobodyCam | good morning ironic | 13:09 |
| dtantsur | morning NobodyCam | 13:09 |
| NobodyCam | morning dtantsur | 13:09 |
| NobodyCam | hey hey | 13:09 |
| dtantsur | NobodyCam, aaaaaand before you wake up, I'd like your update on bug https://bugs.launchpad.net/ironic/+bug/1300589 :) | 13:10 |
| NobodyCam | so that bug I have not been able to dig in like I would like to | 13:10 |
| Shrews | So, our changes have broken the gate?? | 13:12 |
| * Shrews tries to catch up on scrollback in -infra | 13:12 | |
| NobodyCam | Shrews: huh | 13:13 |
| NobodyCam | our changes? | 13:13 |
| Shrews | NobodyCam: sdague has taken many approved ironic changes out of the gate | 13:13 |
| Shrews | trying to figure out what's going on | 13:13 |
| *** jbjohnso has joined #openstack-ironic | 13:14 | |
| NobodyCam | oh | 13:14 |
| Shrews | <sdague> I also just kicked all the ironic changes out of the gate pipeline | 13:14 |
| NobodyCam | ahh I guess that would explain this error : Cannot query patchset information | 13:16 |
| dtantsur | NobodyCam, Shrews our unit tests were broken by Nova change | 13:16 |
| dtantsur | waiting for fix to land: https://review.openstack.org/#/c/97447/ | 13:16 |
| Shrews | dtantsur: ah. thanks for the summary | 13:16 |
| *** shakamunyi has joined #openstack-ironic | 13:17 | |
| NobodyCam | is that link working for others, | 13:17 |
| NobodyCam | may be my bandwidth this morning | 13:17 |
| rloo | morning and afternoon to everyone. | 13:18 |
| rloo | NobodyCam: which link? 97447? works for me. | 13:18 |
| NobodyCam | good moring rloo | 13:18 |
| dtantsur | morning, rloo | 13:18 |
| Shrews | <sdague> Shrews: there were enough ironic changes in flight that something that was approved and landed merged in a way that all ironic changes now fail their unit tests | 13:18 |
| NobodyCam | ok so /me is having bandwidth issues | 13:18 |
| Shrews | yay | 13:19 |
| rloo | Shrews: guess that's a good reason for not approving stuff until jenkins is happy. | 13:19 |
| dtantsur | Shrews, rloo, as I said, that's not even we :) | 13:19 |
| NobodyCam | brb | 13:19 |
| Shrews | rloo: yes. take today off :) | 13:20 |
| *** jistr|eng is now known as jistr | 13:20 | |
| dtantsur | rloo, oh yeah, provided we have failures in 90% cases... | 13:20 |
| rloo | Shrews: c'mon; I have time to review and approve stuff today :-) | 13:20 |
| rloo | seems like we haven't had anything merge for 1+ days now. I wonder if it was all due to the same bug. | 13:21 |
| dtantsur | rloo, who knows... there was a bug in infra that prevented getting meaningful logs | 13:22 |
| dtantsur | rloo, and also we have this: https://bugs.launchpad.net/ironic/+bug/1316773 | 13:22 |
| dtantsur | rloo, and now we have Nova change broken everything | 13:22 |
| Shrews | i blame dtantsur for creating such an effecient dashboard that changes are now approved too quickly | 13:22 |
| * rloo thinks taking whole week off might not be a bad idea | 13:22 | |
| Ng | hey folks | 13:22 |
| Ng | is there a corresponding change in Ironic to match up with https://review.openstack.org/#/c/94043/2 ? | 13:22 |
| dtantsur | Shrews, WOW SUDDENLY O_O | 13:22 |
| Shrews | :) | 13:22 |
| rloo | Shrews: yeah +1 | 13:23 |
| Ng | seeing this atm, suggesting that a corresponding change is needed: | 13:23 |
| Ng | 2014-06-04 12:57:53.303 4596 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/ironic/nova/scheduler/http://ironic_host_manager.py", line 94, in new_host_state | 13:23 |
| Ng | 2014-06-04 12:57:53.303 4596 TRACE oslo.messaging.rpc.dispatcher return host_manager.HostState(host, node, capabilities, service) | 13:23 |
| Ng | 2014-06-04 12:57:53.303 4596 TRACE oslo.messaging.rpc.dispatcher TypeError: __init__() takes at most 4 arguments (5 given) | 13:23 |
| dtantsur | Ng, yes and we're discussing it https://review.openstack.org/#/c/97447/ | 13:23 |
| Ng | aha :) | 13:23 |
| Ng | that'll teach me to not quickly check scrollback ;) | 13:25 |
| dtantsur | :) | 13:27 |
| *** shausy has quit IRC | 13:27 | |
| lucasagomes | morning NobodyCam Ng Shrews :) | 13:30 |
| Shrews | lucasagomes: morning/afternoon | 13:30 |
| *** matty_dubs|gone is now known as matty_dubs | 13:33 | |
| rloo | Shrews: hey, just trying to figure out what Sean did. He resubmitted the changes as another patch set, to wipe out the 'Approve' for the previous patch? | 13:33 |
| Shrews | rloo: looks like it | 13:34 |
| Ng | hey lucasagomes :) | 13:34 |
| openstackgerrit | Imre Farkas proposed a change to openstack/ironic: Make driver validation asynchronous https://review.openstack.org/97789 | 13:34 |
| dtantsur | rloo, yeah. Once the fix lands, we'll need to reapprove and clean up the commit message | 13:35 |
| Shrews | I hope he kept a list of which ones he pulled :) | 13:37 |
| rloo | Shrews: he doesn't need to keep a list. we have dtantsur's dashboard. should be easy to figure out. | 13:38 |
| dtantsur | :) | 13:38 |
| dtantsur | ... and this change is still in check pipleline >_< | 13:39 |
| rloo | makes me wonder if we should have a 'status' thing somewhere other than the 'topic' thing here. | 13:39 |
| rloo | dtantsur: wondering if that is due to the other bug 1324793. things must be churning away in there... | 13:39 |
| derekh_ | I got 3 ironic nodes each iwth 2 instances associated with them https://bugs.launchpad.net/ironic/+bug/1326364 | 13:41 |
| derekh_ | I now can't nove delete them | 13:41 |
| derekh_ | any ideas ? or is a manual DB edit | 13:41 |
| rloo | derekh_: i'm guessing manual db edit. | 13:43 |
| derekh_ | rloo: ok, thanks, thats what I thought | 13:48 |
| matty_dubs | Uh-oh. Someone has cited a blog post of mine from January on openstack-dev. | 13:49 |
| rloo | matty_dubs: why Uh-oh? Isn't that your 15-minutes, er secs or whatever, of fame? | 13:49 |
| matty_dubs | Well he was having issues. And I wrote it as a "notes to self" type thing. | 13:49 |
| dtantsur | Folks, seems like it now takes ~10 hours for gate pipeline to consider a change. We can relax and find something useful to do, except for approving patches >_< | 13:53 |
| *** jdob has quit IRC | 13:53 | |
| rloo | dtantsur: bug bash day :D | 13:54 |
| dtantsur | rloo, I'm having a bug bash week :D | 13:54 |
| *** jgrimm has joined #openstack-ironic | 13:56 | |
| rloo | dtantsur: and we all thank you very much for that! | 13:56 |
| *** jdob has joined #openstack-ironic | 13:56 | |
| dtantsur | :) | 13:56 |
| *** k4n0 has quit IRC | 13:58 | |
| *** romcheg has quit IRC | 14:01 | |
| * lucasagomes wrote ironic maiden on spotify instead of iron maiden. I'm working too much >.< | 14:03 | |
| matty_dubs | LOL | 14:04 |
| dtantsur | :D | 14:07 |
| dtantsur | not too bad name for some band though | 14:07 |
| devananda | g'morning, all | 14:10 |
| Mikhail_D_ltp | devananda: good morning :) | 14:10 |
| rloo | morning devananda. | 14:10 |
| dtantsur | devananda, morning :) don't try to approve any patches, please :) | 14:12 |
| dtantsur | until this lands: https://review.openstack.org/#/c/97447/ | 14:12 |
| matty_dubs | Morning devananda | 14:12 |
| devananda | dtantsur: is the problem specific to ironic, or is infra still having issues? | 14:12 |
| dtantsur | devananda, it's nova change that broke our unit-tests | 14:13 |
| devananda | dtantsur: ahhh, lovely. is someone working on it? | 14:13 |
| dtantsur | devananda, we have a patch https://review.openstack.org/#/c/97447/ and we're waiting for it to merge | 14:13 |
| lucasagomes | devananda, rloo matty_dubs morning | 14:13 |
| lucasagomes | Mikhail_D_ltp, too :) | 14:13 |
| rloo | hi lucasagomes | 14:14 |
| dtantsur | rloo, btw, are you working on this: https://bugs.launchpad.net/ironic/+bug/1288363 ? You're assigned | 14:16 |
| rloo | dtantsur: oh yeah. guess i'll do that today ;) | 14:16 |
| openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Wipe any metadata from a nodes disk https://review.openstack.org/93133 | 14:17 |
| rloo | dtantsur: thx for the reminder! | 14:17 |
| devananda | dtantsur: awesome | 14:17 |
| dtantsur | rloo, ok :) | 14:17 |
| NobodyCam | good morning devananda and lucasagomes | 14:17 |
| devananda | dtantsur, lucasagomes: also, sorry about that. I didn't realize the respective nova patch would break ironic (should have checked the jenkins results...) | 14:17 |
| Mikhail_D_ltp | lucasagomes: morning :) | 14:17 |
| dtantsur | NobodyCam, are you doing anything on https://bugs.launchpad.net/ironic/+bug/1300589 ? | 14:18 |
| *** rakesh_hs has quit IRC | 14:19 | |
| NobodyCam | dtantsur: It is on my list.. I have not had a chance last couple of days to work on it | 14:19 |
| dtantsur | NobodyCam, ok, will not touch it :) | 14:19 |
| rloo | dtantsur lies, waiting to pounce on folks as they enter IRC... | 14:19 |
| NobodyCam | dtantsur: if someelse has the time.. I'm happy to release it to them.. if not I will get back to it | 14:20 |
| * dtantsur is dangerous | 14:20 | |
| dtantsur | NobodyCam, I haven't heard about candidates, just cleaning up bugs list | 14:20 |
| * rloo glad that dtantsur is on our side! | 14:20 | |
| jroll | morning ironic | 14:21 |
| NobodyCam | dtantsur: Thank you for that!!!! :) | 14:21 |
| NobodyCam | good morning jroll | 14:21 |
| jroll | dtantsur: saw your comment, I'll have JayF take a look | 14:21 |
| lucasagomes | devananda, no worries :) | 14:21 |
| matty_dubs | Heya jroll | 14:21 |
| dtantsur | morning jroll, thanks :) I've created a bug for tracking that and assigned JayF | 14:21 |
| jroll | dtantsur: I expect that code to be run in the next week or so, so we'll definitely find out then | 14:21 |
| jroll | oh awesome | 14:21 |
| jroll | thank you! | 14:21 |
| dtantsur | it would be really cool to have some automatic end-to-end testing of IPA, even before merging with Ironic | 14:22 |
| jroll | of course | 14:22 |
| jroll | dwalleck is (supposed to be) working that | 14:23 |
| lucasagomes | comstud, ping re 1314732 | 14:25 |
| devananda | still reading scrollback.... busy morning ... | 14:26 |
| ryanpetrello | anybody know what the `check-tripleo-ironic-seed-precise` failure in https://review.openstack.org/#/c/97475/ is about? | 14:27 |
| ryanpetrello | Jenkins still seems to have given a +1 | 14:27 |
| devananda | ryanpetrello: 2014-06-03 17:01:39.508 | ERROR: Element 'common-venv' not found in '/opt/stack/new//tripleo-image-elements/elements:/opt/stack/new//diskimage-builder/bin/../elements:/opt/stack/new//diskimage-builder/bin/../elements' | 14:28 |
| ryanpetrello | right, seems related to https://bugs.launchpad.net/openstack-ci/+bug/1325689, which was fixed | 14:28 |
| devananda | ryanpetrello: i'm not sure what caused that. short version: that is only testing if triplo can /build/ the ironic images. it's not exercising them yet. | 14:29 |
| ryanpetrello | what’s odd is that seems to have failed, but didn’t re-run | 14:29 |
| ryanpetrello | do I need to recheck? | 14:29 |
| ryanpetrello | I guess what I’m confused about is jenkins shows a +1 despite a failing job | 14:29 |
| *** coolsvap|afk is now known as coolsvap | 14:30 | |
| devananda | ryanpetrello: yea. that jenkins doesn't get a real vote yet | 14:31 |
| ryanpetrello | gotcha | 14:31 |
| ryanpetrello | okay | 14:32 |
| devananda | or that test pipe. i'm not sure the right word (and haven't had coffeeyet) | 14:32 |
| ryanpetrello | heh | 14:32 |
| ryanpetrello | I know we discussed freezing python-keystoneclient the other night | 14:32 |
| ryanpetrello | but upon investigation, the actual change in keystone client seems to have happened over a year ago | 14:32 |
| ryanpetrello | so it seems to me like this does actually need a direct backport | 14:32 |
| NobodyCam | humm I dont see 97447 in the zuul queue | 14:36 |
| lucasagomes | NobodyCam, it's in the gate queu | 14:37 |
| dtantsur | NobodyCam, 6th from the end... | 14:37 |
| lucasagomes | queue* | 14:37 |
| agordeev | good morning Ironic | 14:37 |
| dtantsur | morning, agordeev | 14:38 |
| agordeev | dtantsur: morning :) | 14:38 |
| lucasagomes | morning agordeev | 14:40 |
| *** rwsu has joined #openstack-ironic | 14:40 | |
| NobodyCam | oh there it is.. tooka page reload :-p | 14:41 |
| * NobodyCam blames his bandwidth | 14:41 | |
| devananda | ryanpetrello: ack | 14:41 |
| agordeev | morning lucasagomes :) | 14:41 |
| NobodyCam | morning agordeev | 14:42 |
| agordeev | morning NobodyCam, devananda, jroll and everyone else! | 14:43 |
| jroll | \o agordeev | 14:44 |
| agordeev | jroll: could you take a quick look at comments from my changes in IPA? | 14:46 |
| jroll | sure | 14:46 |
| *** sysexit has joined #openstack-ironic | 14:48 | |
| lucasagomes | dtantsur, that patch will take a looooooooong time to get merged, still queued :( | 14:48 |
| jroll | agordeev: on this first one, I'm not sure I understand https://review.openstack.org/#/c/92399/8/ironic_python_agent/hardware_utils.py | 14:48 |
| jroll | agordeev: I just mean those couple of lines could be condensed, not the entire except block | 14:48 |
| *** matsuhashi has quit IRC | 14:49 | |
| NobodyCam | lucasagomes: the gate seems quite backed up atm | 14:49 |
| lucasagomes | yeah :( | 14:49 |
| dtantsur | lucasagomes, provided patches gated now are 11 hours old... yes, it will! And if it fails for whatever stupid reason... that will be bad, really bad :) | 14:49 |
| rloo | there ought to be a way to fasttrack patches. | 14:50 |
| agordeev | jroll: ah, you just meant to use one-lined `if smth: <>`, right? | 14:51 |
| jroll | agordeev: lines 71-74 can just be: | 14:52 |
| jroll | if not k: | 14:52 |
| jroll | info[section][multiline_values].append(k) | 14:52 |
| agordeev | jroll: nah, it can't. That change breaks the test. | 14:53 |
| jroll | added a comment | 14:54 |
| jroll | what, how | 14:54 |
| dtantsur | rloo, that's what I am thinking. The whole Ironic is broken from approving changes... | 14:54 |
| * jroll is confused | 14:54 | |
| NobodyCam | brb ... quick walkies | 14:54 |
| rloo | dtantsur: something to ask infra? | 14:54 |
| jroll | errrr | 14:55 |
| rloo | dtantsur: although it seems like sean knew about it... | 14:55 |
| jroll | agordeev: I guess I mean, if k: maybe | 14:55 |
| dtantsur | well, maybe. Like is it possible with PTL approval | 14:55 |
| jroll | idk, maybe it's fine | 14:55 |
| jroll | agordeev: added comments on that one | 14:55 |
| rloo | dtantsur: guess the first question is whether it is possible regardless of who approves. although one would think so. | 14:55 |
| agordeev | jroll: i agree, that it's very tricky to do parsing like that. Also, i'd found that 76-77 is not really needed. | 14:55 |
| jroll | agordeev: cool | 14:56 |
| jroll | agordeev: also, "it isn't a list due to current limitation in ironic IPA driver code. It has a problem with the lists nested into the dict." | 14:56 |
| jroll | I don't think this is true | 14:56 |
| jroll | if it is, then that is a bug | 14:56 |
| jroll | there's code to flatten nested lists/dicts | 14:56 |
| jroll | we use it already for network interfaces | 14:57 |
| jroll | it ends up something like hardware/interfaces/0/blablabla | 14:57 |
| devananda | dtantsur: think i'm caught up on scroll back but i dont understand the comment "is it possible with PTL approval" | 14:58 |
| jroll | agordeev: line 225 here https://review.openstack.org/#/c/84795/96/ironic/drivers/modules/agent_utils.py | 14:59 |
| dtantsur | devananda, we discuss the theoretical way of fast-forwarding some critical patches | 14:59 |
| dtantsur | devananda, so that we don't wait ~11 hours before it is evaluated by gate | 14:59 |
| devananda | ah. nope | 15:00 |
| devananda | the gate is a ruthless overlord | 15:00 |
| jroll | dtantsur: idk if that's a real problem, because nothing will get merged before that change anyway, right? | 15:00 |
| agordeev | jroll: yup, i have tested the flatter code and found. https://review.openstack.org/#/c/92847/9/ironic_python_agent/mainline.py | 15:00 |
| rloo | devananda: seems like every overlord can be ... bribed... | 15:01 |
| devananda | dtantsur: best thing we can do is take advantage of the check queue to test patches and re-order them deterministically | 15:01 |
| jroll | agordeev: well, that's a bug then :P | 15:01 |
| agordeev | jroll: L45 right after dtantsur's comment | 15:01 |
| devananda | so we know that, when they hit the gate, assuming no other project's cjhanges break our code, they'll all land in a line | 15:01 |
| jroll | agordeev: can you write a test case that calls flatten_dict directly? | 15:01 |
| jroll | agordeev: and I'll fix it for you :) | 15:02 |
| dtantsur | devananda, ok, let me try to reapprove a couple of the most important patches, so that tomorrow we maybe get something landed | 15:02 |
| jroll | agordeev: I have to run off and get to the office, talk to you soon :) | 15:03 |
| devananda | dtantsur: feel free to rebase them | 15:03 |
| agordeev | jroll: surely i can. where to send the code? | 15:03 |
| devananda | dtantsur: making it a straight line will, unfortunately, actually help | 15:03 |
| jroll | agordeev: gist.github.com or paste.openstack.org is fine | 15:03 |
| devananda | dtantsur: not that it helps jenkins -- but it helps us make sure jenkins won't hit a merge conflict | 15:03 |
| * dtantsur digs into git magic | 15:04 | |
| jroll | bbl | 15:04 |
| *** Mikhail_D_ltp has quit IRC | 15:04 | |
| *** martyntaylor has quit IRC | 15:06 | |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Add __init__.py for nova scheduler filters https://review.openstack.org/96912 | 15:07 |
| comstud | lucasagomes: pong | 15:09 |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Cleanup mock patch without `with` part 2 https://review.openstack.org/86536 | 15:10 |
| lucasagomes | comstud, hey, was just wondering if ur working on the RPC objects stuff (#1314732) | 15:10 |
| comstud | i am | 15:11 |
| lucasagomes | it's something we want to land in J1 so I thought about jumping on it if ur busy or something | 15:11 |
| lucasagomes | comstud, ack | 15:11 |
| comstud | slowly. I've had some distractions... but I'm getting back to it | 15:11 |
| lucasagomes | right, np :) | 15:11 |
| comstud | all of it is not going to make J1... | 15:11 |
| comstud | there's quite a bit of work left | 15:11 |
| comstud | i mean, J1 is like... 2 weeks away right? | 15:11 |
| lucasagomes | yeah I can figure, we can try to priorize that work on the reviews and all | 15:12 |
| lucasagomes | hmm not sure but I bet that's around 2 weeks yeah | 15:12 |
| comstud | I suppose it's possible it could make it | 15:12 |
| comstud | Node is 90% done | 15:12 |
| comstud | Port needs to be done | 15:12 |
| lucasagomes | yeah | 15:13 |
| lucasagomes | and chassis as well | 15:13 |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Work around iscsiadm delete failures https://review.openstack.org/96577 | 15:14 |
| *** coolsvap is now known as coolsvap|afk | 15:14 | |
| *** Kai14 has joined #openstack-ironic | 15:14 | |
| *** Kai14 is now known as KaiB | 15:15 | |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Work around iscsiadm delete failures https://review.openstack.org/96577 | 15:15 |
| *** KaiB has left #openstack-ironic | 15:16 | |
| *** Kai14 has joined #openstack-ironic | 15:16 | |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Remove 'fake' and 'ssh' drivers from default enabled list https://review.openstack.org/88711 | 15:16 |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Remove 'fake' and 'ssh' drivers from default enabled list https://review.openstack.org/88711 | 15:18 |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Remove 'node' parameter from the Console and Rescue interfaces https://review.openstack.org/91097 | 15:18 |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Remove 'node' parameter from the Console and Rescue interfaces https://review.openstack.org/91097 | 15:19 |
| comstud | lucasagomes: I'll see what I can do in the next few days | 15:20 |
| Shrews | devananda: on your comments here: https://review.openstack.org/#/c/94439/2/tempest/scenario/test_baremetal_advanced_ops.py | 15:21 |
| openstackgerrit | Dmitry Tantsur proposed a change to openstack/ironic: Test for membership should be 'not in' https://review.openstack.org/97412 | 15:21 |
| Shrews | devananda: The PXESSH part of the class name was copied over from the basic test, but I don't see the need for it in the name either. Will remove | 15:21 |
| lucasagomes | comstud, ack, cheers, thank you! | 15:22 |
| Shrews | devananda: on the /mnt part, if the ephemeral partition is never on /mnt, then that test will fail | 15:22 |
| comstud | lucasagomes: So I have this one that I need to rebase and fix up tests I think: https://review.openstack.org/#/c/91419/ | 15:22 |
| comstud | which is unrelated to objects | 15:22 |
| Shrews | devananda: so i'm not convinced another check is needed for that precondition | 15:22 |
| lucasagomes | comstud, ohh that's very important yeah | 15:22 |
| dtantsur | devananda and others: https://review.openstack.org/#/c/97412/ is a head of my terrible review queue. Please keep an eye on them, in case one is broken | 15:22 |
| comstud | lucasagomes: It should probably be rebased on dmitry's patches maybe? | 15:23 |
| comstud | i'm not sure | 15:23 |
| comstud | I'll rebase and see where we are | 15:23 |
| comstud | (rebase on master) | 15:23 |
| dtantsur | comstud, not required, patch queue is for patches we can't merge right now | 15:25 |
| dtantsur | the situation should be solved eventually and we'll return to a normal flow | 15:25 |
| comstud | nod | 15:25 |
| agordeev | jroll: http://paste.openstack.org/show/82789/ here it is. | 15:26 |
| devananda | Shrews: thanks | 15:26 |
| devananda | Shrews: true - the test itself will fail. but not because rebuild itself failed, because a precondition wasn't met | 15:27 |
| *** coolsvap|afk is now known as coolsvap | 15:28 | |
| Shrews | devananda: so, that "touch" test is validating the ephemeral preservation | 15:28 |
| * dtantsur is tired and calling it a day. Please keep an eye on patches in gate pipleline! | 15:28 | |
| NobodyCam | have a good night dtantsur | 15:29 |
| *** vinbs has joined #openstack-ironic | 15:29 | |
| devananda | dtantsur: thanks! g'night! | 15:29 |
| dtantsur | g'night, folks :) | 15:29 |
| *** dtantsur is now known as dtantsur|afk | 15:29 | |
| lucasagomes | dtantsur|afk, night! | 15:30 |
| NobodyCam | brb | 15:30 |
| devananda | Shrews: my point is, eg, if init starts putting /dev/sda4 at /extra instead of /mnt, then the test will fail in a non-obvious way | 15:30 |
| devananda | Shrews: not that the test wont fail | 15:30 |
| devananda | (it will) | 15:30 |
| Shrews | devananda: ok, so you're suggesting we need to delineate between the different causes of faillure there | 15:30 |
| Shrews | devananda: i can see that point | 15:31 |
| devananda | Shrews: i'm suggesting that it would be helpful to someone debugging that test's failure later on | 15:31 |
| agordeev | dtantsur|afk: night! :) | 15:31 |
| *** diga has joined #openstack-ironic | 15:34 | |
| agordeev | jroll: russell_h also i'd noticed the discussion about merge the code into generic hw manager. I like the idea :) | 15:36 |
| NobodyCam | devananda: did you find a taker for 96902? | 15:38 |
| devananda | NobodyCam: no | 15:39 |
| NobodyCam | I can pick it up | 15:40 |
| NobodyCam | just to check your ok with what was talked about yesterday | 15:40 |
| devananda | NobodyCam: let me toss up a skeleton of the chagne i'd like | 15:41 |
| NobodyCam | sure :) | 15:41 |
| devananda | NobodyCam: it may be half-baked, but i think it needs to support both | 15:41 |
| NobodyCam | devananda: ++ for supporting both | 15:42 |
| russell_h | agordeev: awesome, yeah I really like what you did there | 15:42 |
| NobodyCam | *CORES* (and others) thou they will prob require rebaseing on 97447, I would love to get any input on 93133 and 97590 | 15:45 |
| linggao | devananda, ping. | 15:47 |
| NobodyCam | morning linggao | 15:47 |
| linggao | morning NobodyCam. | 15:48 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/ironic-python-agent: Updated from global requirements https://review.openstack.org/88722 | 15:48 |
| linggao | I have a question to the core members. | 15:48 |
| linggao | I'd like start working on the console log feature. but the spec is not approved. | 15:48 |
| linggao | shall I wait or just go ahead prototyping while waiting? | 15:49 |
| devananda | linggao: we are focusing on bug fixes, code cleanup, and landing the nova driver right now | 15:50 |
| devananda | linggao: so other things are not going to get much review attention for a little while. | 15:50 |
| devananda | linggao: I understand that folks want to land new features, but we don't have the review bandwidth to do those yet. you are welcome to help by reviewing other patches or helping to fix bugs (there's plenty of them now!) | 15:51 |
| *** jistr has quit IRC | 15:51 | |
| linggao | devananda, understand. It is just that I'll be out for vacation in July, and come back in Aug. And just hope to finish most of the console log feature before the vacation so that it will be too late for Juno. | 15:52 |
| linggao | devananda, if console log feature can wait, I'll do what you just said, fixing defects and review patches. | 15:54 |
| comstud | ah man, this rebase is a mess | 15:57 |
| lucasagomes | devananda, so one thing I didn't get from that api spec is how the client (by pooling) will know when his request was finished? | 15:57 |
| russell_h | lucasagomes: the async spec? | 15:58 |
| lucasagomes | russell_h, yes | 15:58 |
| russell_h | its not _that_ different from today the way I see it | 15:58 |
| russell_h | like if you set target_provision_state to something, your request isn't "done" when it returns, its done when you see provision_state get where it's supposed to be | 15:59 |
| *** matty_dubs is now known as matty_dubs|lunch | 15:59 | |
| russell_h | my concern is how does a client know if their request was dropped | 16:00 |
| lucasagomes | russell_h, right | 16:00 |
| lucasagomes | russell_h, but | 16:00 |
| lucasagomes | if there's 2 requests in the queue | 16:00 |
| lucasagomes | right after it finishes the first one | 16:00 |
| russell_h | if you have to poll for completion, and your request gets silently dropped... do you just time out after a while? | 16:00 |
| russell_h | right | 16:00 |
| lucasagomes | it's start the other and the second was finishs pretty quickly | 16:00 |
| devananda | so | 16:00 |
| lucasagomes | before the client who submitted the first one pooled the API to verify the current state | 16:01 |
| devananda | let's think about the type of checking that this proposes in the API | 16:01 |
| devananda | maybe it needs to be clearer | 16:01 |
| lucasagomes | now it's in a complete different state than he requested | 16:01 |
| devananda | but eg, if you request a change in power state | 16:01 |
| devananda | the API should block any further changes in power state *until thats done* | 16:01 |
| lucasagomes | ah | 16:01 |
| devananda | and in fact it should prevent changing provision or console or ... states too | 16:01 |
| devananda | but, if i make 5 requests to change instance_info | 16:01 |
| lucasagomes | right, so that's kinda like we do today, so there's no queue for such operations | 16:01 |
| devananda | those can queue up | 16:01 |
| devananda | there's still a queue internally | 16:02 |
| devananda | but the API should prevent new requests that potentially conflict with an ongoing operation | 16:02 |
| lucasagomes | right | 16:02 |
| lucasagomes | tihngs like changing instance_info are only db changes, should it actually go to the conductor? | 16:03 |
| *** vinbs has quit IRC | 16:04 | |
| lucasagomes | well probably yes, in case of a vif change or something | 16:04 |
| lucasagomes | hmm | 16:04 |
| russell_h | so... | 16:05 |
| russell_h | I agree with all of that except the bit about instance_info | 16:05 |
| *** romcheg has joined #openstack-ironic | 16:05 | |
| russell_h | I'd even say those shouldn't queue up | 16:05 |
| russell_h | well | 16:05 |
| russell_h | hmm | 16:05 |
| lucasagomes | the queue is where I'm struggling more | 16:05 |
| devananda | should we allow the API to driectly write to the database? or push changes over RPC to the conductor, where it writes to the DB? | 16:06 |
| russell_h | I think the API should write directly to the db | 16:06 |
| *** diga has quit IRC | 16:06 | |
| NobodyCam | I would vote for the latter :-p | 16:06 |
| russell_h | otherwise you're back to blocking on the conductor | 16:06 |
| lucasagomes | devananda, if we want the API to control locks and all before passing it to the conductor we might want ti to write to the db as welll | 16:06 |
| devananda | russell_h: while faster, that's going to lead to a complication. how does conductor know to apply some change if the change just appears in the DB? | 16:07 |
| russell_h | devananda: the way I've been thinking about this, the API puts it in the db and casts a "converge" message to the conductor | 16:07 |
| NobodyCam | we also dont have many checks inplace for data changing out from under the conductor | 16:07 |
| russell_h | the conductor reads out the db, and makes things so | 16:07 |
| russell_h | the conductor can also periodically converge every node, sort of like sync power states today | 16:08 |
| devananda | i'm suggesting the API write to the db only to set a requested state, and if there is a non-null requested state, the API responds to subsequent "change state" requests with a 409 CONFLICT | 16:09 |
| russell_h | +1 | 16:09 |
| lucasagomes | I think I'm having the same view as russell_h... even for the queue, maybe the api should queue it to the conductor and the conductor just reads from the db what it should do next... even because having a queue means having a limit of items to be queue, if not controlled by the API how we are going to tell the client "look the queue is already full" | 16:09 |
| devananda | I think "api writes to db, conductor converges" will run into problems with any of the data that is used to control a node | 16:10 |
| devananda | eg, driver_info, instance_info, etc | 16:10 |
| lucasagomes | yeah +1 for that state change, 409 seems appropriated | 16:11 |
| russell_h | right, so I don't think the conductor should probably need to do anything in response to a change in driver_info | 16:11 |
| devananda | lucasagomes: storing a queue in a db is an antipattern. we have a queue already | 16:11 |
| russell_h | except use the new info I guess | 16:11 |
| russell_h | might be thinking about that wrong though, I'm unfortunately deep in IPMI-land | 16:12 |
| devananda | russell_h: well, it may want to validate that info and update some other state field | 16:12 |
| russell_h | for instance_info... it should probably just be invalid to change instance_info at all unless the state is nostate and no changes are pending | 16:12 |
| russell_h | devananda: mm, yes | 16:12 |
| Shrews | ugh, yes, let's NOT use the db as a queue | 16:13 |
| russell_h | devananda: if you want the driver to validate driver_info though, aren't you going to want to block on that? | 16:13 |
| devananda | so all this tells me i need to be MUCH more specific in my spe | 16:13 |
| devananda | spec | 16:13 |
| lucasagomes | heh right I take back the db queue | 16:13 |
| devananda | russell_h: absolutely not | 16:13 |
| devananda | russell_h: validating driver_info could take an hour | 16:13 |
| devananda | that's precisely the sort of thing that should be queued | 16:13 |
| russell_h | heh, you people and your fancy computers | 16:13 |
| devananda | :) | 16:14 |
| russell_h | so if you make a change to driver info, how do you know if your change failed? | 16:14 |
| *** jcoufal has quit IRC | 16:17 | |
| lucasagomes | or when it's completed? | 16:17 |
| lucasagomes | the same e.g with 2 operations, imagine the first one include the attr 'test' in the db, and the second one removes it | 16:17 |
| devananda | lucasagomes: why would you issue the second before validating the first has completed? | 16:18 |
| *** ewindisch has quit IRC | 16:18 | |
| lucasagomes | devananda, well, it seems logical to not do it, but just thinking about a race condition here | 16:19 |
| devananda | seriously - anything can be broken. and it should be able to recover. what i'm trying to find is a way to do this that doesn't break under NORMAL use | 16:19 |
| lucasagomes | just thinking about how to have each opearation detail separated from the rest | 16:19 |
| devananda | and that can heal when someone does something (intentionally) dumb | 16:19 |
| *** ellenh has joined #openstack-ironic | 16:19 | |
| devananda | if I send the same PATCH request 100 times, it's reasonable for it to only succeed ONCE | 16:19 |
| devananda | and the rest t obe ignored by the server | 16:19 |
| devananda | that sort of thing is often done because of misbehaving proxies anyway | 16:20 |
| lucasagomes | right, but the how you know if the requested change failed? | 16:20 |
| devananda | ditto if a client sends PATCH requests doing this: A, NOT A, A, NOT A, A ... the final state should be only that -- the final state. at any point in the middle, some may or may not have been applied yet | 16:21 |
| devananda | lucasagomes: the solution there is to only accept whole documents | 16:21 |
| devananda | if there's taht much concern about minor requests failing AFTER the API has validated them and placed them on the RPC bus, then we should stop allowing PATCH requests at all | 16:22 |
| devananda | (i'm being a bit extreme to make a point) | 16:22 |
| openstackgerrit | Chris Behrens proposed a change to openstack/ironic: TaskManager: Only support single node locking https://review.openstack.org/91419 | 16:23 |
| comstud | ok, i think that ^ is fixed up | 16:23 |
| comstud | ah crap, i have some comments from rloo in ps3 to address | 16:24 |
| devananda | or implement a task queue for PATCH requests, returning a unique resource link in each 202 response | 16:24 |
| rloo | comstud: yeah, and i'm keeping an eye on you ;) | 16:24 |
| comstud | oh | 16:25 |
| comstud | it looks like i hit some of them | 16:25 |
| * comstud double checks them all | 16:25 | |
| lucasagomes | devananda, right yeah... when i think about queuing up requests I think about that sort of queue where clients could check the status of each request | 16:26 |
| lucasagomes | but yeah, maybe it's just overcomplex and I didn't grasp the whole spec | 16:26 |
| * lucasagomes will read again after this talk | 16:26 | |
| devananda | lucasagomes: https://restful-api-design.readthedocs.org/en/latest/methods.html#asynchronous-requests | 16:26 |
| *** hemnafk is now known as hemna | 16:26 | |
| NobodyCam | brb | 16:28 |
| lucasagomes | devananda, sounds good, so the status monitor link is a unique URI | 16:31 |
| lucasagomes | that the gb can remove it after it's accessed at least once after the completion/failure of that request | 16:32 |
| *** Kai14 has quit IRC | 16:34 | |
| *** ndipanov has quit IRC | 16:35 | |
| devananda | right | 16:36 |
| *** ewindisch has joined #openstack-ironic | 16:36 | |
| *** martyntaylor has joined #openstack-ironic | 16:36 | |
| openstackgerrit | Chris Behrens proposed a change to openstack/ironic: TaskManager: Only support single node locking https://review.openstack.org/91419 | 16:37 |
| *** matty_dubs|lunch is now known as matty_dubs | 16:37 | |
| lucasagomes | devananda, cool, +1 for that | 16:38 |
| openstackgerrit | Chris Behrens proposed a change to openstack/ironic: TaskManager: Only support single node locking https://review.openstack.org/91419 | 16:39 |
| comstud | alrighty | 16:39 |
| * comstud done with that one | 16:39 | |
| devananda | comstud: heh | 16:39 |
| devananda | russell_h: how have ya'll dealt with the flood-of-requests-chokes-RPC issue? | 16:40 |
| devananda | russell_h: i mean, has that continued to be a problem in your lab, or...? | 16:41 |
| JayF | dtantsur|afk: sorry that got merged without you getting another look at it. I'll push up a fix to the bug you filed. | 16:46 |
| *** Alexei_987 has quit IRC | 16:46 | |
| *** digambar has joined #openstack-ironic | 16:48 | |
| *** ewindisch has quit IRC | 16:48 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Rework make_partitions logic when preserve_ephemeral is set https://review.openstack.org/97590 | 16:51 |
| openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Wipe any metadata from a nodes disk https://review.openstack.org/93133 | 16:52 |
| openstackgerrit | Ruby Loo proposed a change to openstack/ironic: Driver interface's validate should return nothing https://review.openstack.org/97855 | 16:52 |
| NobodyCam | ^^^ just rebasing | 16:52 |
| JayF | devananda: I think we might have had some environmental issues contributing to that (i.e. some BMCs registered with Ironic that were not responding) | 16:55 |
| JayF | devananda: I don't think we've seen it since updating our environment, so perhaps testing with failed bmcs would be an interesting way to find bugs along the lines of that | 16:55 |
| JayF | jroll: ^ right? | 16:55 |
| *** derekh_ has quit IRC | 16:56 | |
| jroll | devananda, JayF: right, we haven't seen any issues (running master as of monday(?)) | 16:56 |
| jroll | also running with 4 workers iirc | 16:56 |
| *** vinbs has joined #openstack-ironic | 16:56 | |
| * jroll checks configs quickly | 16:57 | |
| *** dwalleck has joined #openstack-ironic | 16:57 | |
| *** dwalleck has quit IRC | 16:57 | |
| jroll | oh no, we're running with defaults | 16:57 |
| jroll | oh | 16:59 |
| jroll | [conductor] | 16:59 |
| jroll | heartbeat_timeout=6000 | 16:59 |
| jroll | cool | 16:59 |
| jroll | devananda: apparently that's how we've dealt with it :/ | 16:59 |
| devananda | jroll: i see. any other changes? | 17:01 |
| devananda | dtantsur|afk: i'm reopening https://bugs.launchpad.net/ironic/+bug/1321494 as we just hit it in your rebase-tree | 17:02 |
| *** zdiN0bot has joined #openstack-ironic | 17:02 | |
| lucasagomes | jroll, even after https://review.openstack.org/#/c/93083/ | 17:02 |
| lucasagomes | the heartbeat still a problem? | 17:02 |
| jroll | devananda: I don't think so, as far as that goes | 17:02 |
| *** zdiN0bot has left #openstack-ironic | 17:02 | |
| jroll | worker limits are defaults etc | 17:02 |
| jroll | lucasagomes: we're running that but apparently haven't checked :) | 17:03 |
| jroll | I'll let you know later today | 17:03 |
| lucasagomes | jroll, right, if you can set the default timeout back to see if that fix the problem it would be great | 17:03 |
| lucasagomes | jroll, alright thanks! | 17:03 |
| JayF | That is a good looking change, thanks for that | 17:03 |
| digambar | Hello | 17:04 |
| digambar | I am assigning one bug to myself | 17:04 |
| digambar | https://bugs.launchpad.net/ironic/+bug/1282836 | 17:04 |
| NobodyCam | hi digambar | 17:04 |
| digambar | Hi | 17:04 |
| digambar | will this be fine ? | 17:04 |
| JayF | I suspect the answer is yes, as long as you then fix it :) | 17:05 |
| devananda | ^ :) | 17:05 |
| lucasagomes | heh | 17:05 |
| digambar | okay | 17:05 |
| NobodyCam | digambar: great Thank you | 17:05 |
| digambar | WC | 17:06 |
| *** zdiN0bot1 has joined #openstack-ironic | 17:06 | |
| digambar | Nobody | 17:06 |
| devananda | Shrews: hi! want to poke at tempest some more? | 17:06 |
| digambar | can you suggest any blueprint also to me ? | 17:06 |
| devananda | Shrews: I suspect https://bugs.launchpad.net/ironic/+bug/1321494 is not really fixed -- take a look at http://logs.openstack.org/12/96912/2/check/check-tempest-dsvm-ironic/9253a95/console.html#_2014-06-04_16_08_26_359 | 17:06 |
| *** Mikhail_D_ltp has joined #openstack-ironic | 17:07 | |
| NobodyCam | lucasagomes: still here | 17:08 |
| lucasagomes | NobodyCam, hey yes, just for a couple of minutes (gotta go to the shop buy some food soon) | 17:08 |
| NobodyCam | :) wanted to just quickly chat about 96558 | 17:08 |
| *** dwalleck has joined #openstack-ironic | 17:09 | |
| NobodyCam | it is covering the case of multiable ipmi commands ie (off then on) | 17:09 |
| NobodyCam | -N -R are for a single command | 17:10 |
| lucasagomes | NobodyCam, sure, I didn't understand that change | 17:10 |
| lucasagomes | NobodyCam, right, so then, do we need -N -R at all? | 17:10 |
| NobodyCam | yes for how to handle a single command | 17:10 |
| Shrews | devananda: sorry, back from lunch now | 17:10 |
| devananda | Shrews: fwiw, logstash says that was only hit once in the last 48 hrs | 17:10 |
| NobodyCam | 96558 is for a sequence of commands | 17:11 |
| NobodyCam | while the -N -R patch is for a single command | 17:11 |
| NobodyCam | (retry if you will) | 17:11 |
| Shrews | devananda: i'll take a look | 17:11 |
| devananda | Shrews: thanks much | 17:12 |
| NobodyCam | lucasagomes: we are also going to work support for older versions of ipmi that do not support -N -R | 17:13 |
| NobodyCam | in to the code | 17:13 |
| *** shakamunyi has quit IRC | 17:13 | |
| lucasagomes | NobodyCam, right, think I understand it better | 17:17 |
| NobodyCam | :) | 17:17 |
| lucasagomes | I will take a look at the patch in a bit | 17:17 |
| lucasagomes | NobodyCam, just out of curiosity, how we are going to work around the -N -R problem for old versions? | 17:17 |
| NobodyCam | sure... the dependent parch will be reworked | 17:17 |
| lucasagomes | checking the version string of ipmitool? | 17:17 |
| lucasagomes | or do we actually need to support old versions | 17:18 |
| lucasagomes | I checked the ipmitool in RHEL 6.4 and it does support -N -R | 17:18 |
| lucasagomes | oddly it's the same version as in the ubuntu 12.04, 1.8.11 (I think) idk why ubuntu 12.04 doesn't support it | 17:18 |
| NobodyCam | lucasagomes: on driver load check ipmi command support for -n -r if not there loop in code if there set -N -R | 17:18 |
| lucasagomes | I see | 17:19 |
| lucasagomes | ok | 17:19 |
| lucasagomes | I will keep an eye on the patch, NobodyCam thanks for the clarification | 17:19 |
| NobodyCam | then throw WARN: Deprecated typ msg if ipmi dose not support it | 17:20 |
| NobodyCam | :-p | 17:20 |
| lucasagomes | ah heh | 17:20 |
| lucasagomes | fair | 17:20 |
| NobodyCam | guilt distros to updating it | 17:20 |
| lucasagomes | "UPDATE UR OS" | 17:20 |
| lucasagomes | :P | 17:20 |
| lucasagomes | yeah | 17:20 |
| NobodyCam | hehehe | 17:21 |
| NobodyCam | WARN: Deprecated: your ipmitool is OLD nag someone | 17:21 |
| lucasagomes | hah | 17:22 |
| lucasagomes | alright, I gotta run | 17:23 |
| lucasagomes | have a good night everybody! | 17:23 |
| NobodyCam | have a great inner lucasagomes | 17:23 |
| JayF | night lucas | 17:23 |
| NobodyCam | dinner even | 17:23 |
| lucasagomes | thank u | 17:23 |
| *** lucasagomes is now known as lucas-dinner | 17:24 | |
| *** vinbs has quit IRC | 17:24 | |
| openstackgerrit | Jay Faulkner proposed a change to openstack/ironic-python-agent: Perform manual error checks instead of set -e https://review.openstack.org/97860 | 17:26 |
| openstackgerrit | Jay Faulkner proposed a change to openstack/ironic-python-agent: Perform manual error checks instead of set -e https://review.openstack.org/97860 | 17:27 |
| openstackgerrit | Jay Faulkner proposed a change to openstack/ironic-python-agent: Perform manual error checks instead of set -e https://review.openstack.org/97860 | 17:28 |
| * JayF fails at commit messages and trailing whitespace | 17:29 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Enforce a minimum time between all IPMI commands https://review.openstack.org/96558 | 17:31 |
| jroll | dtantsur|afk: you should look at https://review.openstack.org/#/c/97860/ | 17:31 |
| *** jcoufal has joined #openstack-ironic | 17:33 | |
| adam_g | oh no http://no-carrier.net/~adam/openstack/ironic_gate_status.html | 17:33 |
| *** zdiN0bot1 has quit IRC | 17:37 | |
| *** digambar has quit IRC | 17:37 | |
| *** harlowja_away is now known as harlowja | 17:39 | |
| rloo | adam_g, what's wrong, you don't like the colour red? Aren't those mostly going to be fixed by https://review.openstack.org/#/c/97447/ | 17:39 |
| devananda | adam_g: we're aware of the break. fix is in flight, but gate queue is ~13hr behind | 17:40 |
| adam_g | ah cool, yea i just found that patch | 17:40 |
| NobodyCam | only ten hours to go for 97447 | 17:40 |
| rloo | do you guys think it might be useful to put latest-status/useful-info somewhere? problem is where, and keeping it updated/cleared. | 17:42 |
| rloo | or let people ask on irc ;) | 17:42 |
| NobodyCam | rloo: it always end being that last bit that so hard | 17:43 |
| NobodyCam | I like the let them ask approch | 17:43 |
| rloo | ok, go with the current approach then ;) | 17:43 |
| *** eghobo has joined #openstack-ironic | 17:44 | |
| matty_dubs | I think that in sdague's talk at Summit he said they were very receptive to people pinging them and asking them to promote jobs that were blocking development | 17:44 |
| devananda | matty_dubs: aiui, that is w.r.t. devstack and tempest patches needing review/approval | 17:46 |
| devananda | matty_dubs: these are patches already in the gate queue for merge :) | 17:46 |
| devananda | unless you're talkinga bout sometjhing else | 17:46 |
| devananda | rloo: status of what? | 17:46 |
| matty_dubs | Oh, merge stuff probably can't be preempted. | 17:47 |
| rloo | devananda: status of anything people deem important. like our patches failing. | 17:47 |
| rloo | why our patches are failing. don't approve until xxx lands, ... | 17:47 |
| rloo | recheck bug xxx ... | 17:48 |
| rloo | coast is clear now. | 17:48 |
| devananda | rloo: do ppl look at the IRC channel status msesage? | 17:48 |
| rloo | devananda: i do sometimes. don't know about others. | 17:49 |
| matty_dubs | I do, in channels where it's used to communicate status. The one here already scrolls off my screen, though | 17:49 |
| NobodyCam | I have it set as a title for every channel | 17:50 |
| rloo | maybe we should use that. is there a max length to the msg? | 17:50 |
| matty_dubs | IMHO there should be a ~80-character limit, since a lot of IRC is still terminal-based. But if there's any server-enforced limit, it must be a lot longer | 17:51 |
| *** ChanServ sets mode: +o devananda | 17:52 | |
| NobodyCam | what no pep8 check | 17:52 |
| *** devananda changes topic to "ATTN: Ironic's gate queue is broken, and a fix is in the queue: https://review.openstack.org/#/c/97447" | 17:52 | |
| jroll | matty_dubs: you can always type /topic to get the full topic | 17:53 |
| matty_dubs | Ah, good point | 17:54 |
| matty_dubs | Though I'm willing to bet that I'm far from the only one that never actually does that | 17:54 |
| *** jistr has joined #openstack-ironic | 17:55 | |
| *** dwalleck has quit IRC | 17:55 | |
| jroll | of course | 17:56 |
| * NobodyCam loves irssi | 17:56 | |
| NobodyCam | http://paste.openstack.org/show/wfBSFxW5pPKAJdScLD0N/ | 17:56 |
| jroll | announcements can always be first in the topic | 17:56 |
| rloo | i think if people get in the habit of doing something cuz they find it useful ;) | 17:57 |
| *** dwalleck has joined #openstack-ironic | 17:58 | |
| matty_dubs | NobodyCam: Hmm, consistent HTTP 500 from paste.openstack.org for me on that | 18:00 |
| NobodyCam | matty_dubs: try another browser | 18:01 |
| jroll | weird | 18:01 |
| NobodyCam | its cookies | 18:01 |
| jroll | matty_dubs: https://gist.github.com/jimrollenhagen/02141590b21b26d81e5a | 18:01 |
| matty_dubs | Cookies causing a 500? What is this world coming to? | 18:02 |
| matty_dubs | Thanks jroll | 18:02 |
| *** dwalleck_ has joined #openstack-ironic | 18:02 | |
| *** dwalleck has quit IRC | 18:02 | |
| matty_dubs | NobodyCam: What does this do? Make the /topic bar auto-size? | 18:03 |
| *** Mikhail_D_ltp has quit IRC | 18:03 | |
| NobodyCam | and always at the top of the window | 18:03 |
| *** zdiN0bot has joined #openstack-ironic | 18:03 | |
| matty_dubs | Mine's always at the top, but only one line | 18:03 |
| matty_dubs | Though I sort of like that | 18:03 |
| *** dwalleck_ has quit IRC | 18:04 | |
| NobodyCam | oh I misstook ... I thought you did not have topic bar | 18:05 |
| *** dwalleck has joined #openstack-ironic | 18:05 | |
| NobodyCam | in that case NM | 18:05 |
| adam_g | http://paste.openstack.org/show/82817/ | 18:07 |
| adam_g | hmm | 18:07 |
| adam_g | NobodyCam, ^ ever seen anything like that? | 18:07 |
| *** pelix has quit IRC | 18:08 | |
| NobodyCam | adam_g: yep | 18:08 |
| *** athomas has quit IRC | 18:08 | |
| NobodyCam | adam_g: https://review.openstack.org/#/c/93133/ | 18:09 |
| NobodyCam | also dep: https://review.openstack.org/#/c/97590 | 18:10 |
| *** zdiN0bot has left #openstack-ironic | 18:12 | |
| adam_g | NobodyCam, okay cool. those apparently weren't being pulled in as they should | 18:13 |
| NobodyCam | lol have been trying to land them :-p | 18:13 |
| *** jbjohnso has quit IRC | 18:13 | |
| Shrews | NobodyCam: geez, hurry up already LOL | 18:16 |
| NobodyCam | lol | 18:16 |
| Shrews | NobodyCam: oh! apparently, i owe you a party for passing 50 patch sets | 18:16 |
| Shrews | NobodyCam: your party is tonight at my place if you can make it :-P | 18:17 |
| NobodyCam | lol.. mostly just rebasing | 18:17 |
| NobodyCam | lol | 18:17 |
| NobodyCam | thats the wrong direction ... | 18:17 |
| NobodyCam | :) you'll have to drink my share | 18:17 |
| *** zdiN0bot1 has joined #openstack-ironic | 18:18 | |
| Shrews | well... if i *have* to | 18:18 |
| * Shrews invites pcrews, too, and reluctantly offers to drink *his* share as well | 18:19 | |
| jroll | 11:16:37 Shrews | NobodyCam: oh! apparently, i owe you a party for passing 50 patch sets <- we're approaching 100, do we get a double party? :) | 18:20 |
| openstackgerrit | Russell Haering proposed a change to openstack/ironic-python-agent: Add missing methods to base HardwareManager class https://review.openstack.org/97631 | 18:20 |
| openstackgerrit | Russell Haering proposed a change to openstack/ironic-python-agent: Add a HardwareManager method to erase devices https://review.openstack.org/97871 | 18:20 |
| NobodyCam | lol.. now I see what the game is... and the game is a-foot | 18:20 |
| Shrews | lol | 18:20 |
| * NobodyCam would invite folks to his place... but he's not quite sure where that will be tonight | 18:21 | |
| *** romcheg has quit IRC | 18:27 | |
| *** jdob has quit IRC | 18:28 | |
| *** jdob has joined #openstack-ironic | 18:28 | |
| * pcrews salutes Shrews for his selflessness and wishes his liver well ;) | 18:28 | |
| *** max_lobur has quit IRC | 18:30 | |
| devananda | pcrews: speaking of livers, we should get a drink some time | 18:33 |
| pcrews | devananda: ++! | 18:33 |
| Shrews | can't we just have the nova/ironic meeting in seattle??? | 18:34 |
| *** dwalleck has quit IRC | 18:35 | |
| NobodyCam | ok folks dropping off for a bit.. back later (and Shrews +++ for seattle) | 18:35 |
| *** dwalleck has joined #openstack-ironic | 18:37 | |
| *** zdiN0bot1 has quit IRC | 18:39 | |
| *** r0j4z0 has quit IRC | 18:41 | |
| *** CAP1089 has joined #openstack-ironic | 18:41 | |
| *** CAP1089 is now known as r0j4z0 | 18:41 | |
| *** jbjohnso has joined #openstack-ironic | 18:42 | |
| *** jcoufal has quit IRC | 18:42 | |
| * devananda stabs ipmitool | 18:47 | |
| devananda | -R 1 -N -1 => 6 packets | 18:48 |
| devananda | -R 1 -N -1 => 6 packets in 5 seconds | 18:48 |
| devananda | -R 1 -N -2 => 6 packets in 10 seconds | 18:49 |
| *** dwalleck has quit IRC | 18:49 | |
| devananda | -R 1 -N -4 => 6 packets in 20 seconds | 18:49 |
| devananda | \o/ | 18:49 |
| devananda | -R 2 -N -1 => 12 packets in 16 seconds | 18:50 |
| devananda | :-( | 18:50 |
| *** romcheg has joined #openstack-ironic | 18:50 | |
| *** romcheg has quit IRC | 18:50 | |
| *** romcheg has joined #openstack-ironic | 18:51 | |
| devananda | -R 2 -N -2 => 12 packets in 24 seconds | 18:52 |
| *** ellenh has quit IRC | 18:53 | |
| *** jistr has quit IRC | 18:55 | |
| devananda | default # packets sent by version: 1.8.11 == 16 packets. 1.8.13 == 24 | 18:58 |
| devananda | tested with: ipmitool -I lanplus -H 127.0.0.1 -U admin -P fake chassis power status | 18:58 |
| JayF | new plan: make pyghmi better, add a cli tool to it called ipmitool, then have a third divergent ipmitool project, ???, profit! | 19:08 |
| devananda | ++ | 19:09 |
| matty_dubs | "ipymitool" | 19:10 |
| linggao | adam_g, ping | 19:10 |
| adam_g | linggao, hey | 19:11 |
| linggao | Hi, I am looking at this bug, https://bugs.launchpad.net/ironic/+bug/1301279 | 19:11 |
| linggao | did your patch https://review.openstack.org/#/c/88476 fix it or not? | 19:12 |
| adam_g | linggao, no that patch addressed a different bug | 19:15 |
| JayF | matty_dubs++ | 19:15 |
| adam_g | linggao, IIRC 1301279 requires some bigger changes in nova | 19:16 |
| linggao | So basically, for fixing this one. I am thinking, if the node has instance associated with it, it should report the numbers from associated flavor instead of getting it from nodes tables | 19:17 |
| *** ellenh has joined #openstack-ironic | 19:17 | |
| linggao | is this a right way? Or do you have something in mind? | 19:19 |
| *** mgagne has left #openstack-ironic | 19:22 | |
| *** romcheg has quit IRC | 19:23 | |
| adam_g | linggao, yeah that would probably work, but then doesn't the problem move to nova. what happens when if the flavor specs change? | 19:26 |
| adam_g | i don't know the expected behavior there TBH | 19:27 |
| linggao | adam_g, yeah, you are right. | 19:28 |
| *** romcheg has joined #openstack-ironic | 19:29 | |
| *** romcheg has quit IRC | 19:29 | |
| linggao | adam_g, looks like Ironic need to keep the 'original' resources somewhere :( | 19:30 |
| *** max_lobur has joined #openstack-ironic | 19:30 | |
| linggao | adam_g, just like vm hypovisors do for VMs. | 19:31 |
| adam_g | linggao, which drivers do that? | 19:31 |
| linggao | adam_g, I am thining kvm must save this information somewhere. | 19:32 |
| adam_g | linggao, i *think* thats all dervied from libvirt. the original specs of the instance are stored in the outdated domain configuration | 19:35 |
| *** rloo has quit IRC | 19:37 | |
| linggao | adam_g, where is the "outdated domain configuration" stored? | 19:37 |
| *** rloo has joined #openstack-ironic | 19:37 | |
| adam_g | linggao, in the libvirt configuration for the running instance. libvirt is able to report on exactly how much resources are used by instance $foo and whats available | 19:39 |
| *** rloo has quit IRC | 19:40 | |
| *** rloo has joined #openstack-ironic | 19:40 | |
| adam_g | whereas (last i looked) ironic was just reporting the properties of the node | 19:40 |
| linggao | adam_g, I see. | 19:40 |
| adam_g | so yeah, maybe ironic needs to track the specs of the running instance. IOW a snapshot node resources when the instance was spawned | 19:41 |
| linggao | adam_g, yes. | 19:42 |
| linggao | adam_g, do you know if nova keep track of the instances spawned? | 19:42 |
| *** foexle has quit IRC | 19:43 | |
| adam_g | linggao, yes. it also tracks reservations and uses those in calculating resources, to take into account instances that are scheduled but not running on the hypervisor yet | 19:44 |
| linggao | adam_g, hmm, wonder if Ironic can call nova function to get that info for the node. | 19:47 |
| linggao | this way, there is no need to change in nova. | 19:47 |
| jroll | I'm going to throw it out there that my proposed fix for that bug would be "don't change the node's properties while it is active". | 19:48 |
| adam_g | jroll, yeah | 19:51 |
| linggao | jroll, that should be a valide requirement. | 19:52 |
| jroll | ok, I'm not the only one. good. | 19:52 |
| jroll | so let's enforce that in the api | 19:52 |
| jroll | and call it a day? | 19:52 |
| linggao | jroll, thought we have agreed something like this at the Juno summit. | 19:54 |
| jroll | so why are we talking about keeping track of specs etc? | 19:56 |
| jroll | changing flavors? | 19:56 |
| jroll | so if flavor specs change, it will show free resources? | 19:57 |
| jroll | (I guess I misread this conversation to begin with) | 19:57 |
| linggao | jroll, adam_g and I was discussing this bug https://bugs.launchpad.net/ironic/+bug/1301279 | 19:57 |
| jroll | ok, right | 19:58 |
| jroll | so what's the issue? | 19:59 |
| linggao | the issue was if someone changed resources (like cpu# or memory size) for a deployed node. the nova will show available resources for that node. | 20:00 |
| openstackgerrit | A change was merged to openstack/ironic-specs: Add instance_info field to Node model https://review.openstack.org/95701 | 20:00 |
| jroll | that's what I'm trying to say. don't allow someone to change those resources while deployed. | 20:00 |
| linggao | jroll, yep. you are right. | 20:01 |
| jroll | so what's this discussion about ironic taking snapshots and such? | 20:01 |
| linggao | jroll, I'll put a note on the bug to see if others agree with it. | 20:01 |
| jroll | heh, ok | 20:02 |
| *** ellenh has quit IRC | 20:04 | |
| *** sysexit has quit IRC | 20:12 | |
| *** zdiN0bot has joined #openstack-ironic | 20:15 | |
| *** zdiN0bot has quit IRC | 20:22 | |
| *** zdiN0bot has joined #openstack-ironic | 20:26 | |
| *** pbrooko has joined #openstack-ironic | 20:33 | |
| devananda | NobodyCam: back? | 20:48 |
| *** zdiN0bot has quit IRC | 20:49 | |
| *** zdiN0bot has joined #openstack-ironic | 20:49 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Let ipmitool natively retry commands https://review.openstack.org/96902 | 20:49 |
| devananda | NobodyCam: take a look at that -- it needs unit tests added to validate the new behavior, but I think it does what we discussed | 20:49 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Enforce a minimum time between all IPMI commands https://review.openstack.org/96558 | 20:51 |
| lifeless | hi | 20:52 |
| devananda | lifeless: each of those ^ should address different problems you are having with ipmitool | 20:52 |
| devananda | hi! | 20:52 |
| lifeless | devananda: cool; adam_g ^ they might help no wedge things | 20:53 |
| devananda | now with non-breakage for ubuntu 12.04 :) | 20:53 |
| *** harlowja has quit IRC | 20:54 | |
| adam_g | lifeless, yeah, ive been keepin an eye on the second one | 20:54 |
| *** harlowja has joined #openstack-ironic | 20:54 | |
| lifeless | devananda: so | 20:56 |
| lifeless | devananda: I have several wedged ironic instancse on the hp1 region | 20:56 |
| lifeless | devananda: could I impose on you to analyse and ensure there are bugs filed before I stomp on the database? | 20:57 |
| devananda | lifeless: that depends on when you need said analysis by -- i have an apartment viewing in 2hr, and need to eat lunch and shower before then | 20:58 |
| lifeless | devananda: if so ssh to stack@bastion.hp1.tripleo.org, then screen -xr will get you into a shared screen with creds etc | 20:58 |
| * devananda risks taking a peek | 20:58 | |
| lifeless | devananda: I mainly don't want to spend time looking at failure modes you already know about | 20:58 |
| devananda | ack | 20:59 |
| lifeless | devananda: one of them is https://bugs.launchpad.net/ironic/+bug/1326279 | 20:59 |
| *** jdob has quit IRC | 21:01 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Enforce a minimum time between all IPMI commands https://review.openstack.org/96558 | 21:02 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Let ipmitool natively retry commands https://review.openstack.org/96902 | 21:02 |
| *** matty_dubs is now known as matty_dubs|gone | 21:02 | |
| *** jbjohnso has quit IRC | 21:02 | |
| devananda | lifeless: i assume you've seen my reply on there | 21:02 |
| *** godp1301 has joined #openstack-ironic | 21:03 | |
| lifeless | devananda: I have and have just replied | 21:05 |
| *** zdiN0bot has quit IRC | 21:05 | |
| devananda | lifeless: how do i source undercloudrc in screen 3? it's balking at the env | 21:09 |
| devananda | urgh ... | 21:11 |
| devananda | matty_dubs|gone: was it you who had hit "instance associated to >1 node" issues? | 21:12 |
| lifeless | devananda: this is adam_g's tripleo-end-to-end stuff, which is how we're working at all since trunk is rather recalcitrant at the moment | 21:12 |
| devananda | ah | 21:12 |
| devananda | adam_g: "source undercloudrc" is failing -- what am i doing wrong? | 21:13 |
| adam_g | devananda, failing with what error? | 21:13 |
| devananda | os-apply-config: command not found | 21:14 |
| devananda | adam_g: ah, different dir | 21:14 |
| adam_g | devananda, not familiar with your workspace.. if you're in the tripleo-end-to-end dir, try NO_SOURCE_PREP=1 source devtestrc && source tripleo/tripleo-incubator/undercloudrc | 21:15 |
| *** ellenh has joined #openstack-ironic | 21:15 | |
| *** godp1301 has quit IRC | 21:18 | |
| *** derekh_ has joined #openstack-ironic | 21:20 | |
| devananda | lifeless: i'm not sure how to detangle this in the time i have today | 21:21 |
| openstackgerrit | Russell Haering proposed a change to openstack/ironic-python-agent: Add a HardwareManager method to erase devices https://review.openstack.org/97871 | 21:22 |
| devananda | lifeless: you have 3 nova instances in ERROR/DELETING, each of which have 2 ironic node's associated | 21:25 |
| devananda | /that/ seems to be the cause of the failure to delete | 21:25 |
| devananda | someone else reported a similar issue yesterday morning, IIRC, but wasn't able to reproduce, and they claimed to have tinkered with their devstack env, so I couldn't validate it | 21:26 |
| *** zdiN0bot has joined #openstack-ironic | 21:26 | |
| devananda | ah, nvm -- there's the bug: https://bugs.launchpad.net/ironic/+bug/1326364 | 21:27 |
| devananda | derekh_: hi! | 21:27 |
| *** zdiN0bot has quit IRC | 21:33 | |
| *** zdiN0bot has joined #openstack-ironic | 21:34 | |
| lifeless | devananda: nice | 21:34 |
| lifeless | devananda: I will stab hard at things to get going again | 21:35 |
| *** ellenh has quit IRC | 21:36 | |
| devananda | lifeless: trace here: http://paste.openstack.org/show/wOdHiJurWONIhlYrejCj/ | 21:38 |
| *** martyntaylor has quit IRC | 21:38 | |
| *** linggao has quit IRC | 21:38 | |
| devananda | recommend folks poke at https://bugs.launchpad.net/ironic/+bug/1326364 | 21:38 |
| devananda | if there are any more details on it | 21:38 |
| * devananda eats food | 21:38 | |
| *** ellenh has joined #openstack-ironic | 21:38 | |
| lifeless | I'm going to clean out things by hand | 21:42 |
| derekh_ | devananda: hi, see you find the bug | 21:48 |
| *** pbrooko has quit IRC | 21:49 | |
| *** jgrimm has quit IRC | 21:59 | |
| *** ellenh has quit IRC | 21:59 | |
| *** ellenh has joined #openstack-ironic | 22:02 | |
| *** zdiN0bot has left #openstack-ironic | 22:11 | |
| lifeless | is the dvsm-virtual job broken right now ? | 22:11 |
| devananda | lifeless: see the room title :( | 22:13 |
| devananda | nova landed a patch that broke it, and the whole queue is ~12hr behind, so we're stuck today | 22:14 |
| *** SpamapS has quit IRC | 22:14 | |
| *** SpamapS has joined #openstack-ironic | 22:18 | |
| *** SpamapS has quit IRC | 22:18 | |
| *** SpamapS has joined #openstack-ironic | 22:18 | |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Use processutils properly https://review.openstack.org/91202 | 22:20 |
| *** max_lobur has quit IRC | 22:35 | |
| openstackgerrit | lifeless proposed a change to openstack/ironic: Fix host manager node detection logic https://review.openstack.org/97447 | 22:37 |
| *** derekh_ has quit IRC | 22:50 | |
| *** ellenh has quit IRC | 23:08 | |
| *** lucas-dinner has quit IRC | 23:10 | |
| *** ellenh has joined #openstack-ironic | 23:10 | |
| *** mrda-away is now known as mrda | 23:24 | |
| mrda | morning ironic | 23:24 |
| jroll | \o mrda | 23:31 |
| *** rloo has quit IRC | 23:34 | |
| *** rloo has joined #openstack-ironic | 23:34 | |
| *** rloo has quit IRC | 23:35 | |
| *** rloo has joined #openstack-ironic | 23:35 | |
| *** godp1301 has joined #openstack-ironic | 23:41 | |
| lifeless | whee https://bugs.launchpad.net/ironic/+bug/1326364 | 23:44 |
| lifeless | its spreading | 23:44 |
| bigjools | like a bad smell | 23:45 |
| jroll | do we know how that's happening yet? | 23:49 |
| lifeless | jroll: NFI | 23:51 |
| jroll | cool | 23:51 |
| lifeless | how does one cancel a state transition ? | 23:51 |
| lifeless | ironic node-update 00130b60-9844-47ef-9df0-b6007823eedd replace /stats/provision=error | 23:51 |
| lifeless | Node 00130b60-9844-47ef-9df0-b6007823eedd can not be updated while a state transition is in progress. (HTTP 409) | 23:51 |
| jroll | I don't think you can? | 23:52 |
| jroll | /etc/init.d/ironic-conductor stop | 23:52 |
| jroll | :P | 23:52 |
| lifeless | doesn't reset it | 23:53 |
| lifeless | did a conductor restart | 23:53 |
| lifeless | ironic node-update 00130b60-9844-47ef-9df0-b6007823eedd remove /instance_uuid | 23:53 |
| lifeless | Node 00130b60-9844-47ef-9df0-b6007823eedd can not be updated while a state transition is in progress. (HTTP 409) | 23:53 |
| jroll | well yeah, it just stops the actual execution of the state transition | 23:54 |
| lifeless | when does wait call-back time out ? | 23:54 |
| jroll | if there's a way to reset it, I don't know it | 23:54 |
| jroll | I've just been going to the database when necessary | 23:55 |
| jroll | and idk when that times out, I think it's in configs | 23:55 |
| lifeless | 1800 default | 23:55 |
| lifeless | 30m | 23:55 |
| *** godp1301 has quit IRC | 23:57 | |
| *** igordcard has quit IRC | 23:59 | |
| *** ellenh has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
