*** mkerrin has quit IRC | 00:04 | |
*** dlaube has quit IRC | 00:17 | |
*** dlaube has joined #openstack-ironic | 00:21 | |
*** penick has quit IRC | 00:23 | |
*** mkerrin has joined #openstack-ironic | 00:44 | |
*** dlaube has quit IRC | 00:54 | |
*** killer_prince has quit IRC | 01:03 | |
*** ellenh has quit IRC | 01:12 | |
*** nosnos has joined #openstack-ironic | 01:15 | |
*** killer_prince has joined #openstack-ironic | 01:15 | |
*** killer_prince is now known as lazy_prince | 01:15 | |
*** nosnos has quit IRC | 01:22 | |
*** chenglch has joined #openstack-ironic | 01:22 | |
*** nosnos has joined #openstack-ironic | 01:23 | |
*** lazy_prince has quit IRC | 01:30 | |
openstackgerrit | Yuiko Takada proposed a change to openstack/ironic: Update API document with BootDevice https://review.openstack.org/114415 | 01:49 |
---|---|---|
*** krtaylor has joined #openstack-ironic | 01:54 | |
*** killer_prince has joined #openstack-ironic | 01:56 | |
*** killer_prince is now known as lazy_prince | 01:56 | |
*** Krast has joined #openstack-ironic | 02:33 | |
*** rloo_afk has quit IRC | 02:39 | |
*** annegentle_ has quit IRC | 03:02 | |
*** Krast has quit IRC | 03:10 | |
*** Krast has joined #openstack-ironic | 03:10 | |
*** nosnos has quit IRC | 03:28 | |
*** Haomeng has quit IRC | 03:38 | |
*** Haomeng has joined #openstack-ironic | 03:38 | |
*** nosnos has joined #openstack-ironic | 03:41 | |
*** harlowja is now known as harlowja_away | 03:42 | |
*** rushiagr_away is now known as rushiagr | 03:56 | |
*** pcrews has quit IRC | 04:04 | |
*** rushiagr is now known as rushiagr_away | 04:56 | |
*** rushiagr_away is now known as rushiagr | 05:03 | |
openstackgerrit | Haomeng,Wang proposed a change to openstack/ironic: WIP: Add send-data-to-ceilometer support for pxe_ipminative driver https://review.openstack.org/112486 | 05:05 |
chenglch | Hello, may i ask a question about ironic driver. If I setup the two nova compute node with ironic driver ? They get the node info from ironic client, ironic client get info from ironic database, so the compute node will return the same resource to nova conductor? | 05:16 |
*** Krast has quit IRC | 05:31 | |
Haomeng | chenglch: sure, welcome | 05:34 |
Haomeng | chenglch: what is your concern? | 05:36 |
Haomeng | chenglch: sorry, can not get your question | 05:37 |
chenglch | hello haomeng | 05:39 |
chenglch | so only one compute node with ironic driver is needed , is that right? | 05:39 |
Haomeng | chenglch: Ironic support multiple compute nodes, they will call ironic client to connect single api and single database | 05:40 |
Haomeng | chenglch: and our ironic conductor support multi-work | 05:41 |
Haomeng | chenglch: can handle multi-request from multi-clients | 05:41 |
openstackgerrit | Haomeng,Wang proposed a change to openstack/ironic: WIP: Add send-data-to-ceilometer support for pxe_ipminative driver https://review.openstack.org/112486 | 05:43 |
chenglch | when the node.set_provision_state is called, ironic api will let ironic-conductor to do the provision work, may i setup the ironic-conductor on multiple node? | 05:43 |
Haomeng | chenglch: no need, we have logic to balance the work for multiple conductors | 05:44 |
chenglch | iornic conductor and neutron network node is on the same machine? | 05:45 |
Haomeng | chenglch: you can get concept via this document I think | 05:46 |
Haomeng | chenglch: http://docs.openstack.org/developer/ironic/deploy/user-guide.html | 05:46 |
Haomeng | chenglch: no, Ironic depends on neutron client directly | 05:46 |
Haomeng | chenglch: the neutron client will find the neutron url from keystone I think | 05:47 |
Haomeng | chenglch: so, ironic conductor and neutron can be different machine | 05:47 |
Haomeng | chenglch: this is following openstack deployment model, all service can be deployed defferent machine/nodes | 05:48 |
Haomeng | chenglch: and register themself into keystone with api url | 05:48 |
chenglch | I meet a problem when I use the pxe_driver. pxe_driver will prepare the kernel and ramdisk in /tftpboot, and pxe_driver will update the dnsmasq opttion. This work is done by ironic-conductor | 05:49 |
Haomeng | chenglch: yes, what problem it is? | 05:49 |
Haomeng | chenglch: ironic conductor will call neutron client I think | 05:50 |
Haomeng | chenglch: let me find the code | 05:50 |
chenglch | but the the client machine is power on, dnsmasq will do the dhcp work, and tell the client baremetal machine the tftp-server | 05:50 |
chenglch | but the ip address of the tftp-server that client machine get is always the neutron network node | 05:51 |
Haomeng | chenglch: one question, why do you setup two ironic compute nodes? I think one single ironic compute nodes is enough, because it is not *real* hypervisor, the real one is the physical machine | 05:54 |
chenglch | I have check the dnsmasq on neutron network node, the option server-ip-address, and tftp-server is set to the ip address I want, but client baremetal machine don't care that ip in --dhcp-optsfile | 05:54 |
Haomeng | by default, the tftp server is from conductor node | 05:55 |
chenglch | Now i only setup up one compute node with one ironic-conductor, but the ironic-conductor and neutron-netowrk node is not the same machine | 05:55 |
Haomeng | got | 05:56 |
Haomeng | you mean the dhcp's tftp address is not conductor's ip? | 05:57 |
chenglch | yes | 05:57 |
Haomeng | chenglch: can you share the log which debug flag enabled? | 05:57 |
Haomeng | chenglch: let me check the log, maybe it is bug | 05:57 |
Haomeng | and let me go through the cod logic about how to set tftp's ip | 05:58 |
chenglch | the tftp-server address in dnsmasq is the conductor's ip | 05:58 |
Haomeng | chenglch: that should be correct | 05:58 |
Haomeng | chenglch: what is wrong? | 05:58 |
Haomeng | chenglch: because, by defaut, tftp server is in ironic conductor side | 05:59 |
chenglch | but when pxe started ,the client baremetal machine get from dnsmasq is not the conductor's ip | 05:59 |
Haomeng | chenglch: you mean get ip is the one same with neutron server's ip? | 06:00 |
chenglch | yes | 06:00 |
Haomeng | chenglch: pxe is trying to connect to neutron's ip to run tftp to download the kernel+ram disk? | 06:01 |
chenglch | yes | 06:01 |
Haomeng | chenglch: ok, got, can you share the conductor's log to http://paste.openstack.org/? | 06:02 |
Haomeng | chenglch: let me trace back the log with code, maybe it is defect | 06:02 |
Haomeng | did you set tftp_server option in ironic.conf? | 06:04 |
Haomeng | chenglch: in [pxe] section | 06:05 |
chenglch | I set the tftp_server to the ironic conductor ip in [pxe] | 06:05 |
Haomeng | chenglch: should be sure tftp_server ip is correct, which ip is runing tftp service | 06:05 |
Haomeng | chenglch: ok | 06:05 |
chenglch | Does dnsmasq need some special config? | 06:06 |
Haomeng | chenglch: no, dnsmasq is controled by neutron | 06:08 |
Haomeng | and our ironic conductor will call neutron api to update dhcp port | 06:08 |
Haomeng | so it is better you can provide the conductor log file | 06:08 |
*** Krast has joined #openstack-ironic | 06:09 | |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/ironic: Imported Translations from Transifex https://review.openstack.org/114135 | 06:10 |
chenglch | Do we need start tftp on ironic conductor manually ? | 06:14 |
Haomeng | chenglch: if you deploy ironic with devstack, tftp will be started by devstack stack scripts, else, have to make sure it is in service by manually | 06:15 |
Haomeng | chenglch: another question, do you have more dhcp server in your lan? | 06:15 |
Haomeng | chenglch: can you run tcpdump to make sure our dnsmasq receive the dhcp request from your physical pxe? | 06:16 |
chenglch | No , i can see the pxe from clinet get the ip from neutron network node | 06:16 |
Haomeng | what code version are you using? is it latest code? or icehouse release? | 06:17 |
chenglch | I get the code from devstack, and kill the conductor service on the network node, Then I start the ironic-conducor on another machie, the code version is the latest | 06:19 |
Haomeng | chenglch: ok, I got | 06:19 |
Haomeng | chenglch: the tftp ip is registered to neutron's dnsmasq from your firt conductor node | 06:19 |
chenglch | Yes, | 06:21 |
chenglch | The question i met from dnsmasq is like this one http://ubuntuforums.org/showthread.php?t=1605715 | 06:22 |
chenglch | So i'm not sure is there any config needed in dnsmasq to support remote tftp | 06:23 |
*** pcrews has joined #openstack-ironic | 06:40 | |
Haomeng | chenglch: no need to config dnsmasq, it is controlled by neutron | 06:40 |
Haomeng | you make sure the ip in such file /opt/stack/data/neutron/dhcp/e04ad51a-e591-4164-b75f-8f7a3ab472ba/opts , it is conductor's ip? but pxe get neutron's ip? | 06:42 |
Haomeng | chenglch: can you share your dnsmasq option file content, such as - /opt/stack/data/neutron/dhcp/e04ad51a-e591-4164-b75f-8f7a3ab472ba/opts | 06:42 |
Haomeng | this is my option file - http://paste.openstack.org/show/95401/ | 06:43 |
*** jcoufal has joined #openstack-ironic | 06:44 | |
mrda | Have a good weekend Ironic! | 06:46 |
*** zz_naotok is now known as naotok | 06:46 | |
*** mrda is now known as mrda-weekend | 06:46 | |
Haomeng | mrda: good night, and nice weekend:) | 06:46 |
*** dtantsur|afk is now known as dtantsur | 06:47 | |
dtantsur | Morning, Ironic | 06:47 |
Haomeng | dtantsur: morning:) | 06:47 |
dtantsur | brb | 07:24 |
*** yuriyz has quit IRC | 07:47 | |
chenglch | Hello Haomeng , this is my option file http://paste.openstack.org/show/95424/ | 07:50 |
Haomeng | chenglch: server-ip-address - 10.1.0.72/10.1.0.241 | 07:59 |
Haomeng | chenglch: I assume one is from your first conductor's ip, another one is 2nd conductor ip right? | 07:59 |
Haomeng | chenglch: so you manually kill first conductor which is same neutron node with conductor in a machine, right? | 08:00 |
chenglch | yes , I kill the first conductor on 10.1.0.72 | 08:02 |
Haomeng | chenglch: can you check the first conductor's ironic.conf tftp_server | 08:02 |
Haomeng | chenglch: I guess the first conductor 's tftp_server = 10.1.0.72 | 08:03 |
chenglch | I think i find the cause in my environment, when the ironic-conductor and neutron network node is on the same machine, I add " enable-tftp tftp-root=/opt/stack/data/ironic/tftpboot" in a config file I create called dnsmasq.conf, and I enable this file in the dhcp-agent.ini , now clinet pxe can get the tftp file | 08:03 |
Haomeng | chenglch: cool:) | 08:03 |
openstackgerrit | Haomeng,Wang proposed a change to openstack/ironic: WIP: Add send-data-to-ceilometer support for pxe_ipminative driver https://review.openstack.org/112486 | 08:04 |
chenglch | but without enable-tftp in dnsmasq,conf when the ironic-conductor and neutron network node on the same machine , the client pxe can't get the right file | 08:05 |
Haomeng | chenglch: I understand by default, devstack will enable tftp in dnsmasq right? | 08:08 |
Haomeng | chenglch: if it is not enabled, will not work, make sense | 08:08 |
chenglch | so , if we want to use remote tftp, must disable tftp in dnsmasq? | 08:11 |
*** jistr has joined #openstack-ironic | 08:12 | |
*** derekh has joined #openstack-ironic | 08:13 | |
Haomeng | chenglch: I have devstack env, let me check the settings | 08:21 |
Haomeng | chenglch: checked, enable-tftp in dnsmasq,conf is false by default, means that we will not use dnsmasq built-in tftp-server | 08:22 |
*** lucasagomes has joined #openstack-ironic | 08:25 | |
chenglch | Haomeng , then you start tftp-server manually? | 08:26 |
Haomeng | chenglch: devstack will setup tftp server - https://github.com/openstack-dev/devstack/blob/a3ecab6618cb7c9ca8089d8b24ac503ed5a719e7/lib/ironic#L407 | 08:26 |
Haomeng | chenglch: so no need to start by manually, if you did not run stack.sh to install openstack, have to setup tftp server by manually I think | 08:27 |
chenglch | Maybe my environment is very strange, when the network node and ironic-conductor is on the same machine, i must enable-tftp in dnsmasq to make the pxe work correct. I will reinstall my environment next time | 08:33 |
chenglch | Thanks Haomeng | 08:33 |
Haomeng | chenglch: welcome:) | 08:34 |
Haomeng | chenglch: nice weekend:) | 08:34 |
*** ndipanov_gone is now known as ndipanov | 08:35 | |
openstackgerrit | Imre Farkas proposed a change to openstack/ironic: Add DracDriver and its DracPower module https://review.openstack.org/104850 | 08:47 |
openstackgerrit | Haomeng,Wang proposed a change to openstack/ironic: WIP: Add send-data-to-ceilometer support for pxe_ipminative driver https://review.openstack.org/112486 | 08:57 |
openstackgerrit | Rakesh H S proposed a change to openstack/ironic: IPMI double bridging functionality https://review.openstack.org/95775 | 08:59 |
*** pelix has joined #openstack-ironic | 09:07 | |
*** yuriyz has joined #openstack-ironic | 09:12 | |
dtantsur | Folks, have you seen these errors recently: http://logs.openstack.org/46/113346/3/check/check-tempest-dsvm-virtual-ironic/8c2f0d7/ ? | 09:14 |
dtantsur | Is it something with the gate? | 09:15 |
*** chenglch has quit IRC | 09:15 | |
*** dtantsur is now known as dtantsur|lunch | 09:24 | |
*** Krast has quit IRC | 09:56 | |
lucasagomes | dtantsur|lunch, yup yeah i've seem that in some patches | 10:22 |
lucasagomes | lemme take a look | 10:22 |
*** naotok is now known as zz_naotok | 10:32 | |
*** Alexei_987 has joined #openstack-ironic | 10:51 | |
Haomeng | lucasagomes: morning | 10:53 |
lucasagomes | Haomeng, morning :) | 10:53 |
Haomeng | lucasagomes: do you know how to debug our unittest code, I have a unittest passed in my venv but failed in Jenkins | 10:54 |
Haomeng | lucasagomes: I try with import pdb;pdb.set_trace(), it does not work in unittest code | 10:55 |
lucasagomes | Haomeng, yeah :/ I usually debug it primarily with strings heh I think that there should be a better way of doing that | 10:55 |
lucasagomes | Haomeng, can I see the error on jenkins? | 10:55 |
Haomeng | lucasagomes: sure | 10:55 |
Haomeng | lucasagomes: this is my patch, | 10:56 |
Haomeng | lucasagomes: https://review.openstack.org/#/c/112486/ | 10:56 |
Haomeng | lucasagomes: I worked on this unittest for some time, it is working as well in my venv, but not working in jekins | 10:57 |
* lucasagomes clicks | 10:57 | |
Haomeng | lucasagomes: I think my code works, I test it outof our ironic code tree also | 10:58 |
Haomeng | lucasagomes: I will be away and back for a while | 10:58 |
Haomeng | lucasagomes: welcome your comments | 10:58 |
lucasagomes | Haomeng, hmm just for reference in my local env it's failing like the gate | 11:01 |
lucasagomes | http://paste.openstack.org/show/95485/ | 11:01 |
Haomeng | lucasagomes: same with jenkins, ok let me recreate my env, thank you Lucas | 11:02 |
Haomeng | lucasagomes: :) | 11:04 |
Haomeng | lucasagomes: nice day:) | 11:04 |
lucasagomes | Haomeng, I will see if I can spot something else | 11:04 |
lucasagomes | but yeah looks like it ur env | 11:04 |
lucasagomes | Haomeng, you too buddy! have a nice day | 11:05 |
ifarkas | Haomeng, you should be able to use pdb with unittests if you are not using test runner | 11:07 |
ifarkas | Haomeng, you can do eg. python -m testtools.run ironic.tests.test_disk_partitioner.DiskPartitionerTestCase | 11:08 |
openstackgerrit | Rakesh H S proposed a change to openstack/ironic: IPMI double bridging functionality https://review.openstack.org/95775 | 11:09 |
*** lucasagomes is now known as lucas-hungry | 11:18 | |
*** rushiagr is now known as rushiagr_away | 11:30 | |
*** rushiagr_away is now known as rushiagr | 11:35 | |
*** dtantsur|lunch is now known as dtantsur | 12:03 | |
*** rushiagr is now known as rushiagr_away | 12:08 | |
*** nosnos has quit IRC | 12:37 | |
*** rakesh_hs has joined #openstack-ironic | 12:44 | |
*** lazy_prince has quit IRC | 12:49 | |
*** lucas-hungry is now known as lucasagomes | 12:49 | |
*** rakesh_hs_ has joined #openstack-ironic | 12:52 | |
*** rakesh_hs has quit IRC | 12:52 | |
*** rakesh_hs_ has quit IRC | 13:02 | |
*** russellb is now known as rustlebee | 13:03 | |
*** rushiagr_away has quit IRC | 13:09 | |
*** lucasagomes_ has joined #openstack-ironic | 13:15 | |
*** pcrews_ has joined #openstack-ironic | 13:15 | |
*** ndipanov has quit IRC | 13:15 | |
*** yuriyz has quit IRC | 13:20 | |
*** lucasagomes has quit IRC | 13:20 | |
*** pcrews has quit IRC | 13:20 | |
*** rushiagr_away has joined #openstack-ironic | 13:27 | |
*** yuriyz has joined #openstack-ironic | 13:37 | |
*** lucasagomes_ is now known as lucasagomes | 13:38 | |
*** yuriyz has left #openstack-ironic | 13:40 | |
*** yuriyz has joined #openstack-ironic | 13:41 | |
*** killer_prince has joined #openstack-ironic | 13:45 | |
*** killer_prince is now known as lazy_prince | 13:46 | |
lucasagomes | gate seems to be broken due some tempest stuff :( | 13:51 |
* lucasagomes dig | 13:51 | |
Shrews | lucasagomes: example? | 13:57 |
lucasagomes | Shrews, morning http://logs.openstack.org/46/114046/4/gate/gate-tempest-dsvm-virtual-ironic/7faaf8f/console.html | 13:57 |
* Shrews helps lucasagomes dig | 13:57 | |
lucasagomes | https://bugs.launchpad.net/openstack-ci/+bug/1311066 | 13:58 |
lucasagomes | maybe... acording to the elastic check... | 13:58 |
lucasagomes | according* | 13:58 |
lucasagomes | maybe that's it cause I can't find a proper error, I looked into what was merged in tempest/ironic none seems to be causing that | 13:59 |
Shrews | yeah, i see the 15m warning in that console log | 14:01 |
*** jgrimm has joined #openstack-ironic | 14:01 | |
*** jistr is now known as jistr|afk | 14:04 | |
*** lazy_prince has quit IRC | 14:12 | |
NobodyCam | good morning Ironic, TGIF :) | 14:14 |
Shrews | morning NobodyCam | 14:14 |
*** jistr|afk is now known as jistr | 14:15 | |
NobodyCam | morning Shrews | 14:15 |
NobodyCam | :) | 14:15 |
* Shrews needs to reboot. brb | 14:16 | |
Shrews | oh lovely. after updating my system, FF no longer wants to work *at all* | 14:20 |
* Shrews thinks he'll start his vacation a day early | 14:20 | |
*** killer_prince has joined #openstack-ironic | 14:21 | |
*** killer_prince is now known as lazy_prince | 14:21 | |
NobodyCam | :-p | 14:21 |
*** mitz has quit IRC | 14:27 | |
lucasagomes | morning NobodyCam | 14:35 |
NobodyCam | morning lucasagomes :) | 14:39 |
openstackgerrit | Dmitry Tantsur proposed a change to openstack/python-ironicclient: Add sort_key and sort_dir parameters to *-list https://review.openstack.org/114555 | 14:40 |
*** pelix has quit IRC | 14:41 | |
dtantsur | NobodyCam, morning | 14:45 |
dtantsur | and TGIF :) | 14:45 |
jroll | morning ironic :) | 14:48 |
Shrews | jroll: morning. how goes the begging? | 14:48 |
Shrews | :) | 14:48 |
jroll | hehehe | 14:50 |
jroll | I got 2/3 approved | 14:50 |
jroll | one landed, one had conflicts | 14:50 |
jroll | had to rebase, should go through when people are around :) | 14:50 |
jroll | what happened was, I put up a new patch, saw ianw -1 it, and jumped on him :P | 14:51 |
dtantsur | jroll, morning | 14:52 |
jroll | morning dtantsur :) | 14:52 |
NobodyCam | morning dtantsur and jroll :) | 14:53 |
*** ellenh has joined #openstack-ironic | 14:59 | |
jroll | heya NobodyCam :) | 14:59 |
Shrews | anyone here use gerrymander for reviewing? | 15:09 |
jroll | Shrews: huh, haven't seen that before | 15:11 |
jroll | I've been using gertty lately, it's nice | 15:11 |
*** ellenh has quit IRC | 15:13 | |
*** Haomeng|2 has joined #openstack-ironic | 15:17 | |
*** Haomeng has quit IRC | 15:18 | |
NobodyCam | brb | 15:27 |
*** ifarkas has quit IRC | 15:30 | |
dtantsur | Calling it a day, have a nice weekend | 15:31 |
jroll | you too, dtantsur :) | 15:32 |
dtantsur | lucasagomes, note that I'm not PTO on Mon, so no bug team report or just use the dashboard to get some stats :) | 15:32 |
*** dtantsur is now known as dtantsur|afk | 15:32 | |
lucasagomes | dtantsur|afk, ack, cheers for the heads up | 15:33 |
lucasagomes | dtantsur|afk, have a great night, enjoy the weekend! | 15:33 |
lucasagomes | jroll, morning | 15:33 |
jroll | morning lucasagomes :) | 15:33 |
Shrews | ALL: Interesting python oddity that QA guys just pointed out to me: https://review.openstack.org/100132 | 15:46 |
jroll | <3 python | 15:49 |
JoshNang | Shrews: ahh yeah, i've had that bite me before. i'm pretty sure all the oslo loopingcalls use that to keep state between loops | 15:49 |
*** HenryG has joined #openstack-ironic | 15:54 | |
Shrews | JoshNang: well it's a good thing the loopingcalls aren't used all over the place </end sarcasm> | 15:55 |
JoshNang | heh | 15:57 |
*** dlaube has joined #openstack-ironic | 15:59 | |
jroll | Shrews: ever seen this? fresh clone of testr http://paste.openstack.org/show/95620/ | 16:22 |
*** comstud is now known as bearhands | 16:26 | |
*** derekh has quit IRC | 16:26 | |
*** chuckC has quit IRC | 16:27 | |
*** ellenh has joined #openstack-ironic | 16:34 | |
openstackgerrit | Angus Thomas proposed a change to openstack/ironic: Removes get_port_by_vif https://review.openstack.org/114595 | 16:37 |
openstackgerrit | Angus Thomas proposed a change to openstack/ironic: Removes get_port_by_vif. https://review.openstack.org/114595 | 16:39 |
*** Hefeweizen has quit IRC | 16:40 | |
openstackgerrit | Angus Thomas proposed a change to openstack/ironic: Removes get_port_by_vif. https://review.openstack.org/114595 | 16:40 |
openstackgerrit | Angus Thomas proposed a change to openstack/ironic: Removes get_port_by_vif. https://review.openstack.org/114595 | 16:42 |
NobodyCam | dtantsur|afk: please ping /me when your back | 16:43 |
openstackgerrit | Angus Thomas proposed a change to openstack/ironic: Removes get_port_by_vif https://review.openstack.org/114595 | 16:44 |
openstackgerrit | Angus Thomas proposed a change to openstack/ironic: Removes get_port_by_vif https://review.openstack.org/114595 | 16:45 |
lucasagomes | woah | 16:45 |
NobodyCam | hey lucasagomes got a question 'bout tests | 16:50 |
NobodyCam | happen to have a free second | 16:50 |
lucasagomes | NobodyCam, hey yes | 16:50 |
lucasagomes | sure | 16:50 |
NobodyCam | :) | 16:50 |
NobodyCam | re: dtantsur's comment on https://review.openstack.org/#/c/114281/8/ironic/tests/conductor/test_manager.py | 16:51 |
NobodyCam | I have attempted to use the decorator but something is not working when I do that? | 16:53 |
lucasagomes | NobodyCam, hmm lemme see | 16:55 |
NobodyCam | node.reservation is not cleared and spawn is called three times | 16:55 |
NobodyCam | :) | 16:55 |
lucasagomes | NobodyCam, oh wondering if it was because if you mock at the method level | 16:57 |
lucasagomes | this line | 16:57 |
lucasagomes | thread = self.service._spawn_worker(lambda: None) | 16:57 |
lucasagomes | will actually return the mock | 16:57 |
lucasagomes | cause u just mocked _spawn_worker | 16:57 |
lucasagomes | but seems unreleated to the reservation thing | 16:58 |
NobodyCam | thats what thru me off | 17:00 |
Shrews | jroll: umm, nope | 17:00 |
Shrews | jroll: were you making changes to one of the tests when that happened? | 17:01 |
jroll | oh wait | 17:01 |
jroll | it's because I don't have a config file :/ | 17:01 |
*** openstackgerrit has quit IRC | 17:02 | |
Shrews | jroll: i have seen tempest vomit similarly one stupid things, like a misspelled import or such | 17:02 |
Shrews | not helpful | 17:02 |
jroll | right | 17:03 |
jroll | testr is awesome like that | 17:03 |
Shrews | really | 17:03 |
*** openstackgerrit has joined #openstack-ironic | 17:03 | |
*** harlowja_away is now known as harlowja | 17:03 | |
lucasagomes | NobodyCam, wondering if when you had that thread = service._spawn_worker before | 17:04 |
lucasagomes | outside the mock | 17:04 |
lucasagomes | was calling release_resources (which will call release_node and clean up the reservation) | 17:04 |
lucasagomes | where when you mock at the method level, you will invoke do_node_deploy which will call reserve_node and set the hostname | 17:05 |
lucasagomes | but release_node will never be called since it's runs on the __exit__() at task_manager | 17:05 |
lucasagomes | correting... release_node will never be called because spawn_worker is now mocked so the __exit__() method will never run (which is where release_node is invoked in the task_manager.py) | 17:06 |
*** chuckC has joined #openstack-ironic | 17:14 | |
lucasagomes | NobodyCam, what if instead of mocking _spawn_worker you mock the deploy() method? | 17:14 |
lucasagomes | NobodyCam, http://paste.openstack.org/show/95650/ | 17:16 |
NobodyCam | ahh, I see ... let me try that... Ty lucasagomes :) | 17:17 |
lucasagomes | NobodyCam, it sounds even better, cause by mocking _spawn_worker you would mock the _do_node_deploy() complete... where mocking only the driver.deploy() you just mock the driver specific call | 17:18 |
lucasagomes | and run the manager code stuff | 17:18 |
NobodyCam | ya | 17:18 |
NobodyCam | that was it TY lucasagomes :) | 17:22 |
lucasagomes | NobodyCam, np yw! | 17:22 |
*** shakamunyi has joined #openstack-ironic | 17:24 | |
Shrews | lucasagomes: grrr... hitting that same bug you encountered earlier | 17:24 |
lucasagomes | Shrews, :( which one? | 17:25 |
Shrews | https://bugs.launchpad.net/openstack-ci/+bug/1311066 | 17:25 |
lucasagomes | Shrews, >.< | 17:25 |
lucasagomes | Shrews, well at least it's a resource problem not a bug in the code | 17:25 |
Shrews | top bug in the recheck list | 17:25 |
openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Allow rebuild of node in ERROR and DEPLOYFAIL state https://review.openstack.org/114281 | 17:26 |
lucasagomes | yeah... it's causing pretty much all patches to fail in the gate | 17:26 |
* lucasagomes clicks on this bug affects me | 17:29 | |
lucasagomes | :) | 17:29 |
lucasagomes | aight, I will call it a day | 17:30 |
lucasagomes | it's late and friday :) | 17:30 |
lucasagomes | have a good night everyone! enjoy the weekend | 17:30 |
NobodyCam | have a great weekend lucasagomes | 17:30 |
NobodyCam | :) | 17:30 |
Shrews | later lucasagomes | 17:30 |
NobodyCam | and Ty again | 17:31 |
lucasagomes | np :) | 17:31 |
lucasagomes | laterz | 17:31 |
*** lucasagomes is now known as lucas-dinner | 17:31 | |
JoshNang | ~.~.~~. | 17:32 |
JoshNang | ...whoops. frozen screen :/ | 17:33 |
NobodyCam | :-p | 17:33 |
jroll | heyyyyy so. | 17:36 |
jroll | I think our gate might be busted | 17:36 |
jroll | http://logs.openstack.org/75/95775/15/check/check-tempest-dsvm-virtual-ironic/e15e2fa/logs/devstacklog.txt.gz#_2014-08-15_14_31_20_386 | 17:36 |
jroll | I'm having this problem with a fresh install of devstack as well | 17:36 |
Shrews | hrm, so... i was thinking that was due to 1311066, but maybe there is something else? seems to only be affecting the ironic jobs | 17:39 |
Shrews | ugh | 17:39 |
jroll | Shrews: yeah, see that log there... seems like maybe we need to pass container format or something | 17:39 |
jroll | I don't see evidence of something related to that changing, but who knows | 17:39 |
jroll | oh, we use glanceclient master | 17:40 |
* jroll looks | 17:40 | |
*** shakamunyi has quit IRC | 17:43 | |
openstackgerrit | Chris Krelle proposed a change to openstack/ironic: Allow rebuild of node in ERROR and DEPLOYFAIL state https://review.openstack.org/114281 | 17:43 |
*** Alexei_987 has quit IRC | 17:45 | |
jroll | Shrews: I'm going to try this again with --container-format | 17:46 |
*** krtaylor is now known as krtaylor_away | 17:46 | |
jroll | still curious what broke this | 17:47 |
Shrews | jroll: is this due to a recent change in glanceclient? | 17:47 |
Shrews | that would suck | 17:47 |
* Shrews checks logs | 17:48 | |
jroll | it would | 17:49 |
jroll | I poked around but couldn't find anything obvious | 17:49 |
jroll | and.. glanceclient gets "schemas" from glance api | 17:49 |
jroll | presumably defaults are there | 17:49 |
*** penick has joined #openstack-ironic | 17:52 | |
*** jistr is now known as jistr|afk | 17:52 | |
Shrews | f*** | 17:53 |
Shrews | jroll: https://review.openstack.org/111538 | 17:53 |
Shrews | notice, once again, the ironic job failed, but it was +A'd | 17:53 |
NobodyCam | nice | 17:54 |
NobodyCam | :( | 17:54 |
jroll | oh | 17:54 |
jroll | ffs | 17:54 |
jroll | Shrews: want me to do the bug filing and kicking and screaming, or you got it? | 17:55 |
Shrews | jroll: trying to force feed a pb&banana down my mouth hole while debugging... can you kick it off? | 17:56 |
jroll | Shrews: I believe I have the bugfix right now | 17:56 |
jroll | sure | 17:56 |
Shrews | can we have a voting job now????? plz??? | 17:56 |
jroll | srsly | 17:57 |
jroll | I bet this broke nova-bm, too :P | 17:57 |
*** shakamunyi has joined #openstack-ironic | 17:57 | |
jroll | ugh, it's really hard to not be an asshole right now | 18:02 |
* Shrews updates whiteboard | 18:03 | |
*** ellenh_ has joined #openstack-ironic | 18:04 | |
adam_g | :( | 18:04 |
jroll | https://review.openstack.org/114619 | 18:06 |
*** ellenh has quit IRC | 18:06 | |
Shrews | awesome | 18:06 |
jroll | how the $%^&* do you land a change to lib/ironic without ironic's job passing | 18:06 |
Shrews | maybe he was in the middle of debugging the change when it was approved? /me says sheepishly | 18:07 |
jroll | lol | 18:07 |
jroll | I mean, as a core | 18:07 |
jroll | I'm going to ping them all once jenkins runs | 18:08 |
Shrews | i think the meat of the problem is that there are so many transient gate bugs, that unless a job is voting, it's very easy to assume that a non-voting job failed b/c of one of the other bugs | 18:11 |
Shrews | tl;dr -- unless someone is forced to look at it, it won't be looked at | 18:12 |
jroll | I think the problem is people assume | 18:13 |
jroll | like, I'm not mad if you change lib/nova and it breaks ironic | 18:13 |
jroll | but the change was to lib/ironic | 18:13 |
jroll | and ironic failed | 18:13 |
jroll | you can't ignore that | 18:14 |
*** ellenh_ has quit IRC | 18:18 | |
NobodyCam | well can't and shouldn't are two diffenent things | 18:24 |
jroll | >:( | 18:24 |
adam_g | we should probably be taking a bit of responsibility for catching those errors in the devstack review queue. i try to add a -1 to reviews for tempest/devstack if its consistently failing the ironic job | 18:27 |
adam_g | hmm | 18:27 |
Shrews | oh, adam_g, that reminds me. once https://review.openstack.org/112134 lands, we need to put up another patch to disable cinder with ironic again | 18:28 |
adam_g | i can modify http://no-carrier.net/~adam/openstack/ironic_gate_status.html to show the project for each review, that may help us catch these earlier and | 18:28 |
adam_g | Shrews, that would need to go to devstack-gate or config? | 18:28 |
Shrews | adam_g: it would need to be this: https://review.openstack.org/105599 (which was reverted) | 18:29 |
Shrews | devstack-gate | 18:29 |
adam_g | Shrews, ohh right | 18:29 |
Shrews | we need those jobs converged first | 18:30 |
adam_g | Shrews, that'll be a fun one to coordinate | 18:30 |
*** ellenh_ has joined #openstack-ironic | 18:39 | |
*** ellenh_ is now known as ellenh | 18:40 | |
Shrews | woohoo. finally got a +A on the tempest rebuild test. | 18:48 |
NobodyCam | :) | 18:48 |
*** jistr|afk has quit IRC | 18:50 | |
ellenh | silly question: does ironic use openstack/stevedore or dreamhost/stevedore? | 18:53 |
NobodyCam | i would think openstack | 18:54 |
NobodyCam | :-p | 18:54 |
kylestev | I would hope openstack | 18:54 |
kylestev | that would be confusing | 18:54 |
ellenh | i'm guessing openstack/stevedore, but the behavior isn't quite matching up with what the docs say... | 18:54 |
ellenh | or maybe i just misspelled things :P | 18:54 |
NobodyCam | https://github.com/openstack/ironic/blob/master/requirements.txt#L21 | 18:55 |
NobodyCam | should be from here: https://pypi.python.org/pypi/stevedore/1.0.0.0a1 | 18:56 |
NobodyCam | source is listed as http://git.openstack.org/cgit/openstack/stevedore while the home page is listed as https://github.com/dreamhost/stevedore | 18:57 |
NobodyCam | so your guess is as good as mine :-p | 18:57 |
ellenh | that's kinda awkward... | 18:58 |
ellenh | hrm. thanks, i'll poke around some more :) | 18:58 |
NobodyCam | maybe ask dhellmann? | 18:59 |
Shrews | openstack took over hosting stevedore, iirc | 18:59 |
Shrews | same code base, but doubtful the github.com/dreamhost/stevedore code is updated | 19:00 |
ellenh | yeah, dreamhost's hasn't been updated since 2013 | 19:00 |
Shrews | pypi should be published from the openstack repo | 19:00 |
ellenh | i'm trying to use an argument that exists in github/openstack/stevedore but not github/dreamhost/stevedore, and it's not doing anything | 19:01 |
ellenh | so that's why i was suspicious | 19:01 |
ellenh | https://github.com/openstack/stevedore/blob/master/stevedore/driver.py#L22 vs https://github.com/dreamhost/stevedore/blob/master/stevedore/driver.py#L22 | 19:02 |
ellenh | i have a on_load_failure_callback argument, but i'm seeing runtime exceptions instead of the exception i want, so... | 19:03 |
Shrews | ellenh: what's the exception you're seeing? | 19:05 |
*** penick has quit IRC | 19:06 | |
openstackgerrit | Kyle Stevenson proposed a change to openstack/ironic-specs: Add spec for pluggable distributed synchronization https://review.openstack.org/114632 | 19:07 |
ellenh | same one as http://paste.openstack.org/show/95453/ | 19:08 |
Shrews | ellenh: is 'blah' defined in setup.cfg? | 19:09 |
ellenh | no, that's the error i want to catch | 19:09 |
Shrews | ah, i see | 19:09 |
Shrews | is the code up for review somewhere? | 19:10 |
ellenh | yeah, let me just push up a WIP | 19:11 |
openstackgerrit | Ellen Hui proposed a change to openstack/ironic: Make DHCP provider pluggable https://review.openstack.org/112351 | 19:13 |
ellenh | Shrews: ^ :) | 19:13 |
jroll | kylestev: you should poke ironic folks to see what they think | 19:16 |
Shrews | ellenh: so, i *think* that callback is called only if the defined entrypoint file is not found (i.e., it must still be defined in setup.cfg) | 19:20 |
Shrews | so, that's a different exception | 19:20 |
ellenh | hmm. yeah, i'm now 99% sure that it's not a problem with stevedore, since the driver_factory uses on_load_failure_callback | 19:21 |
jroll | I think it's if an exceptino is raised when instantiating the class | 19:21 |
Shrews | jroll: right | 19:21 |
* jroll goes afk for a while | 19:22 | |
ellenh | yeah, you're right. https://github.com/openstack/stevedore/blob/master/stevedore/extension.py#L169 | 19:23 |
kylestev | Hey devananda and NobodyCam, want to take a look at https://review.openstack.org/114632 when you get a chance? | 19:24 |
NobodyCam | kylestev: ironic is in feature proposal freeze | 19:25 |
NobodyCam | that will need to go into K cycle | 19:25 |
NobodyCam | ahh just saw your comment | 19:25 |
kylestev | NobodyCam: right, I put it in kilo and set the workflow to -1 | 19:26 |
kylestev | :P | 19:26 |
*** jcoufal has quit IRC | 19:34 | |
*** Hefeweizen has joined #openstack-ironic | 19:34 | |
ellenh | alright, things make sense now. Thanks Shrews :) | 19:36 |
Shrews | np | 19:36 |
*** athomas has quit IRC | 19:37 | |
*** Mikhail_D_ltp has joined #openstack-ironic | 19:51 | |
*** aignatov has quit IRC | 19:51 | |
*** lucas-dinner has quit IRC | 19:52 | |
*** tatyana has joined #openstack-ironic | 19:53 | |
*** aignatov has joined #openstack-ironic | 19:57 | |
NobodyCam | one +2 on 114619 | 20:01 |
*** aignatov has quit IRC | 20:07 | |
*** Mikhail_D_ltp has left #openstack-ironic | 20:13 | |
Shrews | no vote from the cantankerous Mr. J yet, though | 20:15 |
*** aignatov has joined #openstack-ironic | 20:16 | |
Shrews | the ironic jobs are all SUCCESS though | 20:16 |
NobodyCam | just three more jobs to go | 20:17 |
Shrews | hrm, i'm guessing jeblair is going to be the only core around to approve that for us | 20:24 |
* Shrews pokes around for a core... | 20:26 | |
NobodyCam | ty Shrews :) | 20:26 |
* NobodyCam needs mo coffee brb | 20:28 | |
*** ellenh has quit IRC | 20:30 | |
*** tatyana has quit IRC | 20:30 | |
*** igordcard has joined #openstack-ironic | 20:31 | |
NobodyCam | woo hoo | 20:35 |
NobodyCam | 114619 on its way down for landing | 20:35 |
NobodyCam | TY Shrews | 20:35 |
Shrews | just glad it didn't take us 3 days :) | 20:37 |
*** igordcard has quit IRC | 20:45 | |
*** igordcard has joined #openstack-ironic | 20:47 | |
NobodyCam | lol it still has to get past the gate :-p | 20:48 |
*** ellenh has joined #openstack-ironic | 20:49 | |
openstackgerrit | Ellen Hui proposed a change to openstack/ironic: Make DHCP provider pluggable https://review.openstack.org/112351 | 20:51 |
Shrews | calling it a day here, folks. see you all in a week | 20:51 |
NobodyCam | have a good one Shrews | 20:51 |
*** rwsu has quit IRC | 20:51 | |
*** aignatov has quit IRC | 20:52 | |
NobodyCam | :) | 20:52 |
*** aignatov has joined #openstack-ironic | 20:54 | |
NobodyCam | kylestev: do you have another locking schema in mind? | 21:16 |
kylestev | NobodyCam: yes. ZooKeeper. | 21:17 |
kylestev | implementing locking with ephemeral znodes is quite trivial and will mitigate deadlocks caused by the db-backed locking when a conductor dies | 21:19 |
NobodyCam | I would like to see that as part of the spec, more so I can under stand the why do this question I generate while reading that spec | 21:24 |
kylestev | jroll: ^ thoughts? | 21:25 |
chuckC | hi all: I'm working on a bug in neutron to allow neutron's port mac_address to be updatable so nic failure on an ironic server can be more easily handled | 21:29 |
chuckC | what I need to know is how to simulate a nic failure on an ironic server that's really a VM | 21:30 |
chuckC | thoughts? | 21:31 |
*** todd_dsm has joined #openstack-ironic | 21:36 | |
adam_g | chuckC, what do you mean by nic failure? | 21:38 |
NobodyCam | ifdown the nic | 21:38 |
NobodyCam | ? | 21:38 |
openstackgerrit | Ellen Hui proposed a change to openstack/ironic: Make DHCP provider pluggable https://review.openstack.org/112351 | 21:39 |
chuckC | It would need to be a way to effect the simulated hardware, not something inside the server OS | 21:40 |
*** todd_dsm has quit IRC | 21:40 | |
chuckC | What I'm struggling with is if nic hardware fails on an ironic server, will neutron notice in any way? | 21:42 |
chuckC | So I wanted to try it, and of course I don't have baremetal hardware, just VMs | 21:44 |
NobodyCam | chuckC: would hacking the xml work ? you could add <link state='down'/> for the nic | 21:46 |
chuckC | NobodyCam: Thanks for the idea, I'll try it. | 21:47 |
*** coolsvap has quit IRC | 21:50 | |
NobodyCam | quick walkies ...brb | 21:58 |
*** coolsvap has joined #openstack-ironic | 22:04 | |
NobodyCam | ugh 114619 fail :( | 22:19 |
*** igordcard has quit IRC | 22:25 | |
*** shakamunyi has quit IRC | 22:30 | |
*** igordcard has joined #openstack-ironic | 22:37 | |
*** chuckC has quit IRC | 22:37 | |
openstackgerrit | Kyle Stevenson proposed a change to openstack/ironic-specs: Add spec for pluggable distributed synchronization https://review.openstack.org/114632 | 22:47 |
kylestev | NobodyCam: https://review.openstack.org/#/c/114632/1..2/specs/kilo/pluggable-distributed-synchronization.rst - Let me know what you think. If you’d prefer not to deal with this on a Friday afternoon, I can remind you on Monday :) | 22:52 |
*** rwsu has joined #openstack-ironic | 23:06 | |
NobodyCam | :) please do... :) | 23:07 |
*** chuckC has joined #openstack-ironic | 23:08 | |
kylestev | :) | 23:09 |
openstackgerrit | Ellen Hui proposed a change to openstack/ironic: Make DHCP provider pluggable https://review.openstack.org/112351 | 23:23 |
NobodyCam | ugggh | 23:34 |
*** rwsu has quit IRC | 23:35 | |
*** ellenh has quit IRC | 23:39 | |
NobodyCam | reverify started... will check back later ... :) | 23:49 |
*** shakamunyi has joined #openstack-ironic | 23:51 | |
*** igordcard has quit IRC | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!