Thursday, 2015-06-25

*** david-ly_ has joined #openstack-ironic00:06
*** ijw_ has joined #openstack-ironic00:07
*** david-lyle has quit IRC00:08
*** ijw__ has quit IRC00:10
*** ijw_ has quit IRC00:29
*** dprince has joined #openstack-ironic00:37
*** rwsu has quit IRC00:38
*** boris-42 has quit IRC00:42
*** r-daneel has quit IRC00:48
*** zhenguo has joined #openstack-ironic00:52
*** naohirot has joined #openstack-ironic00:53
*** amotoki has joined #openstack-ironic01:01
*** amotoki has quit IRC01:05
*** smoriya has joined #openstack-ironic01:06
*** thrash is now known as thrash|g0ne01:19
*** dprince has quit IRC01:22
*** david-ly_ is now known as david-lyle01:36
*** mitchjameson has quit IRC01:40
*** dontalton has quit IRC01:56
*** Nisha has joined #openstack-ironic01:57
*** yuanying has joined #openstack-ironic02:02
*** Nisha has quit IRC02:06
*** ramineni has joined #openstack-ironic02:40
*** mitchjameson has joined #openstack-ironic02:43
*** Sukhdev has joined #openstack-ironic03:01
*** amotoki has joined #openstack-ironic03:02
*** amotoki has quit IRC03:06
*** amit213 has quit IRC03:12
*** amit213 has joined #openstack-ironic03:13
*** yuanying has quit IRC03:16
*** mtanino has quit IRC03:31
*** korean101 has quit IRC03:42
*** yuanying has joined #openstack-ironic03:47
*** rameshg87 has joined #openstack-ironic03:56
*** yuanying has quit IRC03:59
*** coolsvap|away is now known as coolsvap04:04
*** yuanying has joined #openstack-ironic04:09
*** amotoki has joined #openstack-ironic04:10
*** hemna has joined #openstack-ironic04:19
*** achanda has joined #openstack-ironic04:23
*** mitchjameson has quit IRC04:35
*** lazy_prince has joined #openstack-ironic04:39
*** hemna has quit IRC04:40
*** hemna has joined #openstack-ironic04:42
*** chenglch has joined #openstack-ironic04:44
*** saripurigopi has joined #openstack-ironic04:51
saripurigopimorning Ironic04:54
lazy_princesaripurigopi: morning..04:54
saripurigopilazy_prince: o/04:54
*** jrath4 has joined #openstack-ironic04:54
openstackgerritShivanand Tendulker proposed stackforge/proliantutils: Adding RIS support for virtual media devices  https://review.openstack.org/19457004:58
*** Sukhdev has quit IRC05:01
*** Marga_ has joined #openstack-ironic05:02
*** jrath4 has quit IRC05:03
*** hemna has quit IRC05:05
*** achanda has quit IRC05:06
*** Nisha has joined #openstack-ironic05:21
*** radek__ has joined #openstack-ironic06:01
*** yuikotakada has joined #openstack-ironic06:03
*** yuanying_ has joined #openstack-ironic06:15
*** yuanying_ has quit IRC06:17
*** yuanying_ has joined #openstack-ironic06:18
*** yuanying has quit IRC06:18
*** yuanying_ has quit IRC06:19
*** jefabo has joined #openstack-ironic06:23
*** jefabo has left #openstack-ironic06:26
*** ifarkas has joined #openstack-ironic06:27
*** yuanying has joined #openstack-ironic06:32
*** yuanying has quit IRC06:32
openstackgerritNaohiro Tamura proposed openstack/ironic: Add iRMC Virtual Media Deploy module for iRMC Driver  https://review.openstack.org/15195806:38
*** romcheg has joined #openstack-ironic06:39
*** blixt_ is now known as hblixt06:41
*** romcheg has quit IRC06:43
*** achanda has joined #openstack-ironic06:44
*** ukalifon1 has joined #openstack-ironic06:44
*** jefabo_ has joined #openstack-ironic06:46
*** jefabo_ has quit IRC06:51
*** jefabo_ has joined #openstack-ironic06:52
*** jefabo_ is now known as jefabo06:53
openstackgerritRamakrishnan G proposed stackforge/proliantutils: Check if virtual media is attached before ejecting  https://review.openstack.org/19543206:53
rameshg87ramineni: ^^^06:54
*** yog_ has joined #openstack-ironic06:54
*** ifarkas has quit IRC06:57
*** achanda has quit IRC06:58
*** bradjones has quit IRC07:02
*** Nisha has quit IRC07:02
*** yuanying has joined #openstack-ironic07:02
*** bradjones has joined #openstack-ironic07:04
*** bradjones has quit IRC07:04
*** bradjones has joined #openstack-ironic07:04
openstackgerritNaohiro Tamura proposed openstack/ironic: Add localboot support for iscsi_irmc driver  https://review.openstack.org/16373807:05
*** yuanying has quit IRC07:05
*** yuanying has joined #openstack-ironic07:05
*** athomas has joined #openstack-ironic07:06
*** achanda has joined #openstack-ironic07:08
*** gzhai1 has joined #openstack-ironic07:13
*** dasm is now known as dasm|afk07:13
*** dasm|afk is now known as dasm07:14
*** viktors|afk is now known as viktors07:15
*** romainh has joined #openstack-ironic07:21
*** gzhai1 has quit IRC07:25
*** lsmola has joined #openstack-ironic07:28
*** natorious is now known as zz_natorious07:39
*** dtantsur|afk is now known as dtantsur07:50
dtantsurMorning Ironic!07:50
yuikotakadadtantsur, morning :)07:56
openstackgerritZhenguo Niu proposed openstack/ironic: When boot option is not persisted, set boot on next power on  https://review.openstack.org/17764207:56
*** gzhai1 has joined #openstack-ironic08:01
*** lucasagomes has joined #openstack-ironic08:03
dtantsuryuikotakada, o/08:03
*** ndipanov has joined #openstack-ironic08:06
gzhai1dtantsur:hi08:07
dtantsuro/08:07
gzhai1dtantsur:Do you have time to revisit my bp? https://review.openstack.org/#/c/176109/08:07
dtantsurprobably :) I'm pretty busy recently, will try to find time (it's on my radar already)08:08
gzhai1dtantsur: Thanks!08:09
*** coolsvap is now known as coolsvap|away08:16
*** coolsvap|away is now known as coolsvap08:19
*** dguerri` is now known as dguerri08:21
*** e0ne has joined #openstack-ironic08:32
huatshuats08:32
lucasagomesmorning folks08:34
lucasagomesif you have a time please review https://review.openstack.org/#/c/19413208:34
lucasagomesneeds another +2 :-)08:34
huatsI am having some issues too be able to have both vm instances and ironic one on the same OpenStack, is it just me ?08:36
huats(and a misconfiguration)08:36
*** e0ne has quit IRC08:37
saripurigopilucasagomes: o/08:37
openstackgerritLucas Alvares Gomes proposed openstack/ironic: Add iBoot driver documentation  https://review.openstack.org/19354608:37
lucasagomeshuats, usually people use host aggregates to separate both clouds08:37
lucasagomeshuats, or cells08:37
lucasagomesthe problem is scheduling, for baremetal we need to tweak the scheduler to behave differently08:38
huatslucasagomes:  yeah that was my understanding08:38
huatsin that case do you have just one list of filters ?08:38
lucasagomessince we can't allocate partial resources like you could with a hypervisor. Baremetal you always get 100% of the resources08:38
lucasagomeshuats, you also have to tweak memory allocate ration08:38
lucasagomesto 1.0 instead of 1.5  and use a custom host manager for ironic08:39
*** stegranet has joined #openstack-ironic08:39
lucasagomesso reporting available resources are more accurate08:39
lucasagomeshuats, 1 sec08:39
huatslucasagomes: I have ssen that the memory allocation can be specified by host aggregate yes08:39
lucasagomeshuats, http://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-compute-service-to-use-the-bare-metal-service08:39
huatsbut I am not sure for the schedulling part08:39
*** romcheg has joined #openstack-ironic08:41
huatslucasagomes: in the doc you are refering it is the same settings for the whole openstack08:42
lucasagomeshuats, right, so the scheduling part I think it depends on ur use case08:45
lucasagomesby default we do exact matching of the resources (those exact filters are enabled by default when you set the IronicHostManager)08:45
lucasagomesbecause we don't want users to ask for X and get Y, if one ask for 4 CPUs and 2G of ram08:45
*** jcoufal has joined #openstack-ironic08:46
lucasagomeswe want to give him that and not 8 CPUs and 8 GB of ram (because we can't allocate part of the resources)08:46
huatsyep08:46
lucasagomesbut if that's OK for you, sure yeah you can tweak the scheduler to behave that way too08:46
huatsi have seen that08:46
huatsthat is why I am asking if I can have a different list of filters based on hostaggregates, so that I can have one hostaggregate for vm (and thus their own filters) and another for baremetal (with another set of filters with exact types in particular)08:48
lucasagomesI believe you can, because you'r going to have 2 scheduler services running right? One for each host aggregates I beleive08:48
lucasagomesI would ask in #openstack-nova for specifics08:48
huatsok08:49
huatsthanks !08:49
*** derekh has joined #openstack-ironic08:51
*** stendulker has joined #openstack-ironic09:05
*** pelix has joined #openstack-ironic09:05
openstackgerritShivanand Tendulker proposed stackforge/proliantutils: Adding RIS support for virtual media interfaces  https://review.openstack.org/19457009:06
sambettsMorning all o/09:09
stendulkerGood morning sambetts :)09:09
*** e0ne has joined #openstack-ironic09:10
yuikotakadasambetts, stendulker: o/09:10
huatslucasagomes: apparently running 2 diffents scheduller (with different configuration I mean) is not really doable09:10
stendulkeryuikotakada: o/09:10
huatsif someone is using hostaggregates to have ironic and instances I am really really interested :)09:11
sambettso/ yuikotakada09:11
huatsdevananda apparently that is what you told me to do :)09:11
lucasagomeshuats, oh :-/09:14
lucasagomesperhaps cells would be the only way them?09:14
huatsI imagine :(09:15
huatswhich is not really nice09:15
*** siva has joined #openstack-ironic09:15
huatsI have to figure it out :(09:15
*** siva is now known as Guest6147009:15
*** e0ne is now known as e0ne_09:16
*** Guest61470 has left #openstack-ironic09:17
openstackgerritTan Lin proposed openstack/ironic-specs: Add clean task field to node  https://review.openstack.org/19547009:19
*** e0ne_ has quit IRC09:22
dtantsursambetts, morning!09:25
openstackgerritRamakrishnan G proposed stackforge/proliantutils: Check if virtual media is attached before ejecting  https://review.openstack.org/19543209:31
sambettsMorning dtantsur09:31
rameshg87lucasagomes: was checking https://review.openstack.org/#/c/194132/609:32
rameshg87lucasagomes: hi there09:32
lucasagomesrameshg87, hi there09:32
rameshg87lucasagomes: do you think it makes sense to set last_error there ?09:32
*** e0ne has joined #openstack-ironic09:33
lucasagomesrameshg87, we could, to say that the node was put into deploy fail state?09:33
openstackgerritYuriy Zveryanskyy proposed openstack/ironic: Do not preserve partitions after ephemeral  https://review.openstack.org/18268809:34
* rameshg87 checks 09:36
rameshg87lucasagomes: oh, so node goes to deleted and then cleaning and back to available09:36
rameshg87okay, got it. I thought we were going to fail deploy09:36
rameshg87that seems not the case09:36
lucasagomesyeah it will be ready for available again09:36
lucasagomesdid you test it locally?09:37
stendulkerlucasagomes, dtantsur: Good morning !09:45
dtantsurstendulker, morning09:46
*** alexpilotti has joined #openstack-ironic09:46
stendulkerlucasagomes, dtantsur: Can you please review this spec when you get some time https://review.openstack.org/#/c/174295/09:47
lucasagomesstendulker, morning09:47
lucasagomesack09:47
stendulkerlucasagomes, dtantsur: It's about adding grub2 bootloader for UEFI and secure boot support for pxe_ilo driver09:47
stendulkerlucasagomes, dtantsur: thank you09:48
openstackgerritDmitry Tantsur proposed openstack/ironic: Make task_manager logging more helpful  https://review.openstack.org/19461909:51
dtantsurlucasagomes, no more dirty hacks ^^^09:51
openstackgerritLucas Alvares Gomes proposed openstack/ironic: Add IPMI 1.5 support for the ipmitool power driver  https://review.openstack.org/19515709:55
*** naohirot has quit IRC09:58
openstackgerritSergey Vilgelm proposed openstack/ironic: Switch to oslo.service  https://review.openstack.org/19500810:11
*** MattMan has quit IRC10:13
*** rameshg87 has left #openstack-ironic10:14
*** athomas has quit IRC10:15
*** husanu4 has joined #openstack-ironic10:19
openstackgerritLucas Alvares Gomes proposed openstack/ironic: Clean nodes stuck in DEPLOYING state when ir-cond restarts  https://review.openstack.org/19413210:21
*** rameshg87 has joined #openstack-ironic10:22
*** husanu4 has quit IRC10:25
*** husanu1 has joined #openstack-ironic10:26
*** husanu1 has quit IRC10:28
*** athomas has joined #openstack-ironic10:29
*** husanu1 has joined #openstack-ironic10:30
*** husanu1 has quit IRC10:31
*** e0ne is now known as e0ne_10:31
*** MattMan has joined #openstack-ironic10:33
*** thrash|g0ne is now known as thrash10:33
*** e0ne_ has quit IRC10:36
zhenguolucasagomes dtantsur, can you help to review this https://review.openstack.org/#/c/177642/ if you have time.10:37
dtantsurif we have time...10:38
dtantsur:)10:38
openstackgerritMerged openstack/ironic: Add iBoot driver documentation  https://review.openstack.org/19354610:38
zhenguodtantsur: thanks :)10:39
lucasagomeszhenguo, will take a look10:39
zhenguothanks lucasagomes10:39
openstackgerritVladyslav Drok proposed openstack/ironic: Image service should not be set in ImageCache constructor  https://review.openstack.org/19475210:40
*** rameshg87 has left #openstack-ironic10:41
openstackgerritSam Betts proposed openstack/ironic-inspector: Deprecate authenticate opt in favor of auth_strategy  https://review.openstack.org/19508210:44
openstackgerritMerged openstack/ironic: Add unit test for ilo_deploy _configure_vmedia_boot()  https://review.openstack.org/19404710:46
*** erelwell has joined #openstack-ironic10:46
*** erelwell is now known as bethelwell10:46
*** Marga_ has quit IRC10:50
*** ramineni has quit IRC10:53
*** e0ne has joined #openstack-ironic10:55
openstackgerritMerged openstack/ironic: Add ENROLL and related states to the state machine  https://review.openstack.org/19226911:09
*** chenglch has quit IRC11:10
*** yuikotakada is now known as yuikot_go_home11:12
sambettsdtantsur, yuikotakada: the reason I changed it do get_auth_strategy instead of just simply returning true and false is that it'll reduce the amount of reworking we need to do in the future, and we could even add new strategies before completely deprecating authenticate11:15
*** e0ne is now known as e0ne_11:15
dtantsursambetts, well, we're not planning on adding new strategies yet, but if we do, amount of reworking will still be high - it won't be enough to just return new string value :)11:15
sambettsdtantsur: We shouldn't have to modify the util function though, only the parts of the code actually dealing with the auth11:17
*** david-lyle has quit IRC11:17
*** jrist has quit IRC11:18
dtantsurwell, I don't mind both variants actually11:18
*** jrist has joined #openstack-ironic11:19
*** david-lyle has joined #openstack-ironic11:21
*** lucasagomes has quit IRC11:22
*** achanda has quit IRC11:25
*** e0ne_ has quit IRC11:26
*** e0ne has joined #openstack-ironic11:28
*** Marga_ has joined #openstack-ironic11:32
*** lucasagomes has joined #openstack-ironic11:34
*** bradjones has quit IRC11:38
*** EmilienM|off is now known as EmilienM11:38
*** saripurigopi has quit IRC11:39
*** bethelwell has quit IRC11:40
*** bradjones has joined #openstack-ironic11:41
*** bradjones has quit IRC11:41
*** bradjones has joined #openstack-ironic11:41
*** bethelwell has joined #openstack-ironic11:41
*** lucasagomes is now known as lucas-hungry11:43
*** bethelwe_ has joined #openstack-ironic11:45
*** Marga_ has quit IRC11:48
*** bethelwell has quit IRC11:48
*** bethelwe_ has quit IRC11:49
*** stegranet has quit IRC11:52
*** stegranet has joined #openstack-ironic11:53
*** amotoki has quit IRC11:57
*** yuanying has quit IRC11:57
jefabohuats: thansk for asking :)11:58
huatsjefabo: do you have any clue ?11:59
jefabohuats: not at all, unless using a single flavor / signle type of hardware12:03
*** zhenguo has quit IRC12:04
*** stegranet has quit IRC12:08
*** stegranet has joined #openstack-ironic12:09
openstackgerritShivanand Tendulker proposed stackforge/proliantutils: Adding RIS support for virtual media interfaces  https://review.openstack.org/19457012:17
*** killer_prince has joined #openstack-ironic12:18
*** killer_prince has quit IRC12:21
*** lazy_prince has quit IRC12:22
*** stegranet has quit IRC12:24
*** stegranet1 has joined #openstack-ironic12:24
*** coolsvap is now known as coolsvap|away12:26
*** stegranet1 is now known as stegranet12:27
*** stendulker has quit IRC12:27
*** bethelwell has joined #openstack-ironic12:28
*** dprince has joined #openstack-ironic12:29
*** dtantsur is now known as dtantsur|brb12:32
*** coolsvap|away is now known as coolsvap12:35
*** david-ly_ has joined #openstack-ironic12:35
*** coolsvap is now known as coolsvap|away12:36
*** david-lyle has quit IRC12:39
*** trown|outttypeww is now known as trown12:41
TheJuliaGood morning!12:44
*** lucas-hungry is now known as lucasagomes12:47
*** bradjones has quit IRC12:47
*** bradjones has joined #openstack-ironic12:50
*** bradjones has quit IRC12:50
*** bradjones has joined #openstack-ironic12:50
*** husanu5 has joined #openstack-ironic12:50
*** husanu5 has quit IRC12:52
*** rloo has joined #openstack-ironic12:55
*** absubram has quit IRC12:55
*** husanu1 has joined #openstack-ironic12:56
*** romcheg has quit IRC12:58
*** romcheg has joined #openstack-ironic12:58
lucasagomesTheJulia, good ugt morning12:59
*** smoriya has quit IRC12:59
*** coolsvap|away is now known as coolsvap13:00
*** husanu1 has quit IRC13:00
*** e0ne is now known as e0ne_13:04
*** e0ne_ is now known as e0ne13:04
*** yuikotakada has joined #openstack-ironic13:06
openstackgerritJulia Kreger proposed openstack/bifrost: Add delete role for dynamic inventory testing  https://review.openstack.org/19369013:06
openstackgerritJulia Kreger proposed openstack/bifrost: Individual playbook for creating a test VM  https://review.openstack.org/19369113:06
openstackgerritJulia Kreger proposed openstack/bifrost: Update the dynamic test playbook  https://review.openstack.org/19369213:06
openstackgerritJulia Kreger proposed openstack/bifrost: Add dynamic inventory description to README  https://review.openstack.org/19370913:06
openstackgerritJulia Kreger proposed openstack/bifrost: Addition of dynamic enrollment and deployment playbooks  https://review.openstack.org/19373713:07
openstackgerritJulia Kreger proposed openstack/bifrost: Change Bifrost over to using dynamic inventory  https://review.openstack.org/19388513:07
*** dtantsur|brb is now known as dtantsur13:07
dtantsurmuch code wow ^^^13:07
TheJuliayeah... :(  Next time I chain a bunch of stuff together, I'm going to do Depends-On13:08
TheJuliaand just make big list in the last few commits. :(13:08
openstackgerritVladyslav Drok proposed openstack/ironic: Update cached images based on update time  https://review.openstack.org/19501713:12
*** husanux7 has joined #openstack-ironic13:13
*** husanux7 has quit IRC13:13
*** husanux6 has joined #openstack-ironic13:14
*** degorenko has quit IRC13:16
openstackgerritVladyslav Drok proposed openstack/ironic: Update cached images based on update time  https://review.openstack.org/19501713:17
*** husanux6 has quit IRC13:18
*** husanux1 has joined #openstack-ironic13:20
*** e0ne is now known as e0ne_13:20
*** romcheg has quit IRC13:20
*** husanux1 has quit IRC13:21
*** coolsvap is now known as coolsvap|away13:21
*** e0ne_ is now known as e0ne13:22
*** Marga_ has joined #openstack-ironic13:22
*** romcheg has joined #openstack-ironic13:22
*** martini has joined #openstack-ironic13:22
martiniGood morning Ironic13:23
*** husanux5 has joined #openstack-ironic13:25
rloohi ironickers, dtantsur. g'morning martini, TheJulia13:25
dtantsurmorning rloo, martini13:26
*** husanux5 has quit IRC13:29
*** Marga_ has quit IRC13:31
*** Marga_ has joined #openstack-ironic13:31
martiniTheJulia- For a full offline install, scripts/env-setup needs to be able to clone ansible from an alternate url.  Any suggestions on how to store/pass that?13:34
*** degorenko has joined #openstack-ironic13:34
*** absubram has joined #openstack-ironic13:35
TheJuliaWe store it as an variable with a default, if it is already defined, then we use the dfined variable13:39
TheJuliadefined variable, in the environment13:39
TheJuliaso you could set it by doing something like export ANSIBLE_GIT_URL="/tmp/ansible.git" && scripts/env-setup13:40
martiniCool, thanks13:41
TheJulianp13:42
*** bethelwell has quit IRC13:44
*** jcoufal has quit IRC13:49
bcornecHello everybody13:49
TheJuliaGood morning!13:49
bcornecHave a question around bifrost. Still trying to call it in a self-sufficient and isolated docker container.13:49
TheJuliaokay :)13:50
TheJuliaWhats up?13:50
bcornecAt a moment during the install I get the following error:13:50
bcornecTASK: [ironic-install | Starting MySQL] ***************************************13:50
bcornec<localhost> REMOTE_MODULE service name=mysql state=started13:50
bcornec<localhost> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1435239030.01-233799260644744 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1435239030.01-233799260644744 && echo $HOME/.ansible/tmp/ansible-tmp-1435239030.01-233799260644744']13:50
bcornec<localhost> PUT /tmp/tmpjvZIm0 TO /home/bruno/.ansible/tmp/ansible-tmp-1435239030.01-233799260644744/service13:50
bcornec<localhost> EXEC /bin/sh -c 'sudo -k && sudo -H -S -p "[sudo via ansible, key=farvusjvqpxqrnxscqfwmwsjuskluvup] password: " -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-farvusjvqpxqrnxscqfwmwsjuskluvup; LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /home/bruno/.ansible/tmp/ansible-tmp-1435239030.01-233799260644744/service; rm -rf /home/bruno/.ansible/tmp/ansible-tmp-1435239030.01-233799260644744/ >/dev/null 2>&1'"'"''13:50
bcornecfailed: [localhost] => {"failed": true}13:50
bcornecmsg:  * Starting MySQL database server mysqld13:50
bcornec   ...done.13:50
bcornecFATAL: all hosts have already failed -- aborting13:50
bcornecPLAY RECAP ********************************************************************13:50
bcornec           to retry, use: --limit @/home/bruno/install.yaml.retry13:50
bcorneclocalhost                  : ok=21   changed=19   unreachable=0    failed=113:50
bcornecINFO[0846] The command [/bin/sh -c su - $BIFROST_UID -c "./launch-bifrost.sh $http_proxy $https_proxy"] returned a non-zero code: 213:50
*** dasm is now known as dasm|afk13:50
bcornecHowever I already stopped my local MySQL so have no real clue why it's not startng correctly here13:51
NobodyCamgood (UGT) Morning Ironicers13:51
NobodyCambcornec: anything in the mysql log?13:53
TheJuliabcornec: what distribution/distribution version are you using.  That seems reminiscent of an error I found ages ago on TripleO13:55
*** stegranet has quit IRC13:58
*** athomas has quit IRC13:58
*** stegranet has joined #openstack-ironic13:59
lucasagomesNobodyCam, morning14:07
*** r-daneel has joined #openstack-ironic14:09
NobodyCammorning lucasagomes :)14:10
bcornecMy distro is Mageia, but the container is Ubuntu 15.0414:11
bcornecProblem is that it's difficult to get logs as the container dies... I'll try o reproduce manually, but thought I could ask before if that was a known problem. Last time I did an update from your git repo it fixed the previous issue I had, so guessed it could be handled similarly...14:12
*** e0ne is now known as e0ne_14:13
NobodyCambcornec: I have not seen that in my testing.14:13
*** romainh has quit IRC14:15
*** romainh has joined #openstack-ironic14:15
TheJuliahmm14:17
TheJuliabcornec: have you tried 14.04?14:18
bcornecI also have other error msgs which seem less important:14:18
bcornec- name: "Install Ironic using pip"14:18
bcornec  pip: name=/opt/stack/ironic state=latest extra_args='--proxy http://web-proxy.fra.hp.com:8080'14:18
bcornecerror: Your local changes to the following files would be overwritten by checkout:14:18
bcornecPlease, commit your changes or stash them before you can switch branches.14:18
bcornecAborting14:18
bcornecCleaning up...14:18
bcornecmkdir: cannot create directory '/opt/stack': Permission denied14:18
bcornecCloning into 'ansible'...14:18
bcornecreading manifest file 'lib/ansible.egg-info/SOURCES.txt'14:18
bcornecreading manifest template 'MANIFEST.in'14:18
bcornecno previously-included directories found matching 'v2'14:18
bcornecno previously-included directories found matching 'docsite'14:18
bcornecno previously-included directories found matching 'ticket_stubs'14:18
bcornecno previously-included directories found matching 'packaging'14:18
bcornecno previously-included directories found matching 'test'14:18
bcornecno previously-included directories found matching 'hacking'14:18
bcornecno previously-included directories found matching 'lib/ansible/modules/core/.git'14:18
bcornecno previously-included directories found matching 'lib/ansible/modules/extras/.git'14:18
bcornecwriting manifest file 'lib/ansible.egg-info/SOURCES.txt'14:18
TheJuliaplease use paste.openstack.org14:19
lucasagomesfolks http://paste.openstack.org14:19
NobodyCambcornec: please use service lisk paste.openstack.org14:19
NobodyCamlol14:19
lucasagomeshah14:19
lucasagomesNobodyCam, PixeBot should detect it14:19
NobodyCamhummm :)14:19
dtantsurNobodyCam, morning!14:20
bcornecAre you working with 14.04 as a preferred distro ?14:20
NobodyCammornign dtantsur14:20
NobodyCam:)14:20
bcornecRelaunching with 14.04. Invalidates the full docker cache of course, so we'll have to wait a bit before having the result ;-)14:21
devanandamorning, all14:21
NobodyCamgood morning devananda14:21
TheJuliabcornec: yes, we developed and run ci on trusty, presently working on rhel/centos support14:21
bcornecmorning dvvdv14:22
TheJuliabcornec: give me a few and i'll look at that error14:22
bcornecTheJulia: easy to change in the docker file, longer to have it run again completely. You have a few to look at the error if you have time to do so :-)14:23
dtantsurdevananda, morning! I've update the task_manager debugging patch https://review.openstack.org/#/c/194619/ as we agreed14:23
*** e0ne_ has quit IRC14:23
bcornecTheJulia:  My modifications are at https://github.com/bcornec/bifrost in case you want ot have a look at what I'm doing...14:24
devanandadtantsur: at a read through, I like it!14:26
dtantsurcool :)14:28
yuikotakadadtantsur, hi, what does 'NOT_REGISTERED' mean in https://review.openstack.org/#/c/174231/, do you know?14:29
yuikotakadasambetts, around here?14:30
sambettsyuikotakada: Yup :)14:31
yuikotakadasambetts, o/ I'm reading your patch and I'm a little bit confused >_<14:31
sambettsyuikotakada: oh? whats up?14:32
yuikotakadasambetts, Even if  CONF.authenticate is 'noauth', utils.add_auth_middleware(app) will be executed in main.py?14:32
devanandadtantsur: heh, tiny nit - there's no punctuation14:33
devanandaAttempting to get exclusive lock on node 34184b36-bae7-47a1-91a0-75dcdd7ec28e for power state sync __init__ /opt/source/openstack/ironic/ironic/conductor/task_man14:34
sambettsyuikotakada: No, if CONF.authenticate = False then get_auth_strategy should return 'noauth' and therefore add_auth_middleware(app) will be skipped14:35
*** e0ne has joined #openstack-ironic14:36
TheJuliabcornec: Interesting... seems like starting with a new base, I suspect you've been tracking with the same image from the earlier bifrost code, maybe nuking /opt/stack/ansible?  But if your rebuilding with 14.04, I suspect it will largely disappear except for the mkdir I think14:36
dtantsurdevananda, you mean dot in the end? because everything starting with __init__ is not my fault :)14:36
NobodyCamgood (UGT) morning sambetts14:36
TheJuliabnemec: we've also put a number of fixes in, you may wish to refresh your master branch and rebase your docker branch when you have a little time :)14:37
devanandadtantsur: oslo debug logging always appends method name and file name14:37
*** athomas has joined #openstack-ironic14:37
dtantsuryuikotakada, probably job is not ready to run yet. I've rechecked to figure out14:37
devanandadtantsur: so yea, we need some separator there. since you've got (...) in the other two messages, wdyt about using paren like this: (for $purpose)14:37
devanandadtantsur: so that all three logs have "some words  (...) {func} {file}"14:38
huatsdevananda: yesterday I was asking about how to use separate scheduler filters (some dedicated to vm and some dedicated to baremetal) and you told me to use host aggregate. But I clearly don't see how... I have been searching and I fear it is not possible :(14:38
huatsdo you have anythng precise in mind ?14:38
dtantsurdevananda, why not, will do14:38
devanandadtantsur: also, this is really really a lot of logs now. I think these are very helpful debug logs, but it might be good to use a CONF flag to disable by default?14:38
huatsactually I just want to be able to use at the same time ironic and kvm for vm on the same openatck14:38
devanandadtantsur: that ^ is a longer discussion about cleaning up our logging in general -- not related to your patch14:39
dtantsurdevananda, it's actually let then before, when we had 5-10 attempts... I believe that's purpose of DEBUG - to show everything14:39
dtantsurdevananda, re our logs, I think we should make INFO useful, not strip DEBUG14:40
devanandadtantsur: ++14:40
devanandahuats: http://docs.openstack.org/kilo/config-reference/content/section_compute-scheduler.html#host-aggregates14:42
openstackgerritDmitry Tantsur proposed openstack/ironic: Make task_manager logging more helpful  https://review.openstack.org/19461914:42
dtantsurdevananda, ^^^ with (for purpose)14:43
*** mitchjameson has joined #openstack-ironic14:43
devanandaeep. fat fingered a +A on that14:43
yuikotakadasambetts, Ah! CONF.authenticate is boolean! :D I misunderstood it is 'keystone' or 'noauth' same as auth_strategy. Thanks a lot!14:43
NobodyCamTheJulia: got a sec for https://review.openstack.org/#/c/19489314:44
sambettsyuikotakada: :D hehe yup14:44
dtantsurdevananda, lol, Alt+3 instead of Alt+2? ;)14:44
yuikotakadadtantsur, Can I believe that it will work well someday? I'm worrying about I wrote job wrongly.14:45
* dtantsur is checking zuul14:46
yuikotakadasambetts, I will read your patch again carefully and write reply (maybe within 1h), so please wait for a while...14:46
dtantsuryuikotakada, the job is queued in zuul this time, so at least it's going to run :)14:47
sambettsyuikotakada: Thanks :D14:47
bcornecTheJulia: Going further with 14.04 !! Now I have an error with14:47
bcornecTASK: [ironic-install | RabbitMQ - Fixing /etc/hosts] *************************14:47
bcornecfailed: [localhost] => {"changed": true, "cmd": ["sed", "-i", "s/localhost/d3659c5e113e localhost/", "/etc/hosts"], "delta": "0:00:00.096540", "end": "2015-06-25 14:45:04.390499", "rc": 4, "start": "2015-06-25 14:45:04.293959", "warnings": ["Consider using template or lineinfile module rather than running sed"]}14:47
bcornecstderr: sed: cannot rename /etc/sedyP8Egx: Device or resource busy14:47
bcornecMore logs at http://paste.openstack.org/show/320990/14:48
yuikotakadadtantsur, could you please teach me how to check zool?14:48
dtantsuryuikotakada, go to http://status.openstack.org/zuul/ and enter your change number (see in gerrit URL, 174231 in this case)14:49
dtantsuryuikotakada, when you see your patch, click on it to expand14:49
TheJuliabcornec: wow... that.... just... is no fun :(14:49
huatsdevananda: hum. This is how aggregates are created. But the thing is that the filters are shared accross all you aggregates AFAIK so you cannot have some specific filter list to some hostaggregate14:49
BadCubdevananda: I have added the date confirmation for mid-cycle to meeting14:49
bcornecTheJulia: why no fun ? If it was working out of the box it would not have been fun ;-)14:50
TheJuliabcornec: Due to rabbitmq, the node hostname has to be resolvable with localhost *facepalm*, thats what that change does14:50
JayFBadCub: is the date finalized? E14:50
bcornecNow I can learn something and you can fix something hopefully !14:50
BadCubJayF: yep. We got confirmation from HP Facilities for Aug 12-1414:50
NobodyCammorning JayF :)14:51
*** mitchjameson has quit IRC14:51
JayFmorning14:51
NobodyCam:)14:51
yuikotakadadtantsur, I see. Thank you.14:52
dtantsurmorning JayF, BadCub14:52
JayFmorning14:52
BadCubmorning dtantsur14:52
*** david-ly_ is now known as david-lyle15:01
*** mtanino has joined #openstack-ironic15:05
*** martini has quit IRC15:12
*** mtanino_ has joined #openstack-ironic15:15
*** mtanino has quit IRC15:16
sambettsThanks for the review yuikotakada :D15:20
*** ijw_ has joined #openstack-ironic15:21
yuikotakadasambetts, you're welcome :)15:21
rlooHi BadCub, wrt the midcycle. If it is definitely "on", maybe send email about it so people can start planning for it sooner.15:24
*** hblixt has quit IRC15:27
*** mtanino has joined #openstack-ironic15:28
rloohi dtantsur, devananda: wrt is_hostname_safe(). https://review.openstack.org/#/c/193587/15:30
rloodtantsur, devananda: I guess 'we' should decide once and for all. is this something to bring up at the meeting?15:31
*** mtanino_ has quit IRC15:31
rloodtantsur, devananda: or at least decide for the near-future :)15:31
dtantsurI'm just against breaking changes without serious reason. I don't care about names too much, though having underscore would be good IMO15:31
JayFLike I get what Chris is saying though. We're in the worst of all worlds right now15:32
JayFwe don't restrict to the standard properly, and we don't allow whatever folks want15:32
JayFwe've invented a new standard15:32
rloodtantsur: i looked at the past changes to that. The underscore was added for 'domain' part, not hostname part.15:32
openstackgerritJulia Kreger proposed openstack/bifrost: WIP: Test shade microversion lock ***DO NOT MERGE***  https://review.openstack.org/19563715:32
JayFwhether or not it's worth breaking changes to fix is still the question; but I do think the way it is may not be awesome15:33
dtantsurrelaxing rules is not a breaking change, so I would go for that one :)15:33
rlooJayF: ++15:33
devanandarloo: that's the plan (sending an email today)15:33
rloothx devananda15:33
dtantsurcool!15:33
* rloo wonders if we need to spend more time reading specs in detail but i don't know that it would help much and might cause more frustration.15:34
dtantsurrloo, are you following [Nova] discussion about dropping specs? :)15:35
dtantsurseems like it's easy to overdo15:35
rloodtantsur: no, didn't know they were having a discussion. but i'm behind in my emails too.15:35
*** e0ne is now known as e0ne_15:35
rloodtantsur: the whole 'ungrammatical' way of "writing' things/projects in OpenStack has me a bit depressed15:36
dtantsurme too15:36
dtantsurnot only is it ungrammatical, it's also confusing a lot15:36
rloodtantsur: http://lists.openstack.org/pipermail/openstack-docs/2015-June/007020.html15:37
dtantsurimagine user seeing "ironic inspector" in the middle of a sentence. how are they supposed to understand that this is a project name15:37
*** mark-os has quit IRC15:37
rloodtantsur: some good comments there but i don't know how/who decides what there.15:37
BadCubrloo: I will do that today15:37
*** e0ne_ is now known as e0ne15:37
rloothx BadCub15:37
dtantsurrloo, yeah, we're not the only confused15:38
*** mtanino has quit IRC15:40
openstackgerritVladyslav Drok proposed openstack/ironic: Update cached images based on update time  https://review.openstack.org/19501715:43
BadCubrloo: email sent :)15:45
*** yog_ has quit IRC15:46
lucasagomesrloo, heh luckily I got my pop corn to watch the comments on that patch changing "Bare metal" to "Bare Metal"15:50
lucasagomesI remember when we were discussing whether the first line of the commit message should end with a period or not15:50
lucasagomesat end we allowed both :-)15:50
rloolucasagomes: when you originally said you were getting popcorn (was it only a few days ago), I thought, what do you mean? Now I know :-(15:51
lucasagomesrloo, heh yeah watching the show15:51
NobodyCammorning rloo :)15:52
rloomorning NobodyCam!15:52
NobodyCam:)15:52
rloolucasagomes: I added a bit more comedy to it ;)15:52
lucasagomeshah will check15:53
lucasagomesI love the gif http://i.imgur.com/deAYzZe.gif15:53
rloolucasagomes: ha ha15:53
NobodyCam:-p15:53
*** yuikotakada has quit IRC15:54
lucasagomesthe cool thing is that people already stopped talking about whether it Bare Metal or Bare metal, they are talking about whether the nick name of the projects (Ironic) is a proper noun or not15:54
lucasagomesand so it should be capitalized15:54
rloodtantsur: we need a spec for any change to a driver interface? or only certain kinds of changes? https://review.openstack.org/#/c/188466/15:54
jlvillalGood morning Ironic.  Four more work days until my (4 week) sabbatical :)15:54
lucasagomesjlvillal, w00t!15:55
rloolucasagomes: what I don't understand (and doesn't seem to be documented) is where they came up with those 'conventions' in the first place.15:55
rloojlvillal: stop making us jealous! :)15:55
lucasagomesrloo, well it's probably around for quite while, probably the docs team does make those conventions15:55
jlvillallucasagomes, rloo: Thanks.  But I have this feeling I will still come here to hang out at times :)15:56
lucasagomesjlvillal, you welcome, bring beer15:56
jlvillallucasagomes: I will, if I can figure out what the beer emoticon is15:56
*** e0ne is now known as e0ne_15:56
rloojlvillal: you're always welcome ;)15:56
jlvillalThanks rloo!15:57
rloojlvillal: btw, before you take off, what's the status with testing? maybe you can update the etherpad.15:57
NobodyCammorning jlvillal15:57
NobodyCamjlvillal: 4 weeks of sabbatical wow15:58
dtantsurrloo, my intuition says yes. driver interface is a stable one in Ironic15:58
dtantsurat least we maintain it as such15:58
dtantsurso I'd like more visibility on it's changes, especially when we're in risk of breaking compatibility15:58
rloodtantsur: so any change to an existing method there, or adding a new one, requires a spec?15:58
JayFjlvillal: I'm counting down too; something like 3 weeks until my 1m sabbatical15:58
dtantsurrloo, I believe so (others might disagree)15:58
*** kozhukalov_ has joined #openstack-ironic15:59
rloodtantsur: I do agree that visibility is important.15:59
jlvillalrloo: Okay.  I ended up spending far too much time trying (and failing) to get the Ubuntu Cloud image to work to run devstack.  I want to spend more time trying to figure out the issue why the scheduler says it can't schedule.15:59
jlvillalJayF: Congrats! :)15:59
rloodtantsur: I want to make sure we're all 'in agreement' so we are consistent. I sometimes feel sorry for our developers.15:59
dtantsuryeah, it's annoying, I agree16:00
dtantsurif we had more spec cores, we could move faster16:00
dtantsurrloo, e.g. Ruby Loo does wonderful reviews, do you think we could convince here to become a specs-core? ;)16:01
rloodtantsur: oh, I was thinking more like 'all spec cores' need to review :D16:01
dtantsur:)16:02
*** Sukhdev has joined #openstack-ironic16:03
rloodtantsur: the official name is 'Bare metal inspector'16:03
TheJuliaJayF: Have fun!  Will you be visiting NC?16:03
JayFTheJulia: nope; 1 week in Italy (Rome, then Venice); some time at home; and 5 days in Chicago16:04
dtantsurrloo, it sounds reasonable, but it's not official16:04
TheJuliaJayF: Have fun!16:04
JayFTheJulia: there's any chance at all I'd go to NC to surprise family/friends, but I doubt I'll wanna travel that much as I'll also be away from hoem the week before and after my sabbatical16:04
TheJuliaJayF: don't blame you there at all16:05
*** e0ne_ has quit IRC16:06
rloodtantsur: it would be nice, but not sure it is that reasonable :)16:08
JayFmorganfainberg: asking about the new policy.json framework in the v3 middleware  - would policys configured using the new middleware still apply for folks using keystone v2?16:08
JayFs/ys/ies/16:09
morganfainbergV3 uses policy.json, v2 is really "is an admin", "is not an admin" or "no access"16:09
morganfainbergNot much beyond that.16:09
morganfainbergAnd mostly hard-coded-ish16:10
morganfainbergIn keystone that is.16:10
morganfainbergSo we'd implement new things on v3 only16:10
morganfainbergV2 is froZen from a keystone perspective.16:10
JayFmorganfainberg: so if I have a keystone v2 server, and using keystone v3 middleware to implment policies16:10
JayFwill I be able to use those policies?16:10
morganfainbergSo v2 and v3 keystone only Impact auth (minimally) and keystone crud16:11
*** e0ne has joined #openstack-ironic16:11
devanandamorganfainberg: context here is JayF is interested in making security things around password access/storage/change-mgmt better, and I think we need to have policy in place before it makes any sense to start adding crypto features that we want to restrict access to16:11
morganfainbergAs a non Keystone Service you shouldn't care which one is used. You would need to have middleware source the policy stuff from the v3 interface.16:11
morganfainbergBut that should be done by our middleware so less of something you have to directly implement.16:12
morganfainbergdevananda: sure.16:12
*** greghaynes has joined #openstack-ironic16:12
morganfainbergJayF: I recommend focusing on v3. In (every?) deployment that is adhering to keystone and defcore, v3 is available.16:13
morganfainbergEven if they use v2 primarily.16:13
morganfainbergFor auth/crud16:13
JayFmorganfainberg: well, I'm at Rackspace; so I can't change my "keystone" very easily16:13
JayFmorganfainberg: mainly curious if work to implement policies in a "v3" way would lend me no direct value until that identity service was upgraded16:13
morganfainbergJayF: I'll just say that any new apis in keystone will be v3 only.16:14
JayFmorganfainberg: and it sounds like the answer is yes?16:14
morganfainbergSo. Again, you don't have a lot from ironic to deal with from a v2 or v3 difference. Unless you are asking keystone for crud things. (Or care about domains)16:15
morganfainbergYou get the same roles, scope, etc data from the middleware.16:15
devanandamorganfainberg: we dont have a lot -today-. we're discussing adding more16:15
morganfainbergV2 vs v3 in keystone should have no impact on ironic unless you cared about hierarchical multi tenancy or domains16:16
devanandamorganfainberg: and I think JayF's question is, if we add the sorts of things we're talking about, using v3 APIs and the new keystonemiddleware, will his deployment be able to derive any benefit?16:16
devanandaahh16:16
devanandaafaik, we do not16:16
JayFsweet16:16
morganfainbergdevananda: unless you are doing crud work and interacting with keystone's crud interface (heat)16:16
morganfainbergIt wouldn't matter much.16:17
morganfainbergI recommend assuming v3 is used if there is ever a question, but it shouldn't matter to you beyond how keystone middleware is configured ;)16:17
devanandathink: create three "roles" with different "access controls" implemented for business reasons16:17
morganfainbergIf ironic is trying to create roles in keystone vs operator leveraging roles. Then we can talk.16:18
morganfainbergAnd it's more interesting (use v3 :P)16:18
devanandasomeone's gotta CRUD those roles, but I imagine that ironic is just checking them from a middleware in our API layer16:18
morganfainbergYah. You'll know if the user has those roles based on keystone middleware decoding the token.16:18
devanandaor keystone is checking them via the keystonemiddleware, and maybe ironic doesn't know anything about them16:19
devanandaright16:19
devanandaso we add keystonemiddleware as a  Pecan hook16:19
morganfainbergYep. We will just pass on "roles are [x, y, z]16:19
devanandaif there are no roles, it just passes through16:19
morganfainbergShould already be done when you use auth_token16:19
morganfainbergIt gets added to the context and / env cars from __call__16:20
devanandaif there are roles (and an auth_toekn) then keystonemiddleware checks them -- this actually doesn't require any changes to ironic's logic, beyond just adding the keystonemiddleware hook16:20
devanandaand the operator creating those roles and setting the policy16:20
*** subscope has quit IRC16:20
morganfainbergYou already have the hook. Afaik16:20
devanandamorganfainberg: how does said operator declare what Roles have what CRUD acces to which of Ironic's API resources?16:20
morganfainbergIt is handed in the auth_protocol wrapper you have.16:20
morganfainbergThat is done via policy.json (today)16:21
morganfainbergAnd the enforcers used that compare the rule to the auth context.16:21
devanandaJayF: sounds like there may not be much work to do here16:21
morganfainbergSo you pass the context to the enforcer which has loaded the rules from policy.json, and then you say "enforce on rule/api X"16:21
JayFdevananda: just basically hooking up roles to endpoints16:22
devanandayup16:22
JayFdevananda: sounds like 90% design 10% implementation16:22
JayFmariojv: around?16:22
morganfainbergMost projects use Oslo.policy already and you can just use them as an example.16:22
lucasagomesare we thinking about exposing the ironic api to the end user?16:22
lucasagomes(non operator)16:22
morganfainbergNova has the more complex uses where they enforce on ownership of a resource from the dB as well16:22
JayFlucasagomes: well; "operator" is a broad term16:23
devanandalucasagomes: not really -- more like different types of operator16:23
JayFlucasagomes: at Rackspace I might have different "level" of operators I want to have different access16:23
lucasagomesdevananda, JayF right, that sounds pretty neat16:23
devanandathat ^ is a request i've seen from several large customers, too16:23
JayFlucasagomes: think also about the ability to have dashboard users which can't see credentials16:23
JayFwell; I think we have that policy already16:23
JayFbut more things like that+++16:23
devanandaJayF: it's a CONF option, not a policy, which is bad IMO16:24
devanandabut yes, more things like that ++16:24
lucasagomes++16:24
lucasagomessounds good indeed16:24
JayFdevananda: I think we have it in policy as well?16:24
morganfainbergdevananda, JayF: so you just need to add in Oslo.plolicy and make sure the auth token middleware is properly adding to your request context, then just do enforce where you want to enforce.16:24
JayFdevananda: you're going to make me wanna check to see if this is downstream-only now, lol16:24
devanandaJayF: I believe some of those things are downstream, yea16:25
morganfainbergJayF: if it is part of your "public" offering it likely uses repose. Which isn't compatible with the upstream policy.json stuff16:25
JayFdevananda: https://github.com/openstack/ironic/blob/master/etc/ironic/policy.json#L316:25
morganfainbergRepose acts before policy.json16:25
JayFdevananda: I think the show_password role is what matters there, but imbw16:25
JayFmorganfainberg: we don't front ironic with repose16:25
morganfainbergOk cool16:26
morganfainbergThat makes it easier.16:26
*** stegranet has quit IRC16:26
JayFI never put java in my production if given a choice :P16:26
morganfainbergLol16:27
openstackgerritMerged openstack/bifrost: Add the rest of the cookiecutter-generated files for testing  https://review.openstack.org/19489316:27
openstackgerritMerged openstack/bifrost: Add a dynamic prepare for test role  https://review.openstack.org/19200116:27
devanandaJayF: ah - you may be correct16:27
openstackgerritMerged openstack/bifrost: Add README note about cleaning  https://review.openstack.org/19478316:28
mariojvJayF: yeah, stepped away for a minute16:30
lucasagomesdtantsur, ping any reason why not +a https://review.openstack.org/#/c/194132/ ?16:31
*** achanda has joined #openstack-ironic16:31
JayFmariojv: if you read scrollback, Deva + me + Morgan (PTL of Keystone) were talking about policy stuff. I think that might be a good quick win instead of the barbican integration you're working on now.16:31
dtantsurlucasagomes, was assuming devananda might want to have a look16:31
JayFmariojv: WDYT? (once you get more context)16:31
mariojvlooking16:31
lucasagomesdtantsur, fair16:31
lucasagomesdevananda, yeah please, if you have 1 min to take a look at https://review.openstack.org/#/c/194132/16:32
* devananda looks16:32
lucasagomesit mitigates the problem of having to go to the database and unstuck the nodes in DEPLOYING16:32
lucasagomesdevananda, on top of that, a timeout will be added16:32
JayFlucasagomes: we were talking some yesterday about possibly allowing a transition from almost any state into error for similar reasons16:33
lucasagomesJayF, yeah we def should do something16:34
JayFlucasagomes: in our case we have some things getting stuck cleaning (still not found a root cause) and right now it takes a DB update16:34
*** romainh has left #openstack-ironic16:34
lucasagomesJayF, actually our ERROR state can leave us stuck too16:34
JayFand everytime someone has to touch a DB to fix production software I die a little inside :P16:34
devanandaJayF: I've hit that tuck-in-cleaning seeral times too16:34
lucasagomesJayF, because, from ERROR we only can move to DEPLOYING (rebuild) or DELETING (delete)16:34
JayFlucasagomes: o, really? even without locks?16:34
lucasagomesJayF, yup16:35
JayFlucasagomes: hrmph16:35
lucasagomesJayF, if something fails there you get in a loop where you can't get out of there16:35
devanandalucasagomes: huh? error -> deleting should be ok. why is that not?16:35
lucasagomesdevananda, we had a case where ipmi commands where failing, tear_down was trying to power off the node16:35
lucasagomeserroring out and sending it back to error16:35
*** romainh has joined #openstack-ironic16:36
lucasagomesbut problem was that (IIRC) it was in provision_state = ERROR and target_provision_state = DEPLOYED16:37
lucasagomesso any updates to the node was denied because it was "in transition"16:37
lucasagomesthere's something there, I've a TODO to look into it a bit more and investigate16:37
devanandaahh :(16:37
devanandaso yea, you may not have been able to UPDATE the ipmi creds16:37
devanandathat's silly of us16:37
lucasagomesyeah exactly16:37
*** romainh has quit IRC16:38
openstackgerritMerged openstack/bifrost: Add os_ironic_facts module  https://review.openstack.org/19109316:38
lucasagomesI will start looking into some of those stuck problems16:38
openstackgerritMerged openstack/bifrost: Correct license header on zuul change parsing code  https://review.openstack.org/19507316:38
lucasagomesdevananda, like having timeouts etc... that patch for DEPLOYING is just to mitigate the problem, since at that point we don't have any API that allows us to get out of it16:39
openstackgerritMerged openstack/bifrost: Addition of shade support to bifrost_inventory.py  https://review.openstack.org/19468416:39
lucasagomesdevananda, also, there's something that may require some thought... In nova it's allowed to do a destroy() when the instance is "spawning"16:40
lucasagomesbasically you could abort the deployment, which is not allowed in Ironic16:40
*** derekh has quit IRC16:40
*** romainh has joined #openstack-ironic16:40
lucasagomes(unless you're at DEPLOYWAIT)16:40
mariojvJayF: using keystone here sounds like a good idea b/c we do need to have roles defined for accessing the secrets before we even worry about how they're securely stored16:41
JayFmariojv: well basically what we have to do is come up with more "roles" for Ironic and assign endpoints/actions to the roles16:41
JayFmariojv: so we could make a user that could, say, only have READ access to insecure information in the API16:41
JayFmariojv: vs an Admin who can CRUD any endpoitn16:41
JayFmariojv: the pieces are all in place (that's what the scrollback was figuring out)16:42
mariojvright, that sounds like a good idea16:42
JayFmariojv: honestly the biggest amount of work here is likely getting everyone to agree on roles for the policies :)16:42
mariojvlol16:42
devanandalucasagomes: on 194132, i think there's an edge case here16:42
*** ndipanov has quit IRC16:42
devanandalucasagomes: if I am running 2 conductors (A and B) and I kill -9 both of them, then only start up conductor A16:42
devanandalucasagomes: self.dbapi.clear_node_reservations_for_conductor(self.host) gets called for that host (A)16:43
devanandalucasagomes: so half the nodes fail16:43
devananda*half hte nodes that were in DEPLOYING move to DEPLOYFAIL16:43
lucasagomesright16:44
devanandalucasagomes: if i do not start conductor B, we still hvae no way to recover the other half of the nodes w/o a DB update16:44
lucasagomesyeah the other half will still locked16:44
lucasagomesthat will be solved with a timeout16:44
devanandaright16:44
lucasagomesso conductor A may check the timeout (when added)16:44
lucasagomesand will unbreak those nodes16:44
devanandawhat if I dont restart either A or B, but instead I start a new conductor C ?16:45
devanandanone of the deploys are failed16:45
lucasagomesthen nodes still stuck in DEPLOYING state16:45
mariojvJayF: do you think that addresses the issue of driver pw storage / retrieval though? i see how it's a good idea but don't understand how that would help us if we needed to do something like upgrade BMC credentials across part of a fleet16:45
devanandathis is not solving the root of the problem -- it's making an assumption that the operators will restart conductors with the same hostname16:45
lucasagomesdevananda, yeah the final solution is the timeout16:45
JayFmariojv: it doesn't address the same issue; but it addresses a similar issue that probably should be handled first16:45
lucasagomesdevananda, yes, that's the assumption. It just mitigates because right now16:45
JayFmariojv: if we store the info not awesomely it's not quite as bad if we can at least isolate some api users from it16:46
mariojvi see16:46
lucasagomesdevananda, if the operator starts a conductor with the same hostname what happens is that the nodes are in DEPLOYING state without lock16:46
lucasagomesdevananda, and you just can't do anything there unless you change the db16:46
devanandamariojv: and in order to benefit from storing the passwords in a better way, you also need to control who has access to them (or else why did you secure them if anyone can access them?)16:46
lucasagomesdevananda, at least with that patch, I can tell one to "restart the conductor and it will clean up the nodes it was managing"16:47
lucasagomesthe timeout is on the way btw, that will solve the broader case16:47
devanandalucasagomes: right - so I agree this mitigates one issue today, but it's very limited. the real solution (timeouts) would actually solve both sets of problems16:47
devanandayea16:47
lucasagomesyeah it will16:47
lucasagomesthis just fails fast and fails safe for that case16:48
lucasagomeswhile not perfect, it helps16:48
devanandalucasagomes: mind if I edit the commit message to word that a bit more clearly?16:48
lucasagomesdevananda, go ahead16:48
devanandaI agree it helps, just want it to be clear what it is doing16:48
devanandathx16:48
mariojvdevananda: that makes sense, we ought to define some sort of rbac before jumping ahead and storing all passwords in one place, i think what you were discussing with policies + keystone will do that16:49
*** ijw_ has quit IRC16:49
devanandalucasagomes: one more thing -- this isn't actually FAILing the deploys from the conductor that restarted16:50
devanandalucasagomes: it's FAILing any node that was in DEPLOYING state and had no lock16:50
devanandalucasagomes: while the end result is probably the same, there's still a race if >1 conductor restarts at the same time16:50
lucasagomesyeah, but we only clean the locks for the nodes it was managing16:50
devanandamariojv: exactly16:51
JayFlucasagomes: I think a node being deployed by agent, it's not locked 100% of the time it's in DEPLOYING state16:51
lucasagomesJayF, yeah nodes are not locked on DEPLOYWAIT state16:51
lucasagomesdevananda, perhaps I should handle InvalidTransitionState there?16:52
*** pelix has quit IRC16:52
*** rwsu has joined #openstack-ironic16:52
lucasagomesInvalidState*16:52
openstackgerritMerged openstack/bifrost: Remove un-necessary checks from the dynamic test  https://review.openstack.org/19368716:52
openstackgerritMerged openstack/bifrost: Add dynamic node unprovision role  https://review.openstack.org/19368816:52
openstackgerritMerged openstack/bifrost: Correct ipv4 networking config issue in dynamic path  https://review.openstack.org/19368916:52
openstackgerritMerged openstack/bifrost: Add example deploy all available playbook  https://review.openstack.org/19510016:53
openstackgerritMerged openstack/bifrost: Follow-up fix for H306 in bifrost_inventory.py  https://review.openstack.org/19532416:53
*** romcheg has quit IRC16:55
*** romainh has left #openstack-ironic16:56
*** athomas has quit IRC16:57
*** trown is now known as trown|lunch16:58
openstackgerritDevananda van der Veen proposed openstack/ironic: Clean nodes stuck in DEPLOYING state when ir-cond restarts  https://review.openstack.org/19413216:59
devanandalucasagomes: let me know what you think of wording ^16:59
devanandalucasagomes: if good, i'll +A it16:59
*** igordcard_ has joined #openstack-ironic17:01
* lucasagomes looks17:01
devanandaugh. one sec. formatting got mangled17:01
lucasagomeshhe yeah17:01
openstackgerritDevananda van der Veen proposed openstack/ironic: Clean nodes stuck in DEPLOYING state when ir-cond restarts  https://review.openstack.org/19413217:03
devanandathat'll teach me to use gerrit's web ui17:03
lucasagomesdevananda, just one thing, we also call clean_up and stuff ( see the  callback_method=utils.cleanup_after_timeout)17:03
lucasagomesdevananda, that will make the node not stay in DEPLOYFAIL, it will actually clean it up17:03
lucasagomesand put it back to available17:03
lucasagomesso it will be read for scheduling again17:03
lucasagomesready*17:03
devanandaah, right17:03
lucasagomeswhich is neat (IMO)17:03
devanandaoooh17:04
devanandalucasagomes: so, a) what happens if it hasn't exceeded the timeout? b) what does that do to Nova?17:04
lucasagomesdevananda, we currently don't have the timeout, but once we do since the node will leave the DEPLOYING state17:05
lucasagomesthe loop that checks the timeout won't filter that node anymore17:05
lucasagomesdevananda, the instance in nova will go to ERROR when spawning, so you can delete it17:05
*** hemna_ has joined #openstack-ironic17:06
devanandalucasagomes: so this is factually incorrect17:06
devanandalucasagomes: I can start a deploy, then restart conductor, and within a few seconds of starting the deploy it will log: Timeout reached while waiting for callback for node %s17:06
*** jefabo has quit IRC17:06
devanandaand set node.last-error to that17:07
devanandasee ironic.conductor.utils.cleanup_after_timeout17:07
* lucasagomes looks17:07
devanandawe should not call that from conductor.startup17:07
*** jefabo has joined #openstack-ironic17:07
dtantsurg'night folks17:07
*** dtantsur is now known as dtantsur|afk17:07
NobodyCamnight dtantsur|afk17:07
lucasagomesdevananda, oh yeah hmm the log will be misleading17:08
lucasagomesdevananda, yeah perhaps we should leave it in DEPLOYFAIL anyway17:08
*** achanda has quit IRC17:08
*** e0ne has quit IRC17:08
*** ukalifon1 has quit IRC17:09
*** hemna_ has quit IRC17:09
*** Marga_ has quit IRC17:09
devanandalucasagomes: i think that's better. comment posted17:09
lucasagomesdevananda, ack17:10
*** Marga_ has joined #openstack-ironic17:10
devanandatime to move coffee shops -- bbi15min17:10
lucasagomesdevananda, thanks for looking into it, yeah good discussion. I will see if I can get the timeout soon too17:10
devanandalucasagomes: cheers17:10
lucasagomesdevananda, enjoy17:11
lucasagomesI will call it a day as well17:11
lucasagomeshave a good night everyone!17:11
*** lucasagomes is now known as lucas-dinner17:12
bcornecTheJulia: well, I think bifrost is doing the right stuff by using sed to modify /etc/hosts. THe problem I think is that /etc/hosts is generated by docker, so I think that's why we get the error stderr: sed: cannot rename /etc/sedyP8Egx: Device or resource busy. So I'd need to find a workaround to that....17:12
*** hblixt has joined #openstack-ironic17:14
*** blixt_ has joined #openstack-ironic17:14
*** ijw_ has joined #openstack-ironic17:15
TheJuliabcornec: i guess people who have run rabbit in docker might know how to get it to run, and if we can detect it is in a container, or even just use a setting, it would be easy to support17:15
*** hblixt has quit IRC17:18
*** romcheg has joined #openstack-ironic17:19
NobodyCamnight lucas-dinner17:19
*** sambetts has quit IRC17:21
*** sambetts has joined #openstack-ironic17:24
*** achanda has joined #openstack-ironic17:25
*** bradjones has quit IRC17:26
*** bradjones has joined #openstack-ironic17:28
*** bradjones has quit IRC17:28
*** bradjones has joined #openstack-ironic17:28
*** blixt__ has joined #openstack-ironic17:33
*** ukalifon1 has joined #openstack-ironic17:34
*** blixt_ has quit IRC17:36
*** blixt__ has quit IRC17:37
*** sambetts has quit IRC17:38
*** sambetts has joined #openstack-ironic17:40
*** Marga_ has quit IRC17:42
*** dguerri is now known as dguerri`17:43
openstackgerritChris Krelle proposed openstack/bifrost: Fix white space in readme file.  https://review.openstack.org/19568717:52
NobodyCamTheJulia: ^^^^^17:52
NobodyCamsuper minor17:52
TheJuliaThank you!17:53
*** boris-42 has joined #openstack-ironic17:54
rlooNobodyCam: my first bifrost review :)17:54
*** Marga_ has joined #openstack-ironic17:55
*** mitchjameson has joined #openstack-ironic17:58
*** hblixt has joined #openstack-ironic18:00
NobodyCamrloo: w00t!!!18:01
NobodyCambiFrost needs more reviewers :)18:01
*** husanu8 has joined #openstack-ironic18:02
rlooTheJulia: you can +A that (we are good with only one +2 for typos in non-code)18:02
*** dontalton has joined #openstack-ironic18:02
TheJuliaMerci18:02
*** blixt_ has joined #openstack-ironic18:02
*** Sukhdev has quit IRC18:03
*** hblixt has quit IRC18:05
*** Sukhdev has joined #openstack-ironic18:07
*** husanu8 has quit IRC18:14
bcornecTheJulia: I think I found why I have that issue: https://github.com/travis-ci/travis-build/pull/40318:18
*** lsmola has quit IRC18:19
rlooTheJulia: I updated our wiki with that one +2 thing: https://wiki.openstack.org/wiki/Ironic/CoreTeam#Other_notes18:19
rlooothers -- let me know if I misunderstood18:19
NobodyCamrloo: great update Thank you :)18:21
rlooNobodyCam: yw. I need to remember to keep our docs updated.18:21
NobodyCamrloo: we should remember to do that :) *BLUSH*18:22
NobodyCams/we/we all/18:22
rlooNobodyCam: no time for everything and my memory is so bad ;)18:23
rlooNobodyCam: yeah, all of us!18:23
NobodyCam+++18:23
jlvillalBadCub: Do you have an approximate location for the sprint in August in Seattle?  For trying to figure out lodging.18:23
NobodyCam701 pike street18:23
jlvillalNobodyCam: Thanks :)18:24
TheJuliabcornec: looking18:24
rlooBadCub: were you going to look into getting some discount rate? (or maybe it was some other project)18:24
BadCubforgot to put that in the email :(18:24
BadCub rloo I can check into that, but most hotels want a minimum booking commit for group rates.18:25
NobodyCamI think we have like the most of if not the entire 24th floor18:25
*** trown|lunch is now known as trown18:25
rlooBadCub: oh. Don't you have an hp rate that folks can use ;) (I'm not going, just asking, so feel free to ignore me)18:26
TheJuliabcornec: I guess... in a sense, that does make sense.  Just not a fan of using pipe to overwrite instead of in-place editing. :(18:26
*** stegranet has joined #openstack-ironic18:27
NobodyCamahh :(18:27
BadCubrloo:  HP dioes have employee rates at some hotel chains, but it is an employee rate that requires an emp. badge at check-in18:27
BadCuband usually it is actually higher than BAR18:27
rlooBadCub: oh, that is disappointing. Well, I suspect most people can expense the hotels so it doesn't really matter.18:28
BadCubrloo: indeed. Anyone coming to a summit should be expensing travel18:29
*** jhova has joined #openstack-ironic18:29
rlooBadCub: I believe there are OpenStack contributors that don't work for any company, but I don't think we have any such folks in ironicland.18:30
devananda* 21st floor18:31
BadCubrloo: I believe that to be the case as well. If there are any, I might suggest they call the Sheraton and see if there is a "HP Visitor" rate code.18:31
*** blixt__ has joined #openstack-ironic18:31
*** krtaylor has quit IRC18:31
*** blixt_ has quit IRC18:35
*** hblixt has joined #openstack-ironic18:38
devanandaJayF: take a look at gov patch 191892 when you have a sec -- it's proposing the tag:service label to IPA, which I htink is misinformed18:38
devanandaJayF: I will comment as such if you agree18:38
*** blixt__ has quit IRC18:40
openstackgerritVladyslav Drok proposed openstack/ironic: Update cached images based on update time  https://review.openstack.org/19501718:40
JayFdevananda: I'm in agreement18:41
devanandaack18:41
JayFdevananda: definately is not a long running REST API18:41
devanandacomment posted18:43
*** dontalton has quit IRC18:44
jlvillalBadCub: Darn, 6000 attendees in town for the ASA 2015 Joint Statistical Meetings http://www.wscc.com/upcoming-events?field_event_start_date_value_1[value][month]=8&field_event_start_date_value_1[value][year]=201518:46
BadCubWOW18:46
* jlvillal is not sure what the "ASA 2015 Joint Statistical Meetings" is ...18:46
* BadCub is not either18:46
jlvillalBadCub: Statisticians it seems18:47
BadCubjlvillal: lol. it appears so18:47
openstackgerritVladyslav Drok proposed openstack/ironic: Update cached images based on update time  https://review.openstack.org/19501718:52
*** rbrooker has joined #openstack-ironic18:52
*** r-daneel has quit IRC18:54
*** r-daneel has joined #openstack-ironic18:55
*** dguerri` is now known as dguerri18:55
*** zz_natorious is now known as natorious18:58
bcornecTheJulia: I have a workaround. I can add a line to the /etc/hosts file before running bifrost, thus avoiding your modification, thanks to your test, and then I sould go further...19:05
*** kozhukalov_ has quit IRC19:07
*** gzhai1 has quit IRC19:08
*** gzhai1 has joined #openstack-ironic19:09
TheJuliabcornec: good to hear :)19:09
*** achanda has quit IRC19:11
*** dprince has quit IRC19:11
*** ukalifon1 has quit IRC19:15
*** romainh has joined #openstack-ironic19:18
harlowjadtantsur|afk https://pypi.python.org/pypi/futurist19:24
harlowjayaaaa19:24
bcornecTheJulia: I'm now failing at DB user creation. Will have to look at that tomorrow. My repo is up to date if you want to try...19:28
TheJuliabcornec: I don't think I'll have time today, but I'll try to take a look in the morning.  Have a good evening!19:28
harlowjadtantsur|afk so that should enable ironic starting to use it (along with https://review.openstack.org/#/c/195711/ )19:29
harlowjabtw, dtantsur|afk  did u want to be core on that library (not sure i saw a respones to that a while ago)19:29
NobodyCamgood (UGT) morning harlowja19:31
harlowjayo yo yo19:32
NobodyCam:)19:32
harlowjahow goes things19:32
NobodyCamnot to bad today :)19:33
NobodyCamharlowja: LOL: Code from the future, delivered to you in the **now**19:33
NobodyCamluv it19:33
harlowja:)19:33
* harlowja tries to make descriptions that people will not completly say wtf to, so far most of the ones seem to slide under the rug without people noticing, lol19:34
NobodyCam:) +++19:34
rloodevananda: I hope you're ok with my comment, since this is your patch: https://review.openstack.org/#/c/147087/19:34
rloodevananda: and they might 'plagiarize' your slides since I can't given him permission.19:36
devanandarloo: all my slides are CC-BY-SA. so as long as there is attribution, it's fine19:37
devanandawithout attribution, it is plagiarism, and I will be upset19:37
rloodevananda: what does 'attribution' mean, if they are adding that to our ironic docs?19:37
rloodevananda: maybe you should just comment in that patch :)19:37
devanandathat the commit message and, if possible, the file indicate where it was copied from19:37
devanandai did :)19:37
rloodevananda: thx. always one step ahead of me :)19:38
*** natorious is now known as zz_natorious19:44
*** ijw_ has quit IRC19:45
*** zz_natorious is now known as natorious20:01
*** mtanino has joined #openstack-ironic20:01
*** natorious is now known as zz_natorious20:02
*** Sukhdev has quit IRC20:07
*** krtaylor has joined #openstack-ironic20:09
devanandahttps://www.eventbrite.com/e/openstack-ironic-sprint-august-2015-tickets-1753386225420:14
*** gabriel-bezerra has quit IRC20:17
BadCubdevananda: I added ^^ to the meeting agenda too20:19
devanandanext ironic sprint should be hosted by !hp, !rackspace, but somewhere that is warm in the middle of winter.20:19
devanandawho has offices in hawaii? :-D20:19
*** gabriel-bezerra has joined #openstack-ironic20:20
NobodyCamdevananda: plam springs is warm this time of year20:20
BadCub++ on Hawaii!20:20
devanandaNobodyCam: there's no good airport there ...20:20
NobodyCampsp20:20
BadCubPSP is an awesome airport20:20
NobodyCamis most awesome airport20:20
NobodyCamhttps://en.wikipedia.org/wiki/Palm_Springs_International_Airport20:22
NobodyCam:-p20:22
devanandaTheJulia: i tried your new dynamic playbook -- node went straight to 'cleaning' and hit the bug so it's stuck now20:22
devanandaohh20:24
* NobodyCam eats a ham sandwich20:24
TheJuliaoh... :(20:26
TheJuliadevananda: yeah.. cleaning is on by default. perhaps I should update destructions to tell the user they have to explicitly enable it20:27
devanandayea, looks like i'm hitting issues aorund cleaning more than anything else20:27
NobodyCamwe call out cleaning here: https://github.com/openstack/bifrost/tree/master/playbooks/roles/ironic-install20:28
devanandaonce it gets stuck there, it's hard to un-cleaning-it20:28
devanandaespecially since manageable->available transition now goes through cleaning, too20:28
devanandaand cleaning bugs it every time on my nuc20:28
JoshNang:/20:29
TheJulia:(20:30
NobodyCamdevananda: is it a bug in cleaning code or the NUC itself?20:31
JoshNanghow does it get stuck? like is it not getting a list of clean_steps? not starting?20:31
devanandait's at least two things contributing to it20:32
devanandamy "amttool" driver is pretty buggy -- but at least it doesn't block the GIL20:32
devanandaso htings like "reboot" sometimes fail, because AMT is not responsive at times20:32
devanandabut once it enters CLEANING state, it doesn't ever exit it (without me updating the DB)20:33
NobodyCamhummm20:34
JoshNanghrm. do you have any logs? I'll see if anything stands out. like Jay said, we hit a major bug with it, but it turned out to be something with our decom->cleaning transition20:34
devanandathink i just found it20:35
devanandafirewall20:35
NobodyCamahh that could do it20:35
JoshNangah20:35
*** ijw_ has joined #openstack-ironic20:36
devanandamaaaybe20:37
*** ijw_ has quit IRC20:41
lucas-dinnerdevananda, for the next sprint there's a red hat office in Rio de Janeiro :-)20:50
lucas-dinnerthat would be great!20:50
NobodyCamlucas-dinner: +++20:52
NobodyCamfor Rio20:52
devanandaTheJulia: I'm getting this: vagrant-ubuntu-trusty-64 dnsmasq-dhcp[1155]: no address range available for DHCP request via eth220:53
devanandaTheJulia: even though I've modified groupvars/all to reference eth2 instead of virbr0 -- something isn't getting set up correctly20:53
lucas-dinnerNobodyCam, (-: o/20:53
devanandalucas-dinner: rio! yes!20:53
lucas-dinnerI would love to, if we had a quorum I may be able to manage that20:54
lucas-dinnerit will definitely be warm20:54
devanandatravel will be the hardest thing -- if we dont get enough US/EU companie sending ppl, it won't really happen20:55
lucas-dinneryeah, we should put up to a vote just to see how it turns out20:55
lucas-dinnerwho knows20:55
devanandayah20:56
* lucas-dinner brb20:57
*** ijw_ has joined #openstack-ironic21:02
* NobodyCam jumps on a call21:02
*** e0ne has joined #openstack-ironic21:05
*** zz_natorious is now known as natorious21:24
* trown wants to visit Rio21:28
*** dguerri is now known as dguerri`21:28
* BadCub votes for Rio21:38
*** ijw_ has quit IRC21:38
*** mtanino has quit IRC21:38
*** ijw_ has joined #openstack-ironic21:38
mrdaMorning Ironic21:38
NobodyCammorning mrda21:39
hblixt:)21:39
mrdao/21:39
*** e0ne is now known as e0ne_21:39
NobodyCamI see21:40
hblixttoo easy21:40
*** e0ne_ is now known as e0ne21:41
openstackgerritMerged openstack/bifrost: Fix white space in readme file.  https://review.openstack.org/19568721:43
*** romcheg has quit IRC21:46
*** romcheg has joined #openstack-ironic21:46
*** romcheg has quit IRC21:47
*** krtaylor has quit IRC21:47
*** romainh has quit IRC21:47
*** romcheg has joined #openstack-ironic21:47
*** trown is now known as trown|outttypeww21:47
*** romcheg has quit IRC21:48
*** Pradip_ has joined #openstack-ironic21:55
*** Marga_ has quit IRC21:55
SpamapSHm21:56
SpamapSwould it be an accurate statement that approximately 90% of all ironic-conductor's will want ipmitool?21:56
SpamapSthe current packaging in Ubuntu does not even recommend ipmitool21:57
Pradip_Hi, I installed ironic with devstack. But whenever I create a new instance, it doesn't go to the active state. also wen I delete the instance it hangs in that stage. Any ideas?21:58
NobodyCamSpamapS: depending on hardware... but ya i could go with *most*21:58
SpamapSNobodyCam: recommends says most21:58
SpamapSNobodyCam: and suggests is "a lot"21:59
NobodyCamlol21:59
NobodyCamPradip_: hummm... have you checked the compute and conductor logs?21:59
Pradip_NobodyCam: No22:00
Pradip_I will do that. Also once I install the ironic, I can't create normal VM instance. It is saying that no host founf. Is it supposed to happen. i am fairly new in this thing. So I am really struggling22:01
openstackgerritStephanie Miller proposed openstack/bifrost: Disambiguate the roles of ci_testing, ci_testing_zuul etc  https://review.openstack.org/19575922:02
*** jhova has quit IRC22:03
TheJuliaPradip_: Sounds like nova-scheduler is another log you need to check22:09
Pradip_TheJulia: Ok thanks22:11
TheJuliaPradip_: for deployment on baremetal though, you can directly attempt deployment via the ironic command line if you have enough details22:12
TheJuliai.e. instance_info defined22:12
Pradip_i am doing that. But its not working. It is going upto open_callback stae22:15
Pradip_and getting stuck there. Once I issue the delete command it get stuck in deleting mode. i tried waiting for 15-20 min in both cases22:15
TheJuliais that the state nova is reporting?22:16
Pradip_nova is saying "spawning" and No state22:18
*** hblixt has quit IRC22:18
Pradip_also when I try to add a normal VM, it is saying that no host found. Is it standard?22:19
TheJuliaNo, its not.  That really sounds like the scheduler cannot find a host22:20
Pradip_ohh. Then I don't know what to do? I will look into the logs for more details22:21
TheJuliaCould you have applied the nova-compute config for using ironic to all of your hypervisor nodes?22:22
TheJuliathe configurations should be different22:22
TheJuliai.e. ironic requires a separate nova-compute process to allow for scheduling on to baremetal nodes.22:23
Pradip_ah. I guess that may be the reason. I have only one host at present, I am following the instruction in http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html for devstack to create 3 VMs to pose as bare metal nodes22:25
TheJuliaGuys, why does http://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-compute-service-to-use-the-bare-metal-service not seem right to me ?  Or have I lost my mind today?22:25
TheJuliaOH22:25
*** e0ne has quit IRC22:26
Pradip_I am trying to add another host. But before that I wanted to be familiar with the usage concepts. But I am stuck here for long now22:27
TheJuliaokay22:28
TheJuliaso have you defined baremetal nodes in ironic?22:28
*** krtaylor has joined #openstack-ironic22:30
Pradip_I followed the instruction on the link to create 3 VM acting as bare metal node. I can see the loist too with ironic command. But when I am trying to provision them, the problem occcurs22:30
Pradip_on that settings, I can;t add normal Vm too22:31
*** dontalton has joined #openstack-ironic22:32
TheJuliaI understand that, you won't be able to launch VMs unless you configure multiple nova-compute processes.  I guess the big question is what is the conductor log saying when your attempting to deploy on to a baremetal node22:35
Pradip_INFO ironic.conductor.utils [-] Succesfully set node 28141b2a-988e-4f7a-bd3f-1934a4ced10a power state to power on. 2015-06-25 14:42:52.841 DEBUG ironic.conductor.task_manager [-] Attempting to reserve node 1 from (pid=25430) reserve_node /opt/stack/ironic/ironic/conductor/task_manager.py:17922:36
Pradip_and nova log: 2015-06-25 14:45:15.253 DEBUG nova.virt.ironic.driver [-] [instance: 26c17dd5-c4d7-490b-b4b1-d3f58f324602] Still waiting for ironic node 28141b2a-988e-4f7a-bd3f-1934a4ced10a to become ACTIVE: power_state="power on", target_power_state=None, provision_state="wait call-back", target_provision_state="deploy complete" from (pid=23843) _log_ironic_polling /opt/stack/nova/nova/virt/ironic/driver.py:15022:37
TheJuliacan you share with us the ironic node-show output for one of those nodes?22:37
Pradip_instance_uuid          | 26c17dd5-c4d7-490b-b4b1-d3f58f324602                                     | | target_power_state     | None                                                                     | | properties             | {u'memory_mb': u'1024', u'cpu_arch': u'x86_64', u'local_gb': u'10',      | |                        | u'cpus': u'1'}                                                           | | maintenance22:38
TheJuliaplease use paste.openstack.org :)22:38
Pradip_sorry :)22:39
TheJuliaNo problem :)22:39
Pradip_Paste #32120322:40
Pradip_can I share it here?22:41
TheJuliaI just pulled it up22:41
Pradip_ok22:42
TheJuliacan you pull up the console and see whats happening to that VM?  Basically it needs to boot and check-in after booting a deployment image22:42
Pradip_ah yeah. it is saying no bootable device22:45
TheJuliaanother possibility is networking may not be right between the VM and your DHCP and TFTP services22:45
Pradip_yeah I can see it22:45
TheJuliaerr, may not be22:45
TheJuliatry rebooting the VM and see if it attempts to network boot at all22:45
Pradip_it is also saying no more network device, connection timed out22:45
*** dontalton has quit IRC22:45
*** dhellmann has quit IRC22:46
TheJuliaIt would seem, that you likely are missing something with the network plumbing between your test VM and your dhcp/tftp services.  Anything in your dhcp service log?22:46
*** dhellmann has joined #openstack-ironic22:46
TheJuliafor the mac address of that test VM?22:46
Pradip_same output. I guess It is not able to connect to the server for the pxelinux22:47
TheJuliaThat is what it seems22:47
Pradip_yeah . I think so too22:48
jlvillalPradip_: Just a question.  How did you create your VM for devstack?  I'm assuming you are running devstack in a VM22:48
*** lucas-dinner has quit IRC22:48
Pradip_dhcp l;og is repeating same line over and over22:48
*** ijw__ has joined #openstack-ironic22:49
Pradip_no i am usning devstack in real machine22:49
jlvillalPradip_: How did you create your VM for devstack?  I'm assuming you are running devstack in a VM.22:50
jlvillalPradip_: I'm curious because I ran into similar sounding situation.22:50
Pradip_jlvillal: I am running it on a real machine and followed the instruction in http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html22:52
jlvillalPradip_: Oh.  Okay.  Thanks.22:52
*** ijw_ has quit IRC22:52
jlvillalPradip_: Okay.  Different from my case.  Thanks.22:52
Pradip_TheJulia: Thanks a lot for your help, I will check the networking issue22:53
TheJuliaPradip_: Your welcome, we're here to help :)22:59
*** natorious is now known as zz_natorious22:59
*** stegranet has quit IRC23:02
*** ijw_ has joined #openstack-ironic23:10
*** ijw__ has quit IRC23:13
*** zhenguo has joined #openstack-ironic23:18
*** thrash is now known as thrash|g0ne23:21
*** Sukhdev has joined #openstack-ironic23:21
*** rbrooker has quit IRC23:22
SpamapS$ ironic driver-list23:25
SpamapS'NoneType' object has no attribute 'rfind'23:25
SpamapSanybody seen that?23:25
NobodyCamSpamapS: I have not seen that one23:26
SpamapShrm23:27
SpamapS| publicURL | http:infra-cloud-baremetal0.localdomain:6385/ |23:28
SpamapStypeo23:28
devananda:)23:28
devanandaSpamapS: how's it going, aside fromtypos?23:28
SpamapSnot a useful error message tho ;)23:28
SpamapSdevananda: so bifrost was an interesting challenge to get working with these crazy mellanox cards23:28
SpamapSdevananda: they have a weird version of iPXE on them which gave TheJulia fits. ;)23:29
devanandaSpamapS: haha23:29
SpamapSdevananda: but once we worked around that, it's been mostly puppet wrangling.23:29
*** absubram has quit IRC23:29
devanandaSpamapS: that's good!23:29
SpamapS| pxe_ipmitool        | infra-cloud-baremetal0.localdomain |23:29
SpamapSthere we go23:29
*** ijw__ has joined #openstack-ironic23:30
devanandaTheJulia: dhcp issue aside, the deploy-all playbook seems to do the right thing23:31
*** yuanying has joined #openstack-ironic23:32
*** ijw_ has quit IRC23:33
devanandaTheJulia: it's a bit odd that if it finds no nodes "available" it runs all the way through, but skips every step23:33
*** UForgotten has joined #openstack-ironic23:34
TheJuliadevananda: awesome, sorry, didn't get a chance to even think about the dhcp issue23:42
devanandaTheJulia: no worries. it's probably something in my local env23:42
TheJuliaYeah, that could be an issue.  Wonder if I can make it fail easilly if there are no nodes available23:42
devanandaJoshNang: I have a new theory about the clenaing failures -- I think this device requires passing the boot device every time23:43
TheJulias/issue/source of confusion/23:43
JoshNangdevananda: ohhhh23:43
devanandaJoshNang: yah. agent.py 228     manager_utils.node_set_boot_device(task, boot_devices.PXE,23:43
devanandaJoshNang: nope - nvm. that's called from preprae_cleanning too23:44
JoshNang:/23:45
devanandaJoshNang: so there's a thing missing from our state machine -- "clean wait"23:45
JoshNangthis is bifrost and agent-amttool?23:45
TheJuliapxe_amt23:45
devanandaJoshNang: that time when a node isn't online yet, and the driver knows it should be cleaning, but it's waiting for it to finish POST23:45
TheJuliathere is no agent_amt23:45
JoshNangdevananda: yeah that would be a lot cleaner23:45
JoshNangTheJulia: gotcha, thanks!23:45
devanandait's my hacky agent-amttool23:46
devanandaTheJulia: ^ yes there is. but not in trunk23:46
TheJuliaheh, figures :)23:46
devanandaTheJulia: https://review.openstack.org/#/c/193767/1 fyi23:47
devanandait exists because of https://bugs.launchpad.net/ironic/+bug/145449223:48
openstackLaunchpad bug 1454492 in Ironic "openwsman library is not greenthread-aware, leads to driver lockup with both DRAC and AMT drivers" [Medium,Triaged] - Assigned to Imre Farkas (ifarkas)23:48
TheJuliadevananda: ahhhh!23:51
*** ijw_ has joined #openstack-ironic23:51
*** ijw__ has quit IRC23:53
*** Sukhdev has quit IRC23:56
*** smoriya has joined #openstack-ironic23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!