| *** matsuhashi has joined #openstack-ironic | 00:23 | |
| *** matty_du1s|gone has joined #openstack-ironic | 01:09 | |
| *** Hefeweiz1n has joined #openstack-ironic | 01:09 | |
| *** zul has quit IRC | 01:13 | |
| *** krtaylor has quit IRC | 01:13 | |
| *** datajerk has quit IRC | 01:13 | |
| *** ewindisch has quit IRC | 01:13 | |
| *** killer_prince has quit IRC | 01:13 | |
| *** jgrimm has quit IRC | 01:13 | |
| *** matty_dubs|gone has quit IRC | 01:13 | |
| *** Hefeweizen has quit IRC | 01:13 | |
| *** JoshNang has quit IRC | 01:13 | |
| *** ewindisch has joined #openstack-ironic | 01:16 | |
| *** JoshNang has joined #openstack-ironic | 01:20 | |
| *** killer_prince has joined #openstack-ironic | 01:21 | |
| *** jgrimm has joined #openstack-ironic | 01:21 | |
| *** zul has joined #openstack-ironic | 01:21 | |
| *** matsuhashi has quit IRC | 01:24 | |
| *** matsuhashi has joined #openstack-ironic | 01:26 | |
| *** nosnos has joined #openstack-ironic | 01:31 | |
| *** krtaylor has joined #openstack-ironic | 01:31 | |
| *** eghobo has quit IRC | 02:19 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/ironic: Add tox target and script for nova driver tests https://review.openstack.org/84033 | 02:31 |
|---|---|---|
| openstackgerrit | Adam Gandelman proposed a change to openstack/ironic: Update Ironic Nova drivers' unit tests https://review.openstack.org/84043 | 02:40 |
| *** eghobo has joined #openstack-ironic | 03:07 | |
| *** matsuhashi has quit IRC | 03:19 | |
| mordred | adam_g: seems sane to me | 03:24 |
| mordred | adam_g: in adding an jjb entry to run that - I suggest overriding NOVA_REPO to be set to file:///opt/git/openstack/nova - since all of our build slaves will have a mostly recently clone of nova there anyway | 03:25 |
| mordred | adam_g: I mean, we can get MUCH more clever ultimately - but it's probably fine for what you need for now | 03:26 |
| *** nosnos has quit IRC | 03:34 | |
| *** radsy has joined #openstack-ironic | 03:54 | |
| *** matsuhashi has joined #openstack-ironic | 03:57 | |
| *** nosnos has joined #openstack-ironic | 04:02 | |
| *** radsy has quit IRC | 04:08 | |
| adam_g | mordred, it or this needs a bit of tweaking for that to fit nicely into the gate http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/run-unittests.sh#n42 | 04:11 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/ironic: Update Ironic Nova drivers' unit tests https://review.openstack.org/84043 | 04:32 |
| *** matsuhashi has quit IRC | 04:39 | |
| *** lazy_prince has joined #openstack-ironic | 04:39 | |
| *** krtaylor has quit IRC | 04:39 | |
| *** killer_prince has quit IRC | 04:39 | |
| *** lazy_prince is now known as killer_prince | 04:39 | |
| *** matsuhashi has joined #openstack-ironic | 04:43 | |
| *** krtaylor has joined #openstack-ironic | 04:48 | |
| *** killer_prince has quit IRC | 05:19 | |
| *** lazy_prince has joined #openstack-ironic | 05:33 | |
| *** lazy_prince is now known as killer_prince | 05:33 | |
| *** pradipta_away is now known as pradipta | 05:38 | |
| *** nosnos_ has joined #openstack-ironic | 05:53 | |
| *** nosnos has quit IRC | 05:53 | |
| *** matsuhashi has quit IRC | 05:54 | |
| *** matsuhashi has joined #openstack-ironic | 05:55 | |
| openstackgerrit | Jenkins proposed a change to openstack/ironic: Imported Translations from Transifex https://review.openstack.org/83956 | 06:09 |
| dtantsur | Morning Ironic | 06:58 |
| *** matsuhashi has quit IRC | 07:08 | |
| *** matsuhashi has joined #openstack-ironic | 07:08 | |
| *** nosnos_ has quit IRC | 07:09 | |
| *** nosnos has joined #openstack-ironic | 07:10 | |
| *** ifarkas has joined #openstack-ironic | 07:18 | |
| *** matsuhashi has quit IRC | 07:22 | |
| *** matsuhashi has joined #openstack-ironic | 07:28 | |
| *** yuriyz has joined #openstack-ironic | 07:29 | |
| *** jistr has joined #openstack-ironic | 07:40 | |
| *** martyntaylor has joined #openstack-ironic | 07:46 | |
| *** martyntaylor has left #openstack-ironic | 07:46 | |
| *** matsuhashi has quit IRC | 07:46 | |
| *** matsuhashi has joined #openstack-ironic | 07:47 | |
| GheRivero | morning aIronic | 07:52 |
| *** dshulyak has joined #openstack-ironic | 07:55 | |
| *** mkerrin has joined #openstack-ironic | 07:59 | |
| *** derekh has joined #openstack-ironic | 08:09 | |
| *** mdurnosvistov has joined #openstack-ironic | 08:10 | |
| *** athomas has joined #openstack-ironic | 08:12 | |
| mdurnosvistov | Morning folks! =) | 08:21 |
| *** lucasagomes has joined #openstack-ironic | 08:24 | |
| yuriyz | morning all | 08:28 |
| *** eghobo has quit IRC | 08:29 | |
| lucasagomes | morning yuriyz | 08:36 |
| *** Haomeng has joined #openstack-ironic | 08:36 | |
| *** max_lobur has joined #openstack-ironic | 08:37 | |
| mdurnosvistov | Morning lucasagomes, yuriyz, max_lobur, Haomeng! =) | 08:49 |
| Haomeng | mdurnosvistov: morning:) | 08:49 |
| max_lobur | morning Everyone | 08:49 |
| Haomeng | :) | 08:50 |
| Haomeng | one question, who konw if we can use tripleo to deploy openstack understand cloud with our ironic now? | 08:50 |
| Haomeng | Haomeng: I understand it is not ready now | 08:51 |
| *** ndipanov has joined #openstack-ironic | 08:51 | |
| ifarkas | morning all! | 08:53 |
| dtantsur | Haomeng, I thinks lifeless and maybe NobodyCam are working on patches, but yes is seems like they're not ready yet | 08:53 |
| lifeless | Haomeng: dtantsur you can | 08:54 |
| lifeless | just export USE_IRONIC=1 in your environment | 08:54 |
| dtantsur | lifeless, great news! | 08:54 |
| Haomeng | lifeless: ok, I will do, it is very Interesting to have a first try:) | 08:55 |
| Haomeng | lifeless: is there any TripleO documents which can guide us to deploy then undercloud? | 08:55 |
| lifeless | Haomeng: http://docs.openstack.org/developer/tripleo-incubator/devtest.html | 08:56 |
| Haomeng | lifeless: got, thank you:) | 08:56 |
| ifarkas | Haomeng, lifeless, there's another one, specific to Ironic: https://wiki.openstack.org/wiki/Ironic/TripleO | 08:57 |
| lifeless | ifarkas: that may be dated | 08:57 |
| Haomeng | ifarkas: ok, great, thanks | 08:57 |
| ifarkas | lifeless, right, it refers to an old patch | 08:57 |
| lifeless | Haomeng: that wiki page is very incorrect | 08:58 |
| Haomeng | lifeless: :) | 08:58 |
| lifeless | Haomeng: ifarkas: fixed https://wiki.openstack.org/wiki/Ironic/TripleO#Using_Ironic_with_TripleO | 09:03 |
| ifarkas | lifeless, wow, great! thanks! | 09:04 |
| *** pbrooko has joined #openstack-ironic | 09:22 | |
| ifarkas | Haomeng, I am also trying devtest with Ironic but I hit this issue: http://paste.openstack.org/show/74644/ Did you hit the same? | 09:41 |
| *** Haomeng has quit IRC | 09:42 | |
| *** Haomeng has joined #openstack-ironic | 09:44 | |
| lucasagomes | lifeless, ping, can I fix the nova driver to translate the swap mb to gb in ironic in another patch? I know it's very related but I'm trying to avoid having the fix in the Ironic tree and nova driver in the same commit, since the nova driver is only a temporary thing in the ironic tree | 10:29 |
| lifeless | lucasagomes: if you don't, you'll fail CI as soon as the jobs are working again. | 10:30 |
| lifeless | lucasagomes: so, you can either do backwards compat, or fix at the same time | 10:30 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Add DiskPartitioner https://review.openstack.org/83396 | 10:30 |
| lifeless | (welcome to more CI :)) | 10:30 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Use DiskPartitioner https://review.openstack.org/83399 | 10:31 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Get rid of the swap partition https://review.openstack.org/83726 | 10:32 |
| lucasagomes | lifeless, ah, indeed | 10:34 |
| lucasagomes | lifeless, heh ok I will put in the same patch | 10:34 |
| *** pbrooko has quit IRC | 10:46 | |
| *** vkozhukalov has joined #openstack-ironic | 10:57 | |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Use GB instead of MB for swap https://review.openstack.org/83788 | 11:11 |
| openstackgerrit | Yuriy Zveryanskyy proposed a change to openstack/ironic: Add worker threads limit to _check_deploy_timeouts task https://review.openstack.org/78651 | 11:15 |
| mtrisoline | Is there any documentation on Ironic w/ the Havana release? | 11:19 |
| mtrisoline | I was told that the there is a driver for it and it works but is buggy, was interested in checking it out, | 11:19 |
| *** matsuhashi has quit IRC | 11:26 | |
| *** nosnos has quit IRC | 11:27 | |
| *** lucasagomes is now known as lucas-hungry | 11:40 | |
| Haomeng | ifarkas: I did not hit same issue, but think this is network unstable issue to cause "Server returned an unexpected response code. [404]" | 11:42 |
| Haomeng | ifarkas: you can try again, maybe it is not recreatable | 11:42 |
| Haomeng | lifeless: ok, thanks | 11:43 |
| *** saju_m has joined #openstack-ironic | 11:51 | |
| *** overlayer has joined #openstack-ironic | 11:52 | |
| * Shrews attempts to wake up | 11:53 | |
| ifarkas | Haomeng, thanks! It was actually caused by PERCONA_ARCH not being evaluated in source-repository-mysql | 11:57 |
| ifarkas | Haomeng, which is already fixed upstream but I had a old version of diskimage-builder installed in /usr/share of which I was not aware of | 11:58 |
| *** saju_m has quit IRC | 12:00 | |
| openstackgerrit | David Shrewsbury proposed a change to openstack/ironic: Encapsulate Ironic client retry logic https://review.openstack.org/83105 | 12:06 |
| *** saju_m has joined #openstack-ironic | 12:06 | |
| *** saju_m has quit IRC | 12:08 | |
| Haomeng | ifarkas: ok:) | 12:08 |
| *** linggao has joined #openstack-ironic | 12:14 | |
| *** mdurnosvistov has quit IRC | 12:41 | |
| *** lucas-hungry is now known as lucasagomes | 12:41 | |
| *** jdob has joined #openstack-ironic | 12:44 | |
| *** max_lobur1 has joined #openstack-ironic | 12:53 | |
| *** max_lobur has quit IRC | 12:53 | |
| *** dhellmann_ is now known as dhellmann | 13:04 | |
| *** vkozhukalov has quit IRC | 13:05 | |
| *** jdob has quit IRC | 13:06 | |
| *** jdob has joined #openstack-ironic | 13:06 | |
| *** vkozhukalov has joined #openstack-ironic | 13:07 | |
| *** pradipta is now known as pradipta_away | 13:13 | |
| *** jbjohnso_ has joined #openstack-ironic | 13:14 | |
| *** ndipanov_ has joined #openstack-ironic | 13:17 | |
| *** ndipanov has quit IRC | 13:18 | |
| *** matty_du1s|gone is now known as matty_dubs | 13:25 | |
| *** rloo has joined #openstack-ironic | 13:25 | |
| *** jdob_ has joined #openstack-ironic | 13:35 | |
| *** jgrimm has quit IRC | 13:48 | |
| *** zigo has quit IRC | 13:50 | |
| NobodyCam | good morning Ironic, says the man making coffee | 13:52 |
| *** mdurnosvistov has joined #openstack-ironic | 13:53 | |
| linggao | good morning, NobodyCam, tea please. | 13:53 |
| NobodyCam | :) | 13:55 |
| NobodyCam | morning linggao :) | 13:55 |
| lucasagomes | morning NobodyCam | 13:55 |
| linggao | morning lucasagomes. | 13:56 |
| NobodyCam | morning lucasagomes :) | 13:57 |
| NobodyCam | did everyone have a good weekend? | 13:57 |
| jroll | morning y'all | 13:57 |
| *** zigo has joined #openstack-ironic | 13:58 | |
| *** lucasagomes has quit IRC | 13:59 | |
| NobodyCam | good morning jroll :) | 13:59 |
| *** lucasagomes has joined #openstack-ironic | 13:59 | |
| NobodyCam | woo hoo https://review.openstack.org/#/c/83906 landed | 14:04 |
| mdurnosvistov | Good morning/afternoon all! =) | 14:05 |
| NobodyCam | good morning mdurnosvistov | 14:06 |
| *** rwsu has joined #openstack-ironic | 14:06 | |
| lucasagomes | morning linggao mdurnosvistov | 14:14 |
| openstackgerrit | Yuriy Zveryanskyy proposed a change to openstack/ironic: Fix hosts mapping for conductor's periodic tasks https://review.openstack.org/84142 | 14:16 |
| *** ndipanov_ has quit IRC | 14:20 | |
| *** ndipanov_ has joined #openstack-ironic | 14:32 | |
| *** jgrimm has joined #openstack-ironic | 14:36 | |
| *** jdob_ has quit IRC | 14:38 | |
| NobodyCam | humm no bug for that one... | 14:40 |
| *** jdob has quit IRC | 14:41 | |
| openstackgerrit | Pablo Fernando Cargnelutti proposed a change to openstack/ironic: Change admin_url help in ironic driver https://review.openstack.org/84148 | 14:46 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Use GB instead of MB for swap https://review.openstack.org/83788 | 14:48 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Use GB instead of MB for swap https://review.openstack.org/83788 | 14:52 |
| *** mkerrin has quit IRC | 14:53 | |
| NobodyCam | brb | 14:55 |
| *** blamar has quit IRC | 15:07 | |
| *** mkerrin has joined #openstack-ironic | 15:11 | |
| *** BadCub01 has joined #openstack-ironic | 15:11 | |
| *** BadCub01 has left #openstack-ironic | 15:12 | |
| *** BadCub01 has joined #openstack-ironic | 15:12 | |
| *** dshulyak has quit IRC | 15:16 | |
| *** max_lobur1 has quit IRC | 15:22 | |
| *** mkerrin has quit IRC | 15:31 | |
| *** mkerrin has joined #openstack-ironic | 15:38 | |
| NobodyCam | lucasagomes: looking over 83788 driver.py...line 259.. do you see any issues with rounding up? | 15:44 |
| lucasagomes | NobodyCam, so I thought about it, if the user request something like 512 MB | 15:44 |
| lucasagomes | NobodyCam, if we floor divide it it goes to 0 | 15:44 |
| lucasagomes | so I thought about round it up to 1 | 15:45 |
| lucasagomes | as a minimum u know | 15:45 |
| lucasagomes | which is the minimum in Ironic | 15:45 |
| NobodyCam | with this patch we switch to gb input users would have to enter .5 to 512 MB. | 15:45 |
| NobodyCam | :-p s/to 512/to GET 512/ | 15:47 |
| lucasagomes | NobodyCam, hmm I thought we didn't accept fractions | 15:47 |
| NobodyCam | i dont think we do | 15:47 |
| lucasagomes | NobodyCam, if we are going to accept fractions the best would be to have everything in MB instead | 15:47 |
| lucasagomes | which more fine grained | 15:47 |
| lucasagomes | right | 15:48 |
| NobodyCam | so will never get a request for 512 | 15:48 |
| lucasagomes | NobodyCam, ah, well in nova u can do it | 15:48 |
| lucasagomes | NobodyCam, the flavor is mb | 15:48 |
| NobodyCam | oh thru novA | 15:48 |
| NobodyCam | .... | 15:48 |
| lucasagomes | and the driver translate what is in the flavor to ironic | 15:48 |
| NobodyCam | ack | 15:48 |
| NobodyCam | lol sorry | 15:48 |
| NobodyCam | brain not thinking correctly | 15:49 |
| lucasagomes | NobodyCam, ah np! :D | 15:49 |
| NobodyCam | !!! :-p | 15:49 |
| openstack | NobodyCam: Error: "!!" is not a valid command. | 15:49 |
| lucasagomes | hah | 15:49 |
| NobodyCam | so if I pull that it should work? just about to attempt a test with it | 15:50 |
| *** vkozhukalov has quit IRC | 15:53 | |
| Shrews | ah, this reminds me... | 15:54 |
| NobodyCam | morning Shrews :) | 15:54 |
| Shrews | lucasagomes: have there been any recent changes to how we partition disks? | 15:54 |
| Shrews | NobodyCam: morning | 15:54 |
| lucasagomes | Shrews, I got a couple of patches up refactoring that part | 15:54 |
| lucasagomes | Shrews, but we are waiting until juno opens to get them in | 15:55 |
| NobodyCam | Shrews: there are several patch that switch to parted | 15:55 |
| Shrews | lucasagomes: I ask because I discovered last night that our devstack changes no longer work: http://paste.openstack.org/show/74630/ | 15:55 |
| Shrews | I'm not sure what changed | 15:55 |
| lucasagomes | Shrews, ew well it looks like sfdisk didn't tell the kernel to re-read the partition table after creating it | 15:56 |
| Shrews | provisioning an instance fails because of that | 15:56 |
| lucasagomes | Shrews, which can be done with the -R option of sfidsk | 15:56 |
| lucasagomes | or using partprobe | 15:56 |
| lucasagomes | Shrews, I think you won't see that with the new partitioning tho | 15:57 |
| lucasagomes | Shrews, could you try it ? | 15:57 |
| lucasagomes | Shrews, https://review.openstack.org/#/c/83399/ | 15:57 |
| Shrews | lucasagomes: the new changes? shouldn't we fix what we're going to have for Icehouse? | 15:57 |
| lucasagomes | Shrews, and I'm not aware of any changes in that part for a while, so it might be some system related problem (guessing) | 15:58 |
| lucasagomes | Shrews, yeah we can back port it to icehouse | 15:58 |
| lucasagomes | devananda, ^ | 15:58 |
| Shrews | adam_g: have you encountered that error lately? ^^^^ | 15:58 |
| *** eghobo has joined #openstack-ironic | 15:59 | |
| Shrews | or anyone using devstack, for that matter | 15:59 |
| lucasagomes | Shrews, I will try later devstack with trunk | 16:00 |
| lucasagomes | in my env I've my patches applied so that I can see if something breaks | 16:00 |
| Shrews | lucasagomes: great. it would be nice to know if it's something particular to my system, or if it's a true bug | 16:01 |
| NobodyCam | Shrews: devTest has a issue that required some fidling with disk-image-create... | 16:01 |
| NobodyCam | http://paste.openstack.org/show/N4nNohHoZmYqH1v9S9IP | 16:01 |
| NobodyCam | lucasagomes: I am running a devtest run with your parches now | 16:01 |
| lucasagomes | NobodyCam, :D nice, please keep me updated | 16:02 |
| devananda | morning, all | 16:02 |
| NobodyCam | good morning devananda :) | 16:02 |
| lucasagomes | morning devananda | 16:03 |
| devananda | Shrews, lucasagomes: we should confirm what's caused that. icehouse RC1 isn't tagged for us yet | 16:03 |
| devananda | so we don't need to back-port things at this stage - -better to fix if we can | 16:03 |
| Shrews | lucasagomes: also, thx for your comments on 83105. I hope my response re: the global seems logical | 16:03 |
| lucasagomes | Shrews, oh I will take a look, about the answer yes it's grand, thanks for that | 16:03 |
| Shrews | devananda++ | 16:04 |
| Shrews | devananda: also, good morning :) | 16:05 |
| devananda | lucasagomes: re: diskpartitioner, that's a pretty big change to be landing this late, so if we can figure out why this is broken and fix just that, i think it'd be better | 16:07 |
| devananda | Shrews: have you opened a bug? | 16:07 |
| lucasagomes | devananda, ack, I will run it against trunk | 16:07 |
| lucasagomes | see if I can reproduce the problem | 16:07 |
| Shrews | devananda: not yet. hoping to eliminate a problem with my system first | 16:07 |
| Shrews | lucasagomes: thanks | 16:08 |
| lucasagomes | np | 16:08 |
| devananda | Shrews: does it seem related to new release of linux-utils? http://changelogs.ubuntu.com/changelogs/pool/main/u/util-linux/util-linux_2.20.1-5.1ubuntu18/changelog | 16:09 |
| NobodyCam | bbt..brb... | 16:09 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:10 | |
| Shrews | devananda: possible, especially since I can't find any recent relevant changes in ironic | 16:12 |
| devananda | Shrews: what version of util-linux is installed? | 16:15 |
| devananda | i have a fairly recent devstack env, going to test it in | 16:15 |
| Shrews | devananda: Version: 2.20.1-1ubuntu3 | 16:16 |
| Shrews | guess that's older than the changelog you pasted | 16:16 |
| devananda | yep | 16:16 |
| lucasagomes | devananda, Shrews ohhhhhh I know what's wrong! https://review.openstack.org/#/c/83131/ | 16:16 |
| lucasagomes | after that nova ironic driver will get the swap_mb from flavor | 16:16 |
| lucasagomes | and if it's 0 it will send 0 to Ironic | 16:16 |
| devananda | ahh | 16:16 |
| devananda | yes | 16:16 |
| lucasagomes | but in Ironic we have a get('pxe_swap_mb', 1) | 16:17 |
| lucasagomes | which will return 0 | 16:17 |
| lucasagomes | because it's set | 16:17 |
| lucasagomes | 1 sec, lemme try to put some quick fix | 16:17 |
| lucasagomes | urgh | 16:17 |
| *** martyntaylor has joined #openstack-ironic | 16:18 | |
| devananda | Shrews: I'm going to untarget https://bugs.launchpad.net/ironic/+bug/1295870 | 16:18 |
| devananda | as there's no way that'll be done today | 16:19 |
| Shrews | devananda: totally done, no. | 16:19 |
| Shrews | but i'm fine with that | 16:19 |
| devananda | Shrews: right. there's some good stuff done, but it doesn' tlook not close-the-bug-done | 16:20 |
| devananda | Shrews: unless you feel it will be later today? | 16:21 |
| NobodyCam | quick morning walkies... bbiafm | 16:21 |
| Shrews | devananda: you mean done as in replacing "all" calls, or done with this particular review? because i think since we don't have good testing of this code, we need burn-in time | 16:22 |
| devananda | Shrews: close-the-bug done would be, for me, when all the calls are wrapped. | 16:23 |
| Shrews | devananda: nah, i wouldn't feel comfortable with that for Icehouse | 16:24 |
| Shrews | it's not THAT critical, imo, anyway | 16:24 |
| devananda | right | 16:24 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: If no swap is specified default to 1MB https://review.openstack.org/84167 | 16:24 |
| devananda | lucasagomes: re: https://review.openstack.org/#/c/83572/1/ironic/common/driver_factory.py | 16:25 |
| lucasagomes | Shrews, that would fix ur problem ^ | 16:25 |
| lucasagomes | devananda, right | 16:25 |
| devananda | lucasagomes: I agree with both your and lifeless' comments. Things work as they are now for Ironic, but as it is, it breaks tripleo | 16:25 |
| devananda | lucasagomes: because tripleo is parsing the log for error/exception and stopping | 16:26 |
| Shrews | lucasagomes: i'll give that a try | 16:26 |
| lucasagomes | Shrews, ack :) | 16:26 |
| devananda | lucasagomes: so we can suppress that error now to unbreak tripleo, or we can add a new config option ... but it's way late to do that | 16:26 |
| lucasagomes | devananda, right... hmmm it's complicated because we are basically introducing a bug in Ironic to overcome a problem in tripleo | 16:28 |
| devananda | heh | 16:29 |
| * lucasagomes thinks to see if there's another approach we could take | 16:29 | |
| devananda | tripleo has an easy fix | 16:29 |
| devananda | if they install the python-seamicroclient pip, the error goes away | 16:29 |
| lucasagomes | devananda, right, what the other suggestion to remove the seamicro driver from the default setup.conf? | 16:30 |
| lucasagomes | since it does have external dependencies it shouldn't be a driver which is "enabled" by default | 16:30 |
| NobodyCam | lucasagomes: + | 16:30 |
| devananda | lucasagomes: if we do that, it would effectiely be not included in the release | 16:31 |
| devananda | it'll affect packagers | 16:31 |
| devananda | how would someone consuming the ironic .deb/.yum get that driver? | 16:32 |
| devananda | as it is today, they'll get the driver, and an exception in the startup log, and they can "pip install python-seamicroclient" and then it works. Gah. we need to document that ... | 16:32 |
| lucasagomes | yeah I see... yeah that just enforce yet more lifeless comment that we need a enable/disable mechanism within Ironic for drivers | 16:32 |
| NobodyCam | devananda: what about patch 83572 | 16:32 |
| lucasagomes | but if I could I would prefer the easy fix in tripleo (installing python-seamicroclient) than what that patch suggest | 16:33 |
| devananda | lucasagomes: I totally agree that we need a CONF.manager.drivers=... default="pxe_ssh, pxe_ipmitool" | 16:33 |
| lucasagomes | idk if there's any push back from lifeless for installing that as default, at least until we have a mechanism in place for that | 16:33 |
| lucasagomes | devananda, + | 16:33 |
| lucasagomes | (my english is really going down today hah, I'm tired :P) | 16:34 |
| lucasagomes | I mean a mechanism to disable/enable drivers | 16:34 |
| devananda | right | 16:35 |
| devananda | lucasagomes: also, https://review.openstack.org/#/c/84167/1 should have a unit test and a bug report | 16:35 |
| lucasagomes | devananda, https://pypi.python.org/pypi/python-seamicroclient/0.1.0 it's apache licensed so it might be fine | 16:35 |
| devananda | or be tagged to https://bugs.launchpad.net/ironic/+bug/1297937 | 16:35 |
| lucasagomes | devananda, ah, ack will add some tests and tag it | 16:36 |
| devananda | lucasagomes: it is fine. and fwiw, it's currently in global-requirements, though it shouldn't be | 16:36 |
| lucasagomes | devananda, heh ++ I agree with that too | 16:36 |
| *** martyntaylor has quit IRC | 16:36 | |
| lucasagomes | it's not really a requirement, idk how infra didn't push back on that | 16:36 |
| devananda | yea | 16:39 |
| *** bearhands is now known as comstud | 16:40 | |
| Shrews | lucasagomes: currently 'nova boot'-ing... should know soon if that fix works | 16:41 |
| Shrews | soon-ish... slow vm :( | 16:41 |
| NobodyCam | lucasagomes: undercloud deployed from seed with ironic and 83788 | 16:42 |
| lucasagomes | Shrews, ack :) | 16:42 |
| lucasagomes | NobodyCam, yay! | 16:43 |
| *** urulama has joined #openstack-ironic | 16:43 | |
| lucasagomes | NobodyCam, and the swap partition is not created right? (in case u didn't specify any swap) | 16:43 |
| devananda | lucasagomes: on http://paste.openstack.org/show/74647/ -- I see you used "node-update .. add propeties/.." | 16:44 |
| Shrews | wow, i *really* want to cleanup the output in conductor logs | 16:44 |
| devananda | lucasagomes: so "add" works as "replace" now too? | 16:44 |
| devananda | Shrews: ++ | 16:45 |
| lucasagomes | devananda, it does, max_labour was even investigating it | 16:45 |
| lucasagomes | devananda, it's part of the RFC | 16:45 |
| devananda | awesome, yea. i saw that in the RFC | 16:45 |
| devananda | didn't realize we already fixed it | 16:45 |
| devananda | lucasagomes: how long did it take for that update to propagate out to nova? | 16:46 |
| NobodyCam | lucasagomes: correct swap not set | 16:46 |
| lucasagomes | devananda, hmm I tricked it :P I restart the n-cpu to make the propagation faster | 16:46 |
| lucasagomes | so I idk the right time :( | 16:46 |
| lucasagomes | NobodyCam, yay! | 16:46 |
| lucasagomes | devananda, I can investigate it tho, I've a env here | 16:47 |
| devananda | lucasagomes: hehe | 16:47 |
| devananda | lucasagomes: I suspect ~ 2 minutes | 16:47 |
| devananda | lucasagomes: also, that patch from rohan lgtm. good to land it now? | 16:47 |
| lucasagomes | devananda, the filters? yup I tested it here | 16:48 |
| lucasagomes | works | 16:48 |
| devananda | lucasagomes: great. approving. also https://review.openstack.org/#/c/83851/ could use a quick check from you | 16:48 |
| lucasagomes | devananda, as I said in the comment, its a bit trick because we have to copy it to the nova's path | 16:48 |
| *** harlowja_away is now known as harlowja | 16:48 | |
| lucasagomes | but yeah, works, it all will be gone after we move the driver and filters back to nova | 16:49 |
| devananda | right | 16:49 |
| lucasagomes | devananda, will take a look at 83851 soon | 16:49 |
| devananda | and since scheduler filters are not loaded via stevedore or any sort of plugin means | 16:49 |
| devananda | i dont think we have any option but to copy it | 16:49 |
| devananda | will suck for tripleo | 16:49 |
| lucasagomes | yeah ;( | 16:49 |
| * devananda files a bug | 16:49 | |
| * devananda updates the existing bug instead | 16:50 | |
| *** jdob has joined #openstack-ironic | 16:50 | |
| devananda | also landing this is a bit in odd form | 16:52 |
| devananda | as it's adding a CONF option | 16:52 |
| devananda | but adding it to nova | 16:53 |
| devananda | this whole "carry a chunk of nova code outside of nova" is really awkward | 16:53 |
| devananda | and likely to cause us a huge headache in Juno | 16:53 |
| NobodyCam | :( | 16:53 |
| lucasagomes | yeah, it's being a painful process :/ | 16:54 |
| devananda | NobodyCam: https://review.openstack.org/#/c/83851/1 could also use your eyes | 16:55 |
| devananda | the exact disk match filter needs that change to work | 16:55 |
| devananda | *I think ... | 16:55 |
| *** martyntaylor has joined #openstack-ironic | 16:56 | |
| *** matty_dubs|lunch is now known as matty_dubs | 16:57 | |
| Shrews | lucasagomes: sorry to report, same error: http://paste.openstack.org/show/74671/ | 16:57 |
| lucasagomes | Shrews, ouch :( | 16:57 |
| * Shrews needs lunch now... biab | 16:57 | |
| lucasagomes | ack | 16:57 |
| lucasagomes | will see if I can reproduce it here then | 16:57 |
| matty_dubs | How many hours until our meeting? My sense of which timezone governs it, and whether Europe is on summer time yet or not, is off. | 16:59 |
| NobodyCam | devananda: just to confirm we are holding off on lucasagomes parted / used gb patches untill after J opens? | 16:59 |
| devananda | NobodyCam: yea | 16:59 |
| devananda | matty_dubs: 2 | 16:59 |
| lucasagomes | matty_dubs, 2 hours | 16:59 |
| NobodyCam | matty_dubs: 2 | 16:59 |
| lucasagomes | heh | 16:59 |
| NobodyCam | :) | 16:59 |
| matty_dubs | Thanks devananda, lucasagomes, and NobodyCam ;) | 17:00 |
| NobodyCam | devananda: 83851 LGTM | 17:02 |
| *** derekh has quit IRC | 17:04 | |
| *** martyntaylor has left #openstack-ironic | 17:05 | |
| lucasagomes | Shrews, I think it didn't work because 0 came as a string from nova... I will test it here locally | 17:05 |
| lucasagomes | I could reproduce the problem | 17:05 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: If no swap is specified default to 1MB https://review.openstack.org/84167 | 17:06 |
| *** rwsu has quit IRC | 17:11 | |
| *** jistr has quit IRC | 17:14 | |
| *** vkozhukalov has joined #openstack-ironic | 17:26 | |
| *** eghobo has quit IRC | 17:27 | |
| NobodyCam | open call for meeting agenda items? anyone have anything they would like to see on hte agenda? | 17:28 |
| *** eghobo has joined #openstack-ironic | 17:28 | |
| *** overlayer has quit IRC | 17:29 | |
| JayF | Should we put the agent arch stuff back on the agenda since it didn't really happen last meeting? | 17:30 |
| JayF | jroll: ^ ? | 17:30 |
| jroll | probably | 17:30 |
| jroll | well | 17:30 |
| NobodyCam | JayF: this one: Driver architecture for Chassis object (Allow vendors to define behavior for the Chassis object like Auto node discovery etc) | 17:30 |
| lucasagomes | Shrews, I put another patch set up | 17:30 |
| * lucasagomes takes a break before meeting | 17:30 | |
| *** lucasagomes is now known as lucas-afk | 17:30 | |
| jroll | JayF: we kinda talked about it afterward in this channel and made some decisions | 17:30 |
| JayF | I'm OK with it happening or not, just wanted to make sure it wasn't needed vs being forgotten :) | 17:31 |
| jroll | they're documented on the wiki | 17:31 |
| jroll | yeah | 17:31 |
| jroll | I don't think I have anything I want to discuss on the agent topic | 17:31 |
| NobodyCam | :) | 17:33 |
| jroll | unless I make magic happen in the next 1.5 hours :P | 17:34 |
| *** martyntaylor has joined #openstack-ironic | 17:40 | |
| *** athomas has quit IRC | 17:41 | |
| *** rustlebee is now known as russellb | 17:47 | |
| NobodyCam | lucas-afk: just fyi: devtest.sh completed. with parted patches! | 17:50 |
| *** ndipanov_ has quit IRC | 17:53 | |
| openstackgerrit | A change was merged to openstack/ironic: Adds exact match filters to nova scheduler https://review.openstack.org/83734 | 18:00 |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic-python-agent: Adding DynamicLoopingCall around lookup https://review.openstack.org/83604 | 18:04 |
| NobodyCam | begal time... brb | 18:05 |
| Shrews | lucas-afk: it works! woot | 18:09 |
| Shrews | lucas-afk: +1 from me when you un-WIP it | 18:10 |
| devananda | Shrews: you can vote anytime | 18:13 |
| Shrews | devananda: i can, but if he's planning on pushing another change, i'd have to vote again :-P | 18:14 |
| devananda | ah | 18:14 |
| adam_g | Shrews, i seem to be having other issues this morning /w devstack + ironic: http://paste.ubuntu.com/7186094/ | 18:15 |
| Shrews | adam_g: eek. not seen that. | 18:15 |
| Shrews | adam_g: what's HEAD on your nova? | 18:16 |
| Shrews | i'm working off of ec4d88c2 | 18:16 |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic: Adding swift temp url support https://review.openstack.org/81391 | 18:17 |
| devananda | urgh | 18:17 |
| devananda | tempest random failure because of NodeLocked by periodic task | 18:17 |
| devananda | http://logs.openstack.org/72/83572/1/check/check-tempest-dsvm-ironic/8fe5c11/logs/screen-ir-api.txt.gz#_2014-03-31_16_42_30_801 | 18:17 |
| NobodyCam | adam_g: should Loading compute driver 'nova.virt.ironic.IronicDriver' be Loading compute driver 'ironic.nova.virt.ironic.IronicDriver' | 18:17 |
| devananda | this is why we need retry in the python client, not just in nova | 18:17 |
| devananda | or we should disable periodic tasks for the tempest tests | 18:17 |
| NobodyCam | devananda: I read that as we DO need retry logic in the client otherwise we will end up with random fails | 18:19 |
| devananda | NobodyCam: correct | 18:19 |
| devananda | there was a discussion last week about putting that retry into the nova driver code | 18:19 |
| devananda | because that's where eg. the glance retry logic is | 18:19 |
| devananda | my point is, anyone using ironic is going to want to retry against a set of temporary things like NodeLocked | 18:20 |
| adam_g | Shrews, c032ab8696aa4b2cb0cebd5f711e123b1e3b0ed7 | 18:20 |
| devananda | perhaps we should implement that inside the API service | 18:20 |
| NobodyCam | ya, but if we put it there then cli users will just have to live with random failures.. which I think is wrong | 18:20 |
| devananda | (that's a terrible idea) | 18:20 |
| devananda | NobodyCam: ... no. they do now | 18:20 |
| devananda | NobodyCam: right now, CLi users live with random NodeLocked failures | 18:20 |
| devananda | which sometimes causes tempest to fail | 18:21 |
| devananda | eg, http://logs.openstack.org/72/83572/1/check/check-tempest-dsvm-ironic/8fe5c11/logs/screen-ir-api.txt.gz#_2014-03-31_16_42_30_801 | 18:21 |
| adam_g | NobodyCam, hmm yeah. nova.conf's compute_driver was not set correctly. odd | 18:21 |
| Shrews | devananda: so, if you put it inside the client itself, you're then making decisions for the user/dev on which exceptions you'll do a retry. perhaps a dev would want a "one and done" call? | 18:22 |
| devananda | Shrews: right. another issue is we don't know how long the node will stay locked | 18:22 |
| devananda | so we can't retry-until-complete. that could be minutes today, possibly hours, depending on the type of work being done (eg, rebuild a raid) | 18:23 |
| Shrews | exactly. so an interactive user, say using the 'ironic' command line client, may wait for X period of time | 18:23 |
| NobodyCam | devananda: is there a way to retry only for NodeLocked by periodic task | 18:23 |
| devananda | NobodyCam: not today. and i dont think that's what matters | 18:24 |
| lucas-afk | Shrews, un-wipped :) | 18:24 |
| devananda | hypothetically a periodic task could also take a long lock | 18:24 |
| Shrews | +1'd | 18:25 |
| devananda | perhaps we should disable periodic tasks while tempest is running? | 18:25 |
| NobodyCam | devananda: that would solve the testing issues | 18:25 |
| devananda | and mean we dont test periodic tasks | 18:25 |
| devananda | heh | 18:25 |
| NobodyCam | but not user experence | 18:25 |
| * lucas-afk afk again | 18:26 | |
| devananda | I think a user can handle a "node locked, please retry" | 18:26 |
| devananda | *a user of the CLI | 18:26 |
| devananda | but what about horizon? | 18:26 |
| JayF | If it's OK for a CLI user, why isn't it OK for Horizon to throw a similar error? | 18:27 |
| devananda | if someone using horizon wants to make a change to a node, when they hit "save", and the node happens to be locked at that point .... | 18:27 |
| devananda | right | 18:27 |
| NobodyCam | also I hope we dont get any long running periodic tasks | 18:27 |
| devananda | so horizon needs to inform them of the error and allow them to hit "save" again w/o re-enterring all the data | 18:27 |
| Shrews | Then horizon should present a similar message to the user. "Please try again" | 18:27 |
| Shrews | or, code the retry logic themselves | 18:28 |
| Shrews | either way, it should be under their control | 18:28 |
| devananda | Shrews: "or code the retry logic themselves" -- no. | 18:28 |
| Shrews | por que? | 18:28 |
| NobodyCam | what if periodic_interval=60 got set to 1 the node would never unlock | 18:28 |
| JayF | NobodyCam: the task itself determines how often it runs, not that interval | 18:29 |
| devananda | Shrews: re-inventing the wheel leads to lots of duplicated/wasted effort, and lots of bad wheels | 18:29 |
| Shrews | devananda: i don't think it's possible to have a "one size fits all" solution here | 18:29 |
| JayF | NobodyCam: since my patch, the periodic_interval is just the minimum number of seconds that a periodic task will be checked for (i.e., check every 60s for something to do) | 18:29 |
| * Shrews needs to change location before the meeting. biab | 18:30 | |
| NobodyCam | :) | 18:30 |
| JayF | NobodyCam: although it's possible for someone to make a periodic task that wants to run ridiculously often (every few seconds) and that would be respected, I'd hope things like that would be nack'd in code review and prevented from locking when not actively making a change | 18:30 |
| adam_g | dguerri, around? | 18:31 |
| *** pcargnel has joined #openstack-ironic | 18:32 | |
| devananda | we could also add a Retry-After header to the response | 18:33 |
| *** agordeev2 has joined #openstack-ironic | 18:35 | |
| * Shrews back | 18:42 | |
| NobodyCam | WB Shrews | 18:42 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/ironic: Check that all po/pot files are valid https://review.openstack.org/84210 | 18:44 |
| Shrews | So, my point was, moving the retry logic to the client will likely *reduce* the occurrences of that exception from the user's viewpoint, but unless we retry forever, it will never completely eliminate the possiblity of it happening. So handling that failure it is still the responsibility of the calling party. So you really haven't reduced the duplicated effort there. | 18:45 |
| Shrews | But, it is entirely possible I'm waaaay offbase and severely caffeine deprived | 18:45 |
| * mordred hands Shrews coffee | 18:45 | |
| NobodyCam | morning mordred :) | 18:46 |
| * Shrews checks said coffee for suspicious chemicals | 18:46 | |
| * mordred hands Shrews suspicious chemicals | 18:46 | |
| Shrews | yay! | 18:46 |
| NobodyCam | lol | 18:46 |
| *** tatyana has joined #openstack-ironic | 18:48 | |
| Shrews | mordred: plz to place a bet on 31 red for me. all your monies. kthx | 18:50 |
| NobodyCam | ten minutes till meeting | 18:50 |
| NobodyCam | last chance stops... :) | 18:50 |
| *** dkehn_ has joined #openstack-ironic | 18:51 | |
| Shrews | mordred: assuming you're in vegas now | 18:51 |
| mordred | Shrews: safe assumption | 18:52 |
| matty_dubs | rloo: So your resubmitted ironic-install.rst starts calling Ironic "The Bare Metal Service." Where did that come from? | 18:52 |
| matty_dubs | I feel like I missed a memo. | 18:52 |
| NobodyCam | matty_dubs: thats what we are | 18:54 |
| NobodyCam | its the service we register with keystone | 18:54 |
| adam_g | Shrews, if you're still poking devstack beware of this change: https://review.openstack.org/#/c/82160/ | 18:55 |
| matty_dubs | NobodyCam: Ah-ha. I wasn't aware that it was a proper name registered with Keystone, vs. a description. | 18:55 |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic-python-agent: Adding oslo testing to improve debugging https://review.openstack.org/84212 | 18:55 |
| devananda | adam_g: wtf? | 18:56 |
| Shrews | adam_g: if that was wrong, how was it ever working? | 18:56 |
| adam_g | devananda, yeah. /me makes a note to review relevant devstack changes from now on | 18:57 |
| openstackgerrit | Pablo Fernando Cargnelutti proposed a change to openstack/ironic: Overwriting node_is_available in IronicDriver https://review.openstack.org/84214 | 18:57 |
| adam_g | Shrews, it wasn't wrong | 18:57 |
| adam_g | https://review.openstack.org/#/c/84209/ | 18:57 |
| Shrews | oh. haha! | 18:57 |
| devananda | adam_g: also, we need to get the virtual-ironic test working -- even if that means stripping out tests | 18:57 |
| devananda | adam_g: so that we can start trusting it | 18:57 |
| *** lucas-afk is now known as lucasagomes | 18:57 | |
| * NobodyCam heads over to -meeting start setting up chairs | 18:59 | |
| adam_g | devananda, yea | 19:00 |
| adam_g | devananda, finally got some feedback on that tempest scenario thing, which i was hoping to address when i started hitting devstack issues. | 19:02 |
| devananda | adam_g: awesome. please share your thoguhts in the meeting shortly | 19:04 |
| *** pcargnel has quit IRC | 19:04 | |
| *** eguz has joined #openstack-ironic | 19:06 | |
| *** eghobo has quit IRC | 19:10 | |
| *** john-n-seattle has quit IRC | 19:11 | |
| *** max_lobur has joined #openstack-ironic | 19:15 | |
| *** john-n-seattle has joined #openstack-ironic | 19:16 | |
| devananda | lucasagomes: side ping re https://review.openstack.org/#/c/83851/1 | 19:21 |
| devananda | lucasagomes: related to the exact match filters | 19:22 |
| lucasagomes | devananda, oh sure! told u I would test it | 19:22 |
| *** tatyana has quit IRC | 19:22 | |
| lucasagomes | devananda, will give it a quick go in y env | 19:22 |
| lucasagomes | (takes 10 min) | 19:22 |
| lucasagomes | and vote :) code lgtm just want to test it quickly | 19:22 |
| devananda | lucasagomes: ah, right. sorry to nag -- forgot :) | 19:22 |
| devananda | (was going trhough open tabs and saw it ...) | 19:23 |
| lucasagomes | devananda, np, I'm sorry I forgot it opened somewhere | 19:23 |
| adam_g | Shrews, so more things seem broken today. https://bugs.launchpad.net/ironic/+bug/1300396 in what logs were yous eeing the partitioning errors? | 19:31 |
| Shrews | adam_g: lucasagomes has a fix in for that | 19:32 |
| Shrews | adam_g: https://review.openstack.org/84167 | 19:32 |
| Shrews | adam_g: but it was in the conductor logs | 19:32 |
| adam_g | Shrews, hmm maybe something different | 19:33 |
| lucasagomes | devananda, | d75e5a38-09cf-4c53-a74e-305693103ae3 | ironictest | ACTIVE | - | Running | public=172.24.4.3 | | 19:33 |
| lucasagomes | worked +a ur patch :) thanks! | 19:34 |
| JoshNang | So, I made this bug https://bugs.launchpad.net/ironic/+bug/1297019 and closed it with this patch https://review.openstack.org/#/c/83604/, but it's not showing up in the bug as being fixed. I'm guessing its because ironic and ironic-python-agent are different repos? Any have any ideas how to get around that? | 19:36 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/ironic: Check that all po/pot files are valid https://review.openstack.org/84210 | 19:37 |
| devananda | lucasagomes: thanks much! | 19:38 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/ironic: Check that all po/pot files are valid https://review.openstack.org/84210 | 19:45 |
| Shrews | adam_g: i don't see that error in my nova compute logs at all. likely unrelated, as you suggested | 19:45 |
| adam_g | Shrews, hmm seems somewhat related. pulling in that patch fixes it.. now to figure out why | 19:57 |
| JayF | any idea on JoshNang's ^ bug above? I wonder if it's that we're using the ironic bugtracker for ironic-python-agent? | 19:58 |
| devananda | JoshNang: it may be that every project that has a unique repo needs a unique project in LP | 20:00 |
| JayF | that's what I was suspecting/afraid of | 20:00 |
| NobodyCam | great meeting all Thank you | 20:00 |
| JayF | something I should take up with -infra? | 20:00 |
| lucasagomes | aight, I'm done for today | 20:00 |
| devananda | JayF: mm, nope. I can create it | 20:00 |
| lucasagomes | have a good night everyone! | 20:00 |
| *** romcheg has joined #openstack-ironic | 20:00 | |
| JayF | devananda: sweet, thanks | 20:00 |
| lucasagomes | night devananda NobodyCam JayF Shrews adam_g etc etc etc :) | 20:01 |
| *** lucasagomes is now known as lucas-dinner | 20:01 | |
| NobodyCam | night lucas-dinner :) | 20:01 |
| jroll | see ya lucas-dinner :) | 20:01 |
| Shrews | night | 20:01 |
| devananda | g'night lucas-dinner | 20:01 |
| adam_g | cheers | 20:01 |
| JoshNang | awesome thanks devananda | 20:01 |
| devananda | lifeless: hi! have a minute to disucss https://review.openstack.org/#/c/83572/ ? | 20:04 |
| lifeless | for you, two | 20:07 |
| *** romcheg has quit IRC | 20:07 | |
| lifeless | JoshNang: infra have a system for mappig code -> projects when the names differ | 20:08 |
| * NobodyCam starts playing jeperody music | 20:08 | |
| lifeless | JoshNang: ask in #openstack-infra | 20:08 |
| Shrews | NobodyCam, devananda, JoshNang, et.al.: The bug for the unit test memory consumption problem: https://bugs.launchpad.net/ironic/+bug/1300420 | 20:08 |
| NobodyCam | Shrews: TY :) | 20:09 |
| devananda | Shrews: http://logs.openstack.org/05/83105/3/check/gate-ironic-python26/dab25e8/testr_results.html.gz | 20:10 |
| NobodyCam | brb... | 20:10 |
| devananda | Shrews: any of your local reproductions of that bug point to any particular unit tests? | 20:11 |
| Shrews | devananda: i *believe* i have noticed the test_distribution_three_replicas mentioned in another failure on my machine | 20:11 |
| Shrews | i started looking at that particular test for a bit, but then i realized it was sunday evening and stopped :) | 20:12 |
| *** max_lobur has quit IRC | 20:12 | |
| *** max_lobur has joined #openstack-ironic | 20:12 | |
| *** vkozhukalov has quit IRC | 20:14 | |
| devananda | Shrews: hah | 20:14 |
| JoshNang | lifeless: ack | 20:15 |
| JoshNang | Shrews: awesome, thanks | 20:15 |
| devananda | lifeless: so I had posted a response on https://review.openstack.org/#/c/83572/ -- awaiting your reply, when you have two minutes :) | 20:16 |
| lifeless | devananda: replied | 20:17 |
| devananda | lifeless: hmm. i had the impression from SpamapS that the status quo was broken by that exception | 20:18 |
| devananda | lifeless: otherwise I wouldn't have worried about it for the RC | 20:18 |
| lifeless | SpamapS: ^ | 20:18 |
| lifeless | devananda: there are folk that really don't like error messages except when there is well, an error. | 20:19 |
| lifeless | however, I'm one of them - and this *is* an error. | 20:19 |
| devananda | lifeless: I agree, and would like to fix it. And I agree this isn't an ideal fix. | 20:19 |
| lifeless | I don't think its a fix at all, because once the option exists, this should be backed out | 20:19 |
| lifeless | because failing to load a requested driver isn't a 'warning', its a zomg. | 20:20 |
| devananda | yes, conductor not able to find an expected driver during start-up should cause the conductor to bail | 20:21 |
| russell_h | JoshNang: ok, I remember why I did this weird math in the agent. I wanted to apply jitter to the heartbeats themselves, even when they were working correctly. I played around with a few things, but this seemed like the simplest way to factor things: first decide a "deadline" for the next attempt based on either a backoff factor (in case of an error) or the heartbeat-before from the API, then apply some multiplier that worked in | 20:21 |
| russell_h | either case to that. | 20:21 |
| devananda | conductor failing to find a driver requested via the API which the conductor does not know about -- user error. ironic was conflating these two until now. | 20:21 |
| *** rwsu has joined #openstack-ironic | 20:22 | |
| JoshNang | russell_h: ooooo that makes sense | 20:22 |
| russell_h | JoshNang: there are a lot of fine ways to do this though, I'm not overly attached to that one | 20:23 |
| devananda | lifeless: adding a CONF option now is something I would strongly prefer to avoid | 20:24 |
| lifeless | devananda: is a CONF option the right way to fix it ? | 20:25 |
| devananda | lifeless: and this was not a fatal error before, so that's another large change to make right now | 20:25 |
| devananda | lifeless: yes | 20:25 |
| devananda | lifeless: allow deployer to set what drivers the conductor is expected to load | 20:25 |
| devananda | lifeless: with default set to upstream reference drivers | 20:25 |
| devananda | that would hide the fake* from production | 20:25 |
| devananda | and allow deployers to add more if they want to (eg, seamicro, ilo, etc) install those deps | 20:25 |
| devananda | drivers would live in trunk, but be disabled by default, and enabled in third-party CI systems only | 20:26 |
| lifeless | so its up to you, like I say, functionally it works right now | 20:26 |
| lifeless | the bug can be paraphrased as 'its scary' | 20:26 |
| *** agordeev2 has quit IRC | 20:26 | |
| lifeless | unless you do the right fix, its going to be wrong for some users in I no matter what | 20:26 |
| JoshNang | russell_h: ok, i'll mull it over a bit and throw up a patch | 20:28 |
| russell_h | JoshNang: ok. Dont' depend on it in the current change IMO, it can be refactored later. | 20:28 |
| JoshNang | russell_h: definitely. | 20:29 |
| devananda | lifeless: ack | 20:29 |
| lifeless | devananda: given that the least risk thing is to do nothing. | 20:29 |
| devananda | lifeless: so it may not be quite as much an issue as you think, but IMBW | 20:30 |
| devananda | lifeless: let me clarify | 20:30 |
| devananda | lifeless: the DriverNotFound exception that is caught in the patch I proposed is only beign raised in one spot | 20:31 |
| devananda | https://github.com/openstack/ironic/blob/master/ironic/drivers/pxe.py#L102 | 20:31 |
| devananda | it's not going to hide "zomg ipmitool isn't installed" | 20:31 |
| devananda | or it shouldn't... i'll run it to check | 20:32 |
| lifeless | devananda: its going to hide seamicro-can't-be-loaded, right ? | 20:32 |
| devananda | right | 20:32 |
| lifeless | thats the problem | 20:32 |
| devananda | that and only that | 20:32 |
| lifeless | for seamicro users thats a fatal error not a chiding | 20:33 |
| devananda | ok | 20:33 |
| *** martyntaylor has left #openstack-ironic | 20:33 | |
| devananda | it's cleaning up the printed tracebacks | 20:33 |
| devananda | what if it logged a more clear ERROR | 20:33 |
| devananda | eg, "Failed to load %s because %y" | 20:34 |
| NobodyCam | I like more info to the user :) | 20:34 |
| russell_h | wouldn't this mean that the 99.99% of users _not_ using seamicro are going to have a ton of error logs unless they install all the seamicro dependencies? | 20:35 |
| devananda | russell_h: that's what I'm trying to avoid | 20:35 |
| *** martyntaylor has joined #openstack-ironic | 20:35 | |
| russell_h | right | 20:35 |
| jroll | +1 for being a scary (one line) error message | 20:35 |
| lifeless | devananda: so do the right fix? | 20:35 |
| devananda | lifeless: um... no ... | 20:35 |
| devananda | lifeless: I'm suggesting cleaning up a traceback and logging a helpful error message | 20:35 |
| jroll | +1 for a config option to die/warn/suppress when that happens | 20:36 |
| devananda | lifeless: as a fix for right now | 20:36 |
| *** dwalleck has joined #openstack-ironic | 20:36 | |
| devananda | and a CONF option as the "right" fix for Juno | 20:36 |
| lifeless | devananda: if you hide the traceback, how will a seamicro user with the failing driver debug the failure cause? | 20:36 |
| devananda | lifeless: by logging a helpful error message | 20:36 |
| devananda | :) | 20:36 |
| *** eguz has quit IRC | 20:36 | |
| lifeless | devananda: no like | 20:36 |
| devananda | taht exception is being intentionally thrown | 20:36 |
| lifeless | devananda: say they install the librrary | 20:36 |
| devananda | it doesn't need a traceback | 20:36 |
| lifeless | but it still fails to load | 20:36 |
| lifeless | how do they diagnose that? | 20:37 |
| devananda | lifeless: it's LESS helpful today | 20:37 |
| devananda | lifeless: http://paste.openstack.org/show/74697/ | 20:37 |
| lifeless | some of the most frustrating debugging situations I've had is where someone has caught an exception to give a friendly experience, and destroyed the evidence of the actual problem. | 20:38 |
| russell_h | I actually think adding a CONF option now is going to be less painful than doing it later | 20:38 |
| devananda | that traceback is totally not helpful when diagnosing "oh I forgot to install python-seamicroclient" | 20:38 |
| russell_h | its pretty annoying that in a "production" Ironic deployment you have a ton of "fake_" drivers and whatnot visible | 20:38 |
| devananda | russell_h: I agree | 20:38 |
| lifeless | devananda: you agree that its annoying, or that adding an option now is less painful ? | 20:38 |
| devananda | that it's annoying | 20:39 |
| NobodyCam | devananda: what about something like : http://paste.openstack.org/show/7K7N4aoUQs0fRRFJm125/ | 20:39 |
| devananda | adding the option now would also be less painful | 20:39 |
| NobodyCam | at least for I | 20:39 |
| russell_h | devananda: what about adding an option, and logging an error but defaulting to the current behavior if its not specified | 20:39 |
| devananda | but i'm trying to avoid adding a CONF option on the day we tag the RC, yanno? | 20:39 |
| lifeless | better than the day after | 20:40 |
| rloo | how urgent is this? I don't see why it can't be backported the day/week after rc | 20:40 |
| russell_h | or warning maybe | 20:40 |
| rloo | it == the 'right' solution | 20:40 |
| devananda | NobodyCam: no, won't work. | 20:40 |
| NobodyCam | :( | 20:40 |
| devananda | NobodyCam: that'll cause it to be unavailable in ironic packages | 20:40 |
| devananda | rloo: it's an issue of changing CONF stuff in a release | 20:41 |
| devananda | rloo: *during the release. down stream folks are going to be affected. | 20:42 |
| rloo | devananda: then what about doing the fix before RC, and delaying RC til later this week? | 20:42 |
| rloo | or not doing anything and leaving this as-is. I'm not convinced this is that urgent to fix for RC (the trace etc) | 20:42 |
| rloo | this isn't meant for production as-is anyway | 20:43 |
| openstackgerrit | A change was merged to openstack/ironic: If no swap is specified default to 1MB https://review.openstack.org/84167 | 20:43 |
| rloo | it is a one-time error in the log files when ironic starts up. | 20:44 |
| devananda | ack | 20:45 |
| rloo | matty_dubs, wrt the install and 'Bare Metal Service', NobodyCam's comments made me realize that it wasn't 'Ironic Service' (apparent with the other Services like Image Service, not Glance Service'). That's why I changed but forgot to mention. Sorry. | 20:47 |
| *** romcheg has joined #openstack-ironic | 20:49 | |
| matty_dubs | rloo: Yeah, it makes sense now. I hadn't realized or considered the distinction now. | 20:49 |
| matty_dubs | It still looks odd to me, though, but I guess I'll get used to it. ;) | 20:49 |
| NobodyCam | matty_dubs: & rloo: my next queston would be is it "Baremetal" or "Bare Metal" | 20:50 |
| *** romcheg has quit IRC | 20:51 | |
| openstackgerrit | A change was merged to openstack/ironic: ironic_host_manager was missing two stats https://review.openstack.org/83851 | 20:52 |
| NobodyCam | any reason to keep Integration with TripleO on the agenda? | 20:53 |
| *** urulama has quit IRC | 20:54 | |
| rloo | NobodyCam: that was what I was hoping someone else would tell me! | 20:55 |
| rloo | NobodyCam: I mean Bare Metal, not the agenda stuff. | 20:55 |
| NobodyCam | hehehe :) I have seen it both ways several times | 20:55 |
| rloo | matty_dubs: yeah, I'd rather it just said 'ironic' and forget the service part too, but all the other install guides have XX Service. | 20:55 |
| *** jdob has quit IRC | 20:56 | |
| rloo | NobodyCam: I've seen it both ways too, and maybe neither are English cuz it is a new term? 'Baremetal' would be easier then. | 20:56 |
| NobodyCam | devananda: do you know ? ^^^ is it "baremetal" or "bare metal"? | 20:56 |
| *** max_lobur has quit IRC | 20:57 | |
| rloo | NobodyCam: or 'bare-metal', but I don't know that I've seen it hyphenated. | 20:57 |
| devananda | i haven't seen it hyphenated anywhere | 20:57 |
| *** max_lobur has joined #openstack-ironic | 20:57 | |
| devananda | proper way is with a space | 20:57 |
| devananda | http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n124 | 20:57 |
| NobodyCam | same here... thou it dose make the most sense to me | 20:57 |
| NobodyCam | ahh | 20:58 |
| NobodyCam | with the space :) | 20:58 |
| rloo | thx devananda! | 20:59 |
| *** harlowja is now known as harlowja_away | 21:00 | |
| JayF | JoshNang: devananda: lifeless: https://review.openstack.org/#/c/84250/ here is the fix the infra guys prescribed for fixing the issue with Josh's commit not being reflected in his bug. | 21:03 |
| *** eghobo has joined #openstack-ironic | 21:03 | |
| JoshNang | thanks for grabbing that JayF | 21:03 |
| lifeless | devananda: I think putting up the real patch and letting deployers than want a pre-integrated Ironic backport it is a reasonable compromised | 21:04 |
| devananda | lifeless: i didn't intend to not put up the patch :) | 21:04 |
| devananda | lifeless: it's more a question about the release / RC | 21:05 |
| devananda | in this case, writing it isn't taking much longer than talking about it, s oi'm jsut writing it | 21:05 |
| lifeless | devananda: yeah, I am talking about the release interactions | 21:06 |
| devananda | lifeless: ah, gotcha | 21:07 |
| SpamapS | lifeless: I have read the backscroll and agree with the position you have taken. :) It's just scary, not something to bend over backwards fixing. | 21:10 |
| SpamapS | devananda: ^^ | 21:10 |
| devananda | SpamapS: thanks | 21:10 |
| *** linggao has quit IRC | 21:14 | |
| *** jbjohnso_ has quit IRC | 21:17 | |
| NobodyCam | interresting POV: http://gigaom.com/2014/03/29/rip-the-server-its-time-to-breathe-the-air-of-cloud-connection/ | 21:17 |
| *** harlowja_away is now known as harlowja | 21:19 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Manually update all translated strings https://review.openstack.org/84259 | 21:22 |
| NobodyCam | devananda: thts a one time patch, and we wont have to back it out? | 21:23 |
| devananda | NobodyCam: see -infra. clarkb thinks it will be fine | 21:24 |
| NobodyCam | if I read the commit message correctly | 21:24 |
| devananda | yea. there's an issue with the jenkins periodic job | 21:24 |
| devananda | globally, not just affecting ironic | 21:24 |
| devananda | clark proposed a fix to infra, but i dunno when that'll land | 21:24 |
| devananda | since, you know, it affects everyone | 21:25 |
| *** jgrimm has quit IRC | 21:25 | |
| NobodyCam | devananda: ack .. these files look good to me!!! | 21:26 |
| NobodyCam | at least the ones I checked | 21:26 |
| *** lucas-dinner has quit IRC | 21:32 | |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic-python-agent: Adding oslo testing to improve debugging https://review.openstack.org/84212 | 21:36 |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic-python-agent: Adding DynamicLoopingCall around lookup https://review.openstack.org/83604 | 21:38 |
| *** max_lobur has quit IRC | 21:44 | |
| *** max_lobur has joined #openstack-ironic | 21:45 | |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic-python-agent: Adding DynamicLoopingCall around lookup https://review.openstack.org/83604 | 21:54 |
| NobodyCam | brb | 21:59 |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Add kernel parameter support https://review.openstack.org/84264 | 22:03 |
| *** matty_dubs is now known as matty_dubs|gone | 22:05 | |
| *** dwalleck has quit IRC | 22:06 | |
| *** max_lobur has quit IRC | 22:10 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Better handling of missing drivers https://review.openstack.org/83572 | 22:16 |
| devananda | I need to step out for food... | 22:17 |
| devananda | if anyone feels like testing i18n support | 22:17 |
| devananda | in theory, this patch https://review.openstack.org/#/c/84259/1 plus this patch https://review.openstack.org/#/c/74626/ might do the trick | 22:18 |
| openstackgerrit | A change was merged to openstack/ironic-python-agent: Adding DynamicLoopingCall around lookup https://review.openstack.org/83604 | 22:18 |
| devananda | I'll test when I get back. but I think that's the last thing blocking RC | 22:18 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Better handling of missing drivers https://review.openstack.org/83572 | 22:21 |
| NobodyCam | thats not looking for food.. thats ushing patches up :-p | 22:22 |
| NobodyCam | hehhehehe | 22:22 |
| devananda | heh. well it needed a quick fix ... | 22:23 |
| * devananda really goes looking for food | 22:23 | |
| NobodyCam | :) enjoy :) | 22:23 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/ironic: Add tox target and script for nova driver tests https://review.openstack.org/84033 | 22:33 |
| openstackgerrit | Jay Faulkner proposed a change to openstack/ironic-python-agent: Updated from global requirements https://review.openstack.org/83944 | 22:33 |
| openstackgerrit | A change was merged to openstack/ironic-python-agent: Updated from global requirements https://review.openstack.org/83944 | 22:38 |
| *** radsy has joined #openstack-ironic | 22:39 | |
| NobodyCam | rebuilding with 84259 then just need to figure out how to set the lang to test :) | 22:50 |
| *** uberj has joined #openstack-ironic | 22:51 | |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Add kernel parameter support https://review.openstack.org/84264 | 23:09 |
| *** Haomeng|2 has joined #openstack-ironic | 23:15 | |
| *** Haomeng has quit IRC | 23:15 | |
| *** rloo has quit IRC | 23:16 | |
| *** rloo has joined #openstack-ironic | 23:17 | |
| *** dwalleck has joined #openstack-ironic | 23:17 | |
| *** rloo has quit IRC | 23:19 | |
| *** rloo has joined #openstack-ironic | 23:20 | |
| *** rloo has quit IRC | 23:20 | |
| *** rloo has joined #openstack-ironic | 23:20 | |
| *** rloo has quit IRC | 23:20 | |
| *** rloo has joined #openstack-ironic | 23:21 | |
| dwalleck | Is anyone aware of any breaking issues with devstack + ironic at the moment? | 23:23 |
| NobodyCam | check compute driver | 23:23 |
| NobodyCam | in nova.conf | 23:24 |
| dwalleck | Thanks, checking | 23:24 |
| dwalleck | nova.virt.ironic.IronicDriver | 23:25 |
| dwalleck | Which I'm guessing is wrong at this point | 23:25 |
| NobodyCam | ironic.nova.virt.ironic.IronicDriver | 23:25 |
| NobodyCam | :) | 23:25 |
| dwalleck | Should I be overwriting that in my conf for devstack? | 23:26 |
| NobodyCam | ya when did you last pull... https://review.openstack.org/#/c/84209/ that if I recall is the fix | 23:28 |
| NobodyCam | and it looks like it landed an hour or so ago | 23:28 |
| NobodyCam | :-p | 23:28 |
| dwalleck | Early this afternoon | 23:28 |
| dwalleck | Good, not going crazy then! Thanks | 23:28 |
| NobodyCam | lol it all in the timeing | 23:29 |
| NobodyCam | timing even | 23:29 |
| NobodyCam | lol | 23:29 |
| NobodyCam | :-) | 23:29 |
| Shrews | dwalleck: also need this patch, which landed today: https://review.openstack.org/84167 | 23:29 |
| NobodyCam | nope your not nuts | 23:29 |
| NobodyCam | Shrews: TY | 23:29 |
| dwalleck | Thanks, I'll give it a go again | 23:30 |
| dwalleck | I _will_ make this work lol | 23:30 |
| NobodyCam | dwalleck: just set ironic.nova.virt.ironic.IronicDriver | 23:30 |
| NobodyCam | as compute driver | 23:30 |
| NobodyCam | I think that in localrc / stackrc | 23:31 |
| NobodyCam | or pull laest :) | 23:31 |
| *** rloo has quit IRC | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!