Thursday, 2014-03-13

NobodyCamlol, theres got to be a better way to get what I want00:00
NobodyCamcat $TE_DATAFILE| jq -r '.nodes[] | {("mac"): .mac, ("type"): .virt_type} |  tostring ' | grep $MAC | jq '.type'00:00
*** jbjohnso_ has quit IRC00:07
*** lazy_prince has quit IRC00:12
*** lazy_prince has joined #openstack-ironic00:13
*** lazy_prince is now known as killer_prince00:13
*** rpodolyaka has quit IRC00:17
*** Lotus907efi is now known as Lotus907efi-00:23
*** MarkAtwood has quit IRC00:24
*** matsuhashi has joined #openstack-ironic00:27
*** rpodolyaka has joined #openstack-ironic00:27
*** EmilienM has quit IRC00:30
*** EmilienM has joined #openstack-ironic00:35
lifelessNobodyCam: what do you want00:48
*** JoshNang has quit IRC00:50
*** JoshNang has joined #openstack-ironic00:50
*** JayF has quit IRC00:52
*** JayF has joined #openstack-ironic00:52
NobodyCamlifeless: to get per node virsh_type from $TE_DATAFILE only having the mac address01:06
lifelessNobodyCam: wait, why?01:07
NobodyCamjust sitting down for dinner... back in a bit :)01:08
*** matsuhashi has quit IRC01:11
*** matsuhashi has joined #openstack-ironic01:14
NobodyCamsorry 'bout that lifeless01:24
NobodyCami'm looking at lines 47 thru 50 of https://review.openstack.org/#/c/72969/14/scripts/register-nodes01:25
*** jbjohnso_ has joined #openstack-ironic01:26
*** matsuhashi has quit IRC01:31
*** matsuhashi has joined #openstack-ironic01:31
lifelessNobodyCam: ok01:32
*** matsuhashi has quit IRC01:32
lifelessNobodyCam: so that should build on the refactoring arc stevenk is on01:32
lifelessNobodyCam: which will make it massively simpler01:32
NobodyCamyes, :) that just seeing if I could get it.01:34
*** matsuhashi has joined #openstack-ironic01:35
NobodyCamI was going to at os-apply-config to if I could filter better01:36
lifelessNobodyCam: you don't need to start with the mac01:37
lifelessNobodyCam: a for loop over the nodes vector is all that you 'll need01:37
NobodyCamya :) I'll push up a new rev in the morning.. :)01:39
*** jbjohnso_ has quit IRC01:41
*** blamar has quit IRC01:41
*** nosnos has joined #openstack-ironic01:46
openstackgerritJenkins proposed a change to openstack/ironic: Updated from global requirements  https://review.openstack.org/7933401:48
*** max_lobur has quit IRC01:50
openstackgerritRuby Loo proposed a change to openstack/ironic: Delete node while waiting for deploy  https://review.openstack.org/8014101:52
*** rpodolyaka has quit IRC01:58
openstackgerrityangxurong proposed a change to openstack/ironic: Do not use __builtin__ in python3  https://review.openstack.org/8014402:00
*** killer_prince is now known as lazy_prince02:07
*** rpodolyaka has joined #openstack-ironic02:08
*** rpodolyaka has quit IRC02:21
*** vkozhukalov has quit IRC02:25
openstackgerritFengqian Gao proposed a change to openstack/ironic: Use range instead xrange to keep python 3.X compatibility  https://review.openstack.org/7987502:29
*** matsuhashi has quit IRC02:45
*** matsuhashi has joined #openstack-ironic02:45
*** killer_p- has joined #openstack-ironic02:45
*** killer_p- is now known as killer_prince02:46
*** matsuhashi has quit IRC02:46
*** matsuhashi has joined #openstack-ironic02:46
*** matsuhashi has quit IRC02:47
*** matsuhashi has joined #openstack-ironic02:47
*** matsuhashi has quit IRC02:48
*** lazy_prince has quit IRC02:48
*** matsuhashi has joined #openstack-ironic02:49
*** matsuhashi has quit IRC02:49
*** matsuhashi has joined #openstack-ironic02:50
*** rpodolyaka has joined #openstack-ironic02:53
*** rloo has quit IRC02:53
*** rpodolyaka has quit IRC02:54
*** rpodolyaka has joined #openstack-ironic02:54
openstackgerrityangxurong proposed a change to openstack/ironic: Do not use __builtin__ in python3  https://review.openstack.org/8014403:34
*** matsuhashi has quit IRC03:49
*** vkozhukalov has joined #openstack-ironic03:51
*** rpodolyaka has quit IRC03:51
*** harlowja is now known as harlowja_away04:20
*** rpodolyaka has joined #openstack-ironic04:22
*** harlowja_away is now known as harlowja04:24
*** eghobo has joined #openstack-ironic04:26
*** matsuhashi has joined #openstack-ironic04:47
*** lazy_prince has joined #openstack-ironic04:49
*** blamar has joined #openstack-ironic05:15
*** blamar has quit IRC05:19
*** blamar has joined #openstack-ironic05:20
*** mrda is now known as mrda_away05:36
*** harlowja is now known as harlowja_away05:53
*** blamar has quit IRC06:00
*** blamar has joined #openstack-ironic06:01
*** killer_prince has quit IRC06:05
openstackgerritJenkins proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/7886206:07
openstackgerritRussell Haering proposed a change to openstack/ironic: Expose API for fetching a single driver  https://review.openstack.org/8018706:39
*** pradipta_away is now known as pradipta06:41
*** eghobo has quit IRC06:41
*** killer_prince has joined #openstack-ironic06:46
openstackgerritRussell Haering proposed a change to openstack/ironic: Expose API for fetching a single driver  https://review.openstack.org/8018707:00
*** killer_prince has quit IRC07:05
*** rpodolyaka has quit IRC07:12
*** vkozhukalov has quit IRC07:16
*** loki_ has joined #openstack-ironic07:18
*** max_lobur has joined #openstack-ironic07:31
*** matsuhashi has quit IRC07:32
*** matsuhashi has joined #openstack-ironic07:32
*** nosnos has quit IRC07:32
*** nosnos_ has joined #openstack-ironic07:32
*** linuxgeek has joined #openstack-ironic07:39
*** linuxgeek has quit IRC07:40
*** killer_p- has joined #openstack-ironic07:42
*** killer_p- is now known as killer_prince07:42
GheRiveromorning all07:47
lifelesso/08:03
*** saju_m has joined #openstack-ironic08:12
*** jistr has joined #openstack-ironic08:21
*** yuriyz has joined #openstack-ironic08:42
openstackgerritGhe Rivero proposed a change to openstack/ironic: Fix misspelled impi to ipmi  https://review.openstack.org/8021608:56
*** derekh has joined #openstack-ironic09:00
*** matsuhashi has quit IRC09:07
*** romcheg has joined #openstack-ironic09:07
*** athomas has joined #openstack-ironic09:08
*** nosnos has joined #openstack-ironic09:11
*** nosnos_ has quit IRC09:11
*** jrist has quit IRC09:11
*** matsuhashi has joined #openstack-ironic09:12
*** lucasagomes has joined #openstack-ironic09:24
*** jrist has joined #openstack-ironic09:24
openstackgerritGhe Rivero proposed a change to openstack/ironic: SeaMicro use device parameter for set_boot_device  https://review.openstack.org/8022109:30
*** martyntaylor has joined #openstack-ironic09:34
*** matsuhashi has quit IRC09:44
*** matsuhashi has joined #openstack-ironic09:47
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Make the Nova Ironic driver to wait for ACTIVE  https://review.openstack.org/7990609:48
*** tatyana has joined #openstack-ironic09:50
*** max_lobur has quit IRC09:50
*** ndipanov_gone is now known as ndipanov09:51
*** killer_p- has joined #openstack-ironic09:58
*** killer_prince has quit IRC09:59
*** killer_p- is now known as killer_prince09:59
*** athomas has quit IRC10:03
*** killer_prince has quit IRC10:03
*** killer_p- has joined #openstack-ironic10:04
*** killer_p- is now known as killer_prince10:04
*** athomas has joined #openstack-ironic10:11
mdurnosvistovMorning all :)10:12
yuriyzmorning Ironic10:12
*** blamar has quit IRC10:24
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Make the Nova Ironic driver to wait for ACTIVE  https://review.openstack.org/7990610:34
lucasagomesyuriyz, morning11:00
lucasagomesyuriyz, r u going to update 68697?11:00
*** matsuhashi has quit IRC11:11
*** matsuhashi has joined #openstack-ironic11:12
*** max_lobur has joined #openstack-ironic11:20
yuriyzmorning lucasagomes, plan to update today11:21
*** pradipta is now known as pradipta_away11:25
openstackgerritA change was merged to openstack/ironic: Set the database.connection option default value  https://review.openstack.org/8000311:32
*** jgrimm has joined #openstack-ironic11:39
openstackgerritGhe Rivero proposed a change to openstack/ironic: SeaMicro use device parameter for set_boot_device  https://review.openstack.org/8022111:41
*** lazy_prince has quit IRC11:45
*** loki_ has quit IRC11:46
openstackgerritYuriy Zveryanskyy proposed a change to openstack/ironic: Check BMC availability in ipmitool 'validate' method  https://review.openstack.org/6869711:52
*** matsuhashi has quit IRC12:11
*** matsuhashi has joined #openstack-ironic12:12
*** romcheg has quit IRC12:14
*** nosnos has quit IRC12:15
Shrewsgood morning ironic12:15
*** matsuhashi has quit IRC12:16
agordeevShrews: morning!12:17
Shrewsagordeev: hi! i was able to successfully connect to devstack vm's yesterday! it was a tftp problem12:18
agordeevShrews: great, i'd read the backlog. It's still possible to use atftpd. Actually the problem is in deploy-agent script, but it's possible to work around it on tftp side.12:20
Shrewsagordeev: yes. i got it to work with a simple symlink in $IRONIC_TFTPBOOT_DIR/tftpboot12:21
Shrewsbut adam_g's patch makes devstack align more closely with tripleo testing12:22
agordeevmore terrible thing is that i've already fixed it in my GH repo. About month ago, and totally forgot to file bug/publish comment/etc12:23
Shrewslol! we wondered why it was working for you  :)12:23
agordeevhttps://github.com/ninja-cat/diskimage-builder/commit/fe16251f4419c0d5d2e12bbd79f36db3edfe52f7 due to that, my deploy-agent works successfully.12:23
dtantsurifarkas, hi. Did you try making notes on running devstack+ironic on Fedora? It looks like a lot of differences with Ubuntu..12:24
agordeevShrews: also devstack can be fixed without switching to tftpd-hpa. Just with 3 additional lines12:25
*** matsuhashi has joined #openstack-ironic12:26
ifarkasdtantsur, hey. I had no trouble running devstack with ironic on fedora. do you want to setup ironic with the nova driver?12:26
dtantsurifarkas, I'm not sure, just following https://etherpad.openstack.org/p/IronicAndDevstackAgain12:27
dtantsurifarkas, I wonder how you could experience no problems with ssh opening 2222 port... did you use setenforce 0?12:27
agordeevShrews: http://paste.openstack.org/show/73375/ something like that.12:27
ifarkasdtantsur, right. so that etherpad is specifically testing the CI patch, which sets up devstack with the nova driver12:27
ifarkasdtantsur, you might rather want to use this one: https://wiki.openstack.org/wiki/Ironic/Devstack12:28
dtantsurifarkas, from what I already encountered: no libvirtd group under F20, SELinux prevents access to 2222 port by SSH, now SSH started, but I cannot log into it using ironic key12:29
dtantsurifarkas, is https://wiki.openstack.org/wiki/Ironic/Devstack different with respect to all these?12:29
Shrewsagordeev: i did it with one line, though it is a hack: http://paste.openstack.org/show/73376/   :)12:29
ifarkasdtantsur, yes, the latter guide works for me on Fedora too12:30
dtantsurifarkas, ok, thanks12:30
ifarkasdtantsur, I am not sure if the CI patch is supposed to work on Fedora. If yes, it might worth reporting the issues12:31
ifarkasdtantsur, looking at the patch, there are certain parts which are not specific to the CI, so I would suggest to report the issues on the patch12:40
lucasagomesyuriyz, ack thanks :)12:40
*** lucasagomes is now known as lucas-hungry12:40
*** linggao has joined #openstack-ironic12:53
*** rloo has joined #openstack-ironic12:54
*** rloo has quit IRC13:00
*** rloo has joined #openstack-ironic13:00
*** rloo has quit IRC13:02
*** romcheg has joined #openstack-ironic13:02
*** rloo has joined #openstack-ironic13:03
*** rloo has quit IRC13:03
*** toure has quit IRC13:03
*** rloo has joined #openstack-ironic13:04
*** rloo has quit IRC13:09
*** rloo has joined #openstack-ironic13:09
*** rloo has quit IRC13:09
*** rloo has joined #openstack-ironic13:10
*** matty_dubs|gone is now known as matty_dubs13:12
*** rloo has quit IRC13:13
*** rloo has joined #openstack-ironic13:13
*** mdurnosvistov has quit IRC13:22
*** jdob has joined #openstack-ironic13:26
*** yuriyz has quit IRC13:41
*** matsuhashi has quit IRC13:42
*** mdurnosvistov has joined #openstack-ironic13:54
devanandamorning, all13:57
devanandaagordeev: hi! just read scrollback -- has that /tftpboot fix been applied to the devstack patch?13:59
devanandaifarkas: the CI/devstack patch should work on both fedora & ubutu. where it doesn't, it should be fixed :)14:00
*** jbjohnso_ has joined #openstack-ironic14:01
ifarkasdevananda, good to know, thanks!14:02
ifarkasdevananda, and good morning! :-)14:02
agordeevdevananda: hi! yup, it was applied by Adam14:03
ifarkasdtantsur, ^14:03
devanandafwiw, I just added a note at the top of https://wiki.openstack.org/wiki/Ironic/Devstack14:03
dtantsurdevananda, I've reported a couple of issues, but honestly I gave up in the middle trying to fight ssh14:04
devanandadtantsur: were you trying based on the devstack patch (https://review.openstack.org/#/c/70348/) or ..?14:05
agordeevdtantsur: what's the problem with ssh? could it be default firewall setting?14:06
dtantsurdevananda, yes, patchset 22 using https://etherpad.openstack.org/p/IronicAndDevstackAgain14:06
dtantsuragordeev, SELinux first, then - don't know. I can start from scratch, capturing everything that does not work14:06
dtantsuragordeev, ssh is only allowed to use 22 by default14:06
dtantsuragordeev, you need like semanage port -a -t ssh_port_t -p tcp 2222 (not sure about correctness, need to check)14:07
agordeevdevananda: on what fedora version should CI/devstack work?14:08
devanandadtantsur: a few google searches suggest that devstack in general doesn't work with selinux14:09
devanandadtantsur: not that google search results are authoritative, but ... http://devstack.org/faq.html14:09
rloodevananda: when doing tear-down, is that 'unprovisioning' or 'deprovisioning'?14:11
dtantsurdevananda, honestly, I can't figure out what they mean in this FAQ: do ther support SEL or not; anyway, the remaining devstack seems to work _somehow_ with SEL14:11
devanandarloo: neither word is in miriam-webster14:12
devananda*merriam-webster14:12
rloodevananda: what about oxford? :-) So when we use the term 'provisioning' does that include deleting/tear-down? I don't think so.14:13
rloodevananda: https://review.openstack.org/#/c/80141/1/ironic/api/controllers/v1/node.py14:14
rloodevananda: yuriy pointed out that we're using 'Node.. already being provisioned', even if the instance is being deleted14:15
agordeevrloo: FYI, teeth-agent (python deploy agent) uses 'decomissioning' word14:15
devanandaagordeev, dtantsur: re: what RH dist to target, CentOS 6.514:16
rlooagordeev: I hope they didn't misspell it. 'decommissioning' ;)14:16
dtantsurdevananda, so no Fedora here?14:17
agordeevrloo: ah, yeah, it's just me. 'decommissioning' to be exact14:17
devanandadtantsur: i'm sure there are folks who use fedora (and plenty of other versions of things)14:17
devanandadtantsur: but my priority is seeing devstack support for Ironic added to the openstack CI system14:18
devanandadtantsur: which uses ubuntu 12.04 and centos 6.514:18
dtantsurdevananda, ok then, may I still report issues with Fedora as I find them?14:18
devanandadtantsur: absolutely :)14:18
devanandadtantsur: feel free to fix them too, if you're so inclined ;)14:18
dtantsurdevananda, btw, doesn't centos have selinux as well?14:19
devanandadtantsur: fwiw, though, it's really more helpful to the project right now if we are all focusing on the same goals (namely, getting CI into the gate and pivoting TripleO)14:21
devanandadtantsur: fwiw, based on a conversation in -infra jsut now, it looks like fedora isn't even properly supported by devstack -at all-14:21
devanandadtantsur: folks are working on support for F20 .. but it's not complete14:21
dtantsurdevananda, in fact, I'm new to Ironic and just trying to find some place here :) And as I'm @redhat.com I started with Fedroa by default :)14:22
devanandadtantsur: lucas-hungry may have some advice as to what to use, as he's also @RH14:22
*** saju_m has quit IRC14:22
devanandaIIRC, tripleo supports fedora, but IMBW14:23
*** lucas-hungry is now known as lucasagomes14:24
lucasagomesdevananda, dtantsur morning14:24
dtantsurlucasagomes, hi14:24
lucasagomesdtantsur, yeah selinux is a bit problematic14:24
lucasagomesdtantsur, I disable it for now (tripleo set it to permissive, they have an element for that)14:24
NobodyCamGood Morning Ironic14:25
devanandamornin!14:25
NobodyCam:)14:25
agordeevNobodyCam: morning14:26
dtantsurlucasagomes, did you try the patch in question with Fedora? Or what do you use for development?14:26
NobodyCammorning agordeev :)14:27
lucasagomesdtantsur, nop I haven't tried on fedora because as infra is going to use ubuntu precise I think the focus was to get it working on precise first14:27
lucasagomesdtantsur, but my dev env for all the rest is fedora based14:27
openstackgerritYuriy Zveryanskyy proposed a change to openstack/ironic: Fix 'spacing' parameters for periodic tasks  https://review.openstack.org/8030014:29
dtantsuragordeev, devananda will it be possible to at least ensure SELinux is in permissive mode (also helpful for centos) and install missing packages? That should not take long, but will be a great step towards fedora14:29
lucasagomesdtantsur, +114:31
*** ndipanov has quit IRC14:32
dtantsuragordeev, apart from syslinux you may also want to install libvirt14:33
*** ndipanov has joined #openstack-ironic14:34
GheRiveromorning all14:38
agordeevGheRivero: morning14:38
openstackgerritRuby Loo proposed a change to openstack/ironic: Delete node while waiting for deploy  https://review.openstack.org/8014114:38
NobodyCamdtantsur: I beleieve TripleO has RH/Fedora elements that set permissive mode for SELinux14:38
lucasagomesdtantsur, yeah there's things like PATHs as well, the pxelinux.0 is in a diff path14:39
NobodyCammorning GheRivero :)14:39
lucasagomeson ubuntu and fedora14:39
dtantsurlucasagomes, I think it found a correct path after installing syslinux for me14:40
lucasagomesright, ok14:40
NobodyCambrb... quick morning walkies :)14:40
agordeevdtantsur: k. could you please collect all issues you faced into one place? etherpad or just making inline comments for patch. That would be very helpful, thanks in advance!14:47
dtantsuragordeev, sure, I'm now starting from scratch so that I don't forget anything14:48
rlooGheRivero: funny to see you -1 your patch https://review.openstack.org/#/c/79892/14:49
GheRivero:)14:52
lucasagomesyuriyz: hey I just -2 ur patch fixing the periodic parameters for the periodic tasks, I don't think that's the correct way to fix the problem I left a comment with some suggestion that I think that would be a better way to fix that15:10
lucasagomesyuriyz: I'm open for discussion as well, thanks for the patch anyway15:10
lucasagomesdevananda: ^ can u take a quick look and see if u agree w/ me or not? https://review.openstack.org/#/c/80300/15:10
dtantsurMaybe stupid question, but still: while using Fedora, do I need to change BM_DEPLOY_FLAVOR="-a amd64 ubuntu deploy-ironic" (from localrc template)?15:11
* devananda looks15:12
NobodyCamdtantsur: I dont think ubuntu is correct, I'd bet on s/ubuntu/fedora/15:12
Shrewsdtantsur: yes. just change "ubuntu" to "fedora"15:12
devanandadtantsur: well. it should be fine as ubuntu, i think15:13
lucasagomeswhat does BM_DEPLOY_FLAVOR does? create the image that it's going to boot? and the deploy ramdisk?15:13
devanandathat creates the deploy ramdisk15:13
lucasagomesif so it might be fine to be ubuntu... but if u want to have a full fedora stack15:13
devanandanot the user image15:13
lucasagomesu might want to change that to fedora15:13
devanandaand I think DIB can cross-distro-build15:13
lucasagomesdevananda, yeah it can15:13
devanandabut changing to fedora wouldn't hurt :)15:14
Shrews"fedora" didn't work for me on "ubunut"15:14
Shrewsubuntu, even15:14
lucasagomesShrews, didn't work to create a fedora image on a ubuntu hot?15:14
lucasagomeshost*15:14
openstackgerritRussell Haering proposed a change to openstack/ironic: Expose API for fetching a single driver  https://review.openstack.org/8018715:14
lucasagomesor the image didn't work?15:14
Shrewscreating the image15:15
Shrewsiirc15:15
devanandawhat's the flag to DIB to tell it not to rebuild images?15:15
NobodyCam-c i think15:16
dtantsurShrews, do you remember, was it that devstack is unable to build fedora image or it was due to fedora-ubuntu interop?15:16
Shrewsdtantsur: i don't recall15:16
devanandaNobodyCam: hah! you tricked me15:16
devanandaNobodyCam:     -c -- clear environment before starting work15:16
lucasagomesShrews, right it might be a bug in dib then, cause dib customize an image, it downloads it and chroot into it to do the work15:17
NobodyCamahhh15:17
devanandahmm. so there is --offline now, which doesn't download new image, but I don't see an option to not _build_ an image if one already exists15:17
NobodyCamI'm sorry :(15:18
lucasagomesshould work with cross-distros15:18
*** rpodolyaka has joined #openstack-ironic15:18
devanandaagordeev: within the devstack patch, when building the ramdisk, we should include --offline if $OFFLINE is true15:19
devanandathere's one other option i'm still trying t ofind15:20
NobodyCamahh -c is for boot-seed-vm script15:20
NobodyCamsorry 'bout that devananda :-p15:21
devanandagotta love consistency :)15:21
lucasagomes:P15:21
NobodyCamsomething about many chefs one kitchen15:21
devanandalucasagomes: so Nova's periodic task defs include a lot of spacing=CONF.xxx as well15:22
lucasagomesdevananda, yes, and it's broken15:22
devanandalucasagomes: in nova?15:22
lucasagomesdevananda, yeah15:22
devanandahah!15:22
devanandaum15:22
lucasagomesI had a chat with markmc about it15:22
devanandathat's a pretty serious bug15:22
devanandawhere's teh LP bug?15:23
lucasagomeslemme see if there's one15:23
devanandaI'd think that would matter to a LOT of people15:23
*** ndipanov has quit IRC15:24
lucasagomesdevananda, yeah def15:24
lucasagomesthat's why I think it should be fixed in oslo15:24
lucasagomesand not on specific projects15:24
devanandayep15:24
*** ndipanov has joined #openstack-ironic15:25
devanandaagordeev: ok, DIB doesn't actually support an offline mode. i'll add a comment to the review about hwo to work with it15:30
devanandaposted15:34
*** rpodolyaka has left #openstack-ironic15:35
Shrewsdevananda: did you delete the bit about sshd jumping on the bridge from the etherpad?15:38
devanandaShrews: is that still a problem?15:38
Shrewsdevananda: yes. agordeev just encountered it on the hp instance i loaned him15:39
devanandagreat15:39
devanandaadding it back15:39
devanandadone15:41
*** vkozhukalov has joined #openstack-ironic15:41
devanandaShrews: tftpd-hpa: unrecognized service15:42
devanandaya'll had some notes on tftpd and symlinks?15:42
Shrewsdevananda: i haven't tried adam_g's latest patch15:42
agordeevdevananda: i'm making it to work with regular linux bridges. It will allow us not to update libvirt on U12.04. What do you think does it worth adding?15:43
Shrewsdevananda: so, if you use patchset 21, you can hack it to work with this: http://paste.openstack.org/show/73376/15:44
devanandaagordeev: aiui, we are using neutron's ovs support, so we need to continue using ovs, not regular linux bridge15:45
devanandaagordeev: but I may be totally wrong15:45
devanandaShrews: oh, i see my problem. OFFLINE=True && files/apts/ironic changed15:46
devanandaso stack.sh didn't install the updates15:46
*** eghobo has joined #openstack-ironic15:48
romchegMorning folks15:49
romcheglucasagomes: Are you around?15:49
agordeevdevananda: I'm talking about fake berametal nodes. They can use regular bridge without any problem. Neutron still continues to use ovs regardlessly15:49
agordeevromcheg: morning15:49
lucasagomesromcheg, hey, yes15:49
NobodyCammorning romcheg :)15:49
romchegI was looking on these patches https://review.openstack.org/#/c/73005/6 and https://review.openstack.org/#/c/80187/315:50
romchegI'm concerned about do we need the last one?15:50
*** matty_dubs is now known as matty_dubs|lunch15:52
dtantsurifarkas, https://etherpad.openstack.org/p/jjWcLDThTK15:52
dtantsur^^^ collection issues with F20 here15:52
ifarkasdtantsur, thanks, I will add the issues for centos15:53
devanandaagordeev: neutron needs to be able to assign IPs to the those VMs, etc -- not just resond to DHCP BOOT request.15:53
devananda*respond15:53
*** vkozhukalov has quit IRC15:54
agordeevdevananda: it will able to do. There's no difference in what (virtual) network type used by fake baremetal nodes since it can be properly connected to neuron network15:57
devanandalucasagomes, romcheg: on 73005, I think we should take a close look at it and decide if we're going to land it this week, or hold off entirely15:58
devanandauntil Juno opens15:58
devanandait's a valuable addition to the API, but I haven't given it enough review time myself15:58
devanandadtantsur: 15:56:50 < pleia2> devananda, sdague fedora work is being tracked here:  https://etherpad.openstack.org/p/fedora-on-gate last review outstanding to mostly get us there is  here (needs reviews) https://review.openstack.org/#/c/69510/15:59
dtantsurdevananda, thanks16:00
pleia2I've also been trying to make sure new nodepool commits going in won't break fedora16:01
rloofwiw, romcheg, devananda. I think 73005 should be punted cuz I'm not sure you'll have time and it is Thurs already. Whatever changes you might want, I most likely won't get done before the end of this week (I'm off Fri).16:02
devanandaagordeev: k, then it sounds OK. I would suggest checking with lifeless though - he's got a better grasp of the neutron bits here than I do at this point16:03
lucasagomesromcheg, looking16:06
openstackgerritA change was merged to openstack/ironic: Check BMC availability in ipmitool 'validate' method  https://review.openstack.org/6869716:08
devanandarloo: only issue i see so far is a) it needs a rebase, b) there is no API endpoint for /v1/drivers/{NAME}/16:08
romcheglucasagomes: I just think that 80187 does not bring new fetures and can be easily replaced with rloo's patch16:08
lucasagomesromcheg, yeah... me too to be honest, idk it's fine to get only one driver but I don't see the real value on it16:08
lucasagomesdevananda, that's what 73005 is bringing16:09
devanandarloo: it impelements /v1/drivers/{NAME}/properties  but this URI doesn't appear to be discoverable from /v1/drivers/ or /v1/drivers/{NAME}16:09
devanandalucasagomes: i'm looking at 7300516:09
openstackgerritA change was merged to openstack/ironic: Use range instead xrange to keep python 3.X compatibility  https://review.openstack.org/7987516:09
openstackgerritA change was merged to openstack/ironic: Fix misspelled impi to ipmi  https://review.openstack.org/8021616:09
lucasagomesdevananda, ack maybe 80187 should add a dependency on 7300516:09
*** hemna has joined #openstack-ironic16:09
devanandaahh right16:09
devanandalucasagomes: if 80817 also included a /properties link16:10
devanandathen yes16:10
romcheg+116:10
lucasagomesyeah16:10
devanandaok -- so there's clearly room for optimization in *how* 80187 is getting its info from the DB, but that aside16:11
lucasagomes+1, he left a note there16:11
devanandaare we comfortable enough with this API to land it this late?16:11
romchegI'm also giving agordeev's patch to devstack a heavy test. Already found a few critical problems16:11
NobodyCamhahahahha: http://paste.openstack.org/show/TUBrrvg0iFZbEuYqQCK9/16:11
devananda/v1/drivers  -> the list we have today16:12
devananda^ is missing links16:13
devananda/v1/drivers/NAME  -> details of a driver plus link to properties16:13
devananda/v1/drivers/NAME/properties  -> list of required & optional parameters16:13
romcheg+1, because right now the only information I got from /drivers/name is this: http://paste.openstack.org/show/73418/16:13
romchegGiving a note that it queries all the drivers in the implementation, it's quite useless16:14
devanandaseems like an incomplete API, but also, very useful information16:14
devanandawe'll need to maintain API compatibility in Juno with this API -- thus my hesitation to land an incomplete /drivers API16:15
devanandaeven though I want to expose the list of required/optional driver params16:15
romchegLet's postpose this to Juno16:15
romchegThen we'll have enough time to think on in16:15
devanandalucasagomes: ?16:16
lucasagomesreading16:16
lucasagomesdevananda, I would postpone as well, unless we are landing the properties patch as well16:18
lucasagomesbut as rloo pointed she prefer it to be punted16:18
lucasagomesdevananda, why we have to maintain the compability? cause of the icehouse release?16:18
lucasagomesI think that adding links and properties won't break that compatibility anyway, it's add more stuff not removing what we already have16:19
lucasagomesso I'm fine in postponing it16:19
*** rpodolyaka has joined #openstack-ironic16:19
devanandasee L115 of https://review.openstack.org/#/c/68258/5/reference/incubation-integration-requirements16:20
* NobodyCam git's mo coffee.. brb16:21
lucasagomesdevananda, right but we didn't graduate16:21
devanandalucasagomes: so technically, assuming we dont graduate, then yea, we dont have to maintain compat16:21
lucasagomesyea16:21
devanandalucasagomes: however we will have users, and they will probably care .... :)16:21
lucasagomeswell I don't think we are going to break the compability by adding more stuff to that resources16:21
lucasagomesdevananda, +116:21
romchegWho cares about users? :-P16:21
lucasagomesdevananda, btw, I was thinking about proposing a session to the design summit about this16:22
lucasagomesdevananda, why we didn't graduate, and how we can make sure that we are on the right path to graduate in juno16:22
lucasagomesset's some goals/expectatives there16:22
lucasagomesassign tasks to the active members of the community etc16:22
devanandalucasagomes: right - so, add drivers/NAME and drivers/NAME/properties now, so users of Icehouse can see this info w/o reading the code? -- seems valuable to me16:22
devanandalucasagomes: ++16:22
romcheglucasagomes: +1 for the session16:22
lucasagomesdevananda, romcheg ack I will propose it16:23
lucasagomesdevananda, right, but u think people will benefit from seem these info?16:23
lucasagomesdevananda, I think that the real benefit in having the properties in the api16:24
lucasagomesis for the driver16:24
lucasagomesthat has to know which properties it needs to populate for the driver16:24
lucasagomesso we can kill that driver_parameters_map we have today in the Ironic driver16:24
devanandalucasagomes: you mean the nova driver16:24
devanandaah16:24
lucasagomesyeah16:24
devanandaso16:24
lucasagomesI don't see a real benefits for the users in looking at that16:24
devanandaIMO, that is orthogonal16:24
devanandaanything which the nova driver needs to change *per-instance* needs to be mvoed to node.instance_info16:25
lucasagomesdevananda, right16:25
lucasagomeshmmm16:25
devanandaand the node.driver_info should never be changed by nova -- only by the deployer as a precondition to using Ironic with that chunk of hardware16:25
lucasagomesright16:25
devanandawhcih is why exposing that in the API -- to users -- seems valuable to me16:25
devanandabut it's a bit tangled up right now16:25
lucasagomesdevananda, it might be exposing the wrong stuff tho16:25
lucasagomeswhile we don't have the intance_info16:25
devanandayep16:26
devanandait is16:26
lucasagomesmany of those parameters will be moved afterwards16:26
lucasagomesit might be misleading16:26
devanandaok -- i'm convinced :)16:26
lucasagomesheh ok16:26
*** vkozhukalov has joined #openstack-ironic16:26
devanandajroll: you guys are working on the instance_info stuff, ya?16:26
jrolldevananda: yeah, I have a review up. lucasagomes has some comments but I haven't addressed them yet. have been distracted with other things16:27
lucasagomesdevananda, jroll right yeah, I was mostly questioning the reason why we have nested dicts and lists for that field instead of a simply keypair=value kind of attribute16:29
lucasagomesand why we have some database field I think it was dictlarge or something16:29
devanandaahh right16:29
jrollright16:29
devanandawhen we last talked about that16:29
devanandathe reason for nested dicts was ~ to stash all the config drive data *in* ironic and build it there16:30
lucasagomesbuild the instance metadata?16:30
devanandato which, my takeaway from the ensuing lengthy conversation was, ironic shouldn't be building a config drive16:30
devanandaso it doesn't need a nested dict16:30
lucasagomes+116:30
lucasagomesnova does it for us no?16:30
devanandaright16:30
lucasagomesI mean we just "save" it16:30
devanandaand ironic should expose any info back via the nova driver which nova needs to build the config drive16:31
devanandaeg, what ports are bonded, etc16:31
lucasagomesack yeah16:31
lucasagomessounds more reasonable to me this approach16:31
devanandajroll: so you may want to read some backscroll -- this API patch https://review.openstack.org/#/c/73005/ is going to be affected by the instance_info work slightly16:32
devanandajroll: in as much as, moving params out of driver_info will change what that API exposes16:32
lucasagomesas an addon, the instance_info might also affect the validate of the drivers16:32
lucasagomesright now pxe looks for things like root_gb etc... to validate16:33
jrolldevananda: right. I'd like to get the instance_info patch done, and the patch to move params out of driver_info done, before juno opens16:33
jrolllucasagomes: yep16:33
NobodyCamit will also impact the DIB work too16:33
lucasagomesdib?16:33
NobodyCamdisk image builder16:33
lucasagomesyeah16:33
*** Alexei_987 has joined #openstack-ironic16:34
lucasagomesthe instance_info will affect dib?16:34
lucasagomesto build the image?16:34
NobodyCamwell tripleo-incubator really16:34
lucasagomesah16:34
NobodyCamhttps://review.openstack.org/#/c/72969/15/scripts/register-nodes16:34
devanandayea, tripleo-incubator // devtest16:34
lucasagomesright yeah16:34
devanandait'll also impact the nova driver16:34
lucasagomesyeah heh the whole chain16:34
devanandawhich, since it's in our tree, can be changed synchronously for now :)16:35
lucasagomes:D16:35
lucasagomesyay16:35
openstackgerritGhe Rivero proposed a change to openstack/ironic: Set boot device to PXE when deploying  https://review.openstack.org/7989216:35
openstackgerritGhe Rivero proposed a change to openstack/ironic: Driver utils should raise unsupported method  https://review.openstack.org/7996416:35
lucasagomesdevananda, NobodyCam btw, another thing I see that in our tox.ini the pep8 tests for the nova driver is disabled16:35
devanandajroll: can you do it without nested dict?16:35
lucasagomesbut should it be disabled? I think unittests disabled grand, but maybe we should leave pep816:36
lucasagomesit's good to identify small syntax problems16:36
jrolldevananda: if we go the route of "nova generates configdrive", yes. I need to look at all the details of that.16:36
lucasagomesand things like that variable ur trying to access doesn't exist16:36
jrolldevananda: I just don't want a bunch of chatter back and forth between nova and ironic just to get the configdrive built16:36
devanandalucasagomes: ah. to run pep8 tests on it, i think we need to actually install all of nova in the .venv and then pep8 *that*16:36
devanandalucasagomes: otherwise imports will fail, etc16:37
lucasagomesdevananda, ohh16:37
lucasagomesfeck16:37
lucasagomestrue16:37
* lucasagomes checks16:37
devanandalucasagomes: imbw :)16:37
* lucasagomes googles imbw16:37
lucasagomesheh16:37
lucasagomesok16:37
lucasagomesnp lemme check16:37
devanandajroll: fwiw, I would like to see all the info that nova needs be returned in a single API call16:37
jrolldevananda: of course16:39
devanandaShrews: were you working on 80022?16:43
Shrewsdevananda: yes, and it makes me look forward to the day python 2.6 is no longer supported16:44
devanandaindeed!16:44
lucasagomesShrews, yeah! dict comprehension!!!16:45
jrollif I'm using auth_strategy=noauth, should ironic still be asking me for username/etc?16:46
jrolls/ironic/ironicclient16:46
*** killer_prince is now known as lazy_prince16:46
devanandajroll: openstack still wants either a username or a token16:48
devanandajroll: export OS_AUTH_TOKEN=fake16:49
jrollgot it, thanks16:49
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Enable pep8 tests for the Nova Ironic Driver  https://review.openstack.org/8033516:49
devanandacritical patches in need of reviews: https://review.openstack.org/#/c/79964/ and https://review.openstack.org/#/c/79892/16:49
lucasagomesseems it works :)16:49
devanandalucasagomes: sweet! I'll approve once it passes J16:50
lucasagomesdevananda, cheers16:50
devanandaNobodyCam: https://review.openstack.org/#/c/79906/ could use eyes16:50
* NobodyCam looks16:51
NobodyCamdevananda: lucasagomes is InstanceNotFound the correct execption for a deleted node. I can see how it fits, maybe worth adding a InstanceDeleted execption for? thoughts?16:56
Shrewsooh, testtools assertRaises returns the exception. /me hugs lifeless16:57
devanandaNobodyCam: see the conversation wtih comstud16:57
lucasagomesNobodyCam, yeah according to comstud that's the right exception because nova will set instance ERROR to any other exception but InstanceNotFound which it will treat as a race with delete16:57
devanandaNobodyCam: tldr - if spawn() raises taht error, Nova knows how to handle it16:57
NobodyCamahhh :) ok that makes sense16:58
NobodyCamwhat hit me as odd was that is the same error rasied for not findng the ironic host16:59
*** martyntaylor has quit IRC17:02
agordeevfor unknown reason neutron router doesn't work properly. And i don't see qrouter-<uuid> net namespace17:05
agordeevhas anybody else faced this issue? ^17:06
devanandalifeless: ^ ?17:06
devanandaadam_g: agordeev: I am now getting an error where devstack tries to start n-obj twice, naturally the second one fails with port already in use17:06
NobodyCambbt...brb17:06
*** snakehunter has joined #openstack-ironic17:07
adam_gdevananda, that should be addressed by https://review.openstack.org/#/c/78581/17:08
agordeevdevananda: fix for n-obj was merged into devstack not a long time ago.17:09
agordeevseems the patchset need to be rebased17:09
devanandaah17:09
* agordeev have clicked on rebase17:10
*** matty_dubs|lunch is now known as matty_dubs17:10
devanandaFilter ComputeCapabilitiesFilter returned 0 hosts17:10
*** harlowja_away is now known as harlowja17:10
devanandafailing extra_specs requirement17:13
devanandait's failing the "if cap is None:" check17:16
devanandainside compute_capabilities_filter.py17:17
*** romcheg has quit IRC17:17
NobodyCampost bbt walkies ... bbiafm17:21
devanandaanyone have devstack with a booted node right now?17:22
devanandai'm curious if my flavor / node defs differ from yous17:22
*** tatyana has quit IRC17:24
NobodyCamI have tripleO / dib booted version17:24
*** derekh has quit IRC17:25
NobodyCamhttp://paste.openstack.org/show/B8zvO56Cf0Fb7ZiZ8pIV/17:25
devanandahttp://paste.openstack.org/show/73433/17:26
devanandaNobodyCam: and your node properties?17:27
NobodyCamhttp://paste.openstack.org/show/Wr4DMh0hxDB5GgRMRTN0/17:30
NobodyCamlooks like a match to me17:30
devanandaas soon as i remove cpu_arch: from extra_specs, it works17:31
devanandaand yet it looks like a match to me, too17:32
*** blamar has joined #openstack-ironic17:32
adam_gagordeev, any objection to changing the default ssh port from 2222 to 22? someone mentioned in etherpad that 2222 has issues on fedora/SElinux17:33
devanandaadam_g: there is already ssh running on port 22 in the devstack-gate environment17:33
devanandaadam_g: check with infra to see if they're OK with us modifying it. As long as we only add an auth'd key, I think it'll be fine17:34
adam_gdevananda, well, theres already ssh running on 22 usually anyway. im saying just use that instead of reconfiguring ssh at all17:34
*** max_lobur has quit IRC17:35
devanandaadam_g: sure. but infra will care more if we modify sshd in their environment than your average devstack user :)17:36
NobodyCamdevananda: anything odd in your nova.conf ... could there be a left over instance_type_extra_specs setting by change17:38
openstackgerritA change was merged to openstack/ironic: Make the Nova Ironic driver to wait for ACTIVE  https://review.openstack.org/7990617:39
devanandaodd17:39
devanandadevstack didn't change my nova.conf to use ironic17:39
NobodyCamoh that could do it17:41
devanandagah17:41
devanandaNobodyCam: thanks ...17:41
devanandaNobodyCam: i should have realized that issue earlier. Old VM, still had a localrc that enabled Ironic -- but it was missing VIRT_DRIVER=ironic :(17:41
* devananda restacks17:41
*** jistr has quit IRC17:42
NobodyCamhehehe :)17:42
*** snakehunter has quit IRC17:42
lifelessdevananda: agordeev: Hi, neutron q?17:42
NobodyCammorning lifeless :)17:45
*** jbjohnso_ has quit IRC17:46
*** Alexei_987 has quit IRC17:46
*** Hefeweizen has quit IRC17:48
*** Hefeweizen has joined #openstack-ironic17:48
*** athomas has quit IRC17:48
*** ndipanov has quit IRC17:51
devanandaNobodyCam: http://paste.openstack.org/show/1bnYJUpmEGmSXwTNruFs/ -- should perhaps just be caught and not logged?17:51
NobodyCamhumm my power state is none, should be off17:51
NobodyCamhumm I thought we had a try around the remove17:52
lucasagomesack, I'm off of the day17:55
lucasagomesgoing home, came to the office today17:55
NobodyCamhave a good night lucasagomes17:55
NobodyCamhave a beer for me17:55
devanandag'night lucasagomes!17:55
lucasagomesNobodyCam, devananda thanks! g'night!17:56
*** lucasagomes has quit IRC17:56
NobodyCamahh guess I should have checkd node validate ...doh17:56
NobodyCamSSH key file /mnt/state/var/lib/ironic/virtual-power-key not found.17:56
NobodyCamkinda a biggie17:56
*** rpodolyaka has quit IRC17:57
lifelessNobodyCam: you shouldn't need that file for Ironic17:57
lifelessNobodyCam: IMNSHO17:57
lifelessNobodyCam: the power key should be passed into the API, since its per node17:58
*** Lotus907efi- is now known as Lotus907efi17:58
devanandaNobodyCam: the name of the key is passed into the API. the actual privatekey file is not17:58
devanandalifeless: ^ meant that for you17:59
NobodyCamlifeless: it is passed in ironic takes a file for ssh user key17:59
lifelessdevananda: I know, I'm saying that seems like a bug to me.17:59
devanandalifeless: "not storing a private key in a DB" is an interesting bug17:59
lifelessdevananda: since it means we have to reconfigure all the conductors via heat to add another VM host17:59
devanandahuh?18:00
lifelessdevananda: its either in the seed Heat DB or its in the Ironic DB18:00
*** rpodolyaka has joined #openstack-ironic18:00
lifelessdevananda: you wouldn't suggest passing in a path reference to a file with IPMI password in it, would you?18:00
devanandalifeless: fwiw, I think passing the private key contents via APIs is a bug18:01
lifelessdevananda: credentials are credentials18:02
devanandaclearly, though, it was a design choice which I do not undersatnd yet :)18:02
devanandaNobodyCam: have a minute for https://review.openstack.org/#/c/79964/ ?18:02
lifelessdevananda: I'm not arguing that things are 'right', but I don't see why IPMI creds should be special vs ssh creds, both are equivalent as far as Ironic is concerned.18:03
devanandalifeless: sure. and you can pass in the user/pass for SSH too18:03
lifelessdevananda: ok, so why is a key file handled differently?18:04
devanandalifeless: i have a strong adverse reaction to storing a private key in a database and exposing it via a REST API18:04
devanandalifeless: whether or not it's functionally equivalent in this particular case18:04
lifelessdevananda: do you disagree that the key file is equivalent in this case?18:04
devanandahere's how it is different18:05
devanandain my opinion18:05
devanandatoday, a "user" with API access can find out the name of the key file18:06
NobodyCamdevananda: was there a bug # for 79964?18:06
devanandabut unless they have access to the ironic-conductor host(s), they can not get the content of that key, and therefor cn not get access to the host which is managing the VMs18:06
devanandaNobodyCam: same bug as https://review.openstack.org/#/c/79892/ -- it's a prereq for that patch18:06
lifelessdevananda: and if someone uses a user+pass rather than key file ?18:07
devanandalifeless: so for providers who want to separate SSH access to the hosts on which the VMs are being managed, this is imortant18:07
devanandalifeless: well, then they're exposing that risk.18:08
lifelessdevananda: and anyone using IPMI exposes it too18:08
lifelessdevananda: but I don't know if you know this, user+password is *less* safe than keys, because keys we can lock down to limited operations.18:08
devanandanetwork isolation would mitigate that18:08
lifelessdevananda: I don't understand18:09
lifelessdevananda: you seem to be saying that an org might permit users to deploy to baremetal and run the own DB and API but not the conductors18:12
devanandalifeless: s/the/their/  ?18:12
lifelessdevananda: so for any real environment they'd have complete ability to run anything on baremetal anywhere18:13
devanandalifeless: i'm suggesting that some orgs might separate "users of nova" and "users of ironic" and "operators of the cloud infra" into three distinct privilege groups18:14
devanandaand the "users of ironic" would not have physical or network access to the ir-cond hosts or the ironic database18:14
devanandathe "users of nova" would not have access to ir-api, either18:15
devanandalifeless: but I suspect we've side tracked sufficiently at this point18:16
lifelessdevananda: but the creds are only accessible to operators of cloud infra + users of Ironic18:16
lifelessdevananda: so nova is irrelevant18:16
lifelessdevananda: (in principle, I know keystone v3 etc etc)18:16
lifelessdevananda: users of Ironic add and remove machines18:17
lifelessdevananda: in the above definition?18:17
devanandalifeless: your concern is not having to update a key FILE stashed on each condcutor when you bring a new VM host online, yes?18:17
lifelessyes18:17
lifelesse.g. to have an API :)18:17
devanandalifeless: could the new VM host share the same public key list as existing VM hosts?18:17
lifelessdevananda: I don't think thats a reasonable constraint18:18
devanandawhy not?18:18
devanandathe public key list is, presumably, something you injected via Heat when deploying the VM host(s)18:18
lifelessno18:18
russell_hit feels like if you want Ironic to manage your VM, its not unreasonable to have to put Ironic's public key on your VM18:19
lifelessits generated on the hosts and pushed to jenkins slaves via gear18:19
lifelessso we can obviously change that but that then makes all vm hosts have the same key, which *I* have an adverse reaction to - the same as having the same IPMI admin password on multiple physical hosts18:19
devanandalifeless: a public key list is commonly passed in via ec2 metadata for all nova instances that i've ever used18:20
devanandaahh18:20
devanandawe have different security models18:20
lifelessthe key is *extremely* limited in use18:20
devanandame: guard the key18:20
lifelesswe have admin keys pushed in via ec2 obviously18:20
devanandayou: make the key very limited18:20
lifelessbut they are totally irrelevant for testing, since we don't trust anyone to have baremetal access to these machines18:21
devanandaright. so create a second limited-access key, just for ironic's SSH driver, and share that18:21
lifelessthe key will let you start / stop / enumerate vms and copy a seed disk file to a seed vm18:22
openstackgerritA change was merged to openstack/ironic: SeaMicro use device parameter for set_boot_device  https://review.openstack.org/8022118:22
lifelessdevananda: we do that18:22
devanandait would be no less secure than what you have now, except the private key wouldn't be publicly accessible18:22
lifelessdevananda: but you said 'make the key the same on all hosts'18:22
lifelessdevananda: which as I said, adverse reaction.18:22
devanandaor generate N keys where N == num hosts18:22
lifelessdevananda: and the private key would be totally accessible since jenkins slaves still need it18:22
devanandaor where N >> num hosts18:22
lifelessso I feel like you're arguing on behalf of some unknown environment, when for ssh driver AIUI there are precisely three users today - tripleo-ci, devstack, and local developers.18:23
lifelessfor tripleo-ci, the VM hosts already exist, we think that giving jenkins slaves access to heat to inject more public keys is a terrible idea18:24
lifelesssince it is equivalent to BM access, so we chose to hand out limited access ssh keys.18:24
lifelessFor devstack-gate you have root on the machine, and its one machine so pretty much anything goes18:26
* NobodyCam bangs head on keyboard... and with a sigh says "Remember to add the ssh key to the virsh server and not your local system."18:26
devanandalifeless: ok, so for tripleo-ci, i get why you do not want to change the list of allowable public keys18:27
NobodyCam| power     | True18:28
devanandalifeless: what i dont get is why you need to inject the private key via ir-api, as opposed to copying the privkey into that host18:28
lifelessdevananda: I said above that we either pass it in via Heat on the seed (to pass into the undercloud), or we pass it in via Ironic18:28
russell_hcopying the public key into the host18:28
lifelessdevananda: either way its an API18:28
lifelessrussell_h: private key18:29
russell_hoh, the conductor host18:29
lifelessrussell_h: think like IPMI, I have an appropriately configured security context already, and Ironic needs to play nice18:29
russell_hgotcha18:29
russell_hlifeless: right, I think I'm with you on this, its really no different than IPMI creds18:29
*** dshulyak_ has joined #openstack-ironic18:30
devanandarussell_h: yea, ir-cond needs the pprivkey to SSH to the host where the (fake vbaremetal) VMs are18:30
*** romcheg has joined #openstack-ironic18:30
russell_hright18:31
russell_hI just don't see why there isn't just one private key18:31
devanandalifeless: so I dont qute followwhat you mean by "pass it in via Heat on the seed"18:31
russell_hand you put a public key with restricted access on any box you want Ironic to manage18:31
lifelessdevananda: Using the Heat API for this seems ugly because it means that a different administrative domain has access to the key, vs passing it directly to the thing that needs it18:31
russell_hat least, if I was building this in isolation thats how I'd do it18:31
russell_hbut given Ironic's model for managing IPMI creds, I'm with lifeless that it sort of makes sense to just put it in the database18:32
russell_hfor a limited definition of "makes sense"18:32
lifelessrussell_h: that means writing an API to take public keys and configure them with secured access on the vm host18:32
lifelessrussell_h: note that many tests use one host at any point in time - we divide up the host into logical partitions18:32
lifelessdevananda: we have four ways to get files onto an undercloud host:18:33
lifeless - ssh with the deployer credentials. Ugly. Terrible for Tuskar.18:33
lifeless - bake into the image. Ugly. Makes images include security sensitive contents, making them equivalent to the credentials.18:33
lifeless - supply via metadata (heat or ec2). Known and robust, but means the creds are stored by the *cloud below* persistently.18:34
lifeless - use some relevant API - e.g. we put image files into the undercloud by using glance-API.18:34
lifelessIn the same way that I don't want to put the ramdisk and kernel they undercloud should be using to deploy things into the undercloud via ssh or heat or baking into the image :)18:35
lifelessdevananda: but the big thing for me, is that these sequences of bits are entirely equivalent to IPMI18:36
lifelessdevananda: and I haven't heard *any* explanation of how they are different other than 'well if the IPMI network is firewalled so only conductors can access it'... which applies equally well to SSH access to the VM hosts.18:36
russell_hwell18:37
lifelessdevananda: but, I'm going to drop this - you're trying to get tripleo-incubator working, I was merely trying to point out a way to make it a lot easier and more natural, and if you feel like the tradeoff is worth it, I have *much* bigger things to sort out18:38
russell_hexcept I'm a lot less likely to firewall off SSH than I am IPMI18:38
russell_hgiven that presumably people want access to the gear they provisioned18:38
lifelessrussell_h: huh? no. the vm hosts may (probably are) addressible on an entirely different network to the emulated baremetal machines you deploy on them18:39
lifelessrussell_h: those would (in a multihost setup) likely be on either a VLAN or an overlay network18:39
russell_hah, gotcha18:39
russell_heither way, I buy it18:39
russell_hput all the credentials in the same place18:40
lifelessrussell_h: I mean, sure, you can bridge eth0 of the vm host onto brbm, and have Neutron serving DHCP to both the vm hosts and the emulated BM nodes18:40
russell_hand if that place isn't good enough, lets make it better18:40
lifelessrussell_h: but I think you'd go mad with that as a development environment18:40
lifelessalso, I just realised I've been pushy here. Sorry - 0530 wakeup with Cynthia does not a stable Robert make.18:41
NobodyCamoh in tripleo's setup ironic is running in a different venv then nova18:41
devanandalifeless: stepping back -- the goal being integrate ironic and tripleo -- ironic supports key filenames or passwords today.18:42
lifelessNobodyCam: it may be, its not guaranteed. It might be on a totally different machine.18:42
*** mdurnosvistov_ has joined #openstack-ironic18:42
devanandalifeless: can we make this work? or do we need to retool the ironic API and SSH driver -- either now (during feature freeze) or after Juno opens?18:42
NobodyCamoh that will make getting the nova driver very tuff.. :-p18:43
openstackgerritA change was merged to openstack/ironic: Driver utils should raise unsupported method  https://review.openstack.org/7996418:44
* NobodyCam cheets by manually installing ironic in nova venv18:44
lifelessNobodyCam: you should do that yes18:44
lifelessNobodyCam: its not cheating18:44
lifelessNobodyCam: make a new element, nova-ironic18:44
devanandaNobodyCam: lifeless: for tripleo, you'll need to install ironic on what ever host is running n-cpu18:44
devanandato get the driver18:44
lifelessNobodyCam: it should do that glue.18:44
devanandaright18:44
lifelesssame as e.g. nova-kvm installs libvirt18:44
lifelessdevananda: so, the minimal delta is to support only one physical machine and pass it in like we do the nova-bm key18:45
lifelessdevananda: which is via heat18:45
devanandalifeless: iow, pass the private key contents to ironic-api?18:46
lifelessdevananda: this couples the test environment to our tooling, which I'm quite unhappy about, but it will work18:46
lifelessdevananda: no, pass the private key contents to the heat thats deploying Ironic18:46
devanandak18:46
openstackgerritA change was merged to openstack/ironic: Set boot device to PXE when deploying  https://review.openstack.org/7989218:46
NobodyCamheheheheheh 75a49c8d-f48d-496d-8fa2-368e66dc681c | f168c2a9-f801-4f33-98ac-5edcd3118122 | power off   | deploying18:53
NobodyCampower on    | wait call-back18:54
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Stop logging paramiko's DEBUG and INFO messages  https://review.openstack.org/8036518:55
devanandatiny fix ^18:55
devanandasuper helpful for folks using devstack :)18:55
lifelessoh hmm18:59
lifelessbtw paramiko.AutoAddPolicy18:59
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Suppress conductor logging of expected exceptions  https://review.openstack.org/8002218:59
lifelessI think that really should be don't add18:59
lifeless(and don't load the host keys)18:59
lifelessbut thats a different discussion18:59
Shrewsoh nice. that automatically assigned me devananda's bug. pfft19:03
devanandahaha19:03
Shrewsnow i KNOW you're the devil19:03
devananda;)19:04
lifelessdoes parse_driver_info execute on the conductor ?19:04
devanandayes19:05
NobodyCamhumm ok something up with pxe... getting ip but then cant find configuration file19:05
NobodyCamthou I should rebuild now at 79892 landed19:07
NobodyCambrb19:07
NobodyCammaybe bbiafm19:07
devanandaadam_g: have you changed something in the templates to add a txt console to the VMs?19:09
adam_gdevananda, no, i haven't19:09
devanandaadam_g: have you gotten a text console? :)19:09
devanandai suspect i'm at the "its not accessing tftpboot" stage, even though I thought you fixed that19:10
adam_gdevananda, no, i've just been using virt-manager via ssh to look at console19:10
adam_gdevananda, you should get some tftpd logging to syslog to confirm whether or not the node is hitting it, and if so how far its getting19:10
Shrewsthat bugs me that virt-manager can get to the console, but virsh cannot (rather, i don't know HOW to make virsh do it)19:13
*** rpodolyaka has quit IRC19:13
devanandaright - DHCP ACK and files are where tftpd says they should be19:15
adam_gso the node are successfully pulling their kernel, ramdisk and later token?19:16
adam_gdevananda, also looking at comments @ https://review.openstack.org/#/c/70348/22..23/lib/baremetal... BM_DEPLOY_FLAVOR also needs to point to deploy-ironic element if we're using ironic. is lib/baremetal good to go away at this point?19:18
*** vkozhukalov has quit IRC19:18
devanandaadam_g: looks like it's not pulling the token19:18
adam_gi was hoping to land what we have and then remove lib/ironic's dependency on lib/baremetal.. then lib/ironic can contain all of its own defaults19:18
devanandaadam_g: ++19:19
adam_gdevananda, is it tftp showing a file not found error?19:19
devanandaadam_g: removing lib/baremetal is a very good idea. anything we need (eg, upload_baremetal_ramdisk) should be moved over19:19
devanandahm, no. wondering if this ramdisk doesn't have the deploy-ironic element19:19
*** dshulyak_ has quit IRC19:20
* devananda rebuilds ramdisk, restacks19:22
devanandaafk a bit19:25
* NobodyCam thinks ummm lunch19:26
*** max_lobur has joined #openstack-ironic19:26
*** mdurnosvistov_ has quit IRC19:28
openstackgerritlifeless proposed a change to openstack/ironic: Permit passing SSH keys into the Ironic API  https://review.openstack.org/8037619:29
lifelessdevananda: ^ that should do what I want, without compromising the other use cases you have19:29
*** max_lobur has quit IRC19:36
*** rloo has quit IRC19:37
*** rloo has joined #openstack-ironic19:37
*** max_lobur has joined #openstack-ironic19:38
*** eghobo has quit IRC19:45
openstackgerritRussell Haering proposed a change to openstack/ironic: Expose API for fetching a single driver  https://review.openstack.org/8018719:51
russellbrussell_h: i get an IRC notification every time you post a patch :-)19:51
russellbi'm going to be very on top of what you're working on, hehe19:52
russell_hrussellb: perfect :)19:52
russell_hI had to turn off my "russell" highlight when I joined openstack channels19:53
russellbha19:53
NobodyCamahh good catch devananda i to forgot to use deploy-ironic19:59
NobodyCambrb walkies20:02
devanandastill not getting a tftp fetch of the token20:04
devanandagotta run an errand, bbiab20:04
*** rpodolyaka has joined #openstack-ironic20:14
*** rpodolyaka has quit IRC20:19
*** dwalleck has joined #openstack-ironic20:19
*** rpodolyaka has joined #openstack-ironic20:32
*** vkozhukalov has joined #openstack-ironic20:32
*** eghobo has joined #openstack-ironic20:35
*** rpodolyaka has quit IRC20:39
*** eghobo has quit IRC20:40
*** rpodolyaka has joined #openstack-ironic20:40
lifelessdevananda: I'd like to know if its small enough to FFE (or even doesn't need it); e.g. should I polish it and/or shepard it, or just leave it as a POC ?20:41
openstackgerritlifeless proposed a change to openstack/ironic: Permit passing SSH keys into the Ironic API  https://review.openstack.org/8037620:43
*** eghobo has joined #openstack-ironic20:44
openstackgerritlifeless proposed a change to openstack/ironic: Permit passing SSH keys into the Ironic API  https://review.openstack.org/8037620:45
*** eghobo has quit IRC20:47
*** jdob has quit IRC20:47
*** eghobo has joined #openstack-ironic20:47
openstackgerritDan Prince proposed a change to openstack/ironic: Run ipmi power status less aggressively  https://review.openstack.org/8040020:53
*** mdurnosvistov_ has joined #openstack-ironic20:56
jrollis there documentation on the configs needed to use the ironic driver?20:56
lifelessjroll: should be in the config opts in the driver20:58
jrolllifeless: I mean the nova configuration. e.g. I know I need to set compute_driver, is there other things?20:59
jrolllifeless: I guess I'm just wondering if there's anything written down before I go source-diving21:00
lifelessjroll: yes, thts what I thought you meant.21:03
lifelessjroll: see ironic/nova/virt/ironic/driver.py21:03
lifelessapi version, api endpoint, etc etc21:03
jrollright, probably under an [ironic] section?21:03
lifelessyes21:03
jrollok21:03
lifelessline 45-75 of that file21:03
jrollright, I got that21:04
lifeless:)21:04
openstackgerritlifeless proposed a change to openstack/ironic: Fix typo tenet -> tenant  https://review.openstack.org/8040321:05
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Enable pep8 tests for the Nova Ironic Driver  https://review.openstack.org/8033521:23
*** linggao has quit IRC21:25
*** dwalleck has quit IRC21:26
*** dwalleck has joined #openstack-ironic21:39
lifelessNobodyCam: btw https://review.openstack.org/80403 should allow you to take the nodes vector and shove it straight into Ironic with minimal changes, just map the fields and done.21:42
NobodyCamohhhh /me clicks21:42
NobodyCamlifeless: ^^^ fixes a help string?21:43
lifelessNobodyCam: oh nuts, I mean https://review.openstack.org/8037621:43
NobodyCam:-p21:43
NobodyCamahh neet.. How about on the client end? will we want to show the key_contents data of some kind of isSet flag21:46
NobodyCamI can see some like show the keys comment if set, so that we aren't actually showing the key itself21:47
NobodyCams/some/something/21:47
*** max_lobur has quit IRC21:47
lifelessNobodyCam: well, what do we do for IPMI passwords ?21:47
lifelessNobodyCam: I think we should do the same21:47
*** dwalleck has quit IRC21:48
NobodyCamgood question, I think we just show it21:48
lifelessso see the discussion above for long thoughts on it, but given this is optional, I'd say just do the same as for passwords21:50
lifelessfolk with particular concerns can a) use key paths, b) help make credential storage better for all of Ironic21:50
NobodyCamyay for b !!!21:51
*** matty_dubs is now known as matty_dubs|gone21:52
*** hemna has quit IRC22:14
*** rpodolyaka has quit IRC22:15
*** rpodolyaka has joined #openstack-ironic22:18
openstackgerritlifeless proposed a change to openstack/ironic: Permit passing SSH keys into the Ironic API  https://review.openstack.org/8037622:19
lifelessNobodyCam: ^ tests and pep8 fixed22:21
NobodyCam:)22:21
*** mrda_away is now known as mrda22:26
mrdamorning all22:26
NobodyCammorning mrda22:26
mrdaHi NobodyCam22:26
NobodyCam:)22:27
*** hemna has joined #openstack-ironic22:28
*** mdurnosvistov_ has quit IRC22:29
lifelessNobodyCam: so if this looks useful, I might let you do the client stuff? devananda hasn't weighed in yet though, so it might be a non-starter :)22:34
* devananda looks22:35
devanandaNobodyCam: i'm refactoring the _require_node() method in nova.virt.ironic.driver -- it's not catching cases where ironic has already disassociated the instance but nova has an old cache22:36
devanandaeg, in destroy22:36
NobodyCamack :)22:37
*** eguz has joined #openstack-ironic22:40
*** eghobo has quit IRC22:43
comstudso.. i have an ironic auth related question22:44
comstudwrt to nova22:44
comstudor maybe not even nova22:45
lifelessshoot22:45
comstudbut how do you do noauth ?22:45
comstudassuming that ironic api is on a secured network22:45
lifelessteach keystone I would assume22:46
comstudi'd not want ironic unnecessarily query auth, because it's just extra latency and point of failure22:46
comstudwell, i want keystone out of the way22:46
comstudwe can configure ironic as noauth22:46
comstudbut the clients all require creds :-/22:46
comstudalthough i think you can pass in a fake token and that seems to work22:46
lifelesscomstud: use a signed token they don't need keystone chatter22:47
*** jgrimm has quit IRC22:47
comstudi think the nova ironic driver needs to understand this and currently doesn't22:47
devanandacomstud: ooh. good point22:49
comstudthe driver's _get_client does a get_admin_context()22:49
*** romcheg has quit IRC22:49
comstudand checks for 'auth_token' in it to pass it22:49
devanandacomstud: with noauth in ironic, aiui, it's not checking keystone, but it still wants to receive credentials22:49
comstudbut nova never sets that at all :)22:49
comstudunless you pass it in as a kwarg22:49
devanandasure22:49
devanandaso22:49
comstudyeah22:49
devanandawe've been using noauth only for isolated testing (eg, ironic w/o services)22:50
comstudwe kludged something for now.. passing a fake token22:50
devanandaif you have other services (eg, nova) then i am assuming you also have keystone22:50
comstudbut required hard coding in the driver22:50
devanandaand thereby real aoth22:50
comstudsure22:50
comstudbut i don't want to add more queries to auth22:50
comstud:)22:50
devananda?22:50
devanandais the questiona bout noauth or reducing queries to keystone?22:50
comstudboth?  noauth doesn't query keystone, right?22:50
devanandaafaik, it does not22:51
comstudright22:51
comstudthe problem is:22:51
comstudI want to do noauth with ironicclient, but it doesn't allow it22:51
devanandabut with auth enabled, i think both nova and ironic need to validate with keystone22:51
devanandawe can't skip that22:51
devanandacomstud: oh -- you want ironicclient not to need to pass a fake token :)22:51
comstudi think that's what I'm getting at... and then the corresponding support in nova :)22:52
comstudOr just a reasonable way of doing this in the nova driver without changing ironicclient would be acceptable22:52
comstud:)22:52
comstudbut the other reason I bring this up is...22:53
comstudthe nova driver seems to have support for passing in a pre-generated token22:53
comstudbut that case can never be hit right now22:53
devanandaright22:53
devanandataht's intentional22:53
comstudbecause nova_context.get_admin_context9) never sets .auth_token22:54
comstud(unless you pass it in to that method)22:54
comstudso conceivably I could add a config option that would do that22:54
devanandawell. the "dont use the client's context" is intentional. ironic.driver line 18022:54
comstudadmin_auth_token or something22:54
comstudyeah, maybe i'm not being clear22:54
comstudsec22:54
comstudhttps://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L17522:55
comstudthat 'else' is never hit right now22:55
comstudbecause ctx.auth_token is always None22:55
devanandaahh22:55
devanandaok, didn't realize that22:55
comstudbecause nova_context.get_admin_context9) above22:55
comstudnever passes one in22:55
comstudso I'm thinking about adding a config option to support that22:56
comstudadmin_auth_token or something22:56
devanandaso we explicitly want to avoid passing the existing user context to ironic22:56
comstudcorrect, i'm not proposing changing that22:56
devanandawhether the operator needs to configure a separate user account and supply that to nova, or just fall back to the admin context -- either is fine _for now_.22:56
comstudi'd just like an option to say "here's an admin token that's always good"22:57
devananda++22:57
comstudwhich for the noauth case could just be "fake"22:57
comstud:)22:57
comstudok22:57
devanandacomstud: also see my note there -- we shouldn't need to recreate the token or auth context for each request. it is cacheable, so we should cache it in the driver (or somewhere) IMHO22:57
comstudyeah, although the client is what has the token22:58
devanandaI couldn't find precedent in Nova for that after a brief and not exhaustive search, so i just left a note :)22:58
comstudironicclient i mean22:58
comstudit's getting it from keystone itself22:58
devanandaright22:58
devanandabut that icli object can be cached22:58
comstudgotcha22:58
comstudyep22:58
devanandait's not related to the resources being operated upon22:58
comstudso22:58
comstudthere was a case where we were doing this before22:58
comstudfor quantum at the time22:58
devanandaahhh22:58
comstudbut it didn't use a pool correctly or at all :)22:59
comstudso you'd get multiple greenthreads using the connection at the same time22:59
comstudon random occasions22:59
comstudhehe22:59
devanandaheh22:59
*** rpodolyaka has quit IRC22:59
devanandafun22:59
devanandacomstud: unrelated nova question for you23:00
comstudalthough i'm not sure that would happen here23:00
comstudi'm guessing each ironic_call creates a new connection23:00
comstudbut has the token cached is all23:00
devanandawhen destroy() fails because the instance is already gone from the hypervisor, how is that handled today?23:00
comstudso auth only queried on the first one23:00
*** eguz has quit IRC23:00
comstudwe just clean things up23:00
comstudon the nova side23:00
comstudor should :)23:01
devanandacomstud: if the token is cached and shareable, then i think it's fine for each icli object to establish a new HTTP connection, yea23:01
devanandait's not ...23:01
comstudi haven't looked at it in a while23:01
comstuddevananda: yes, it's fine23:01
comstud(some sort of keep-alive option in our clients would be nice at some point, but that's another issue entirely)23:02
comstudchecking on the destroy case just to refresh myself23:02
devananda_delete_instance -> _shutdown_instance23:03
comstudit looks like if it's gone from the hypervisor23:03
devanandaif that raises any exception, then _complete_deletion is not called23:03
comstudwe try to deallocate network23:03
comstudrollback quota. hm weird23:03
devanandawhere i'm at right now is, the ironic node is already disassociated, so ironic has no record of the instance uuid23:03
comstudyeah23:04
devanandai'm changing our driver to raise InstanceNotFound when nova calls driver.destroy()23:04
devanandabut that doesn't work23:04
*** rpodolyaka has joined #openstack-ironic23:04
comstudi feel like the nova code is not quite correct here23:04
devanandait looks like the only thing that would work is just to return23:04
devanandalet driver.destroy() silently succeed if there is no instance in ironic23:04
comstudi think so ATM23:04
devanandawhich feels dirty ...23:04
comstudyeah it does23:04
comstudjust return because, well, you already did the work23:05
comstudno more work to do23:05
devanandak23:05
comstudbut I think you hit on an interesting potential issue here in nova23:05
comstudthere's most certainly an issue if you just manaully wiped the instance out on the hypervisor23:06
NobodyCamahh seed needs a ram disk build with deploy and undercloud need one built on deploy-ironic23:06
comstudit looks like we do deallocate network23:06
comstudbut we leave it counting against quota23:06
devanandacomstud: yea. there are some odd corners there23:07
devanandathat worked. have to just return23:07
*** dwalleck has joined #openstack-ironic23:09
*** dwalleck has quit IRC23:09
*** dwalleck has joined #openstack-ironic23:09
*** jgrimm has joined #openstack-ironic23:15
*** rpodolyaka has quit IRC23:17
devanandaNobodyCam: ugh, there's a nasty race condition here. I stuck a note in a while ago but didn't fix it23:18
NobodyCam:(23:19
devanandaIronicDriver.spawn()23:19
devanandasee the FIXME at the top23:19
NobodyCamBUT IT DOESN'T! :(23:20
*** rpodolyaka has joined #openstack-ironic23:20
*** dwalleck has quit IRC23:21
devanandathe old nova-bm code used an atomic update-then-select operation23:22
devanandaoooh23:25
devanandanvm23:25
devanandawe did23:25
devanandaironic raises a NodeAssociated error if you try to replace the instance_uuid23:25
devanandathe nova driver jsut isnt' catching it23:26
*** eghobo has joined #openstack-ironic23:34
NobodyCamhumm wait call back and then unable to locate configuration file23:36
lifelessdevananda: so what do you think of my ssh patch?23:42
devanandai think i almost have this nova driver bug fixed :)23:42
devanandaone sec23:42
NobodyCamahh chmod on /tftpboot/pxelinux.cfg dir23:51
devanandaShrews: another exception to add to the wrapper around update_node() -- NodeAssociated23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!