Wednesday, 2014-04-23

*** epim has quit IRC00:13
*** rloo has quit IRC00:23
*** matsuhashi has joined #openstack-ironic00:29
*** zdiN0bot has quit IRC00:32
*** derekh has quit IRC00:37
devanandai'm done for the day. will be out in the morning for some personal errands.. see ya'll around lunch tmw!00:40
*** derekh has joined #openstack-ironic00:43
jrollnight deva :)00:45
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479500:47
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479500:59
*** matsuhashi has quit IRC00:59
*** matsuhashi has joined #openstack-ironic01:01
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479501:03
*** yfujioka has quit IRC01:05
*** derekh has quit IRC01:14
*** eguz has joined #openstack-ironic01:20
*** eghobo has quit IRC01:24
*** nosnos has joined #openstack-ironic01:41
NobodyCamanyone seen this one? HTTPInternalServerError: Timeout while waiting on RPC response - topic: "ironic.conductor_manager.ubuntu", RPC method: "update_node" info: "<unknown>"01:48
*** zdin0bot has joined #openstack-ironic02:31
*** coolsvap|afk is now known as coolsvap02:33
*** dwalleck has joined #openstack-ironic02:35
*** harlowja is now known as harlowja_away02:43
*** dwalleck has quit IRC02:45
*** krtaylor has joined #openstack-ironic02:58
*** dwalleck has joined #openstack-ironic02:58
*** dwalleck has quit IRC03:01
*** dwalleck has joined #openstack-ironic03:02
*** openstackgerrit has quit IRC03:04
*** openstackgerrit has joined #openstack-ironic03:04
*** dwalleck has quit IRC03:07
*** matsuhashi has quit IRC03:14
*** matsuhashi has joined #openstack-ironic03:15
*** matsuhashi has quit IRC03:19
*** nosnos has quit IRC03:29
*** eghobo has joined #openstack-ironic03:48
*** rwsu has quit IRC03:50
*** zdin0bot has quit IRC03:50
*** datajerk has joined #openstack-ironic03:56
*** ilives has joined #openstack-ironic04:10
*** eghobo has quit IRC04:10
*** eghobo has joined #openstack-ironic04:10
*** eguz has joined #openstack-ironic04:13
*** zdin0bot has joined #openstack-ironic04:14
*** killer_prince has quit IRC04:15
*** eghobo has quit IRC04:17
*** dwalleck has joined #openstack-ironic04:17
*** rameshg87 has joined #openstack-ironic04:18
*** matsuhashi has joined #openstack-ironic04:19
*** nosnos has joined #openstack-ironic04:22
*** dwalleck has quit IRC04:31
*** dwalleck has joined #openstack-ironic04:31
*** hemna_ has quit IRC04:42
*** coolsvap is now known as coolsvap|afk04:45
*** zdin0bot has quit IRC04:47
*** dwalleck_ has joined #openstack-ironic04:49
*** lazy_prince has joined #openstack-ironic04:52
*** dwalleck has quit IRC04:52
*** lazy_prince is now known as killer_prince04:52
*** Mikhail_D_ltp has joined #openstack-ironic04:55
*** coolsvap|afk is now known as coolsvap04:57
*** shakamunyi has quit IRC05:14
*** dwalleck_ has quit IRC05:18
*** coolsvap is now known as coolsvap|afk05:28
*** Mikhail_D_ltp has quit IRC05:33
*** coolsvap|afk is now known as coolsvap05:36
*** matsuhashi has quit IRC05:37
*** matsuhashi has joined #openstack-ironic05:37
*** matsuhashi has quit IRC05:43
*** matsuhashi has joined #openstack-ironic05:44
*** zdin0bot has joined #openstack-ironic05:44
*** zdin0bot has quit IRC05:49
*** matsuhas_ has joined #openstack-ironic05:50
*** matsuhashi has quit IRC05:53
*** matsuhas_ has quit IRC05:58
*** matsuhashi has joined #openstack-ironic05:58
*** yonglihe_ has quit IRC05:58
*** ilives has quit IRC06:07
*** ilives has joined #openstack-ironic06:07
openstackgerritOpenStack Proposal Bot proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/8850806:07
*** zdin0bot has joined #openstack-ironic06:28
*** ifarkas has joined #openstack-ironic06:28
*** eguz has quit IRC06:42
*** lazy_prince has joined #openstack-ironic06:44
*** lsmola has joined #openstack-ironic06:48
*** zdin0bot has quit IRC06:54
*** viktors_away is now known as viktors06:59
*** romcheg1 has joined #openstack-ironic07:12
*** martyntaylor has joined #openstack-ironic07:18
*** ilives has quit IRC07:29
*** ilives has joined #openstack-ironic07:30
*** ndipanov_gone is now known as ndipanov07:37
*** rameshg87 has left #openstack-ironic07:40
*** ilives has quit IRC07:47
*** vkozhukalov has joined #openstack-ironic07:51
*** ilives has joined #openstack-ironic07:53
*** derekh has joined #openstack-ironic08:02
*** yuriyz has joined #openstack-ironic08:03
Mikhail_D_wkmorning all! :)08:12
*** ilives has quit IRC08:20
*** jistr has joined #openstack-ironic08:21
*** ilives has joined #openstack-ironic08:22
*** matsuhashi has quit IRC08:27
dtantsurMorning Ironic, morning Mikhail_D_wk08:28
*** matsuhashi has joined #openstack-ironic08:34
*** lucasagomes has joined #openstack-ironic08:40
*** BadCub has quit IRC08:41
*** ilives has quit IRC08:42
*** ilives has joined #openstack-ironic08:43
*** radsy has joined #openstack-ironic08:50
*** radsy has joined #openstack-ironic08:50
*** ilives has quit IRC08:50
*** ilives has joined #openstack-ironic08:52
*** romcheg2 has joined #openstack-ironic09:10
*** romcheg1 has quit IRC09:10
*** romcheg1 has joined #openstack-ironic09:13
*** Alexei_987 has joined #openstack-ironic09:14
agordeevgood morning Ironic :)09:14
*** romcheg2 has quit IRC09:14
*** ilives has quit IRC09:34
*** radsy has quit IRC09:44
*** radsy has joined #openstack-ironic10:11
*** radsy has quit IRC10:11
*** radsy has joined #openstack-ironic10:11
*** matsuhashi has quit IRC10:16
*** radsy has quit IRC10:21
*** matsuhashi has joined #openstack-ironic10:23
*** max_lobur has joined #openstack-ironic10:28
*** foexle has joined #openstack-ironic10:32
*** coolsvap is now known as coolsvap|afk10:38
*** Alexei_987 has quit IRC10:56
*** matsuhashi has quit IRC11:04
*** killer_prince has quit IRC11:10
*** killer_p- has joined #openstack-ironic11:21
*** killer_p- is now known as killer_prince11:21
*** lazy_prince has quit IRC11:31
andreykurilindevananda, hi! can you review my patch? https://review.openstack.org/#/c/71500/11:33
*** lucasagomes is now known as lucas-hungry12:00
*** coolsvap|afk is now known as coolsvap12:01
*** romcheg1 has left #openstack-ironic12:04
*** romcheg1 has joined #openstack-ironic12:04
*** jistr is now known as jistr|english12:06
*** nosnos has quit IRC12:10
dtantsurlifeless, around? Mind looking at https://blueprints.launchpad.net/ironic/+spec/pxe-master-images-caching ? this is result of cache-related discussion last week12:15
*** sabah has joined #openstack-ironic12:17
*** lazy_prince has joined #openstack-ironic12:33
*** killer_prince has quit IRC12:33
*** lazy_prince is now known as killer_prince12:33
*** stephenpearson has joined #openstack-ironic12:40
openstackgerritSandhya Balakrishnan proposed a change to openstack/ironic: Add Ironic User Guide  https://review.openstack.org/8981812:40
*** jgrimm has joined #openstack-ironic12:44
*** rloo has joined #openstack-ironic12:44
*** jdob has joined #openstack-ironic12:48
*** sabah has quit IRC12:48
*** lucas-hungry is now known as lucasagomes13:09
*** jbjohnso has joined #openstack-ironic13:11
*** matty_dubs|gone is now known as matty_dubs13:11
*** rloo has quit IRC13:29
*** rloo has joined #openstack-ironic13:30
yuriyzmorning/evening Ironic13:39
NobodyCamMorning yuriyz and ironic... running a little later this morning13:39
*** jistr|english is now known as jistr13:43
openstackgerritAndrey Kurilin proposed a change to openstack/python-ironicclient: Sync latest code and reuse exceptions from oslo  https://review.openstack.org/7150013:43
*** romcheg2 has joined #openstack-ironic13:45
matty_dubsG'day, Ironic13:46
*** romcheg1 has quit IRC13:47
dtantsurmorning yuriyz, NobodyCam, matty_dubs13:50
* dtantsur bbl13:53
openstackgerritDmitry Tantsur proposed a change to openstack/ironic: Implement more robust caching for master images  https://review.openstack.org/8538713:55
*** martyntaylor has left #openstack-ironic13:56
*** linggao has joined #openstack-ironic13:59
*** romcheg1 has joined #openstack-ironic13:59
*** romcheg2 has quit IRC14:01
*** zdin0bot has joined #openstack-ironic14:08
*** rwsu has joined #openstack-ironic14:14
*** killer_prince has quit IRC14:23
*** dwalleck has joined #openstack-ironic14:24
stephenpearsonHi, Noob question .. Inside a vmware guest, I followed http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html and I have a working dev environment.  What I'd like to do is to try building an external node (i.e. another vmware guest).  Made some progress but got stuck and I'm struggling to find documentation to help me with this.  Any general pointers?14:27
stephenpearsonI think I have dhcp requests getting as far as the dnsmasq namespace, but the dhcp config for my node I enrolled in Ironic doesn't exist.14:28
stephenpearsonOh and my node is stuck in the spawning state.14:29
*** dwalleck has quit IRC14:31
matty_dubsstephenpearson: Is http://docs.openstack.org/developer/ironic/deploy/install-guide.html any help?14:32
*** zdin0bot has quit IRC14:32
*** zdin0bot has joined #openstack-ironic14:32
agordeevmorning/evening Ironic :)14:32
NobodyCammorning dtantsur14:33
stephenpearsonmatty_dubs: Thanks - Think that's more wrt getting the service actually running.  Seems to working already for 'internal' nodes.14:34
matty_dubsstephenpearson: We're working on documentation a good bit right now -- https://etherpad.openstack.org/p/IronicDocumentationTasks is an etherpad to track that, but the start includes links to various existing documentation14:35
stephenpearsonMaybe its something dumb like not setting the correct ram so the n-sch doesn't pick up the node.  matty_dubs: Ok I'll have a rummage through those - ta.14:36
matty_dubsThough I'm not sure any of it's terribly useful to you right now :-\14:36
agordeevstephenpearson: all that you want is to expose another guest as fake baremetal vm for ironic, right?14:37
stephenpearsonagordeev: Yes, but external to the devstack environment.  I used the fake driver as I'm only testing for the moment.14:38
agordeevstephenpearson: well, the fake driver is so fake. It doesn't do anything. It's only for API tests14:39
stephenpearsonok, that's fake all right.14:40
agordeevstephenpearson: i'm sure that it's possible to provision the another guest with ironic. But you need to configure network pieces and choose suitable power driver14:40
stephenpearsonI think (not sure) I may have the network parts working.  So the key thing then is to find another driver?  I don't mind restarting the vmware guest by hand.14:41
agordeevstephenpearson: i've just checked. there's no power driver for vmware.14:42
agordeevstephenpearson: what type of fake driver did you use?14:43
matty_dubsagordeev: There's a reference to vmware in the ssh power driver, but I've no idea if it works14:43
stephenpearsonagordeev: In 'ironic driver-list' I see "fake".  I used that.14:43
agordeevmatty_dubs: ssh driver is limited to virtualbox and virsh14:44
stephenpearsonFine, I'll move over to vbox instead.  Thanks.14:45
NobodyCamagordeev: nope vmware has been added14:45
matty_dubsAh, okay. I saw some code in there for a 'vmware' virt type, but it may very well not work / be supported14:45
matty_dubsSeems to use vim-cmd for management14:46
agordeevstephenpearson: in you case you should choose fake_pxe, so you'll switch the node's power manually14:46
matty_dubsBut I'm wholly clueless about VMware?14:46
agordeevNobodyCam: interesting. where to look?14:46
matty_dubsironic/drivers/modules/ssh.py14:47
*** newell has joined #openstack-ironic14:47
matty_dubsI missed it landing; has it been tested much?14:47
NobodyCamagordeev: https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/ssh.py#L6914:47
agordeevooops, i might be looking at the old code :(14:47
agordeevNobodyCam: matty_dubs stephenpearson sorry for confusing14:47
stephenpearsonagordeev: I don't have a fake_pxe driver.  Ah maybe I need to fiddle with IRONIC_DRIVERS_WHITELIST.14:48
NobodyCamI do not have vmware to test it with so I am not sure about its testing14:48
NobodyCamhehehe14:48
*** killer_prince has joined #openstack-ironic14:52
agordeevstephenpearson: the networking is the trickiest part. Idk how to properly setup the networking between guests.14:56
stephenpearsonagordeev: The fact that tcpdump shows that dhcp requests find their way to dnsmasq is encouraging to me, but perhaps its just a mirage.14:57
agordeevstephenpearson: are you sure that those requests reach the dnsmasq at neutron side?14:58
*** jistr has quit IRC14:59
stephenpearsonNot really.  I assumed that if dnsmasq's namespace can see the packets, so can dnsmasq.  I could be totally wrong.14:59
*** jistr has joined #openstack-ironic15:01
agordeevstephenpearson: aahh, right. just curios to know how did you able to connect the another guest to the neutron's network inside of the first guest?15:02
stephenpearsonagordeev: This is basically what I did - http://paste.openstack.org/show/76790/15:04
dtantsurifarkas, around? Are you still intending to work on https://bugs.launchpad.net/ironic/+bug/1231351 ? If so, you can base it on https://review.openstack.org/#/c/85387/ , I guess. Or I can take it from you, if you don't want to15:10
*** dwalleck has joined #openstack-ironic15:10
ifarkasdtantsur, yeah, I can continue it on your patchset15:10
ifarkasdtantsur, I already have something up, just waiting for your patch to rebase ;-)15:11
ifarkasdtantsur, thanks for the heads up15:11
dtantsurifarkas, ack. And you can start with reviewing my patch, please :)15:11
ifarkasdtantsur, sure ;-)15:11
dtantsurI will consider using checksums then15:11
dtantsurok, I'll be back in ~40 minutes15:12
*** dwalleck has quit IRC15:16
*** viktors is now known as viktors|afk15:16
*** zdin0bot has quit IRC15:20
agordeevjroll: JayF: morning and short question about IPA. does the order matter for encoding.Serializable objects? eg.: https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L68-L7315:31
*** zdin0bot has joined #openstack-ironic15:31
*** foexle has quit IRC15:32
*** yuriyz has quit IRC15:32
*** zdin0bot has quit IRC15:33
*** lazy_prince has joined #openstack-ironic15:33
*** zdin0bot has joined #openstack-ironic15:33
agordeevjroll: JayF: i couldn't see a place where the order matters. Why couldn't it be just dict?15:36
*** zdin0bot1 has joined #openstack-ironic15:39
*** zdin0bot has quit IRC15:39
jrollagordeev: the historic reason for doing that was to generate human-readable json15:41
jrollor more readable json15:41
jrollI'd be fine with just using a dict - do you have a performance concern, or just curious, or?15:42
jrollalso good morning ironic :)15:42
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add ManagementInterface  https://review.openstack.org/8606315:45
agordeevjroll: morning, i'm thinking about eliminating of serialize(). What if base class has serialize() which will be generating the dict with all class attributes besides callable and underscored ones15:45
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMITool to use the new ManagementInterface  https://review.openstack.org/8609215:45
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: SeaMicro to use the new ManagementInterface  https://review.openstack.org/8632815:45
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMINative to use the new ManagementInterface  https://review.openstack.org/8658815:45
NobodyCammornig jroll lucasagomes :)15:47
jrollagordeev: hmm, I'm not opposed but in general I don't like magic that iterates over __dict__ or something. I'd rather define something like `serializable_fields = ['name', 'mac_address', ...]`15:47
jrollfor each class15:47
lucasagomesNobodyCam, yo morning :)15:47
lucasagomesmorning everybody :)15:48
jrollmorning NobodyCam lucasagomes <insert every other nick in this channel> :)15:48
lucasagomes:)15:48
lucasagomesNobodyCam, I'm implementing the set_boot_device for the ssh driver, I did for virsh (will push the patch) and tested it15:49
lucasagomesbut it's been a bit painful to rebase everything, do you think we can start merging some of the base patches?15:49
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMINative set_boot_device persistent  https://review.openstack.org/8574215:49
NobodyCamlucasagomes: yes!15:50
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: SSH virsh to use the new ManagementInterface  https://review.openstack.org/8988415:50
lucasagomesNobodyCam, ^15:51
NobodyCamach.. one minute chatting with suse atm15:51
NobodyCamack even15:51
lucasagomes:)15:52
lucasagomestake ur time15:52
agordeevjroll: makes sense. I'll file a bug againts it.15:53
*** vkozhukalov has quit IRC15:54
jrollok15:54
*** eghobo has joined #openstack-ironic15:55
jrollso, neutron uses dnsmasq for its dhcp server, right?16:00
jrolldoes anybody see a problem with that, especially wrt production?16:01
matty_dubsAre there actual issues with it? It's not the first thing that comes to mind when I think of enterprise-grade DHCP servers, but does it have actual, known limits?16:02
*** matty_dubs is now known as matty_dubs|lunch16:02
jrollthe dnsmasq homepage specificially says it's designed for small networks, to start16:03
*** lazy_prince has quit IRC16:06
jrollI don't personally have a lot of experience with it, but from what I hear, it's horrible especially when used at scale16:07
jrollwas hoping someone here had more insight16:07
*** blamar has quit IRC16:08
lucasagomesjroll, I remember that JayF pointed that out in sunnyvalle... I looked at the neutron code and it seems that it's not even pluggable :( they just support dnsmasq and it's hardcoded16:10
lucasagomeswe would need some work there to make it support other dhcp servers16:10
jrollyeah :/16:10
jrollso, devananda and I discussed that the agent would need to use neutron for dhcp etc, in order to allow for agent and pxe (and other) drivers to coexist16:11
lucasagomesNobodyCam, http://www.meetup.com/OpenStack-Ireland/events/178774492/?a=ea1_grp&rv=ea1&_af_eid=178774492&_af=event16:11
lucasagomes:)16:11
lucasagomessome TripleO stuff in Ireland16:12
jrollbut I have a concern that neutron's dhcp agent isn't production-ready16:12
jrolland I do not want to have a dependency on neutron being fixed16:12
pquerna* i'd say it also has a different deployment model that how ironic wants it16:12
pquernaie, commonly, its on every compute node16:12
lucasagomesone way would be to start ur own dhcp server and not use neutron16:13
lucasagomesto answer the dhcp requests16:13
jrolllucasagomes: right, that's our plan16:14
*** max_lobur has quit IRC16:14
jrolldevananda made a point that he did not want the agent to depend on that16:14
jrollmaybe I should wait for him to be around to talk about this16:14
pquernai'm unsure what you mean, agent depend on it16:15
pquernaagent just wants to be running in ram on abox that booted16:15
lucasagomesbut do the agent care about the dhcp server? (after it's booted)16:15
pquernahow you get it booted, either via neutron-world or your own dhcp16:15
jrollthe agent driver*16:15
pquernadoesn't matter16:15
lucasagomesahh16:16
lucasagomesjroll, I bet a option to work as a switch won't be a bad thing16:16
*** mkerrin has quit IRC16:16
jrollpquerna: so, the pxe driver has code that interacts with neutron to set up dhcp correctly16:16
lucasagomesyou can set neutron by default16:16
lucasagomesor disable it setting a config option16:16
jrollpquerna: it's been said that we should do the same16:16
agordeevjroll: i'm a bit more worried about the agent is being dependent on swift's temp url16:16
jrollagordeev: do you have a better way for the agent to get an image from glance without passing credentials all over the network?16:17
jrolllucasagomes: that might work, I don't love it but we could do that16:18
lucasagomesjroll, yeah, I would prefer to have neutron to support multiple dhcp servers16:18
lucasagomesbut you know, getting things in neutron is really frustraring afaict16:18
lucasagomesso, as a short term, a config option seems ok to me16:18
jrollok16:19
lucasagomeswait for devananda anyway so we can talk more about it16:19
jrollyeah16:19
agordeevjroll: no, i haven't. but it is a nice work around for bypassing missing agent's authentification :) Clever, i like it. But being swift dependent seems overcomplicated for the agent (IMO)16:20
*** zdin0bot1 has quit IRC16:21
jrollagordeev: I'm not a fan of depending on swift either, but I don't see another way at the moment16:21
jrollsuggestions welcome16:21
*** mkerrin has joined #openstack-ironic16:26
*** jistr_ has joined #openstack-ironic16:27
*** zdin0bot has joined #openstack-ironic16:30
*** ndipanov is now known as ndipanov_gone16:30
*** ndipanov_gone has quit IRC16:30
*** zdin0bot1 has joined #openstack-ironic16:31
*** jistr has quit IRC16:31
*** zdin0bot1 has quit IRC16:33
NobodyCamlucasagomes: :) nice meetup :)16:33
lucasagomesNobodyCam, :) yeah cool to see tripleO idea spreading, I hope I can make it to that meetup16:34
*** zdin0bot1 has joined #openstack-ironic16:34
*** zdin0bot has quit IRC16:34
pquernajroll: i think it would be reasonable to have a configuration option that is either swift temp url, OR http-unathenticated-template... where its like http://1.2.3.4/images/${image_id}16:34
pquernabut dunno, not excited.16:35
jroll/shrug16:35
NobodyCamya16:35
jrollI could deal with that16:35
jrollagordeev: wdyt?16:35
NobodyCamwow that sed line is hard to follow16:36
lucasagomesNobodyCam, yeah, editing xml with sed heh16:36
*** jistr_ has quit IRC16:37
lucasagomesbut it's simple, sed -i '/<boot \(dev\|order\)=*\>/d;/<\/os>/i\<boot dev=\"hd\"/>'16:37
russell_hit feels like we just need a generic way to tell the agent that an image is at a URL16:37
russell_honly challenge is any sort of header-based auth16:37
*** zdin0bot1 has quit IRC16:37
lucasagomesNobodyCam, it deletes all <boot dev=> and <boot order=> fields, and then add a <boot dev=[DEVICE]> as a child of the <os>16:37
lucasagomess/fields/elements/g16:38
jrollrussell_h: well, the agent just hits the URL you give it, so we have that. the swift dep is really only in the driver16:38
NobodyCamahh ya16:38
NobodyCameven if its set correct to begain with16:38
russell_hjroll: right, but glance won't even necessarily use swift16:38
lucasagomesyeah16:38
russell_hin fact I'm given to understand that most glance deploys probably don't use swift16:39
jrollright16:39
russell_hso I guess my preference would be, if we see your glance is using swift, we'll opportunistically generate temp urls16:39
russell_hotherwise, we fall back to other options16:39
jrollyeah16:40
jrollbut the question at hand is what other options? :P16:40
russell_hanyway, I think we're in a reasonable position today16:40
russell_hwe can iterate in the future16:40
jrollagreed16:40
russell_hdirect download from glance is probably the most reliable16:40
russell_her, and by "direct" I mean really indirect16:40
jrollyeah, but then you have credentials everywhere16:40
jrollwhich is the main reason we went to this tempurl thing16:41
jrollanyway, we're fine for now16:41
NobodyCamlucasagomes: in the testing I see # vmware does not support (set|get)_boot_device will this be true always? or just until its patched to support it?16:45
lucasagomesNobodyCam, not always, but as it does not support it for now16:46
lucasagomesI put that comment there16:46
lucasagomesonce vmware adds (set|get)_boot_device to that commands mapping16:46
lucasagomesthat test will start failing16:46
lucasagomeswhich is good, so when it's being implemented it won't be missed16:47
*** matty_dubs|lunch is now known as matty_dubs16:48
NobodyCamok we DO need to land the other dep patches16:48
lucasagomesthat would help a lot heh16:48
lucasagomesbecause the linear dependency is getting long16:49
NobodyCamack16:49
*** blamar has joined #openstack-ironic16:50
NobodyCamlucasagomes: do you know (off the top of your head) if the DIB rhel elements work?16:52
lucasagomesNobodyCam, hmm not anymore, long time ago I have worked with the guy who was implementing it so it worked16:53
lucasagomesbut I didn't tried it since then16:53
NobodyCamack :)16:53
openstackgerritOpenStack Proposal Bot proposed a change to openstack/ironic: Updated from global requirements  https://review.openstack.org/8923416:54
lucasagomesNobodyCam, the IRC name of the author is funzo16:54
lucasagomesNobodyCam, he's on the tripleo channel16:55
lucasagomeshe might know better than me16:55
NobodyCamack :)16:57
*** lucasagomes_ has joined #openstack-ironic16:59
NobodyCamjust pinged him OoO16:59
lucasagomes_wow! great, installed vbox, had to stop all my kvm vms because vt was being used... started a machine with vbox and it gave me a kernel panic :D16:59
lucasagomes_on the host machine I mean16:59
lucasagomes_damn oracle17:00
NobodyCamnice.... yep vbox and virsh don't play well togheter17:00
lucasagomes_yeah it was vbox alone17:00
* lucasagomes_ will try again17:00
*** lucasagomes has quit IRC17:00
dtantsurlucasagomes_, interesting, in my experience vbox seems to be much more reliable17:02
*** lucasagomes has joined #openstack-ironic17:02
lucasagomescool again, idk hw I'm adding vbox support now lol17:02
*** dwalleck has joined #openstack-ironic17:02
*** derekh has quit IRC17:02
*** lucasagomes_ has quit IRC17:05
*** harlowja_away is now known as harlowja17:06
*** dwalleck has quit IRC17:11
*** dwalleck has joined #openstack-ironic17:13
lucasagomesok I'm done for the day, have a good night everybody17:16
*** lucasagomes is now known as lucas-dinner17:16
NobodyCamhave a good night lucas-dinner :)17:16
*** eghobo has quit IRC17:17
*** eghobo has joined #openstack-ironic17:19
*** eghobo has quit IRC17:20
*** eghobo has joined #openstack-ironic17:20
*** stephenpearson has quit IRC17:21
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding swift temp url support  https://review.openstack.org/8139117:27
*** vkozhukalov has joined #openstack-ironic17:27
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Drivers may expose a top-level passthru API  https://review.openstack.org/8191917:27
NobodyCambrb17:29
*** zdiN0bot has joined #openstack-ironic17:47
devanandag'morning!17:48
NobodyCamgood morning devananda17:49
jrollhey devananda17:49
*** lsmola has quit IRC17:51
devanandalucas-dinner: re: ironic ssh port, see https://review.openstack.org/#/c/87355/17:52
devanandalucas-dinner: erm, nvm... wrong thing17:54
devanandadtantsur: that was meant for you ^ :)17:54
*** eguz has joined #openstack-ironic18:01
*** dwalleck has quit IRC18:04
*** eghobo has quit IRC18:05
*** epim has joined #openstack-ironic18:05
*** zdiN0bot has quit IRC18:11
devanandajroll: do ya'll have an etherpad up yet for the IPA session?18:12
devanandajroll: if not, i'm going to create one and link it from the session page18:12
devanandajroll: context here is that every session needs an etherpad for collaborative note-taking and recording of action items18:12
jrollwe don't, go for it18:13
devanandajroll: any further changes you want to make to the proposal before i schedule it, now's your chance18:14
devanandaby EOD it will appear on sched.org18:14
jrollI'm fine with it, I think18:14
jrollI haven't had any revelations in the past three days :P18:15
devananda:)18:15
jrolly'all ever seen a scenario where pecan/wsme throws a 500 and all you get back is: {"debuginfo": null, "faultcode": "Server", "faultstring": ""} ?18:16
devanandamatty_dubs, jroll: i've thought a bit more about the scale talk, and i'd like to chat with you a bit18:16
devanandajroll: nice. i dont recall seeing that18:16
jrollthis is happening in the agent but thought I would ask18:16
jrollit's super helpful.18:17
devanandaindeed18:17
jrolldevananda: yeah, I can chat18:17
devanandaso, last summit, we had a similar session proposed18:17
devanandait's a bit like this: without hard numbers, all we can do is theory craft and speculate18:17
devanandawith hard numbers, we know what to fix18:17
devanandain the first case, there's nothing actionable at the end18:18
devanandain the second case, we know what to do, so there's not much to discuss18:18
devanandaas a whole group18:18
jrollsure. although in the second case, if it's an architecture thing, it could use some discussion, no?18:18
devanandawe'l have plenty of time in hallway track // around the project PODs18:18
JayFShouldn't there be some amount of goal setting though? Like we should be able to provision N servers with X conductors in less than 5 minutes, for instance18:18
devanandafor technical discussion of "how do we implement X"18:18
*** epim has quit IRC18:19
jrolldevananda: yeah, that's fair. feel free to axe it if you want18:19
devanandabut we probably don't need to involve 80 - 100 ppl for that technical discussion18:19
jrollJayF: we don't need to all be in a room for 40 minutes for that :P18:19
devanandathat's the difference -- our four slots will draw not just our main upstream developers18:20
devanandabut a lot of vendor folks, and folks from other projects18:20
jrollyep18:20
devanandaJayF: setting/agreeing on project goals is a good use of that time, if they're somewhat contentious goals18:20
devanandaJayF: but that needs to be balanced with the tendency we, as a community, have to rehash the same things (like kexec)18:21
devananda:)18:21
jrollheh18:21
devanandait's great how many folks want to make ironic fast, but hardware is complicated and likes to break18:22
devanandawe should all just go use VMs (lol...)18:22
jroll:P18:23
*** zdiN0bot has joined #openstack-ironic18:23
jrolldevananda: we were talking earlier about dhcp things - a couple questions for you18:23
* NobodyCam wonders if we use openstack based vms instead for virsh :-p18:23
jroll1) are you ok with a config option for the agent driver to *not* use neutron's dhcp?18:24
jroll2) do you have any concerns with the performance / general crappiness of dnsmasq, and depending on that?18:24
devanandajroll: re: multitenancy proposal, that's a key use case for you guys, right?18:25
jroll'tis18:25
devanandajroll: like, you wouldn't be interested in ironic if it doesn't do that at some point18:25
jrolldevananda: more like, we're going to make ironic do that, even if nobody else helps us :)18:26
devanandaheh18:26
lucas-dinnerfolks couple of patches in the python-ironicclient needed another +218:26
lucas-dinnerhttps://review.openstack.org/#/q/status:open+project:openstack/python-ironicclient,n,z18:26
devanandajroll: so, given that i want that (eventually) too, and I know others do as well, I'm going to add a couple items to your proposal's description18:26
jrollthat's fine18:27
* lucas-dinner brb18:27
devanandajroll: 1) make the dhcp dependency pluggable for all drivers, then yes18:28
devanandajroll: 2) for a static external dhcp dependency, ironic shouldn't care what it is18:28
*** blamar has quit IRC18:28
devanandajroll: 2a) if ironic needs to (re)configure some external dhcp, see (1)18:28
jrolldevananda: on 2, I agree that ironic shouldn't care what it is. but given that we know neutron is hardcoded to use dnsmasq, are you comfortable placing a dependency on that?18:29
jrollit's pretty well known that dnsmasq is not designed for large environments18:29
devanandajroll: i'm not sure how 2) results in a dependency on any specific dhcp provider18:30
jrollneutron is hardcoded to use dnsmasq18:30
devanandaperhaps i misinterpreted your (2)18:30
jrollor are you saying it doesn't necessarily depend on neutron?18:31
devanandaright18:31
jrollah18:31
devanandai see two paths18:31
devananda(1) use neutron to dynamically (re)configure dhcp (and other network bits) during provisioning/teardown/etc18:31
devanandathis will eventually also tie into things like switch configuration18:32
devananda(2) flat network with external static DHCP and a common PXE image (eg, the agent)18:32
devanandain (2), ironic won't care what the DHCP provider is18:32
jrollright18:33
jrollso you're saying that should be a single config option for all drivers?18:33
*** zdiN0bot has quit IRC18:33
devanandayes18:33
jrollok, cool18:33
*** max_lobur has joined #openstack-ironic18:33
jrollare you cool with landing the agent driver before this happens?18:33
jroll(last question, promise)18:34
devanandabefore the PXE driver has that option? yes18:35
NobodyCamI have not read the entire scroll back, but seems to me the fix is it expand Neutron to use dhcp providers other then dnsmasq18:35
jrolldevananda: cool. thanks :)18:35
devanandabefore the agent driver supports neutron? ... not so much18:35
jrolloh.18:35
devanandaNobodyCam: iiuc, jroll wants to use flat networking with external static DHCP18:36
jrolldevananda: I'd really prefer to do that in a separate patch18:36
NobodyCamahh :)18:36
devanandajroll: my hesitation is about landing a driver which can not be used in the same environment as the current reference driver18:37
jrollyeah, I understand that18:37
devanandajroll: IMHO, that's an architectural requirement, and for folks testing from trunk, it's going to be broken if we land the agent driver now18:38
jrolleven if the agent driver is not enabled by default?18:38
devanandayep18:38
devanandacause someone can enable it18:38
jrollthose folks will be able to continue using the pxe driver, no?18:38
jrollmmm ok18:38
devanandai have wavered on this point a bit, i know. the implications are becoming clearer as we discuss it and as I review the code more18:39
jrollwe talked before about the agent driver not needing to be 100% complete to land, so I've been working from the assumption18:39
jrollyeah, I hear ya18:39
jrollfrom that assumption*18:40
devanandayea, i definitely agree, doesn't need to be 100% complete, but it should be functional18:40
devanandaas in, i should be able to test it18:40
jrollright18:40
jrollyou can, just not at the same time as the pxe driver :P18:40
devananda:)18:40
devanandaso, for instance18:41
devanandato test the agent in devstack today18:41
devanandai'd need to disable neutron and manually configure dnsmasq18:41
jrollright, I've been there18:42
jrollit's a pain, I agree18:42
devanandait's taken a while to get all that tooling into devstack for Ironic18:42
devananda*just for the pxe driver18:43
jrollnod18:43
jrollneutron might even be worth it just for those purposes18:43
jrolland tempest etc18:43
devanandaright18:43
jrollok18:44
devanandajroll: as an aside, have you seen the thrid-party testing guidelines I posted?18:44
jrollmaybe?18:44
jrollI don't recall them at the moment18:44
devanandathey were an email several months back and a wiki page I just posted yesterday18:44
jrollok, I'll check that out18:44
devanandajroll: https://wiki.openstack.org/wiki/Ironic/Testing18:45
jrollah, right18:45
jrollcool, I'll go off of that as far as this driver goes18:46
devanandajroll: I think it's impiortant that all supported drivers be tested on every commit to ironic (whether upstream or by third-party systems) by the time we graduate18:46
jroll+118:46
devanandajroll: so yes, we can land the driver before we land testing, but i hestitate to land code that is _untestable_ at this point18:47
jrollI'm running off to lunch, anything else you want to chat about before I go?18:47
jrollsure, that's fair18:47
devanandajroll: fwiw, I am holding the ilo and seamicro drivers to that, too, though they're clearly under third-party CI umbrella18:47
devanandajroll: nope. ttyl18:47
jrollsounds good18:47
jrollbbiab18:48
*** jdob_ has joined #openstack-ironic19:10
dtantsurdevananda, hi! I've seen https://review.openstack.org/#/c/87355/, that's awesome :)19:16
dtantsurdevananda, also, if you're still around, could you have a look at https://blueprints.launchpad.net/ironic/+spec/pxe-master-images-caching ? It's a late attempt to make a blueprint for cache-related job I am and will be doing19:18
devanandadtantsur: hi! the BP is a little hard to understand19:20
NobodyCamdtantsur: I am concerned about Consider also Glance checksum when caching (e.g. by adding it to file name), so that when image is updated in Glance, it is redownloaded19:20
dtantsurdevananda, probably, I'm not really good at writing such things :) Something specific to clarify/enhance?19:21
NobodyCamare you saying that if a image is updated in glance it will be redownloaded19:21
devanandadtantsur: it may be a grammar / language issue, i'm not sure, but some things are hard to parse, eg, "so that have to redownload them every time for large deployment"19:21
*** dwalleck has joined #openstack-ironic19:21
devanandadtantsur: the BP should not reference existing code proposals19:22
dtantsurdevananda, oh, that should read as "so that we don't have to". Or is it also too much?19:22
devanandadtantsur: so i dont understand that -- ironic caches the images already, doesn' tit?19:23
dtantsurNobodyCam, redownloaded the next time it is going to be used, not for exiting instances19:23
devanandadtantsur: the code review is to add a TTL and some cleanup jobs19:23
dtantsurdevananda, it does, but only while they are required for exiting instances. Code review is about caching them a bit more + adding TTL and size limit.19:24
NobodyCamdtantsur: what if there are depolyments using the old Image. we don't want to change things on deployed instances19:24
devanandadtantsur: right. the BP is implying that there is no caching at all today19:24
dtantsurNobodyCam, sure, but for them file name and links won't change, only for new deployments19:25
dtantsurdevananda, well, that's why it reads "Enhance" :) and " Store images for some time ***after they're not in use by any instance***"19:25
devanandadtantsur: what NobodyCam points out also concerns me. as does adding a requirement on the glance image *name*19:25
dtantsurdevananda, it's not requirement on image name in Glance, it's requirement on naming master image in cache (currently it is UUID, will be like UUID + checksum)19:26
devanandadtantsur: oh, i see - you'd add the checksum to the local file name, not the glance image name19:26
dtantsurok, I do need to rephrase things :)19:27
devanandathanks :)19:27
dtantsurI'll fix wording for everything mentioned here, thank you. Any more concerns?19:28
*** epim has joined #openstack-ironic19:34
*** ifarkas has quit IRC19:35
*** Antl3r has joined #openstack-ironic19:38
*** max_lobur has quit IRC19:41
*** Antl3r has left #openstack-ironic19:52
jrolldevananda: nice email, thank you for that :)19:54
jrollalthough I see two meetings before the summit - are we skipping the "off week" meeting?19:55
devanandajroll: oh. you're right.19:56
jrollheh :)19:56
devanandajroll: i leave on tuesday for a conference in berlin, that was my confusion19:56
jrollah, got it19:56
NobodyCamlol I made exact same mistake :-p19:56
devanandajroll: replied19:58
devanandajroll: and now i need to  find lunch and hit the road19:58
NobodyCamhit the road?19:58
NobodyCamup at the house?19:58
devanandajroll: i edited http://summit.openstack.org/cfp/details/405 a bit19:58
*** dwalleck_ has joined #openstack-ironic19:59
devanandajroll: if you can prepare the etherpads for that and the agent session over the next two weeks, that'd be great19:59
devanandajroll: the multitenancy one could derail really easily -- i'll probably moderate that even though you proposed it, to keep things on track of what we can actually do upstream20:00
devanandawithout too much reliance on specific hardware vendors (who, yes, will want to steer the talk in their own ways, i expect)20:01
devanandaok, really gotta run now before i get a parking ticket :p20:01
devanandabbiah20:01
jrolldevananda: yeah, np. thanks20:02
*** dwalleck has quit IRC20:02
*** zdiN0bot has joined #openstack-ironic20:07
openstackgerritA change was merged to openstack/ironic: Clean up calls to get_node()  https://review.openstack.org/8457320:08
*** eguz has quit IRC20:19
*** eghobo has joined #openstack-ironic20:20
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Drivers may expose a top-level passthru API  https://review.openstack.org/8191920:27
dtantsurdevananda, NobodyCam I believe I managed to make https://blueprints.launchpad.net/ironic/+spec/pxe-master-images-caching read better, could you look again?20:30
NobodyCambrb20:31
comstudcan someone explain to me a few things about this patch:20:32
comstudhttps://review.openstack.org/#/c/78651/7/ironic/conductor/manager.py,unified20:32
jbjohnsoquestion, so I'm chugging along on my console server, was wondering if there was a request with respect to authorization and authentication20:32
comstudThis merged... and20:32
*** dwalleck_ has quit IRC20:33
jbjohnsocurrently, I authenticate a user and then said user has equal access to all consoles that share the 'tenant'20:33
jbjohnsouser can be authenticated by passphrase or certificate20:34
comstud1) The commit message says nothing about the move of code into utils method (which fixes a different bug).. This is fine, but there was no mention of why20:34
comstud2) It removes 'maintenance' from the filters with no explanation, either.20:34
comstud#2 is what I'm mostly concerned about20:34
jbjohnsowas wondering about either adding another user authentication or implementing a token granting facility20:34
comstuddevananda: you approved this, maybe you know why?20:34
*** jbjohnso has quit IRC20:41
rloocomstud: I believe devananda is away right now but he'll be back shortly20:43
rloocomstud: wrt 1) I think the move was something to do with one of my comments. sec, let me look.20:43
*** harlowja is now known as harlowja_away20:44
*** jdob_ has quit IRC20:44
rloocomstud: wrt 1), see the comment at line 624: https://review.openstack.org/#/c/78651/4/ironic/conductor/manager.py20:46
rloocomstud: wrt 2). hmm, can a node be in maintenance AND in DEPLOYWAIT provision state?20:48
JayFrunning devstack, as indicated here: http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#deploying-ironic-with-devstack is supposed to get me a tftp and dhcp server as well, right?20:51
comstudrloo: No idea.. but these types of things would be helpful in the commit message :)20:52
comstudI'm resolving a conflict with one of my patches.. and I can't find any reason why 'maintenance' was dropped20:53
ShrewsJayF: yep20:53
JayFShrews: where would the tftp root be in that case?20:53
JayFtrying to unravel this configuration to see if ironic pxe and agent drivers could live side by side20:53
JayFbut afaict everything assumes one big network and one big pxe config20:54
* NobodyCam starts a devtest run and says brb20:54
rloocomstud: it isn't really clear to me how much detail should be put in the commit. But I think that if people expect to see more info there, we should try to make sure that it is there.20:54
comstudI could see something going into maintenance while being stuck in DEPLOYWAIT20:54
ShrewsJayF: iirc, there is a map file20:55
comstudrloo: I'm less concerned about #1.. but I don't see anything mentioned about #220:55
rloocomstud: the states and their combinations, need to be clarified I think.20:55
comstudanyway, I'm inclined to keep it in my patch (which means to re-add it)20:55
ShrewsJayF: /tftproot is mapped somewhere else, i believe20:55
JayFyeah I've been looking for it to no avail20:55
JayFgetting the agent running with devstack / neutron dhcp = not terribly difficult. Getting teh agent driver and pxe driver running together with devstack / neutron dhcp = difficult20:56
JayFat least from my looking so far20:56
rloocomstud: so my assumption (and maybe my bad cuz I reviewed it) is that a node in maintenance won't be provisioning. And I think that's why it was removed. But it won't hurt to put it back in.20:56
jrollJayF: ironic docs say /tftproot20:56
JayFnone of this was written in with the ability to have multiple different pxe targets20:56
jrollJayF: also, tftp_root in ironic.conf20:56
JayFjroll: yeah, it may be that JoshNang's devstack was busted, I'm waiting for mine to finish and will check there :x20:57
JayFdevstack has been installing for around an hour or so20:57
jrollyeah20:57
jrollwat20:57
jrollwas 1000s or so for me20:57
jrollbut that was on ethernet20:57
ShrewsJayF: the person to ask is adam_g. he set that stuff up20:57
jrollso /shrug20:57
JayFjroll: also I'm working from a minimal ubuntu image/install, so maybe further to go20:57
jrollah20:57
adam_gJayF, is there a working branch somewhere with the changes you made for agent support? id be intersted in helping out but dont know where to start /w the agent21:01
JayFwe don't have a working branch21:04
JayFin terms of devstack21:04
JayFjroll should be able to let you know what patches to apply on the ironic side to make the driver work21:04
jrolloh hey21:04
*** harlowja_away is now known as harlowja21:05
JayFand now a clean devstack install, from master, with the localrc from the docs is failing21:05
* JayF flips table21:05
JayFfailed to define network from /dev/fd/63 \n Error: XML Error: unexpected virtualport type -121:06
jrolladam_g: one sec21:06
jrolladam_g: master + 81391 + 81919 + 84795 should do it21:08
ShrewsJayF: apparently /opt/stack/data/ironic/tftpboot is removed during unstack'ing, which is news to me, and perhaps why you had a hard time finding it21:09
* Shrews going through a fresh devstack setup right now too21:10
adam_gJayF, you need to be installing a newer libvirt from the cloud archive21:10
*** adam_g has left #openstack-ironic21:11
*** adam_g has joined #openstack-ironic21:11
adam_gJayF, here is a localrc that is close to what gets run in the gate http://paste.ubuntu.com/7317641/21:12
JayFshould the one here work? http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#deploying-ironic-with-devstack21:13
JayFI did see I missed a step to install the cloud archive apt repo21:13
adam_gJayF, yes, it should21:13
ShrewsJayF: that's the one i'm using21:14
*** jdob has quit IRC21:17
*** pradipta_away has quit IRC21:19
Shrewsoh my, 'screen -x -p =' is my new favorite command when working with devstack21:22
JayFnice trick21:23
Shrewssometimes man pages are useful... sometimes21:23
JoshNangooo that is handy21:23
JayFwell it told me what that did, heh21:23
*** pradipta_away has joined #openstack-ironic21:24
*** linggao has quit IRC21:25
JayFShrews: how much ram do you guys give your devstack machines?21:27
JayFand I'll add that to the doc21:27
* JayF just got out of memory erros21:27
ShrewsJayF: i've only ever used it on vm or actual h/w with 8G minimum21:27
JayFthanks, I can spare that much for VMs :)21:28
JoshNangJayF: damn. should have opted for the 16gb laptop :(21:33
vkozhukalovJayF: if you have a couple minutes, please take a look at https://blueprints.launchpad.net/ironic/+spec/ironic-python-agent-partition, I've updated API in etherpad draft https://etherpad.openstack.org/p/ironic-disk-partitioning. Removed granular API, added scheme based API.21:38
ShrewsJoshNang: i would not recommend running devstack on a personal laptop that you actually use21:39
Shrewsunless it's under a vm on the laptop21:39
JoshNangShrews: oh its definitely in a VM. working on a vagrant setup :)21:40
Shrewsoh neat21:40
JoshNangyup!21:41
*** romcheg1 has quit IRC21:45
openstackgerritChris Behrens proposed a change to openstack/ironic: Eliminate races in Conductor _check_deploy_timeouts  https://review.openstack.org/8922321:54
openstackgerritChris Behrens proposed a change to openstack/ironic: Spawn support for TaskManager and 2 locking fixes  https://review.openstack.org/8932821:54
comstuddevananda: ^ (for when you're back later. That spawn taskmanager review should be good to go)21:55
*** matty_dubs is now known as matty_dubs|gone21:58
* devananda is back22:02
openstackgerritChris Behrens proposed a change to openstack/ironic: Eliminate races in Conductor _check_deploy_timeouts  https://review.openstack.org/8922322:03
openstackgerritChris Behrens proposed a change to openstack/ironic: Spawn support for TaskManager and 2 locking fixes  https://review.openstack.org/8932822:03
comstudminor fix22:03
comstudnow that you're here, I'm taking off for a while :)22:03
devanandacomstud: 2 looks like an oversight which, somehow, at least five reviewers missed22:07
comstuddevananda: ok, i've re-added it in one of my reviews above.22:07
comstudhttps://review.openstack.org/#/c/89223/22:08
comstudI hit it when I had to rebase22:08
comstudbbl22:10
devanandacomstud: thanks22:10
*** harlowja has quit IRC22:14
JayFhow and where does /opt/stack/data/ironic/tftpboot/pxelinux.cfg get populated with configs? On demand as machines are provisioned?22:16
JayFI have that dir now but the lack of configurations is confusing22:16
devanandaJayF: iirc (dont have my devstack in front of me at the moment) that dir is created when devstack starts up22:17
JayFyeah I have the dir, and there are a few things populated in tftpboot/ that I'd expect (like pxelinux.0), but there are no configs in pxelinux.cfg/22:17
devanandaJayF: and symlinks with MAC addresses are created that refer to the individual configs and images22:17
JayFaha22:17
devanandawhen an instance is created22:17
JayFoh that's actually kinda awesome22:17
devanandathat way, boot requests from unknown MACs go unanswered22:17
devanandaand multiple machines which boot from the same image don't waste disk space22:18
JayFso in order to make that support the agent, I just have to make sure when it makes that link for an agent-powered mac, that it uses a different config source22:18
devanandayep22:18
JayFdo you happen to know where that happens? assuming in the pxe driver atm?22:18
devanandathere should be some common methods for managing the symlinks22:18
devanandalemme look22:18
JayFyeah that's what I'm looking fro22:18
JayF*for22:18
JayFfwiw neutron isn't even using dnsmasq directly, it's using libvirt which uses dnsmasq22:19
JayFI know we were talking about pluggability earlier, but it's even one more step abstracted22:19
devanandaJayF: i'm going to just paste some paths to things related to this, see what interests you22:19
devanandaironic/common/images.py22:19
devanandaJayF: ironic/drivers/modules/pxe.py actually contains 90% of this code22:20
devanandai thought it was slightly better split up, but it's not22:21
devanandajust those two files22:21
JayFjroll: devananda: ^ so doing those changes, should that be a different merge req? included in the agent driver?22:21
devanandaJayF: also, dtansur is working on refactoring the image caching code (which is what creates said symlinks)22:21
devanandaJayF: see https://blueprints.launchpad.net/ironic/+spec/pxe-master-images-caching22:21
jrollJayF: if you refactor something that's not only in the agent, it should be a seperate patch22:22
JayFyeah so we're going to have to refactor all that out of the pxe driver22:23
JayFso they can share and work together22:23
JayFif the agent and pxe driver are to work on the same machine22:23
JayFbceause the existing assumption seems to be that the pxe driver, when active, owns all pxe/dhcp for a subnet, which clearly can't be true if we need to pxe agent images22:23
devanandaJayF: i think you mean, "the agent driver, when active, owns..."22:24
devanandaJayF: the PXE driver requires neutron to own DHCP ;)22:24
JayFbut pxe driver owns configuration for the pxe, aka the stuff under pxelinux.cfg/22:24
JayFwhich is really all that the agent cares about. We don't care about exactly what dhcp does, as long as it gets the agent on the box22:25
devanandalocal tftp config - yes22:25
JayFand as an MVP, seems like the easiest way to get there is to factor out the tftp config handling bits into something that both the agent and pxe drivers can use simlutaneously22:25
JayFso that both drivers can be active and working in a devstack22:25
devanandaJayF: sure22:26
devanandaone thing to note22:26
devanandaCONF.pxe.tftp_server defaults to the conductor host's IP22:26
devanandaand pxe driver makes sure all the right things are set up locally22:26
JayFAll that stuff would have to move into something both agent and pxe driver could use, no?22:27
JayFfrankly, I wouldn't run the agent driver like this in a production environment, but if it's a requirement that both work under devstack simultaneously, this seems to be the most correct path to take22:27
devanandaand the current assumption is that the deploy ramdisk & kernel IDs are set on the nova flavor22:28
devanandaso that heterogeneous hardware (needing different deploy kernels) can be supported22:29
devanandawhereas, if the deploy kernel*ramdisk were a CONF option, that wouldn't be possible22:29
JayFSo the deploy ramdisk, and kernel options used, are actually baked into the nova flavor?22:29
devanandayep22:29
devanandawell not the kernel OPTIONS22:30
devanandathe kernel image ID22:30
JayFah22:30
*** zdiN0bot has quit IRC22:30
jrollum22:30
jrolldoesn't this tie a driver to a flavor then?22:30
JayFso the kernel command line is something we'll need to be able to modify as well22:30
jrollJayF: that bit is done in ironic afaik22:30
JayFThat doesn't make sense though, as kernel command line opts can vary wildly as the deploy ramdisk/kernel versions vary22:31
devanandaJayF: kernel params passed in via dhcp can be set programatically by ironic, yes22:31
*** mrda_away is now known as mrda22:31
devanandajroll: well, now that you mention it, yes22:31
jrollJayF: ironic knows which ramdisk is happening22:31
JayFright now I'm not even sure libvirt supports passing in those options to dnsmasq via it's intermediate xml format22:31
JayFso much misdirection22:31
jrolldevananda: yeah, I don't like that22:32
devanandajroll: a flavor whose baremetal:deploy_ramdisk_id refers to the agent would not be usable on a node configured for the pxe driver22:32
devanandahmmm22:32
devanandai don't like it either22:33
jrolldevananda: I guess I'm of the opinion that deploy_ramdisk shouldn't be in the flavor22:33
JayFso it's impossible for someone to upgrade from pxe->agent then, in some nebulous future where agent actually works, without a hard cutover?22:33
jrolldevananda: but then again for the different hardware you mentioned, idk what alternatives there are22:33
devanandaJayF: my goal is for that to not only be possible, but be well tested22:33
JayFThen it seems like as an architectural point, this will be something that'll have to have it's behavior change?22:34
devanandajroll: the nova scheduler would, in theory, need to match flavor<->driver22:34
jrollhmm22:34
devanandabut that means, changing a node from pxe -> agent means having a separate flavor for it22:35
devanandawhich i don't like22:35
jrollright22:35
devanandajust curious, how much work do you think it would be to make the agent compatible with the current pxe driver?22:37
devanandaif a particular kernel param is passed in, create iSCSI endpoint, POST that instead, and listen for notice of complete22:38
jrollmmm22:38
jrolldevananda: I know where you're going with this22:38
jrolland I think it's a great idea, in general22:38
jrollbut I also think it is far too early22:39
devanandaupgrade path22:39
devanandasure, it's early. not saying that has to happen now :)22:39
jrollright22:39
pquernaupgrade from what22:39
devanandabut is that possible? is that something we'll need?22:39
devanandapquerna: pxe driver -> agent driver22:39
devanandajroll: i think it's "yes" to both22:39
jrollpquerna: today a nova flavor defines which ramdisk a node boots, for pxe driver22:40
jrollpquerna: if we want feature parity in terms of dhcp things, we'll have to do the same for agent driver22:40
*** zdiN0bot has joined #openstack-ironic22:40
jrollpquerna: which means changing flavors to upgrade from pxe -> agent22:40
jrolldevananda: perhaps. definitely yes to #1. not sure about #2.22:41
jrollyes comes with the question of 'is it worth it?'22:41
devanandajroll: when there are production environments using the pxe driver, i think they'll want to upgrade22:42
jrollsure22:42
devanandajroll: perhaps simply creating a new flavor is sufficient. I don't have enough insight into deployer's future needs to say at this point22:43
jrollyeah22:44
*** zdiN0bot has quit IRC22:44
jrollideally one shouldn't need to do that22:44
*** zdiN0bot has joined #openstack-ironic22:44
jrollbut it's going to be some work and especially testing to migrate as is22:44
jrollcreating a flavor is a really small thing in those terms22:44
devanandajroll: telling all your customers to stop using flavor X and start using flavor X' -- just to get the same results22:45
devanandajroll: that's my concern22:45
jrolldevananda: idk that you would need to22:45
JayFlooking at https://blueprints.launchpad.net/ironic/+spec/pxe-master-images-caching -- does this mean deploy ramdisks are stored in glance for purposes of the pxe driver?22:45
jrollbecause the flavor in this case only matters at provision time, right?22:46
devanandaJayF: yes22:46
jrollso you do something like 'update instances set flavor_id=$newflavor where flavor_id=$oldflavor' and you're done22:46
devanandajroll: correct. you could update the flavor and update all the node's .driver properties -- as long as nothing was provisioned while that update happened22:46
jrollafaict22:46
devanandait'd be ok22:46
jrolls/provisioned/provisioning ?22:46
devanandajroll: that has to be sync'd with the change to node.driver22:47
jrollsure22:47
devanandaunless the agent is backwards compatible :)22:47
jrollstill not a big deal22:47
devanandawhich is why I asked22:47
jrollright22:47
devanandabut yea, not a huge deal22:47
devanandabrief interruption in provisioning (but not of service to existing instances)22:47
devanandareasonable for an upgrade22:47
devanandai agree22:47
jrollyeah22:47
JayFDoes the fact Ironic is incubating and not integrated matter for terms of this discussion? Seems like we should have a lower responsibility for backwards compatibility in something that's not integrated yet22:47
jrollbrief being the execution time of those two UPDATE queries22:48
devanandaJayF: you're correct from the TC's perspective22:48
devanandaOTOH, it shows a lot more maturity if we don't abuse that22:48
JayFThe real question has to be asked of how many people are going to use Ironic now with the pxe driver?22:48
devananda*and* there are downstream consumers who care a lot about it22:48
devanandafolks are going to start deploying things with Ironic now that Icehouse is out22:48
JayFIt's just a little worrying that we're chatting about upgrade paths to a driver that isn't even merged yet22:48
devanandathey'll be impacted if there is no upgrade path to Juno22:48
devanandaJayF: how is that worrying?22:49
jrollJayF: we're hypothesizing for the future, not defining work for today22:49
devanandaJayF: if the plan wasn't to upgrade from pxe to the agent driver, why would we even be working on it?22:49
JayFWell I just would rather things work is all, and it's a little frustrating that it doesn't yet.22:49
JayFRight now the hump I've gotta get over is getting these things running side by side in devstack for the time, and it's just a little more complex and misdirected than I realized22:50
devanandaJayF: fwiw, it took more than six months from when I forked ironic to when it "worked"22:51
JayFyou have bucketloads more patience than I do with such things then :)22:52
devanandai would have suggested you guys take an incremental approach when ya'll started, if you'd been around prior to writing all this code22:52
devanandamerging an existing team, with your own ideas around provisioning and your culture, into this one ... it's going to have a few bumps22:53
devanandahopefully you guys can see how ironic (both the code and the culture) is changing as a result of your input22:53
devanandaanyhow... now back to the technical stuff22:54
devanandajroll: how do you see the agent co-operating with two different deployment styles22:54
devanandaa) use neutron for dhcp and serve tftp from each conductor host, and b) static external dhcp and tftp22:55
jrolldevananda: that shouldn't be a problem, other than time22:56
devanandajroll: code wise, how do you want to do that?22:56
jrolland by time is a problem, I mean it delays the goal of making it work22:56
jrollok, so22:57
jrolldisclaimer: I haven't looked at the pxe/neutron stuff much22:57
devanandaright. but if there's a clear path to (a) and I can see the implementation of (b) is not goign to hinder that, i'm more likely to go ahead without it initially22:57
devanandamy concern was in reviewing the code and seeing that (a) isn't even close to possible22:57
jrollbut I would think factor that stuff out into somewhere common, make config options whether to use that or rely on external, and go22:57
jrollwhy do you say it's not close to possible?22:58
jrollwhat's blocking us from doing (a)22:58
devanandasorry22:58
devanandas/possible/planned or in your awareness/22:58
jrollok, much better :P22:58
jrollit was definitely not something we had our eyes on22:59
devanandayea, sometimes i type faster than i think :(22:59
devananda(heh)22:59
jrollmy main motivation for doing such a thing would be for testing/devstack, because I would never recommend that configuration in production22:59
jrolland I completely see why it's important in that regard22:59
devanandathat's an interesting statement to me22:59
JayFdnsmasq says directly that it's only suitable for small networks23:00
JayFI would not use it in a production sized ironic deployment23:00
devanandabecause it suggests that you think that is a fundamental flaw in the current pxe driver23:00
JayFI would say that23:00
devanandabrb23:01
JayFI've had to replace dnsmasq in production before because it was inadequate, and more specifically, the way it's implemented now, the dnsmasq process that is spun up assumes all nodes are getting IPs in the same network23:02
jrollI would say that *may* be a fundamental flaw in the pxe driver23:02
JayFI'd almost say it seems like we shouldn't support nodes using different drivers inside the same dhcp domain23:02
JayFbut even that doesn't appear to be supported by the dnsmasq spun up by neutron via libvirt23:04
JayFsince it's started with the bind-interfaces flag on23:04
devanandaJayF: that's an interesting idea23:08
devanandaJayF: but I dont see how ironic can enforce that, since it doesn't control the IPs being handed out23:08
JayFIronic, /does/ control, via calls to neutron, what network the nodes are in though, correct?23:10
devanandaJayF: nope. Ironic doesn't determine what IP is put on the node23:10
JayFI'm not talking about IP granularity, I'm talking about network/subnet granularity23:11
JayFDHCP domains only for purposes of this conversation23:11
devanandanope23:11
devanandaironic is only passing one specific DHCP option to neutron23:11
jrollbut it *could* determine these things, no?23:12
devanandaer, 323:12
devanandabootfile-name tftp-server server-ip-address23:12
jrollwhat is server-ip-address?23:12
JayFserver-ip-address?23:12
devanandato control the response that neutron('s dhcp provider) returns to a DHCP BOOT request from that port23:12
JayFyou mean server-mac-address, perhaps?23:13
devanandaCONF.pxe.tftp_server23:13
devanandathe IP address of the TFTP server, ie, ironic-conductor23:13
jrolland then what is tftp-server?23:13
JayFtftp-server is ^ that, correct?23:14
devanandasame thing23:14
devanandadont ask me why neutron has two options that require the same value23:14
jrollum23:14
devanandayea23:14
jrollok23:14
jrollhuh.23:14
jrollyay neutron.23:14
devanandaso. "could ironic determine the subnet that a given node is placed in" -- i believe the answer today is no23:15
devanandabut let me get ar eference23:15
JayFI think that's possibly not true, or if it's true, it's because neutron isn't exposing all of libvirt's API for this23:15
devanandahttp://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/ironic#n33523:16
devanandaat least right now23:16
devanandawhen creating nodes for use with ironic, you have to manuyally create the neutron PORT too23:16
JayFafaict all the config is done via etc/libvirt/qemu/networks/default.xml, and I can't see where that's setting next-server or tftp-server23:16
devanandaJayF: what i mean is, neutron determines that today23:17
devanandanot ironic23:17
devanandaJayF: and to be more precise, it's Nova that creats a neutron port on demand, within the tenant's network range, using the MAC address from the ironic node23:18
JayFand in this case, 'tenant' network is what is used for the provisioning as well?23:18
JayFI guess in my mind, re: deployment, you wouldn't put a node in a 'tenant' network until after it was fully prepared23:19
devanandaJayF: I agree, that's how it should work. i don't believe that's the case today (but IMBW)23:19
devanandahttps://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L36523:20
*** vkozhukalov has quit IRC23:20
devanandahttps://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L55323:20
devanandaso nova has already created neutron ports prior to calling driver.spawn23:21
devanandathe nova.virt.ironic driver is passing those vif_id's down to ironic, so ironic can update them when it needs to change the DHCP options23:21
JayFI guess I need to dig further into neutron23:21
JayFbecuase I can't find any evidence on disk, re: dnsmasq configs, that the options are making it through23:22
devanandahttps://github.com/openstack/nova/blob/master/nova/compute/manager.py#L437423:24
devanandalastly, https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L19123:25
devanandanote the "macs" param -- that bubbles up from ironic's port.address field23:26
adam_gdevananda, is pxe booting from neutron managed dhcp the method thats used for pxe_ipmi /w real hardware?23:32
*** max_lobur has joined #openstack-ironic23:33
devanandaadam_g: yes. and afaik, that's what tripleo is using23:33
adam_ghmm. curious how that works23:34
adam_gnetbooting from a dhcp server running in a namespaced tenant network23:34
JayFadam_g: I'm still heading down the rabbithole, looks like neutron has dhcp-agents, but they aren't well documented. Trying to figure out how that works, given it's spawning dnsmasq instances with bind-interfaces on23:35
JayFI think it's possible at all that the devstack isn't using dhcp-agents, but instead has a dnsmasq spawned by something else that's loading the stuff up23:36
adam_gJayF, it should be writing out a config file and HUP'ing dnsmasq everytime a new networking gets configured23:36
devanandaJayF: quick google search did not yield me results, but i would be surprised if dnsmasq is the /only/ supported dhcp provider for neutron23:36
devanandathen again, neutron often surprises me23:36
JayFadam_g: I don't see /any/ dnsmasq config other than dnsmasq.d/libvirt-bin23:36
JayFdevananda: just what I'm reading in the docs vs what's on my devstack instance don't seem to agree, and I'm trying to figure out what is doingw hat23:37
adam_gJayF, have you booted a server?23:37
JayF1b700153-53d4-4df8-9e21-cdd9181ebcd4 | testing | BUILD  | spawning   | NOSTATE23:37
JayFand it's been in that state for a long time23:37
JayFfollowing the instructions on the ironic developer page23:37
JoshNangsame here23:37
adam_gJayF, is the corresponding ironic node powered on and wait-callback?23:37
JayFadam_g: I seem to be getting 403s from ironic using . devstack/openrc as my creds23:38
devanandaJayF: export OS_USERNAME=admin23:38
adam_gJayF, you need to be admin, do '. ~/devstack/openrc admin admin' instead23:38
devanandaor that23:38
JayFadam_g: 532aafee-a04c-4f3e-83a4-5c3c9c4633b3 | 1b700153-53d4-4df8-9e21-cdd9181ebcd4 | power on    | wait call-back     | False23:39
adam_gJayF, let me re-stack and see if something hasn't broken in devstack with the neutron agents.23:39
adam_gtempest.scenario.test_baremetal_basic_ops.BaremetalBasicOptsPXESSH is failing again in our gate (it was passing yesterday)23:40
* adam_g really really really cant wait till we can actually rely on that devstack check23:40
devanandaugh23:40
adam_ginterestingly, the bug that is blocking it from all passing is related to devstack neutron agent configuration being broken23:41
adam_gso its possible it may be broken wrt dhcp as well23:41
adam_ghttps://review.openstack.org/#/c/89448/23:41
JayFand fwiw, only dhcp driver in mainline if I'm reading the code correctly appears to be dnsmasq, at least for linux23:42
* JayF digs more23:42
adam_gJayF, give me 5 and i can be of more help (devstack running)23:43
JayFokay I'll take 5.23:44
*** zdiN0bot has quit IRC23:47
adam_gdevananda, are there any other places in ironic where we'd benefit from implementing an event callback mechanism similar to novas?  re https://review.openstack.org/#/c/84361/23:47
devanandaadam_g: with ironic as the receiver or the initiator?23:47
devanandaadam_g: that patch suggests ironic would be the initiator. yes, there are others23:47
devanandaadam_g: when a node's status is changed, sending an event notice to nova right away, so that the compute stats can be updated for the scheduler23:47
adam_gdevananda, with ironic as the receiver23:48
adam_gdevananda, in the neutron case, we'd get a callback confirming all ports have been plugged, dnsmasq is HUPd and ready for netboot23:48
devanandaadam_g: ooh. that too.23:51
adam_gthe only other place i can think of is the ramdisk calling back, which is handled thru vendor_passthru atm23:52
adam_gJayF, ugh. might be more than 5, forgot to point DIB it at my local caches23:53
devanandaanteaya: hi! do you mind if I ask you a neutron question (I dont want to bring up too many bad memories...)23:55
openstackgerritA change was merged to openstack/python-ironicclient: node-get-console incorporate the changes in API  https://review.openstack.org/8776923:56
*** zdiN0bot has joined #openstack-ironic23:57
devanandaadam_g: is the event callback REST or RPC based?23:57
adam_gdevananda, REST. for the nova case, supported was added to novaclient and is used by neutron23:58
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Accept new parameters for `prepare_image`  https://review.openstack.org/8672323:58
JayFadam_g: heh, I thought 5m seemed optimistic given how long it took me :)23:59
devanandaadam_g: that could be useful23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!