Tuesday, 2018-02-06

*** bfournie has joined #openstack-ironic00:00
*** mtreinish has quit IRC00:12
*** ijw has quit IRC00:14
*** Goneri has quit IRC00:23
*** yangyapeng has joined #openstack-ironic00:25
*** r-daneel has quit IRC00:29
*** yangyapeng has quit IRC00:30
*** mtreinish has joined #openstack-ironic00:37
*** ijw has joined #openstack-ironic00:38
*** rbudden has quit IRC00:39
*** rloo has quit IRC00:46
*** edleafe has quit IRC00:48
*** edleafe has joined #openstack-ironic00:51
*** edleafe has quit IRC00:55
*** ccollett has joined #openstack-ironic00:59
*** charcol-v2 has quit IRC01:02
*** edleafe has joined #openstack-ironic01:11
*** zshi has joined #openstack-ironic01:17
*** yangyapeng has joined #openstack-ironic01:22
openstackgerritMerged openstack/ironic-python-agent master: Replace use of functools.wraps() with six.wraps()  https://review.openstack.org/54017501:27
*** mjturek has quit IRC01:38
*** etingof has quit IRC01:39
*** ijw has quit IRC01:42
TheJuliaregarding grenade, I've totally lost my mind digging into it tonight. it _looks_ like two different network issues with similar results. the commonality between the node and the qrouter is missing, FIP inbound no reply. one shows no dhcp logs, another two show that the nodes booted with dhcp before the failure... and observed on the subnode console/compute/conductor processes01:48
TheJuliadhcp being on the primary node, meaning traffic is crossing over the vlan between the two01:48
*** zhangfei has joined #openstack-ironic01:51
*** etingof has joined #openstack-ironic01:52
*** zhangfei has quit IRC01:53
*** zhangfei has joined #openstack-ironic01:53
*** sapd__ has quit IRC02:04
*** zhangfei has quit IRC02:04
*** sapd has joined #openstack-ironic02:06
*** zhangfei has joined #openstack-ironic02:18
*** harlowja has quit IRC02:18
*** tuanla____ has joined #openstack-ironic02:53
*** d0ugal_ has joined #openstack-ironic03:16
*** d0ugal has quit IRC03:19
*** gcb has joined #openstack-ironic03:26
*** crushil has joined #openstack-ironic03:48
*** hshiina has joined #openstack-ironic03:56
*** dsneddon has quit IRC03:59
*** dsneddon has joined #openstack-ironic04:03
*** dsneddon has quit IRC04:11
*** devananda has quit IRC04:22
*** vsaienk0 has joined #openstack-ironic04:25
*** dsneddon has joined #openstack-ironic04:29
*** rbudden has joined #openstack-ironic04:30
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Fix for proper deployment when heystone enabled and minimal OS  https://review.openstack.org/54071304:31
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Add creation of openrc file when keystone enabled  https://review.openstack.org/54009604:32
*** rbudden has quit IRC04:34
*** vsaienk0 has quit IRC04:35
*** dsneddon has quit IRC04:37
*** dsneddon has joined #openstack-ironic04:38
*** crushil has quit IRC04:38
*** crushil has joined #openstack-ironic04:42
*** dsneddon has quit IRC04:43
*** harlowja has joined #openstack-ironic04:46
openstackgerritJohn L. Villalovos proposed openstack/ironic master: cleanup: Remove usage of some_dict.keys()  https://review.openstack.org/54112304:46
*** zhangfei has quit IRC04:46
*** dsneddon has joined #openstack-ironic04:59
*** zhangfei has joined #openstack-ironic05:03
*** dsneddon has quit IRC05:08
*** dpawar has joined #openstack-ironic05:10
*** harlowja has quit IRC05:11
*** links has joined #openstack-ironic05:12
*** tuanla____ has quit IRC05:23
*** sdake_ is now known as sdake05:30
*** marios has joined #openstack-ironic05:45
*** dsneddon has joined #openstack-ironic05:53
*** zhangfei has quit IRC05:54
*** hw_wutianwei has quit IRC05:59
*** hw_wutianwei has joined #openstack-ironic05:59
*** gcb has quit IRC06:00
*** e0ne has joined #openstack-ironic06:04
*** zhangfei has joined #openstack-ironic06:07
*** charcol-v2 has joined #openstack-ironic06:08
openstackgerritKaifeng Wang proposed openstack/ironic-inspector master: Introducing an api_workers option to inspector  https://review.openstack.org/54072206:09
*** ccollett has quit IRC06:10
*** d0ugal_ has quit IRC06:14
openstackgerritOpenStack Proposal Bot proposed openstack/ironic-inspector master: Imported Translations from Zanata  https://review.openstack.org/54113806:19
*** d0ugal_ has joined #openstack-ironic06:23
*** ccollett has joined #openstack-ironic06:24
*** charcol-v2 has quit IRC06:26
openstackgerritOpenStack Proposal Bot proposed openstack/ironic-ui master: Imported Translations from Zanata  https://review.openstack.org/54114306:36
*** crushil has quit IRC06:38
*** threestrands has quit IRC06:40
openstackgerritwudong proposed openstack/ironic master: Modify error quotation marks  https://review.openstack.org/54114606:41
*** yolanda has joined #openstack-ironic06:43
*** ianychoi_ has quit IRC06:47
*** ianychoi_ has joined #openstack-ironic06:48
*** ccollett has quit IRC06:57
*** rwsu has joined #openstack-ironic07:01
*** olivierbourdon38 has quit IRC07:06
*** vsaienk0 has joined #openstack-ironic07:09
*** gcb has joined #openstack-ironic07:13
*** dsneddon has quit IRC07:17
*** crushil has joined #openstack-ironic07:19
*** dsneddon has joined #openstack-ironic07:20
*** mjura has joined #openstack-ironic07:22
*** e0ne has quit IRC07:22
openstackgerritwudong proposed openstack/ironic master: Modify error quotation marks  https://review.openstack.org/54114607:25
*** AlexeyAbashkin has joined #openstack-ironic07:32
*** rcernin has quit IRC07:37
*** AlexeyAbashkin has quit IRC07:39
*** rbartal has joined #openstack-ironic07:41
*** olivierbourdon38 has joined #openstack-ironic07:42
*** AlexeyAbashkin has joined #openstack-ironic07:48
*** vsaienk0 has quit IRC07:48
*** vsaienk0 has joined #openstack-ironic07:49
openstackgerritHironori Shiina proposed openstack/ironic master: Update iRMC document for classic driver deprecation  https://review.openstack.org/54117108:02
*** v12aml has quit IRC08:04
*** v12aml has joined #openstack-ironic08:05
*** pcaruana has joined #openstack-ironic08:14
*** tesseract has joined #openstack-ironic08:22
*** dsneddon has quit IRC08:23
*** dsneddon has joined #openstack-ironic08:27
*** ianychoi_ has quit IRC08:27
*** ianychoi_ has joined #openstack-ironic08:28
*** amarao has joined #openstack-ironic08:34
*** dsneddon has quit IRC08:34
*** dsneddon has joined #openstack-ironic08:36
*** frti has joined #openstack-ironic08:48
*** crushil has quit IRC08:49
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: Make the agent distributed using hashring and notifications  https://review.openstack.org/52470908:53
*** mgoddard_ has joined #openstack-ironic08:54
*** priteau has joined #openstack-ironic08:56
*** hshiina is now known as hshiina|afk08:56
*** d0ugal_ has quit IRC08:57
*** d0ugal has joined #openstack-ironic08:57
*** d0ugal has quit IRC08:57
*** d0ugal has joined #openstack-ironic08:57
*** dsneddon has quit IRC08:59
Tenguhello! would a systemd unit file and starting script be of any use for virtualBMC project? Just had to create both today. Would gladly contrib them.09:00
*** hshiina|afk has quit IRC09:03
openstackgerritJames E. Blair proposed openstack/ironic master: Zuul: Remove project name  https://review.openstack.org/54119009:04
*** e0ne has joined #openstack-ironic09:05
*** threestrands has joined #openstack-ironic09:06
openstackgerritMadhuri Kumari proposed openstack/ironic-specs master: Add service management API for ironic  https://review.openstack.org/47121709:07
*** dsneddon has joined #openstack-ironic09:09
sapdHi everyone. I have a supermicro service with Intel Raid support. I don't know how to launch an baremetal instance on this server with raid support. In the guide, I think I have to build agent ramdisk which support raid. But I can't find what driver?09:12
openstackgerritShangXiao proposed openstack/ironic master: [api-ref] Add fault names for the error response codes  https://review.openstack.org/53871709:13
*** dsneddon has quit IRC09:14
*** dsneddon has joined #openstack-ironic09:15
*** dsneddon has quit IRC09:21
*** threestrands has quit IRC09:21
*** frti_ has joined #openstack-ironic09:21
*** frti_ has quit IRC09:25
*** frti has quit IRC09:25
*** dsneddon has joined #openstack-ironic09:27
*** hoonetorg has quit IRC09:28
*** dsneddon has quit IRC09:35
openstackgerritMerged openstack/ironic-inspector master: Imported Translations from Zanata  https://review.openstack.org/54113809:39
*** hoonetorg has joined #openstack-ironic09:41
openstackgerritMaoyangLiu proposed openstack/ironic master: Replace chinese quotes to English quotes  https://review.openstack.org/54120609:43
*** zhangfei has quit IRC09:44
*** derekh has joined #openstack-ironic09:45
*** zhangfei has joined #openstack-ironic09:56
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: Add unit tests for member manager  https://review.openstack.org/54121609:57
*** azaid_ has joined #openstack-ironic10:00
azaid_Hi , I have a problem in ironic which hypvisor have no cpu and no ram10:00
azaid_so when I create instance it return no valid hosts10:01
azaid_any idea ?10:01
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: Add unit tests for member manager  https://review.openstack.org/54121610:01
*** jaianshu has joined #openstack-ironic10:14
*** zshi has quit IRC10:15
*** nmathew has joined #openstack-ironic10:19
*** dtantsur|afk is now known as dtantsur10:25
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Add creation of openrc file when keystone enabled  https://review.openstack.org/54009610:25
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Add creation of openrc file when keystone enabled  https://review.openstack.org/54009610:27
*** zshi has joined #openstack-ironic10:27
dtantsurhi ironic10:28
-openstackstatus- NOTICE: Our Zuul infrastructure is currently experiencing some problems and processing jobs very slowly, we're investigating. Please do not approve or recheck changes for now.10:30
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Fix for proper deployment when keystone enabled and minimal OS  https://review.openstack.org/54071310:31
*** azaid_ has quit IRC10:38
openstackgerritMark Goddard proposed openstack/networking-generic-switch master: Add initial release note  https://review.openstack.org/53420110:43
*** lucas-afk is now known as lucasagomes10:52
*** sambetts|afk is now known as sambetts10:54
sambettsMorning all10:54
sambettsTengu: I think that would be useful thanks!10:55
sambettsdtantsur: Morning o/10:55
dtantsuro/11:01
dtantsurTengu: hi! sync with etingof, he was working on making vbmc a normal service11:01
etingofo/11:02
etingofTengu, here is my patch being reviewed and re-worked -- https://review.openstack.org/#/c/488874/11:03
patchbotpatch 488874 - virtualbmc - multiprocess server, socket-based management cli tool11:03
*** zhangfei has quit IRC11:22
*** skazi has quit IRC11:29
sambettsjohnthetubaguy: Hey, are you around? I have some questions relating to nova to neutron interaction11:32
*** nmathew has quit IRC11:38
*** zhangfei has joined #openstack-ironic11:38
*** links has quit IRC11:41
*** skazi has joined #openstack-ironic11:42
openstackgerritSam Betts proposed openstack/ironic stable/ocata: Use zuul.override_checkout  instead of custom branch_override var  https://review.openstack.org/54083011:45
Tengusambetts: dtantsur etingof oh, if there's alreardy an existing patch, I'll check it and see if I can contrib on it, if you're ok?11:46
Tenguetingof: indeed your patch is (far) more advanced than the quick'n'dirty thing I've made :).11:47
etingofTengu, I welcome your contributions indeed! ;)11:47
Tenguetingof: I'll check that out shortly (I hope) :).11:48
etingofTengu, if you follow the review, a couple of significant changes were suggested ther11:48
etingofTengu, so you may consider working on those if you have time and guts ;)11:48
Tenguyup, saw that in hurry. I'll check deeper later :).11:49
etingofTengu, on the other hand, the patch I linked readily works with systemd so you may just use it as-is11:49
Tenguetingof: good to know - but if I can help to get it merged in some way, that will help a lot more of people than just me ;)11:50
Tenguwill check what dtantsur requests in the latest review. Not sure I can help on that, my knowledge of openstack code is fresh.11:51
etingofTengu, absolutely! for it to get merged we need to 1) address the suggestions in the comments and 2) lure core reviewers to actually review the patch ;-)11:52
Tenguah, will need to learn a bit about cliff - and that will probably help me for certodia :).11:53
Tenguetingof: :] second part is the best I think :)11:53
*** vsaienk0 has quit IRC11:53
Tengubut right now : some rest, then eating, and with a full stomac code will be easier.11:53
*** links has joined #openstack-ironic11:54
*** vsaienk0 has joined #openstack-ironic11:56
etingofsomeone is risking a nap afterward ;)12:00
openstackgerritliumengke proposed openstack/ironic master: Fixed some typos in test code.  https://review.openstack.org/54125112:03
*** fyxim has quit IRC12:11
*** fyxim has joined #openstack-ironic12:11
*** olivierbourdon38 has quit IRC12:11
Tenguetingof: :]12:14
*** absubram has joined #openstack-ironic12:25
*** bfournie has quit IRC12:28
*** bfournie has joined #openstack-ironic12:28
mgoddard_sambetts: hey, I think johnthetubaguy is out today12:33
*** bfournie has quit IRC12:33
sambettsmgoddard_: ah no problem, need his in depth knowledge of nova's internals to answer a question I had12:35
mgoddard_sambetts: what is the question?12:37
sambettsmgoddard_: I wanted to know if and how (if it does) nova resets the mac address on the neutron ports on instance delete if nova has overwriten then during deployment (in cases like SRIOV etc the hardware mac address gets set overriding the neutron generated one)12:39
sambettsbecause if it doesn't then you can end up with ports stuck with the hardware mac address until they are deleted12:41
mgoddard_sambetts: good question. IIRC, being able to unset the MAC of a port is a relatively recent thing. Possibly predating SR-IOV support? So maybe they just don't12:45
mgoddard_If it's done, it will be in nova/network/neutronv2/api.py. Nothing's leaping out at me. I can see where the MAC is set for SR-IOV12:46
sambettsmgoddard_: yeah... I'm trying to bring a fix to Ironic relating to the same thing, basically we overwrite the mac address on the port, but then we leave the port set that way which if the port is deleted breaks deployment the second time around due to "mac already in use" errors12:47
sambettsI was going to set the mac_addres to None when we unbind the port, and I expected neutron would regenerate the mac address, however it seems like you can't unset the mac address like that, so I'm left with having to store the old one12:48
sambettsand restore it12:48
sambettsbut then that concerns me because what if neutron generates a mac matching the mac we've got stored for a different port12:48
sambettsthen we'll end up in a real mess12:49
mgoddard_maybe just generate another one?12:49
sambettsyeah thats the other option... but I was hoping to avoid it12:50
*** jaianshu has quit IRC12:50
sambettswhich is why I wondered if nova had already solved this issue somehow12:50
*** yangyapeng has quit IRC12:52
*** yangyapeng has joined #openstack-ironic12:53
*** yangyapeng has quit IRC12:53
*** yangyapeng has joined #openstack-ironic12:54
*** links has quit IRC12:55
peitolmare the generated MAC addresses based on the underlying hardware's original address (ie assigned by the manufacturer?12:55
*** jlabarre has joined #openstack-ironic12:55
*** takamatsu has joined #openstack-ironic12:56
sambettsin neutron the macs are generated randomly but using a configurable prefix I believe12:58
mgoddard_tricky one. I guess if we can't prevent neutron from reallocating the MAC, we're always going to need a generate code path. In that case, perhaps we should always generate?12:58
sambettsalways generate a new one on port unbind?12:59
mgoddard_could use a similar scheme, allowing the ironic & neutron prefixes to be different?12:59
mgoddard_yes12:59
*** AlexeyAbashkin has quit IRC12:59
sambettsyeah... slightly concerned that that port doesn't have a neutron generated mac any more though :/ I think at the PTG I might asked the neutron folks if they think an ability to set mac to None to ask it to regenerate makes sense as a feature13:00
peitolmif the instances are going on bare-metal, why is the MAC being changed?13:00
*** zhurong_ has joined #openstack-ironic13:01
*** AlexeyAbashkin has joined #openstack-ironic13:01
sambettspeitolm: neutron generates a mac address for a port when it is created, when we deploy onto baremetal we can't change the mac address of the physical baremetal NIC, so we update neutron to match to NIC13:01
sambettspeitolm: when the instance is deleted, the port might not be deleted, so we need to undo the mac address update13:02
*** yangyapeng has quit IRC13:02
sambettsso that it can be used for another instance and won't conflict if someone else redeploys the same baremetal server13:02
peitolmok. so if there is a prefix used as you suggest, (which would/should be different from any hardware vendor), then storing and returning it back to the stored one won't clash13:04
peitolmthe stored one being the original hardware one i'm presuming13:04
sambettsI was suggesting storing the one that neutron generated, so that we can restore it, but if neutron has generated it, it can easily generate it again13:05
peitolmah,13:05
peitolmso nova is going to override a generated one that neutron has set to override the hardware one?13:06
sambettsfor nova VMs using SRIOV, nova overrides the generated one with the hardware one, and in Ironic we override the generated one with the baremetal server's one13:07
*** links has joined #openstack-ironic13:08
peitolm*nods* does the mac already in use error come from neutron?13:09
sambettsyup, when you try either create a port or update a port with a mac that is already in a network13:09
*** jaganathan has quit IRC13:10
*** bfournie has joined #openstack-ironic13:10
peitolmmaybe neutron should be handling this centrally, ie, checking if the mac being set is one that's is already set on that host, (ie is just a null operation)13:11
peitolmrather than, is this mac set anywhere,13:11
peitolmby anywhere, the code suggests it's checking just the network,13:15
peitolmi'm probably missing something though13:16
sambettsall that logic is correct on the neutron side, as far as I know, its just that we have a set a port to have a mac address that can only ever match to single baremetal host, and when we're done with it we need to undo that so that the port not longer has any association to the baremetal it was attached too13:18
*** mrunge_ has joined #openstack-ironic13:22
*** jaganathan has joined #openstack-ironic13:23
*** mrunge has quit IRC13:24
peitolmyou have a configuration where the port has a mac addres, that port is only ever matching a single host, which is valid, correct?13:25
*** AlexeyAbashkin has quit IRC13:25
*** AlexeyAbashkin has joined #openstack-ironic13:27
openstackgerritBob Fournier proposed openstack/ironic-inspector stable/ocata: Remove use of construct lib FieldError exception  https://review.openstack.org/54127113:29
*** yangyapeng has joined #openstack-ironic13:30
openstackgerritBob Fournier proposed openstack/ironic-inspector stable/ocata: Remove use of construct lib FieldError exception  https://review.openstack.org/54127113:33
openstackgerritRuby Loo proposed openstack/ironic master: Fix handling of 'timeout' parameter to power methods  https://review.openstack.org/54015013:33
*** rloo has joined #openstack-ironic13:34
*** yangyapeng has quit IRC13:38
*** AlexeyAbashkin has quit IRC13:38
sambettspeitolm: a user created a port in neutron without specifying a mac address, so neutron generates a mac address for them. then they go to nova and boot an baremetal instance passing in the uuid of the port they created. the nova scheduler selects a baremetal server to deploy their instance on to, and the information including that port uuid gets passed down through nova to ironic. ironic then13:38
sambettsupdates that port with the hardware mac address from the server's NIC so that the information in neutron matches the real server's mac address, so that neutron's DHCP server etc is program with the right mac address. The user uses his instance to do some stuff. The user then deletes his instance via nova. Ironic then powers off the server, and cleans up the user's data. the instances is now13:38
sambettsremoved from nova. However the port in neutron still exists, and the mac address is now set to the mac address of the baremetal server. So now another user does the same set of steps, and ends up on the same baremetal server, not knowing it was used before. when ironic tries to update the port (a different port owned by a different user but assigned to the same baremetal) we get a mac address13:38
sambettsalready in use error, because of the port left behind by the first user13:38
peitolmI see13:39
peitolmthank you, sorry you had to type all of that13:39
sambettsno problem, its easier to explain with the whole picture13:39
peitolmi was missing the first bit, which is probably key13:40
peitolmi'm guessing the instance, may or may not run on ironic13:40
peitolmi wonder if neutron should do something like, "I've been informed that the mac address for this instance has been overriden by the provider, therefore i'll note that fact, along with the original address, so that when the instance goes, i can reset it"13:41
peitolmbut I guess you're trying to avoid modifying neutron?13:42
sambettsyeah right now trying to fix that bug in ironic, without touching nova or neutron, there are two solutions I see if we make changes on the neutron side that could work, either like you suggest neutron keeps track and resets it, or neutron lets a user set the mac address to none which means generate me a new one please13:44
sambettsbut unfortunatly I don't think a change like that will merge soon, so trying to work out how to resolve the bug without those13:44
*** yangyapeng has joined #openstack-ironic13:44
sambettsand so either we can track the old mac in ironic, and restore it, but because neutron then doesn't think the mac is assigned to a port, it might regenerate the same one and assign it to antoher port resulting the in the same conflict13:45
sambettsor we generate a random mac address and change it to that13:46
peitolmi thnk the first one is going to be a problem, depends on the login for generating a mac address13:46
peitolmis the generation of a mac address a function you can call from ironic?13:47
sambettswe'd have to generate it in ironic's code, and then call neutron.port_update to set it13:47
peitolmor you could create a temporary port, grab the mac from that, delete it anad assign that mac to the other port13:48
peitolmbut that's still got a race condition13:48
sambettsyeah, another option I've thought of is deleting the original port, and then reassigning everything including the uuid, to a new one13:49
peitolmfraglie though13:49
*** yangyapeng has quit IRC13:50
sambettsyeah :/ and I'm not sure all the fields are writable13:50
peitolmhave you asked teh neutron team about it?13:50
sambettsnot yet, I was waiting for more of them to wake up13:50
peitolmstoring it and restoreing it sounds like a workaround that'll be forgotten about13:51
sambettsyeah, and I think thats too fragile too13:51
*** zhurong_ has quit IRC13:52
sambettsI think the neatest solution would be to generate new ones until we get one that doesn't conflict with an existing one13:52
sambettswhich should be only one or two13:52
peitolmwell, the neatest solution from your side, would be to not have to deal with it :P13:52
*** jcoufal has joined #openstack-ironic13:52
peitolmbut the neatest solution overwall is to get neutron to handle it13:52
sambettsyeah, I'm definatly going to talk to the neutron team and work out a better solution13:53
*** rbudden has joined #openstack-ironic13:53
peitolmI wonder if a more generic option may be to have a flag for static vs dynamic mac13:53
sambettseven if its just the ability to set mac_addres:"auto" in neutron13:53
peitolmie, if the user sets the mac, then keep it, and if they move the instance to another host, that's their issu13:53
peitolmif it's dynamic, then at deploy time, it's re-generated13:53
peitolmauto is better than none13:54
sambettsyeah, thats similar to whats happening now with defered ip addresses in neutron13:54
peitolmwhat happens if you set it back to blank? is that possible?13:54
*** Sukhdev_ has joined #openstack-ironic13:55
sambettsthat what I mean by setting it to None (null in json), but neutron rejects that request today13:55
sambettsI'd like it if setting it to blank/none/null resulted in a regen13:55
peitolmah, i did wonder if you meant a null, or a literal none13:55
peitolmwonder what would happen if you set it to an invalid mac13:56
sambettsthey have pretty good validation, and it rejects it with a invalid mac address error13:56
*** VisBits has joined #openstack-ironic14:06
TheJuliagood morning everyone14:06
dtantsurmorning TheJulia14:07
peitolmmorning TheJulia14:07
sambettso/ TheJulia14:10
TheJuliaAnyone had any luck with grenade?14:10
*** jrist has quit IRC14:11
*** yangyapeng has joined #openstack-ironic14:11
peitolmsambetts: i found references to net.get_random_mac in the neutron code, but can' seem to pin down its source,  looks like there was a discussion about it back in july 201614:12
*** jrist has joined #openstack-ironic14:12
dtantsurTheJulia: not me. got buried into one "funny" bug14:12
TheJuliais it actually funny? :)14:13
peitolmsambetts: ah, it's in neutron-lib14:13
dtantsurwell.. https://launchpad.net/bugs/164399514:13
openstackLaunchpad bug 1643995 in Ironic "ironic can't find nodes whose name ends in ".<mimetype>"" [High,Triaged] - Assigned to Dmitry Tantsur (divius)14:13
TheJuliaI have a theory, I'm going to get some coffee, and issue a revert on neutron and test patch on our gate14:14
TheJuliadtantsur: ugh14:14
peitolmwow, that's a good definition of unintended consequences14:15
*** Goneri has joined #openstack-ironic14:16
TheJuliapeitolm: which statement?14:17
*** jaganathan has quit IRC14:17
peitolmTheJulia: the lanchpad bug14:18
dtantsuryep14:18
sambettswhat even is a troff-me content type... /me googles14:19
dtantsur.1 is the funniest14:20
peitolmdtantsur: as a random test, trying test.me fails, but test.me.json works14:20
dtantsurpeople really tried that14:20
dtantsuryeah14:20
*** cdearborn has joined #openstack-ironic14:20
peitolmdtantsur: who's the yeah at? me or sambetts14:21
dtantsurat both :)14:21
peitolmoh, did they report that in the redhat bug?14:22
peitolmi'm surprised it actually stripping off the supposed mime type though14:23
* TheJulia tries to wake up14:23
dtantsurpeitolm: this is by design. a feature of the pecan library we use14:24
peitolmso "openstack baremetal node create --driver ipmi --name test.json" followed by "openstack baremetal node show test.json" fails, but if you do, "openstack baremetal node show test.json.json"  it works14:24
*** mjturek has joined #openstack-ironic14:26
*** cdearborn_ has joined #openstack-ironic14:26
dtantsurright, because it only strips the prefix once14:26
*** cdearborn_ has quit IRC14:26
peitolmI wonder if you could sanitity check at node create/update time14:27
peitolmie if the name matches a mimetype, issue a warning14:27
dtantsurwell, I don't want to prevent users from creating such nodes, I'd rather make them work14:27
dtantsurusers don't see warnings, and we don't have any kind of a warning API..14:28
peitolmtrue14:29
*** VisBits has quit IRC14:29
*** jaganathan has joined #openstack-ironic14:31
*** dpawar has quit IRC14:33
*** lucasagomes is now known as lucas-hungry14:34
*** AlexeyAbashkin has joined #openstack-ironic14:35
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Stop guessing mime types based on URLs  https://review.openstack.org/54128914:36
dtantsurpeitolm, TheJulia, sambetts, what do you think about this approach? ^^^14:36
*** dtantsur is now known as dtantsur|bbl14:37
*** links has quit IRC14:38
*** mjturek has quit IRC14:39
*** zhangfei has quit IRC14:39
sambettsdtantsur|bbl: seems sane enough to me14:39
*** cdearborn has quit IRC14:41
TheJuliasane, I'm not really a fan of the json fallback stuff, but I'm not entirely sure it si there... I'm also not fully caffinated yet14:42
peitolmthe tests, test_get_one_with_json_in_name , that's with .json at the end right? rather than something.json.foo14:43
peitolmand test_get_one_with_double_json, shouldn't that be name=node.json.json14:43
peitolmignore that last one, i see what you're testing14:44
*** zhangfei has joined #openstack-ironic14:53
*** davidlenwell has quit IRC14:55
rloogood morning everyone, peitolm, TheJulia, sambetts, dtantsur|bbl14:56
*** davidlenwell has joined #openstack-ironic14:56
rlooTheJulia: I haven't started to look at the grenade issue but it sounds like you've got a handle on it? if it is network related, I'm somewhat useless.14:56
TheJuliarloo: more eyes the better....14:57
rlooTheJulia: ok, i'll look today. just going through priorities now to see if there is anything else that needs attention14:57
rloosambetts: wrt ironic-tempest-plugin failure. is there more work to do, cuz this fails: https://review.openstack.org/#/c/540355/14:59
patchbotpatch 540355 - ironic-tempest-plugin - Switch to override-checkout for stable branch jobs14:59
sambettsrloo: its currently rechecking because of something I missed on the ocata branch backport14:59
sambettsrloo: the latest run (which is still running) should passed on all branches15:00
rloosambetts: ah, ok. is the order wrt merging, master, pike, ocata starting with https://review.openstack.org/#/c/540432/ ?15:00
patchbotpatch 540432 - ironic - Use zuul.override_checkout  instead of custom bran...15:00
openstackgerritJulia Kreger proposed openstack/ironic master: DNM TEST Lets try reverting the suspicious neutron change  https://review.openstack.org/54129815:00
sambettsrloo: yup, we need those three, and then the tempest one15:01
rloosambetts: ok, i'll update the whiteboard with them so i know the order to review.15:01
sambettsrloo: thanks :)15:01
rloosambetts: but... this fails, and it is the first one: https://review.openstack.org/#/c/540432/15:02
patchbotpatch 540432 - ironic - Use zuul.override_checkout  instead of custom bran...15:02
sambettsrloo: from what I can tell the failure isn't related to the change I think its related to the thing TheJulia is investigating15:02
rloosambetts: ugh. i thought it was only multinode grenade that was failing.15:03
TheJuliasambetts: it is, I spotted it on two separate jobs where somehow the connection to the VIP never mapped through...15:03
TheJuliawell, the failure on grenade master to master was actually purely a timeout15:04
TheJuliathe multinode one is the same multinode issue15:04
*** zhangfei has quit IRC15:04
sambettsah so a recheck should fix the normal grenade one15:05
sambettsand multi is just sad15:05
TheJuliaindeed15:05
rlooand we are sad too. thx TheJulia.15:05
*** dsneddon has joined #openstack-ironic15:06
*** zshi has quit IRC15:07
TheJuliaI've not found any way to get additional debugging short of local recreation and interception or hacking on neutron code15:09
*** dsneddon has quit IRC15:09
*** dsneddon has joined #openstack-ironic15:09
*** dtantsur|bbl is now known as dtantsur15:14
dtantsurmorning rloo15:14
*** mgkwill has quit IRC15:14
rloodtantsur: :)15:14
*** mgkwill has joined #openstack-ironic15:15
sambettsTheJulia: hmmm, whats the theory around that patch breaking our grenade?15:15
TheJuliasilent behavior in ovs firewall since we do actually have real ports that it can see/interact with15:16
sambettsoh right... because of our hacks to plug in the bms15:17
TheJuliabingo15:17
*** rpioso|afk is now known as rpioso15:17
sambettsle sigh.. I knew this stuff would bite us on the ass15:17
rpiosoGood morning15:17
*** dsneddon has quit IRC15:18
sambettsTheJulia: hjensas has been working to restore my patches which make our BM connects to OVS less hacky if thats any help15:18
dtantsurmorning rpioso15:18
TheJulialooks like the firewalling defaults to iptables_hybrid.... I'm wondering if we could just..15:18
TheJuliaoh shoot, the failure is on the pike code base, not master15:18
TheJuliaYeah, and that neutron change is only on master15:19
rpiosoHey dtantsur :)15:20
*** zshi has joined #openstack-ironic15:20
TheJuliaso anyway, my revert attempt likely won't do anything15:21
*** trown is now known as trown|outtypewww15:23
*** dsneddon has joined #openstack-ironic15:26
*** lucas-hungry is now known as lucasagomes15:30
dtantsurfolks, do I get it right that only nodes and port groups have user-settable names?15:34
rloodtantsur: yeah, that sounds about right.15:38
rlooTheJulia: :-( do we need to fix on pike?15:38
TheJuliapossibly, I'm hoping reproduce.sh yeilds something I can look at while broken15:38
openstackgerritDmitry Tantsur proposed openstack/ironic master: Stop guessing mime types based on URLs  https://review.openstack.org/54128915:46
dtantsurokay, please consider this bug fix for queens ^^^15:46
dtantsurhuh, I think I need to hack also DELETE operation.. will look later15:48
*** dtantsur is now known as dtantsur|bbl15:49
NobodyCamGood Morning Ironic'ers15:50
*** pcaruana has quit IRC15:51
*** samueldmq has quit IRC15:52
rloodtantsur|bbl: please add link to whiteboard/priorities ^^15:52
rloomorning NobodyCam!15:52
*** samueldmq has joined #openstack-ironic15:52
NobodyCamGood Morning rloo :)15:52
openstackgerritDoug Hellmann proposed openstack/ironic-ui master: Update reno for stable/queens  https://review.openstack.org/54131916:00
*** r-daneel has joined #openstack-ironic16:01
*** Sukhdev_ has quit IRC16:04
*** mjura has quit IRC16:08
*** rloo_ has joined #openstack-ironic16:10
*** rloo has quit IRC16:13
jlvillalGood morning :)16:20
jlvillalThat Zuul check queue is very very long...16:20
*** retr0h has quit IRC16:20
*** retr0h has joined #openstack-ironic16:20
rpiosoGood morning NobodyCam and jlvillal16:21
jlvillal:)16:21
*** e0ne has quit IRC16:22
NobodyCamGood Morning rpioso :)16:25
*** amarao has quit IRC16:27
*** jaganathan has quit IRC16:30
*** rbartal has quit IRC16:40
*** priteau has quit IRC16:42
*** hamzy has quit IRC16:43
jrollgood morning everyone16:55
sambettshey jroll16:56
*** mjturek has joined #openstack-ironic16:57
*** SerenaFeng has joined #openstack-ironic17:02
TheJuliaso.. I think I've reproduced it17:06
TheJuliasambetts: https://paste.openstack.org/show/66364217:07
sambettsTheJulia: oh nice!17:08
sambettsTheJulia: hmmm ... I thought we fixed the issue with us removing the tap interfaces...17:08
sambettsthere was a issue at one point with devstack where libvirt would delete the tap interfaces for the vms17:09
sambettsbut I thought we fixed it, maybe we didn't back port it17:09
TheJuliaso, we have the tap interfaces, only one vm is presently running, was 217:11
TheJulianode-3 has a tag... but I don't see that hooked anywhere else (of course, it is rare for me to have to dig into this stuff)17:11
*** nicodemos has quit IRC17:11
sambettsTheJulia: looks like we didn't backport the fix for the issue with the underlying interfaces getting removed which caused us a mojor headache before17:12
sambettsTheJulia: this --> https://review.openstack.org/#/c/510465/17:12
patchbotpatch 510465 - ironic - [Devstack] Replace tap with veth (MERGED)17:12
*** ricardoas has quit IRC17:12
sambettsit probably needs to be backported17:12
sambettsI think that patch was related to multinode failures (non-grenade)17:13
sambettsso add grenade in running the old version without the fix and *drum rollll* same error17:14
* jroll can't connect to paste.o.o, wtf17:15
sambettsjroll: I coukdn't either until I removed the https://17:15
sambettsnot sure why17:15
jrollahhhhhhhhhhhhh17:15
jroll\o/ thanks17:15
openstackgerritSam Betts proposed openstack/ironic stable/pike: [Devstack] Replace tap with veth  https://review.openstack.org/54137817:16
TheJuliaahh, sam beat me to it17:16
sambetts:-P need a depends on for that ^ on your patch and we'll see if that resolves it17:16
TheJuliaoh... I had to hand type the url because one of my computers is acting wonky17:16
TheJuliayup17:17
sambettsahhh, no problem17:17
openstackgerritJulia Kreger proposed openstack/ironic master: DNM TEST Lets the backport Depends-On: I04ff6a97bebf15636a794b51f748c62eda627d36 Change-Id: I831189540b9730bf4cf4e5de7e0a406f83283475  https://review.openstack.org/54129817:18
TheJuliaso now we wait17:19
sambetts\o/ waiting17:19
*** rwsu has quit IRC17:19
sambettsI've been waiting all day for the results of my recheck on https://review.openstack.org/#/c/540355/17:20
patchbotpatch 540355 - ironic-tempest-plugin - Switch to override-checkout for stable branch jobs17:20
sambetts:(17:20
*** rwsu has joined #openstack-ironic17:20
TheJuliaI just noticed there were rechecks from YESTERDAY still running17:20
sambettsoh dear...17:20
*** SerenaFeng has quit IRC17:20
TheJuliayeah...17:20
*** marios has quit IRC17:21
*** r-daneel has quit IRC17:22
*** r-daneel has joined #openstack-ironic17:22
*** dpawar has joined #openstack-ironic17:23
*** dpawar has quit IRC17:24
*** AlexeyAbashkin has quit IRC17:25
*** AlexeyAbashkin has joined #openstack-ironic17:25
*** AlexeyAbashkin has quit IRC17:29
*** vsaienk0 has quit IRC17:33
*** jlabarre has quit IRC17:34
TheJuliaI was going to ask in infra, but it looks like they are still fighting issues17:35
TheJuliasambetts: you may want to re-review https://review.openstack.org/#/c/534441/17:36
patchbotpatch 534441 - ironic - Don't try to lock for vif detach17:36
openstackgerritSam Betts proposed openstack/ironic-tempest-plugin master: Add [baremetal] available_nodes configuration option  https://review.openstack.org/54138317:37
*** jlabarre has joined #openstack-ironic17:37
sambettsTheJulia: I'll add it to the list :)17:37
*** mtreinish has quit IRC17:43
peitolmi'm trying to double check my install, and my "ovs-vsctl show" doesn't match step 6. of https://docs.openstack.org/ironic/pike/install/configure-networking.html, it's missing the options sections, http://paste.openstack.org/show/663703/17:46
openstackgerritJulia Kreger proposed openstack/ironic master: DNM TEST Lets the backport  https://review.openstack.org/54129817:46
peitolmany chance someone could check the output of ovs-vsctl on their system, and see if you're seeing the same, I'm not sure if it's a mis-configuration at my end, or if it's a change due to the ovs_version17:47
sambettspeitolm: if you have the neutron openvswitch agent running it'll create the phy-br- and int-br devices for you17:47
*** dpawar has joined #openstack-ironic17:47
sambettsif its configured correctly17:48
peitolmok. i'll take a look at that then17:48
peitolmhmm, the br-int and br-eth2 are showing as down as well...17:49
peitolmhmm, " Agent out of sync with plugin!"17:50
*** mtreinish has joined #openstack-ironic17:50
TheJuliawith the current status of the gate, I suspect we are not going to be able to land much in the way of bug fixes17:50
*** rloo_ is now known as rloo17:56
*** crushil has joined #openstack-ironic17:58
*** crushil_ has joined #openstack-ironic17:58
* sambetts ---> home 17:58
peitolmhmmm18:00
*** tesseract has quit IRC18:00
*** crushil has quit IRC18:00
sambettsnight all o/18:01
*** sambetts is now known as sambetts|afk18:01
*** derekh has quit IRC18:02
*** absubram has quit IRC18:03
TheJuliagoodnight sam18:06
peitolm'night sam18:09
*** hamzy has joined #openstack-ironic18:13
*** devananda has joined #openstack-ironic18:13
*** hamzy_ has joined #openstack-ironic18:15
snasirHi can a baremetal instance be suspended and or paused while in "active" state via Nova ? Just like we can suspend or pause VM instances via Nova ? Thanks18:15
*** hamzy has quit IRC18:17
rloosnasir: sorry, it can't be suspended or paused18:20
rloobye sambetts|afk18:20
openstackgerritJulia Kreger proposed openstack/ironic master: Only set default network interface flat if enabled in config  https://review.openstack.org/53583018:21
*** dtantsur|bbl is now known as dtantsur18:22
snasirrloo: If that is the case, then how come we have those options available ? Does nova indeed try and suspend the instance, but then the operation is blocked by ironic-conductor ?18:24
snasirrloo: From horizon, I can see options such as "Migrate, suspend & Pause instances" even for the baremetal instance18:25
TheJuliasnasir: nova is built around those concepts being supported, they simply don't have a matching in the ironic virt driver in nova-compute18:25
rloosnasir: oh, interesting. i suspect from a high level they don't know whether a vm is baremetal or not18:25
TheJuliasnasir: also, horizon has 0 clue that it is baremetal under the hood18:25
rloosnasir: although i agree from the user's point of view, it isn't good.18:26
snasirTheJulia: Thanks for clarifying18:26
rloosnasir: the nova code raises an exception if you try it, not sure what trickles up to horizon18:27
TheJuliacan anyone load the zuul status page?18:29
jrollfwiw, this is what the "capabilities API" we keep talking about is meant to solve. "what actions can be performed on this resource?"18:29
* jroll tries18:30
TheJulia :)18:30
* jroll continues waiting18:30
jrollgonna guess no18:30
dtantsurjroll: do you plan to update https://review.openstack.org/#/c/540402/ or should I?18:31
patchbotpatch 540402 - ironic-inspector - Remove sample policy and config files18:31
dtantsurI'd like it before the release18:31
hjensasTheJulia: looks like the zuul status page took some time off ...18:31
jrolldtantsur: sorry, got distracted, I can do it quickly18:32
dtantsurthanks!18:32
TheJuliainfra seems to be looking into the zuul issue18:33
* TheJulia sighs18:33
*** harlowja has joined #openstack-ironic18:35
snasirTheJulia: In addition, is it possible to create a snapshot for the instance as well, or thats not supported for baremetal either ?18:36
openstackgerritJim Rollenhagen proposed openstack/ironic-inspector master: Remove sample policy and config files  https://review.openstack.org/54040218:36
*** olivierbourdon38 has joined #openstack-ironic18:36
jrolldtantsur: there you are, sorry for the delay18:36
TheJuliasnasir: not supported since the node is a physical server. If your doing something like boot from volume from a cinder volume, you can snap the volume directly18:36
dtantsurjroll: thanks! will wait for the CI now18:37
TheJuliasnasir: we've talked about implementing the ability to take an offline snapshot/image the machine, except such things are a little complicated to do18:37
jroll"a little"18:39
snasirTheJulia: Thanks18:39
TheJulias/a little/very very/18:40
jrollTheJulia: this looks ready to land: https://review.openstack.org/#/c/524709/18:54
patchbotpatch 524709 - networking-baremetal - Make the agent distributed using hashring and noti...18:54
jrollditto the unit tests on top18:57
TheJuliajroll: will look after I eat some lunch18:59
*** AlexeyAbashkin has joined #openstack-ironic19:00
*** lucasagomes is now known as lucas-afk19:02
jrollthanks, no rush19:04
*** r-daneel_ has joined #openstack-ironic19:08
*** r-daneel has quit IRC19:09
*** r-daneel_ is now known as r-daneel19:09
peitolmrloo: is it something that could be supported, if, forexample, you can talk to the BMC to trigger ACPI events?19:10
*** vsaienk0 has joined #openstack-ironic19:11
rloopeitolm: context? support what?19:12
TheJuliapeitolm: you mean like soft power?19:14
peitolmyeah19:14
peitolmi think there's an ACPI event that could trigger suspend or hibernate19:15
dtantsurhighly OS-dependent, I suspect19:17
peitolmmm,19:18
peitolmit's also optional in the IPMI spec19:18
rpiosoI have a silly Gerrit review question. A change I'm reviewing has a Files table, just like every change. For one of its entries, the Comments column contains, 'drafts: 1 new: 1'. What does the 'new: 1' mean? For that line, I made a comment and the owner replied. I've drafted a reply to his reply. What's strange is that another file in the table has two lines with the same review pattern -- comment, reply, and19:19
rpiosodraft. But it doesn't have that 'new' business.19:19
*** Goneri has quit IRC19:19
TheJuliadtantsur: fairly yes19:21
TheJuliapeitolm: we do have soft on/off support, I'm not sure if that can be sent via nova though19:21
*** vsaienk0 has quit IRC19:22
TheJuliarpioso: unread I think19:22
rpiosoTheJulia: I've viewed the file in every patch set, and clicked on each comment and reply.19:28
TheJuliacached data in the browser maybe?19:29
rpiosoTheJulia: Different brower that I rarely use, same result.19:32
openstackgerritDmitry Tantsur proposed openstack/ironic master: Stop guessing mime types based on URLs  https://review.openstack.org/54128919:37
dtantsurthis should be ready ^^^ now a good question is whether it needs a microversion..19:37
dtantsurpros: the fix is discoverable, cons: the fix is not backportable19:37
dtantsuropinions?19:38
dtantsurrloo, TheJulia, afternoon bikeshedding time ^^^ :)19:38
rloodtantsur: dam* you...19:39
dtantsurLOL :D19:39
* TheJulia frowns at the expectation of bikeshedding time19:39
dtantsurit's never bad time for arguing about microversions, is it?19:39
TheJulialol19:40
TheJuliaI need to wrap up an email, give me a couple minutes19:40
dtantsursure19:40
dtantsurme as an API-SIG cores says "microversion it", me as someone who still maintains Kilo says "OH NO"19:40
rloodtantsur: is it a bug? err, was it a bug? I mean, do people intend to specify /v1/nodes.json and expect it to be treated with requested content type Application/json?19:41
*** baoli has joined #openstack-ironic19:41
rloodtantsur: any idea what rest of openstack community does?19:41
dtantsurrloo: /v1/nodes.json is fine. the problem is in /v1/nodes/mynode.119:41
openstackgerritMerged openstack/bifrost master: Fix typo in playbook markdown readme  https://review.openstack.org/54098819:41
openstackgerritMerged openstack/bifrost master: Don't copy ironic's policy.json sample  https://review.openstack.org/54089319:41
dtantsurrloo: API-SIG recommends microversioning bugs. I have no idea what the community actually does19:41
* dtantsur summons edleafe19:41
*** baoli has quit IRC19:42
rloodtantsur: API-SIG recommends, it isn't a 'must' or a 'shall'. let me think/look at the patch...19:42
dtantsurright, we cannot say 'must' :)19:43
dtantsurhuh, nearly 9pm, I guess I'll go19:47
TheJuliagah! complex reply19:47
*** mgoddard_ has quit IRC19:47
dtantsurmmm?19:47
TheJuliadtantsur: In my mind, it is a bug and we really shouldn't try to microversion it19:48
dtantsurI quite agree, even though most of API-SIG things we should version bugs19:48
rlooso is /v1/node/<uuid>.json valid?19:48
dtantsurrloo: it used to be, and still is with my patch19:48
dtantsurpecan magic (and ponies)19:49
TheJuliaour client doesn't go for latest possible maximum... so... we can't logistically version and actually fix the issue at the same time19:49
dtantsuryep19:49
dtantsurand if we version this fix, it's likely to miss queens19:49
* TheJulia wonders if ponies could somehow be added into a slide deck for the project update19:49
dtantsurrloo: pecan by default allows /foo/bar.ext and tries to detect mime type by "ext"19:49
dtantsurwhich is a so-so idea to me, but who knows19:50
rloodtantsur: well, that is pecan. but what does a user expect?19:50
dtantsurso when we have /v1/nodes/small.1, it treats it as a node called "small" returned as a man page :)19:50
TheJuliaoh wow, bifrost changes merged19:50
rloodtantsur: c'mon, that's really cool and is exactly what the user would want! ugh.19:50
dtantsurrloo: I try to not change this aspect. what I want to change is inability to get, patch or delete nodes by name if the name ends with a known extension19:51
dtantsurso, I have to do some trickery around .json, because /v1/nodes/small.json may mean 1. node called "small.json", 2. node called "small" returned as JSON19:51
dtantsurthe former is what we document, the latter is what actually works before my patch19:51
dtantsurI try to keep both19:51
TheJuliayeah, more and more I think about it... unversioned19:52
dtantsurthen please review :)19:52
rloodid we ever talk about using something else instead of pecan?19:53
dtantsurwe did19:53
dtantsurthe problem is to keep compatibility with all these quirks19:53
dtantsurand of course that's quite a lot of work :)19:54
rloonot sure how easy it would be to move over to something else, but i don't think we want to version this and then have to continue supporting both ways19:54
jrollas soon as I don't have to spend a cycle talking about using something else, I'm going to do it19:54
TheJuliadtantsur: just clicked send so I can now :)19:55
jrolldtantsur: I'm inclined to think we don't have users relying on this behavior (other than perhaps .json, which you handle), so I vote no microversion for it19:55
dtantsurI think we had some very early agreement to try Flask (which is something ironic-inspector uses)19:55
dtantsur3:0 for not versioning, nice!19:55
* jroll will review the patch later, it's meetings/lunch time19:56
dtantsurtake your time, I'm leaving for a day anyway19:56
dtantsurcompletely unrelated, but may be curious: a guy on devconf.cz showed me https://github.com/LAKostis/redirector19:56
dtantsur(NSFL warning: a lot of hacks in bash)19:56
rlooit seems to me that other openstack projects use pecan, and will have a similar problem.19:57
rloowe should deal with it the same way.19:57
rloomaybe we are the first to notice...19:57
TheJulianot safe for ?l?19:58
jrolllife19:58
jroll:P19:58
dtantsuryes19:59
dtantsurrloo: we may be the only project with 1. unique names, 2. allowing names to be used in /v1/type/<NAME>, IIRC nova does /v1/servers/?name=<NAME> (which btw does a LIKE query)20:00
dtantsurthough flavors may use the first form, not sure20:00
dtantsurI did not get to implementing flavors in rust-openstack yet :D20:00
*** jtomasek has quit IRC20:00
* dtantsur learns things by implementing them on client side20:00
*** absubram has joined #openstack-ironic20:01
*** dtantsur is now known as dtantsur|afk20:03
dtantsur|afko/20:03
rloobye dtantsur|afk20:06
rlooso sad, that bug was reported in 2016 :-(20:06
* edleafe catches up20:16
edleafedtantsur|afk: rloo: if a bug fix changes a 500 to something else, that doesn't need a microversion20:16
*** r-daneel_ has joined #openstack-ironic20:16
edleafeBut if you're changing what you get back from a given URL, well, that's a prime candidate for a new version20:17
*** r-daneel has quit IRC20:17
*** r-daneel_ is now known as r-daneel20:17
rlooedleafe: so if you did GET /v1/nodes/foo.me and you got 404 (cuz of bug), and with this fix, you get 200 + body in response, it needs to be versioned?20:19
edleaferloo: if the 404 was legitimate, yes. If it should have returned something else, than no20:21
edleaferloo: A lot depends on how it was documented20:21
rlooedleafe: the 404 wasn't legitimate20:21
rlooedleafe: not documented. i mean, the document was that GET /v1/nodes/foo.me would return info about a node named 'foo.me'. which it didn't do.20:22
edleaferloo: if it wasn't doing what it should have, then no new version is needed. If it was doing what it should have, but with an incomplete/inadequate response, then it should be a new version20:23
*** mjturek has quit IRC20:24
*** Goneri has joined #openstack-ironic20:24
edleafeIOW, would anyone have anything that depended on the old behavior?20:24
rlooedleafe: thx, that's what i wanted to hear. It didn't do what it should have, so no versioning.20:24
edleaferloo: cool20:24
*** e0ne has joined #openstack-ironic20:24
rlooedleafe: nope, nothing 'normal' would depend on the old behaviour :)20:24
rlooedleafe: 'normal' cuz one never knows...20:24
*** mgoddard_ has joined #openstack-ironic20:25
rloothank you edleafe for clarifying!20:25
*** rloo has quit IRC20:29
*** hw_wutianwei1 has joined #openstack-ironic20:32
*** hw_wutianwei has quit IRC20:34
*** hw_wutianwei1 is now known as hw_wutianwei20:34
*** mgoddard_ has quit IRC20:38
*** rloo has joined #openstack-ironic20:41
*** mjturek has joined #openstack-ironic20:42
*** mjturek has quit IRC20:45
snasirHi, I'm trying to enable the serial console of the baremetal node, using the instructions here https://docs.openstack.org/ironic/latest/admin/console.html#node-serial-console20:48
snasirAnd keep seeing this error https://thepasteb.in/p/lOhOKAgq71KHB , when i try to do this add this propert" --console-interface ipmitool-socat". And I am using pxe_ipmitool20:49
snasirThe console_interface is already set to "None", and i want to change it to "ipmitool-socat", any suggestions on what I may be doing wrong ? Thanks20:50
*** olivierbourdon38 has quit IRC20:50
TheJuliasnasir: you need to change to a hardware type20:55
snasirTheJulia: how do i go about doing that ? I was just checking the enabled drivers at the moment ? Is that another parameter that needs to be added to the ironic node ?20:56
snasirTheJulia: enabled_hardware_types = ipmi , is set in my ironic.conf at the moment20:57
*** mjturek has joined #openstack-ironic20:57
TheJuliasnasir: set driver to ipmi, set deploy_interface to iscsi, then set console_interface20:58
*** takamatsu has quit IRC21:01
*** e0ne has quit IRC21:01
snasirTheJulia: I have "enabled_drivers = pxe_ipmitool" , And "#enabled_deploy_interfaces = iscsi,direct" in my ironic.conf.... I think iscsi is set by default if you dont specify it explicitly right ?21:01
snasirAnd i also have this set "enabled_console_interfaces = ipmitool-socat,no-console" in my ironic.conf21:01
TheJuliasnasir: so hardware_types are superceeding drivers21:01
TheJuliaipmi is the generic replacement for pxe_ipmitool and agent_ipmitool, the only difference is the deploy interface. iscsi and direct respectively.21:02
TheJuliaI _think_ iscsi would be the default as it is first in the list21:02
*** mjturek has quit IRC21:03
jroll99% sure you're correct there21:13
*** dpawar_ has joined #openstack-ironic21:17
*** e0ne has joined #openstack-ironic21:17
*** dpawar has quit IRC21:19
*** dpawar_ has quit IRC21:21
*** AlexeyAbashkin has quit IRC21:21
TheJuliarloo: https://review.openstack.org/#/c/531120/21:27
patchbotpatch 531120 - ironic - Rework exception handling on deploy failures in co...21:27
rlooneeds a revie?21:27
rloow21:27
rloook, in a few min21:27
TheJuliathanks21:29
TheJuliagrenade multinode also passed on it *blinks*21:29
*** threestrands has joined #openstack-ironic21:32
*** threestrands has quit IRC21:32
*** threestrands has joined #openstack-ironic21:32
* TheJulia muses with the thought of dropping all of the nv jobs util after release... likely too late though21:33
rlooTheJulia: i'd prefer to keep them, just in case something breaks, we can go back to see where it might have happened21:35
rlooTheJulia: what about just dropping patches that we don't care about?21:35
rlooTheJulia: the abandon button will do that21:35
rlooTheJulia: is zuul overwhelmed?21:36
TheJuliarloo: I think there are 400 patches in the check queue21:36
TheJuliaI'd rather not abandon/unabandon... Just have to wait it out I think21:37
rlooTheJulia: whoa! I suspect that there'll be an extension to the RC1 deadline. And really, we are probably safe if we get ironic in 1-2 days after the rest of the projects. i can't recall when things 'broke' last time, maybe a week after rc1.21:38
*** mjturek has joined #openstack-ironic21:39
TheJuliayeah21:41
*** ijw has joined #openstack-ironic21:42
peitolmwhere's the best place to start looking at port binding errors21:45
rlooTheJulia: do we care about string freeze? I'm guessing not?21:46
openstackgerritJulia Kreger proposed openstack/bifrost master: Fix for proper deployment when keystone enabled and minimal OS  https://review.openstack.org/54071321:47
*** snasir has quit IRC21:49
*** snasir has joined #openstack-ironic21:49
peitolm"Refusing to bind due to unsupported vnic_type: baremetal bind_port /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_agent.py:93" hmm...21:49
*** mjturek has quit IRC21:51
*** etingof has quit IRC21:52
*** mjturek has joined #openstack-ironic21:53
openstackgerritMerged openstack/ironic master: Replace chinese quotes to English quotes  https://review.openstack.org/54120621:53
TheJuliarloo: given our release model, we pretty much have to be willing to take the patches as we go, so not really21:53
rlooTheJulia: that's what I thought :)21:54
openstackgerritJulia Kreger proposed openstack/bifrost master: Make Fedora26 CI job voting  https://review.openstack.org/54052421:54
*** ijw has quit IRC21:58
*** etingof has joined #openstack-ironic22:06
*** ijw has joined #openstack-ironic22:07
* TheJulia is watching/waiting for http://zuul.openstack.org/stream.html?uuid=3f91077e2f9f4b7dac39ba03dfa2f37d&logfile=console.log22:08
TheJuliarloo: if there are no disagreements and the backport to stable/pike for devstack changes cleans up the job issues, I'm thinking we should just immediately re-enable the multinode gate22:11
rlooTheJulia: ++22:11
*** mjturek has quit IRC22:13
TheJuliaoooh it is post-stacking22:13
*** e0ne has quit IRC22:15
*** openstackgerrit has quit IRC22:16
TheJulianetworking-baremetal has 2 patches remaining22:17
TheJuliasmoke test running22:26
NobodyCamoh is it 4:20 already :p22:26
* NobodyCam _ducks_22:26
*** Goneri has quit IRC22:27
*** mjturek has joined #openstack-ironic22:28
*** rcernin has joined #openstack-ironic22:28
TheJuliaheh22:31
*** mjturek has quit IRC22:33
*** mjturek has joined #openstack-ironic22:34
*** openstackgerrit has joined #openstack-ironic22:39
openstackgerritMerged openstack/bifrost master: roles: bifrost-ironic-install: Download the iPXE loader file  https://review.openstack.org/53929822:39
* peitolm wonders where to start looking at why neutron can't bind this port, http://paste.openstack.org/show/664104/22:43
openstackgerritMerged openstack/bifrost master: Add creation of openrc file when keystone enabled  https://review.openstack.org/54009622:43
TheJuliapeitolm: oh, sorry, completely got side tracked22:48
peitolmthat's o.k., any help is appreciated, it just seems like a common,but v.difficult issue which boils down to "you've done something wrong"22:49
TheJuliaso which network_interface are you using?22:50
peitolmum, sorry, i'm in a maze of twisty passages with that question,22:51
TheJuliaoh my22:52
peitolm# grep network_inter /etc/ironic/ironic.conf  | grep -v '^#'22:52
peitolmenabled_network_interfaces = flat,noop,neutron22:52
peitolmdefault_network_interface = neutron22:52
TheJuliaI would guess flat, because there is no binding profile sent, which then makes me wonder if you have the required information on the node's port(s), and which network_interface your using22:52
peitolmis that what you mean22:53
peitolmah22:53
TheJuliakind of22:53
peitolmprovider:network_type     | flat22:53
TheJuliaahh, okay22:53
peitolmfrom "# openstack network show 2ca2c8b2-f1e5-4eb2-8d4d-0b180d017f1922:53
peitolm"22:53
TheJuliaawesome22:53
peitolm(which is in the paste :P)22:53
TheJuliaso, your default_network_interface should be likely "flat" in that case22:54
peitolmah22:54
TheJuliaI believe your port binding is likely to always report failed22:56
TheJuliain flat22:56
*** ijw has quit IRC22:57
peitolmok.22:58
peitolmso that may be a red herring then22:58
peitolmi'm trying to get dhcp/bootp packets in22:59
TheJuliapeitolm: this patch might be a little eye-opening. it landed on master about a month ago and describes the situation you with flat networks https://github.com/openstack/ironic/commit/f1e5b92e446a7bf15e052aaab2692c3d4f5710a1#diff-0f2bb064f00c135ecda1486ef58a0584R1722:59
TheJuliain fron where?22:59
TheJuliaand to where?22:59
peitolmfrom the baremetal host to the dhcp server on the cleaning network23:00
TheJuliaokay, and the flat network you referred to. can you ping the interface on that network from the network baremetal nodes are attached to?23:00
TheJulia specifically by network interface in that last statement, I'm referring to the the router that allows connectivity to ironic-api23:01
peitolmhttp://paste.openstack.org/show/664130/ may help23:03
* TheJulia looks23:04
peitolmthe node (pe1950) is trying to get a DHCP to do the initial deploy/cleaning/whatever23:05
peitolmi can see bootp on the raw interface "enx0050b622b509"23:05
peitolmas i understand it, i'd expect to see br-int and br-eth2 showing UP, and tap402407e1-53 to be bound onto the ovs bridge23:07
TheJuliaokay23:09
TheJuliaso neutron is basically looking for physnet1, except you have bridges in OVS and no actual physical interface attached to that23:10
peitolm# grep physnet1 /etc/neutron/plugins/ml2/ml2_conf.ini23:10
peitolmflat_networks = physnet123:11
peitolmbridge_mappings = physnet1:br-eth223:11
TheJuliaokay23:11
peitolmunless that should be something else br-<interfacename>23:12
TheJuliabut br-eth2 doesn't have anything in it physically23:12
*** r-daneel has quit IRC23:12
*** r-daneel has joined #openstack-ironic23:13
peitolmPort "enx0050b622b509" Interface "enx0050b622b509"23:13
peitolmthat's the network interface onto the provisioning/cleaning/tenant network23:16
TheJuliadoh, missed that23:16
peitolmno worries, it looks like line noise :)23:16
peitolmtbh, that's the device i'm tcpdumping23:16
peitolmas the bridge is down, i can't tcpdump that23:17
TheJuliaoh, ifup the bridge23:17
TheJuliawell23:17
TheJuliaip link blah blah blah set up23:18
TheJuliaifconfig br-eth2 up23:18
* TheJulia thinks it is up23:18
peitolm*nods*23:18
peitolmit is now23:18
peitolmwonder why openswitch isn't bringing it up23:18
TheJulia*shrugs*23:19
peitolmo.k. i can see packets on br-eth2, now to step over the link23:20
* TheJulia crosses fingers23:20
peitolmnot seeing it on br-int (even after 'UP'ing it23:20
TheJuliaanything on the tap?23:21
peitolmnope, even after adding it to the bridge23:22
peitolm# ovs-ofctl dump-flows br-eth223:23
peitolm cookie=0x6988756dbafd26d7, duration=7183.245s, table=0, n_packets=16, n_bytes=1327, priority=2,in_port="phy-br-eth2" actions=drop23:23
peitolm cookie=0x6988756dbafd26d7, duration=7183.250s, table=0, n_packets=423, n_bytes=240711, priority=0 actions=NORMAL23:23
TheJuliainteresting23:23
* TheJulia sighs due to grenade failure23:24
peitolmi'll let you get back to that, i'm going to have to go to the land of nod23:24
TheJuliarest well!23:25
peitolmtime to power off the noisy machines :)23:25
TheJulia:)23:25
peitolmthanks for your help Julia, hopefully i'll progress past complete newbie status soon23:27
TheJuliaI'm sure you will23:29
TheJuliaanyway, back to test failures :(23:29
*** mjturek has quit IRC23:29
*** bfournie has quit IRC23:30
*** rbudden has quit IRC23:30
*** charcol-v2 has joined #openstack-ironic23:33
*** MattMan has quit IRC23:37
*** MattMan has joined #openstack-ironic23:38
*** absubram has quit IRC23:38
*** jcoufal has quit IRC23:42
*** absubram has joined #openstack-ironic23:51
*** absubram has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!