Tuesday, 2021-01-12

*** uzumaki has joined #openstack-ironic00:02
openstackgerritMerged openstack/ironic-specs master: Add support for anaconda deploy interface  https://review.opendev.org/c/openstack/ironic-specs/+/74850300:09
*** tosky has quit IRC00:15
*** rloo has quit IRC00:35
*** uzumaki has quit IRC01:19
*** uzumaki has joined #openstack-ironic01:20
*** anuradha1904 has quit IRC01:30
*** pmannidi has quit IRC02:02
*** pmannidi has joined #openstack-ironic02:22
*** Qianbiao has joined #openstack-ironic02:40
*** dsneddon has quit IRC02:57
*** rcernin_ has quit IRC03:14
*** ricolin has quit IRC03:19
*** mkrai has joined #openstack-ironic03:30
*** zzzeek has quit IRC04:06
*** zzzeek has joined #openstack-ironic04:07
*** ricolin_ has joined #openstack-ironic04:17
*** rcernin_ has joined #openstack-ironic04:25
*** rcernin_ has quit IRC04:25
*** rcernin has joined #openstack-ironic04:25
*** anuradha1904 has joined #openstack-ironic05:02
*** tzumainn has quit IRC05:24
*** gyee has quit IRC06:17
*** rcernin has quit IRC06:26
*** rcernin has joined #openstack-ironic06:26
*** hoonetorg has quit IRC06:34
*** mkrai has quit IRC06:42
*** mkrai_ has joined #openstack-ironic06:42
*** uzumaki has quit IRC06:59
*** mkrai_ has quit IRC07:01
*** zzzeek has quit IRC07:02
openstackgerritJacob Anders proposed openstack/ironic-python-agent master: [WIP] Add support for using NVMe specific cleaning  https://review.opendev.org/c/openstack/ironic-python-agent/+/77023707:03
*** zzzeek has joined #openstack-ironic07:07
*** rcernin has quit IRC07:07
openstackgerritMerged openstack/ironic master: Do not enter maintenance if cleaning fails before running the 1st step  https://review.opendev.org/c/openstack/ironic/+/76992307:11
*** tosin has joined #openstack-ironic07:29
*** mkrai has joined #openstack-ironic07:35
*** anuradha1904 has quit IRC07:35
*** mkrai has quit IRC07:57
arne_wiebalckGood morning, ironic!07:59
*** hjensas has quit IRC07:59
*** mkrai has joined #openstack-ironic07:59
openstackgerritankit proposed openstack/sushy master: Fixes issue of redfish firmware update  https://review.opendev.org/c/openstack/sushy/+/76491507:59
*** rpittau|afk is now known as rpittau08:08
rpittaugood morning ironic! o/08:08
*** uzumaki has joined #openstack-ironic08:19
*** tosky has joined #openstack-ironic08:22
jandersgood morning arne_wiebalck rpittau and Ironic o/08:28
rpittauhey janders :)08:29
arne_wiebalckhey janders o/08:35
*** hjensas has joined #openstack-ironic08:35
*** dougsz has joined #openstack-ironic08:49
openstackgerritRiccardo Pittau proposed openstack/ironic master: Replace retrying with tenacity  https://review.opendev.org/c/openstack/ironic/+/37657408:51
openstackgerritwushiming proposed openstack/bifrost master: remove unicode from code  https://review.opendev.org/c/openstack/bifrost/+/77025009:00
*** uzumaki has quit IRC09:03
*** lucasagomes has joined #openstack-ironic09:04
*** derekh has joined #openstack-ironic09:30
*** mgoddard has joined #openstack-ironic09:32
openstackgerritDhuldev Valekar proposed x/sushy-oem-idrac master: Redfish management clean steps  https://review.opendev.org/c/x/sushy-oem-idrac/+/72159609:34
*** ricolin_ has quit IRC09:39
openstackgerritMerged openstack/bifrost master: Fix passing parameters with spaces to bifrost-cli  https://review.opendev.org/c/openstack/bifrost/+/77004009:40
*** dtantsur|afk is now known as dtantsur09:42
dtantsurmorning ironic09:42
openstackgerritDmitry Tantsur proposed openstack/bifrost stable/victoria: Fix passing parameters with spaces to bifrost-cli  https://review.opendev.org/c/openstack/bifrost/+/77018209:42
openstackgerritMerged openstack/bifrost master: remove unicode from code  https://review.opendev.org/c/openstack/bifrost/+/77025009:43
*** uzumaki has joined #openstack-ironic09:52
openstackgerritDmitry Tantsur proposed openstack/ironic stable/victoria: Fix idrac-wsman RAID step async error handling  https://review.opendev.org/c/openstack/ironic/+/77025609:52
openstackgerritDmitry Tantsur proposed openstack/ironic stable/victoria: Update `cleaning_error_handler`  https://review.opendev.org/c/openstack/ironic/+/77025709:53
openstackgerritDmitry Tantsur proposed openstack/ironic stable/victoria: Do not enter maintenance if cleaning fails before running the 1st step  https://review.opendev.org/c/openstack/ironic/+/77025809:53
openstackgerritDmitry Tantsur proposed openstack/ironic stable/victoria: Update `cleaning_error_handler`  https://review.opendev.org/c/openstack/ironic/+/77025709:58
openstackgerritDmitry Tantsur proposed openstack/ironic stable/victoria: Do not enter maintenance if cleaning fails before running the 1st step  https://review.opendev.org/c/openstack/ironic/+/77025809:58
openstackgerritankit proposed openstack/ironic master: Fixes issue of redfish firmware update  https://review.opendev.org/c/openstack/ironic/+/76491410:02
*** uzumaki has quit IRC10:05
*** uzumaki has joined #openstack-ironic10:05
*** adrianc has quit IRC10:12
*** adrianc has joined #openstack-ironic10:13
*** ociuhandu has joined #openstack-ironic10:14
*** adrianc has quit IRC10:17
*** adrianc has joined #openstack-ironic10:18
*** ociuhandu has quit IRC10:24
*** ociuhandu has joined #openstack-ironic10:30
*** ociuhandu has quit IRC10:32
*** ociuhandu has joined #openstack-ironic10:32
*** ociuhandu has quit IRC10:42
*** hjensas is now known as hjensas|afk10:51
dtantsurrpittau: please check also https://review.opendev.org/c/openstack/ironic/+/770257 since it's between the two patches you've approved11:02
rpittauoh whoops I had that open and skipped it!11:02
*** rpittau has quit IRC11:06
*** ociuhandu has joined #openstack-ironic11:07
*** rpittau has joined #openstack-ironic11:07
*** nam-est has joined #openstack-ironic11:14
nam-estHi all,11:14
nam-estI see this error in error log of Ironic-inspector and ironic-conductor11:15
nam-estTraceback (most recent call last):11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 461, in fire_timers11:15
nam-est    timer()11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/hubs/timer.py", line 59, in __call__11:15
nam-est    cb(*args, **kw)11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/hubs/__init__.py", line 151, in _timeout11:15
nam-est    current.throw(exc)11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/greenthread.py", line 221, in main11:15
nam-est    result = function(*args, **kwargs)11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request11:15
nam-est    proto.__init__(conn_state, self)11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 357, in __init__                                                                                                                               self.handle()11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 390, in handle11:15
nam-est    self.handle_one_request()11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 419, in handle_one_request11:15
nam-est    self.raw_requestline = self._read_request_line()11:15
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 402, in _read_request_line11:15
nam-est    return self.rfile.readline(self.server.url_length_limit)11:16
nam-est  File "/usr/lib64/python3.6/socket.py", line 586, in readinto11:16
nam-est    return self._sock.recv_into(b)11:16
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/green/ssl.py", line 241, in recv_into11:16
nam-est    return self._base_recv(nbytes, flags, into=True, buffer_=buffer)11:16
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/green/ssl.py", line 256, in _base_recv11:16
nam-est    read = self.read(nbytes, buffer_)11:16
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/green/ssl.py", line 176, in read11:16
nam-est    super(GreenSSLSocket, self).read, *args, **kwargs)11:16
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/green/ssl.py", line 156, in _call_trampolining11:16
nam-est    timeout_exc=timeout_exc('timed out'))11:16
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/hubs/__init__.py", line 159, in trampoline11:16
nam-est    return hub.switch()11:16
nam-est  File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 298, in switch11:16
nam-est    return self.greenlet.switch()11:16
nam-estssl.SSLError: ('timed out',)11:16
nam-estSorry if this is not a proper way to show the log in this IRC channel11:16
nam-estHow can we fix this error? Can using the newest version of Eventlet fix it?11:16
dtantsurnam-est: please use a paste service, like paste.openstack.org. Timeout sounds like a problem on your end, not in a library.11:20
nam-estdtansur: I will use a paste service next time. Also, I will do more investigation on this error. Thank you11:23
openstackgerritDerek Higgins proposed openstack/ironic master: Add a delay/retry is vmedia insert fails  https://review.opendev.org/c/openstack/ironic/+/77027011:29
*** mkrai has quit IRC11:36
jandersdtantsur rpittau would you have a couple minutes to chat about https://storyboard.openstack.org/#!/story/2008290 (NVMe cleaning)?11:42
dtantsursure11:42
jandersawesome!11:43
* janders is digging out the change URL11:43
jandershttps://review.opendev.org/c/openstack/ironic-python-agent/+/77023711:44
jandersI'm hoping to spend some time on this in the coming days when I'm not working on higher priority items11:44
jandersI wanted to check with you what's the best direction for this change11:44
janders^^ is just a skeleton of a skeleton11:44
janderswhat I'm after is:11:44
*** ociuhandu has quit IRC11:45
openstackgerritdou proposed openstack/ironic stable/ussuri: Feature: Add raid configuration support for ibmc driver  https://review.opendev.org/c/openstack/ironic/+/77018511:45
janders1) I wanted to confirm if we want this in this part of IPA, as opposed to a custom cleaning mechanism11:45
jandersand 2) would it be a good approach to check whether a device looks like a (S)ATA device or an NVMe, based on that try the appropriate "secure erase" method and fall back to shred if it fails, like we currently do with SATA secure erase?11:46
openstackgerritdou proposed openstack/ironic stable/train: Feature: Add raid configuration support for ibmc driver  https://review.opendev.org/c/openstack/ironic/+/77018611:49
jandersI also looked at https://opendev.org/openstack/ironic-python-agent/src/branch/master/examples/custom-disk-erase/example_disk_eraser.py as this is what dking used in his NVMe cleaning work, but I figured that's more for site/operator specific solutions11:50
iurygregorygood morning Ironic!11:54
jandershey iurygregory11:54
jandershow is it going?11:54
iurygregoryhey janders o/ doing good, how about you?11:56
jandersI'm good, thank you! :)11:56
dtantsurjanders: 1) we definitely want it in IPA, it covers a gap in the current secure erase functionality11:58
rpittaujanders: I'm at lunch at the moment, I'll read the scroll later, if you're not here later we can talk tomorrow morning11:59
dtantsur2) sounds about right11:59
dtantsurwe probably need to deprecate options with "ata" in them and use the same options for NVMe and ATA secure erase11:59
jandersI was thinking the same regarding deprecation11:59
dtantsurexample_disk_eraser is an example for operators, yes11:59
openstackgerritMonica Pardhi proposed openstack/ironic stable/ussuri: DRAC: Fix a failure to create virtual disk  https://review.opendev.org/c/openstack/ironic/+/77027712:01
*** ricolin_ has joined #openstack-ironic12:03
jandersgreat, thank you dtantsur ( rpittau if you have any additional feedback I will check the scrollback too - or feel free to comment in the change )12:04
jandersthanks guys!12:04
*** ociuhandu has joined #openstack-ironic12:15
*** ociuhandu has quit IRC12:19
*** ociuhandu has joined #openstack-ironic12:19
*** ociuhandu has quit IRC12:20
*** ociuhandu has joined #openstack-ironic12:20
*** ociuhandu has quit IRC12:23
*** ociuhandu has joined #openstack-ironic12:24
*** ociuhandu has quit IRC12:31
*** ociuhandu has joined #openstack-ironic12:45
*** ociuhandu has quit IRC12:46
*** ociuhandu has joined #openstack-ironic12:47
*** ricolin_ has quit IRC12:47
*** tzumainn has joined #openstack-ironic12:53
*** ociuhandu has quit IRC13:07
*** bfournie has quit IRC13:26
*** ociuhandu has joined #openstack-ironic13:37
openstackgerritVerification of a change to openstack/ironic failed: Fix idrac-wsman RAID step async error handling  https://review.opendev.org/c/openstack/ironic/+/77025613:40
openstackgerritVerification of a change to openstack/ironic failed: Update `cleaning_error_handler`  https://review.opendev.org/c/openstack/ironic/+/77025713:40
openstackgerritVerification of a change to openstack/ironic failed: Do not enter maintenance if cleaning fails before running the 1st step  https://review.opendev.org/c/openstack/ironic/+/77025813:40
arne_wiebalckBare Metal SIG meeting in ~10 mins with tzumainn on multi-tenancy: https://etherpad.opendev.org/p/bare-metal-sig13:48
arne_wiebalckEveryone is welcome!13:48
*** rloo has joined #openstack-ironic13:49
*** ociuhandu has quit IRC13:50
*** paras333 has joined #openstack-ironic13:55
*** paras333 has quit IRC13:56
*** rh-jelabarre has quit IRC13:56
*** ricolin_ has joined #openstack-ironic13:58
*** ricolin_ has quit IRC13:58
*** ricolin has joined #openstack-ironic13:59
rloobaremetal SIG is starting... https://cern.zoom.us/j/94959408650?pwd=dW55WHNaeUd4OGhwSU5BZmR1K2FEZz0914:00
*** paras333 has joined #openstack-ironic14:01
*** rh-jelabarre has joined #openstack-ironic14:03
*** uzumaki has quit IRC14:07
*** ociuhandu has joined #openstack-ironic14:21
arne_wiebalcktzumainn: thanks for the great presentation!14:24
tzumainnarne_wiebalck, thanks, it was fun :)14:24
TheJuliagood morning14:24
rpittautzumainn: thanks, that was really interesting :)14:25
dtantsurmorning TheJulia14:27
openstackgerritMerged openstack/bifrost stable/victoria: Fix passing parameters with spaces to bifrost-cli  https://review.opendev.org/c/openstack/bifrost/+/77018214:37
tzumainndtantsur, er, I haven't written an email to upstream openstack in a while - it's openstack-discuss, correct? and you wanted me to start asking whether the node attributes used for boot-from-volume are something we really want non-admins to update, or if we want some kind of change there?14:39
dtantsurtzumainn: yes and yes14:41
TheJuliatzumainn: something like members?14:48
dtantsurTheJulia: the problem is updating storage_interface. same situation as with deploy_interface: we need a way for non-admin to update it per instance14:49
TheJuliawe could just try to dogpile on rbac stuffs and allow members to do it14:50
dtantsurI don't think allowing members updating interfaces permanently is a good idea..14:50
dtantsurand it will only solve a half of the problem14:50
TheJuliawhat is the other half?14:51
dtantsurdoing a permanent change for something that is image-specific14:52
dtantsurwhich means that a node returns to the available pool with a potentially incorrect deploy interface (in case of ramdisk deploy) or storage interface (in case of boot from volume)14:53
TheJuliahow would we know it is image specifc?14:53
dtantsuras good start is to accept them in instance_info (which is wiped before cleaning)14:53
TheJuliaa this_deployment_only_change_x_interface option?14:54
dtantsurI mean $ baremetal node set <node> --instance_info deploy_interface=ramdisk14:54
guilhermespTheJulia: and ironic team good morning!14:54
dtantsurthis does imply that the caller knows what they are doing, but at least they won't affect subsequent deployments14:54
dtantsurmorning guilhermesp14:55
TheJulialets signify that it is one shot sort of thing in that14:55
guilhermespso TheJulia maybe that will be an option for us instead of using hdpam https://superuser.com/questions/1530363/how-to-securely-erase-an-nvme-ssd/1533183#153318314:55
TheJuliawe don't want people thinking that is the canoical way to change it14:55
guilhermespi can totally put a live cd on my nodes and try things out14:55
dtantsurguilhermesp: you need to sync with janders, he is working (?) on NVMe erasure14:55
dtantsurTheJulia: I hope we have enough education around instance_info at this point..14:55
dtantsurotherwise we need a clean step to return interfaces to (some?) defaults14:56
TheJuliaheh, they kind of ignore that hdparm also works for scsi/sas drives in that superuser answer, but that was about what I was suspecting14:56
guilhermespyeah i can see in the whiteboard and dtantsur and janders are in the loop for that. So i will keep you updated with my results14:56
TheJuliadtantsur: well, we would likely need an automatic clean step anyway.14:56
TheJuliaI think14:56
dtantsurnot with instance_info, it's wiped automatically14:57
TheJuliaso your saying never actually change the node.deploy_interface14:57
TheJuliajust check the field in driver loading for overrides?14:57
dtantsuryep14:59
*** MentalSiege has joined #openstack-ironic15:03
*** MentalSiege has quit IRC15:17
*** lbragstad has quit IRC15:29
*** anuradha1904 has joined #openstack-ironic15:33
*** lbragstad has joined #openstack-ironic15:34
*** lbragstad has quit IRC15:35
*** lbragstad has joined #openstack-ironic15:35
*** Qianbiao has quit IRC15:50
dtantsurajya: so, re BIOS and IPA. I assume we need a way to figure out if applying BIOS setting has finished?16:08
dtantsurdo we even have such a way?16:08
ajyadtantsur: I think yes, I need to check but BIOS update should be regular async op in Redfish, so it can take the same approach as firmware upgrade using task monitor.16:09
dtantsurokay, so we need to implement that to be able to mark these steps as not requiring ramdisk16:10
ajyadtantsur: I don't know the history of that BIOS patch, but I guess at that time task monitors were not available in sushy and current implementation is simpler.16:10
dtantsurprobably16:10
dtantsurajya: do you happen to know if iDRAC reset is the same? i.e. requires a full reboot?16:10
openstackgerritMerged openstack/ironic-specs master: remove unicode from code  https://review.opendev.org/c/openstack/ironic-specs/+/76929416:13
ajyadtantsur: looks like no, reset factory action does not support apply times16:21
dtantsurgreat, thanks for confirming16:21
ajyadtantsur: then solution could be vendor specific, for iDRAC this patch solves the same problem for WS-MAN - how to know when reset is done; https://review.opendev.org/c/openstack/ironic/+/74869616:22
ajyathat reminds me, TheJulia, when you have a time, can you take a look at the last comments of https://review.opendev.org/c/openstack/ironic/+/748696 the code should be as it is16:23
TheJuliaajya: I'll try to look later today16:23
ajyaTheJulia: thanks16:23
tosinTheJulia16:33
TheJuliatosin: eek, got double booked and didn't notice it16:33
tosinTheJulia totally fine :D16:34
*** sshnaidm|ruck is now known as sshnaidm|afk16:47
*** openstackgerrit has quit IRC16:55
*** lucasagomes has quit IRC16:58
TheJuliatosin: I'm back in the call if you want to rejoin17:29
tosinTheJulia: Sure!17:30
*** dougsz has quit IRC17:31
rpittaugood night! o/17:32
*** rpittau is now known as rpittau|afk17:32
*** ociuhandu has quit IRC17:41
*** dtantsur is now known as dtantsur|afk17:51
dtantsur|afko/17:51
zer0c00lhas anyone used cloud-init network_data.json to configure static network interfaces in baremetal?18:00
*** ociuhandu has joined #openstack-ironic18:06
*** ociuhandu has quit IRC18:06
zer0c00lif a server has multiple network interfaces, how do one assign ip addresses using cloud-init to the right network interface (i.e nw interfaces with link)18:14
JayFI believe you have to have the (ironic) port properly configured to map to the correct mac address18:14
*** ianychoi_ has quit IRC18:46
*** ianw_pto is now known as ianw18:58
*** gyee has joined #openstack-ironic19:02
*** dsneddon has joined #openstack-ironic19:48
TheJuliaYes, you do with network_metadata.json20:00
TheJuliaIP gets mapped to the mac.20:00
JayFhttps://seclists.org/oss-sec/2021/q1/31 we may want to ensure this is updated in our published IPA images20:16
JayFnot sure the best way to do that though20:16
JayF(ISCSI target CVE)20:16
JayF(although TBH, I can't think of a threat model that would impact us with the bug...)20:20
*** tosin has quit IRC20:29
TheJuliathere is no real way to do it, just wipe and publish a change to each branch20:31
TheJuliaI guess I'm kind of glad that I don't think our ramdisks even run the LIO based export20:32
*** dsneddon has quit IRC20:37
*** dsneddon has joined #openstack-ironic20:38
*** bfournie has joined #openstack-ironic20:45
eanderssonbtw TheJulia looks like the issue with reboot_requested was user error20:54
eanderssonWe had priority set to 10 (I believe) and that was breaking the logic20:55
eanderssonAfter dtruong bumped the priority it seems to work without an issue20:56
*** derekh has quit IRC20:56
*** derekh has joined #openstack-ironic20:56
*** anuradha1904 has quit IRC21:00
*** ociuhandu has joined #openstack-ironic21:04
*** derekh has quit IRC21:05
*** juanoterocas has joined #openstack-ironic21:06
*** ociuhandu has quit IRC21:07
juanoterocashi guys, could anyone here provide me a small connection example to neutron in ironic.config?21:08
*** hjensas|afk is now known as hjensas21:15
*** tosin has joined #openstack-ironic21:31
tosinTheJulia: about the node validate, should the validate parameter be a boolean?21:33
TheJuliatosin: yes21:33
tosinthank you21:34
TheJuliaeandersson: oh, whew21:34
TheJuliajuanoterocas: like from a production system?21:34
juanoterocasNo, test environment21:34
*** lbragstad has quit IRC21:35
TheJuliaunfortunately I don't have a running devstack handy :\21:36
eanderssonhttps://github.com/openstack/kolla-ansible/blob/master/ansible/roles/ironic/templates/ironic.conf.j2#L10221:36
eanderssonThis is a great example21:36
TheJulia\o/21:36
TheJuliaJayF: rloo: by chance do you run ironic in uwsgi?21:36
rlooI don't think so21:37
*** lbragstad has joined #openstack-ironic21:38
rlooTheJulia: checked. nope.21:40
TheJuliarloo: thanks21:40
*** juanoterocas has quit IRC21:43
*** derekh has joined #openstack-ironic21:50
*** derekh has quit IRC21:50
*** openstackgerrit has joined #openstack-ironic21:54
openstackgerritJulia Kreger proposed openstack/ironic master: CI: Collect a snapshot of network connections  https://review.opendev.org/c/openstack/ironic/+/77048121:54
TheJuliaeandersson: ^^21:54
*** rh-jelabarre has quit IRC22:04
*** nam-est has quit IRC22:16
eanderssonThanks!22:21
JayFTheJulia: Ironic is not capable of running under uwsgi AIUI22:33
JayFTheJulia: per documentation; there are issues running eventlet code under uwsgi ... so maybe in practice it might be OK; but we're sorta violating what the docs say we should do22:34
JayFwait, I might be mixing uwsgi and mod_wsgi22:40
eanderssonFor designate we had more issues with mod_wsgi than uwsgi at least22:41
eanderssonunless I am also mixing them up lol it's been a year22:42
*** ociuhandu has joined #openstack-ironic22:53
*** ociuhandu has quit IRC22:58
*** rcernin has joined #openstack-ironic23:02
*** rloo has quit IRC23:42

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!