Wednesday, 2020-01-29

*** yolanda has quit IRC00:06
*** yolanda has joined #openstack-ironic00:09
*** dtruong has quit IRC00:12
*** dtruong has joined #openstack-ironic00:12
*** k_mouza has joined #openstack-ironic00:17
*** k_mouza has quit IRC00:22
*** threestrands has joined #openstack-ironic00:50
*** billp_ has quit IRC00:53
*** jdandrea has quit IRC00:57
*** TxGirlGeek has quit IRC01:06
*** igordc has quit IRC01:23
*** Lucas_Gray has quit IRC01:26
*** tzumainn has quit IRC01:52
*** gyee has quit IRC01:55
*** hamzy has quit IRC01:56
*** hamzy has joined #openstack-ironic01:56
*** jrist has quit IRC01:57
*** zzzeek has quit IRC02:03
*** TxGirlGeek has joined #openstack-ironic02:04
*** rnoriega_ has joined #openstack-ironic02:05
*** zzzeek has joined #openstack-ironic02:08
*** zul has quit IRC02:12
*** rloo has quit IRC02:24
*** jrist has joined #openstack-ironic02:40
*** TxGirlGeek has quit IRC03:50
*** goldyfruit_ has quit IRC04:04
*** k_mouza has joined #openstack-ironic04:18
*** jdandrea has joined #openstack-ironic04:18
*** k_mouza has quit IRC04:22
*** mbeierl has quit IRC04:24
*** mbeierl has joined #openstack-ironic04:24
*** jdandrea has quit IRC04:41
*** TxGirlGeek has joined #openstack-ironic04:51
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Skip read-only devices  https://review.opendev.org/70472504:55
TheJuliadtantsur|afk: rpittau|afk fyi^^^04:55
*** TxGirlGeek has quit IRC05:12
*** TxGirlGeek has joined #openstack-ironic05:12
*** TxGirlGeek has quit IRC05:17
*** HagunKim has quit IRC05:32
*** hwoarang has quit IRC05:48
*** hwoarang has joined #openstack-ironic05:48
*** mmethot_ has joined #openstack-ironic06:17
*** k_mouza has joined #openstack-ironic06:18
*** mmethot has quit IRC06:20
*** k_mouza has quit IRC06:23
*** dsneddon has quit IRC06:45
*** dsneddon has joined #openstack-ironic06:45
arne_wiebalckGood morning, ironic!07:25
*** dtantsur|afk is now known as dtantsur07:54
dtantsurmorning ironic07:54
*** tesseract has joined #openstack-ironic08:03
*** rpittau|afk is now known as rpittau08:08
rpittaugood morning ironic! o/08:08
*** jtomasek has joined #openstack-ironic08:12
*** rcernin has quit IRC08:20
*** belmoreira has joined #openstack-ironic08:57
*** amoralej|off is now known as amoralej08:58
*** mbeierl1 has joined #openstack-ironic09:04
*** mbeierl has quit IRC09:04
*** mbeierl1 is now known as mbeierl09:04
*** dougsz has joined #openstack-ironic09:14
*** lucasagomes has joined #openstack-ironic09:16
*** iurygregory has joined #openstack-ironic09:40
iurygregorygood morning o/09:41
*** alexmcleod has joined #openstack-ironic09:47
*** threestrands has quit IRC09:47
rpittauhey iurygregory :)09:50
iurygregoryrpittau, o/09:51
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476610:06
*** derekh has joined #openstack-ironic10:13
*** lifeless has quit IRC10:19
openstackgerritMark Goddard proposed openstack/tenks master: WIP: CentOS 8 support  https://review.opendev.org/69588110:27
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476610:27
openstackgerritMark Goddard proposed openstack/tenks master: WIP: CentOS 8 support  https://review.opendev.org/69588110:43
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476610:43
openstackgerritGeorgy Karataev proposed openstack/bifrost master: (WIP) Fix the pre-installation scripts  https://review.opendev.org/70477610:51
*** pcaruana has quit IRC10:57
*** priteau has joined #openstack-ironic10:59
openstackgerritRiccardo Pittau proposed openstack/ironic master: Fix typo in setup-network.sh script  https://review.opendev.org/70477711:05
*** Lucas_Gray has joined #openstack-ironic11:16
*** pcaruana has joined #openstack-ironic11:40
*** rpittau is now known as rpittau|bbl11:53
*** ociuhandu has joined #openstack-ironic11:58
openstackgerritGeorgy Karataev proposed openstack/bifrost master: (WIP) Fix the pre-installation scripts  https://review.opendev.org/70477612:02
*** amoralej is now known as amoralej|lunch12:05
*** ociuhandu has quit IRC12:31
*** ociuhandu has joined #openstack-ironic12:40
*** jawad_axd has joined #openstack-ironic12:54
*** Lucas_Gray has quit IRC12:59
*** goldyfruit_ has joined #openstack-ironic13:01
*** amoralej|lunch is now known as amoralej13:11
*** rh-jelabarre has joined #openstack-ironic13:12
*** ociuhandu has quit IRC13:17
*** ociuhandu has joined #openstack-ironic13:21
*** ociuhandu has quit IRC13:26
dtantsurTheJulia, rpittau|bbl, please check https://review.opendev.org/#/c/704598/13:30
patchbotpatch 704598 - bifrost - Check out global requirements when creating test VMs - 3 patch sets13:30
dtantsurit may be blocking people13:30
*** rpittau|bbl is now known as rpittau13:30
*** khansa has joined #openstack-ironic13:36
*** rloo has joined #openstack-ironic13:36
openstackgerritMark Goddard proposed openstack/tenks master: WIP: CentOS 8 support  https://review.opendev.org/69588113:46
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476613:46
*** bobmel has joined #openstack-ironic13:46
*** jdandrea has joined #openstack-ironic13:46
*** Lucas_Gray has joined #openstack-ironic13:49
*** bobmel has quit IRC13:51
*** strigazi has quit IRC13:56
*** ociuhandu has joined #openstack-ironic14:00
*** strigazi has joined #openstack-ironic14:02
*** khansa has quit IRC14:03
openstackgerritMark Goddard proposed openstack/tenks master: WIP: CentOS 8 support  https://review.opendev.org/69588114:08
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476614:08
*** Lucas_Gray has quit IRC14:14
rpiosoGood morning14:15
TheJuliadtantsur: done14:22
dtantsurthx14:22
dtantsurand good morning TheJulia14:22
*** zzzeek has quit IRC14:22
*** zzzeek has joined #openstack-ironic14:23
*** gkaratae has joined #openstack-ironic14:30
gkarataehi, afaik centos 7 won't be supported in Ussuri, but can I ask why and what is prevents from it? :-)14:30
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Skip read-only devices  https://review.opendev.org/70472514:32
TheJuliagkaratae: python versions 3.6 through 3.8 are what is being tested.... where as the latest version of python for centos that was available is python 3.4. Realistically, Centos also defaults around python2, which many of the libraries and code paths that worked around py2/py3 differences have been removed meaning the code will no longer fire up on python2.14:37
dtantsurgkaratae: as Julia said, a lot of additional work for unclear benefit. Train is just as good for most cases.14:38
*** jawad_axd has quit IRC14:39
*** jawad_axd has joined #openstack-ironic14:39
openstackgerritMerged openstack/bifrost master: Check out global requirements when creating test VMs  https://review.opendev.org/70459814:43
*** jawad_ax_ has joined #openstack-ironic14:43
*** khansa has joined #openstack-ironic14:44
openstackgerritDmitry Tantsur proposed openstack/bifrost stable/train: Check out global requirements when creating test VMs  https://review.opendev.org/70482214:44
*** jawad_axd has quit IRC14:45
openstackgerritDmitry Tantsur proposed openstack/bifrost stable/stein: Check out global requirements when creating test VMs  https://review.opendev.org/70482314:45
*** jawad_ax_ has quit IRC14:48
*** priteau has quit IRC14:50
*** priteau has joined #openstack-ironic14:51
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476614:51
gkarataeTheJulia: thanks, so therefore the same applies for rhel 7x distributions if I understand right?14:54
dtantsurgkaratae: precisely14:55
dtantsurgkaratae: in your specific case, starting a new project on git master may not be a wise thing to do.14:55
dtantsurI'd go with stable/train and backport fixes for all issues you may encounter14:55
dtantsurmaster is VERY much in flux at this point14:55
*** priteau has quit IRC14:56
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent-builder master: [DNM] testing centos8 job  https://review.opendev.org/70482614:58
gkarataeyep, I already tried train, but in my case python3 is need in the future production, so train branch will not make much sense14:58
rpittaueven without considering the potential bug with libvirt in centos8....14:58
TheJuliagkaratae: do you somehow percieve train not working iwth python3?15:00
TheJuliaoh, for bifrost... yeah15:00
* TheJulia makes a sad face15:01
*** priteau has joined #openstack-ironic15:07
dtantsurgkaratae: if you need python 3, why talk about RHEL 7? it has only limited support there.15:09
*** gkaratae has quit IRC15:14
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476615:17
rpittaudtantsur: sorry for change of context, maybe you're aware already, but I'm afraid the centos8 job in ipa-B is broken :/15:18
dtantsurwell, wonderful15:18
dtantsurno, I'm not aware. do you have any ideas?15:18
rpittaudtantsur: https://654d83e20ff6115f815e-89f45418e048e284a0aa8c1a9bb175a2.ssl.cf1.rackcdn.com/704826/1/check/ironic-python-agent-check-image-dib-centos8/c27dde8/job-output.txt15:19
rpittau UnicodeEncodeError: 'ascii' codec can't encode characters in position 72-74: ordinal not in range(128)15:19
rpittauseems some python package screwed up15:19
*** khansa has quit IRC15:19
rpittauI was not able to reproduce locally though15:20
dtantsurrpittau: could you bring it to #openstack-dib?15:20
dtantsurI'm a bit overbooked15:20
rpittauof course, just wanted to check it with you first15:20
*** ociuhandu has quit IRC15:24
rpittauok I was able to reprdouce it locally now15:43
openstackgerritMerged openstack/ironic master: Fix typo in setup-network.sh script  https://review.opendev.org/70477715:45
*** cdearborn has joined #openstack-ironic15:47
*** gkaratae has joined #openstack-ironic15:47
gkarataedtantsur: rhel7 is one of our requirements (machines will run on it and switching to 8 is not planned yet)15:47
dtantsurgkaratae: then I guess stick with python 2 which will be supported by RH for a while?15:48
gkarataedtantsur: yes, you are true, I think I need to discuss this again with colleagues, now I'll play and test with Train, that should be enough for this moment15:49
gkarataethanks a lot for your reply15:50
*** ociuhandu has joined #openstack-ironic15:52
*** billp has joined #openstack-ironic16:07
openstackgerritMark Goddard proposed openstack/tenks master: DNM: Test libvirt-host role ansible_python  https://review.opendev.org/70476616:13
*** gyee has joined #openstack-ironic16:14
*** TxGirlGeek has joined #openstack-ironic16:25
TheJuliarpittau and any other core reviewers: https://review.opendev.org/#/c/704725/ would be greatly appreciated16:50
patchbotpatch 704725 - ironic-python-agent - Skip read-only devices - 2 patch sets16:50
dtantsurdone16:58
dtantsurTheJulia: probably worth backporting16:58
TheJuliaIndeed :(16:58
TheJuliaI'm _REALLY_ surprised this hsa not been hit before, but I was reading that HP changed the firmware to make it non-optional16:58
TheJuliaso people are going to run into it more :(16:58
*** ociuhandu has quit IRC16:59
*** bnemec-ooo has quit IRC17:00
*** bobmel has joined #openstack-ironic17:02
JayFdtantsur: TheJulia: I have a serious concern about the PR, commented on there17:06
TheJuliaI do too, tbh17:06
JayFcan you un-workflow it? I believe it's likely to introduce a security issue17:07
* TheJulia goes back to globally disabling cleaning in another project becuase of it17:07
JayFTheJulia: tl;dr I'm just worried that failing SSDs will be mounted r/o and show up in that list -- leading them to be excluded silently from cleaning17:07
TheJuliaJayF: commented17:09
TheJuliafailed SSDs should still be security locked17:09
TheJuliawhich means they won't even show as devices17:09
JayFif an ssd is security locked, it doesn't do a r/o fallback?17:09
JayFTIL17:09
TheJuliain theory, I mean, I'd have to brick an SSD to know for sure17:10
TheJuliabut basically it doesn't show as a funcctional disk at all17:10
JayFfor non-security-locked home-use ssds (this knowledge is old and dusty) it went into firmware r/o mode17:10
JayFwhich triggered linux to label it r/o after enough IO failures17:10
JayFI don't know if that particular proc flag would trigger for it, but that's the source of my concern17:10
* TheJulia unworkflows it17:11
JayFThis is my experience from ~10 years ago with gentoo, so very old and dusty17:11
* TheJulia goes back to paperwork17:11
JayFbut would have a big impact if that actually happened17:11
TheJuliathe case of IO failure is true, an that should be a hard failure17:11
TheJuliaI wish there was more information to identify these devices17:12
*** lucasagomes has quit IRC17:12
dtantsurI don't know what to do here. We cannot serious refuse to work in the presence of a read-only device..17:13
dtantsurespecially when the alternative is people disabling cleaning17:14
JayFI don't know what other options there are for that particular HP case, but failing "closed" (i.e. never provisioning a device which wasn't properly cleaned) is better than failing "open"17:14
JayFI wish I knew more about the hardware in question, I feel like there has to be a better indicator17:14
dtantsurTheJulia, JayF, I think we can at least allow metadata cleaning to succeed since it's not secure17:14
JayFor maybe even a hardware manager that changes the behavior if it's the expected HP model, to lower the surface17:14
JayF^^ probably a bad idea the more I think about it17:14
TheJuliafew actually do that because of the overhead and never really for general consumption17:15
dtantsurWe may assume that if somebody wants the secure cleaning, they'll find a way to remove "bad" devices17:15
JayFthat assumption broke, in production, in onmetal once17:15
TheJuliadtantsur: I concur on metadata... on full scrub I wonder if we should just fail hard17:15
JayFI like that compromise too, tbh17:15
dtantsurTheJulia: I think it's the reasonable first step. Maybe issue a warning first to help with debugging?17:16
JayFor even make it configurable, so someone can take that security risk if they want17:16
* TheJulia goes back to paperwork since backports are fun17:16
dtantsurTheJulia: if you don't have time today, I can update the patch first(ish) thing in the morning17:16
etingofmay be the best way of cleaning is dropping encryption key from an encrypted volume...?17:16
JayFSorry for being the fun police :(17:16
etingofyou can't fail open with that17:16
dtantsuretingof: won't help with read-only devices17:16
TheJuliadtantsur: I will have time today because this is the #1 priority atm17:17
dtantsurotherwise we already try ATA secure erase17:17
dtantsurk17:17
TheJuliaetingof: that would be entirely unrelated17:17
dtantsurJayF: your comments are as valuable as always!17:17
JayF<317:17
*** gkaratae has quit IRC17:17
JayFNow I'm thinking about how to force an SSD firmware to simulate write failures17:18
*** bobmel has quit IRC17:19
*** gkaratae has joined #openstack-ironic17:19
*** dtantsur is now known as dtantsur|afk17:19
dtantsur|afko/17:19
*** bobmel has joined #openstack-ironic17:19
etingofso my thinking is that clear text data once written onto a media can become unavailable for removal17:22
etingofdue to hardware failure or network disconnect or whatever17:22
etingofso the only way around seems to be not to place clear text data on the media17:23
JayFHere's the case I'm concerned about, this is a real-life example that thankfully didn't end up being a security issue because it was caught in our QE-server-booter:17:26
JayFIn the case of a failing SSD (in this case, it was one that was intermittantly working), you can provision a machine to person and when it boots, the data on that SSD is still there because it wasn't cleaned off17:26
JayFyou can absolutely have this happen without Ironic noticing, depending on how disks in the device are configured17:26
JayFin our case, we worked around that issue by having custom cleaning steps that asserted exactly the disk config expected (so if one was missing or r/o, we'd cleanfail)17:27
JayFOf course Ironic can't do anything about that failing disk if we can't clean it -- but we can make it fail cleaning with a reasonable error so that the operator knows the device needs human intervention17:27
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Use correct command for Python virtualenv  https://review.opendev.org/70485417:33
rpittaugood night! o/17:33
*** rpittau is now known as rpittau|afk17:33
*** amoralej is now known as amoralej|off17:34
*** TxGirlGeek has quit IRC17:45
*** dougsz has quit IRC17:57
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Skip read-only devices with metadata erase  https://review.opendev.org/70472518:03
*** derekh has quit IRC18:03
openstackgerritTzu-Mainn Chen proposed openstack/ironic-specs master: Create node lessee field  https://review.opendev.org/70106118:08
*** TxGirlGeek has joined #openstack-ironic18:09
*** priteau has quit IRC18:33
* etingof has a feeling that cliff does not really handle "nested" CLI commands18:46
*** benj_ has quit IRC18:47
*** benj_ has joined #openstack-ironic18:47
*** iurygregory has quit IRC18:52
*** tesseract has quit IRC18:59
*** igordc has joined #openstack-ironic19:02
TheJuliaetingof: afaik no19:33
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Skip read-only devices with metadata erase  https://review.opendev.org/70472519:41
* etingof wants to do `sushycli system power on` like things19:43
*** gmann is now known as gmann_afk19:57
*** lifeless has joined #openstack-ironic20:01
*** alexmcleod has quit IRC20:44
*** jawad_axd has joined #openstack-ironic20:52
*** jawad_axd has quit IRC20:56
*** Lucas_Gray has joined #openstack-ironic21:10
*** gmann_afk is now known as gmann21:17
*** gkaratae has quit IRC21:25
*** rcernin has joined #openstack-ironic21:32
*** stevebaker_ has quit IRC21:35
*** ociuhandu has joined #openstack-ironic22:01
*** ociuhandu has quit IRC22:06
*** jtomasek has quit IRC22:06
*** k_mouza has joined #openstack-ironic22:15
*** rh-jelabarre has quit IRC22:17
*** k_mouza has quit IRC22:19
*** jdandrea has quit IRC22:53
*** TxGirlGeek has quit IRC23:11
*** cdearborn has quit IRC23:17
*** TxGirlGeek has joined #openstack-ironic23:19
openstackgerritJulia Kreger proposed openstack/bifrost master: Disable inspection power-off  https://review.opendev.org/70446823:42

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!