| *** TxGirlGeek has quit IRC | 00:04 | |
| *** TxGirlGeek has joined #openstack-ironic | 00:07 | |
| *** dsneddon has quit IRC | 00:08 | |
| *** ianychoi_ has joined #openstack-ironic | 00:09 | |
| *** ianychoi has quit IRC | 00:11 | |
| *** jdandrea_ has quit IRC | 00:13 | |
| *** jdandrea has joined #openstack-ironic | 00:14 | |
| openstackgerrit | Merged openstack/ironic-python-agent-builder master: Add efivar https://review.opendev.org/701374 | 00:14 |
|---|---|---|
| openstackgerrit | Merged openstack/ironic master: Remove reference to deprecated [disk_utils]iscsi_verify_attempts https://review.opendev.org/701908 | 00:33 |
| *** jdandrea has quit IRC | 00:37 | |
| *** jdandrea has joined #openstack-ironic | 00:38 | |
| *** jdandrea has quit IRC | 00:41 | |
| *** jdandrea has joined #openstack-ironic | 00:43 | |
| openstackgerrit | Bob Fournier proposed openstack/ironic master: Don't require root partition when installing a whole disk image https://review.opendev.org/702293 | 00:46 |
| *** igordc has joined #openstack-ironic | 00:51 | |
| *** rh-jelabarre has quit IRC | 01:26 | |
| *** k_mouza has joined #openstack-ironic | 01:27 | |
| *** TxGirlGeek has quit IRC | 01:28 | |
| *** k_mouza has quit IRC | 01:32 | |
| *** gyee has quit IRC | 01:34 | |
| *** yedongcan has joined #openstack-ironic | 01:36 | |
| *** igordc has quit IRC | 01:45 | |
| *** rloo has quit IRC | 01:45 | |
| *** rh-jelabarre has joined #openstack-ironic | 03:22 | |
| *** k_mouza has joined #openstack-ironic | 04:18 | |
| *** rh-jelabarre has quit IRC | 04:20 | |
| *** k_mouza has quit IRC | 04:23 | |
| *** jhesketh has quit IRC | 04:23 | |
| *** jhesketh has joined #openstack-ironic | 04:24 | |
| *** tzumainn has quit IRC | 04:32 | |
| *** TxGirlGeek has joined #openstack-ironic | 04:52 | |
| *** jdandrea has quit IRC | 05:15 | |
| *** jdandrea has joined #openstack-ironic | 05:20 | |
| *** jdandrea has quit IRC | 05:36 | |
| *** ociuhandu has joined #openstack-ironic | 05:56 | |
| *** ociuhandu has quit IRC | 06:00 | |
| *** pcaruana has quit IRC | 06:17 | |
| *** jtomasek has joined #openstack-ironic | 06:32 | |
| *** bfournie has quit IRC | 06:32 | |
| *** yedongcan has quit IRC | 06:34 | |
| *** hamzy has quit IRC | 06:49 | |
| *** hamzy has joined #openstack-ironic | 06:50 | |
| *** TxGirlGeek has quit IRC | 07:05 | |
| *** tesseract has joined #openstack-ironic | 07:41 | |
| *** ociuhandu has joined #openstack-ironic | 07:52 | |
| *** ociuhandu has quit IRC | 07:52 | |
| *** rcernin has quit IRC | 07:52 | |
| *** ociuhandu has joined #openstack-ironic | 07:55 | |
| *** ricolin has quit IRC | 07:55 | |
| *** k_mouza has joined #openstack-ironic | 08:15 | |
| *** yedongcan has joined #openstack-ironic | 08:16 | |
| *** k_mouza has quit IRC | 08:20 | |
| openstackgerrit | Merged openstack/ironic-python-agent master: Allow reading root_device from instance_info https://review.opendev.org/701043 | 08:32 |
| *** iurygregory has joined #openstack-ironic | 08:37 | |
| iurygregory | good morning o/ | 08:37 |
| *** khansa has joined #openstack-ironic | 08:45 | |
| gudrutis21 | Morning ironic 0_/ | 08:52 |
| *** dmellado has quit IRC | 08:55 | |
| *** dmellado has joined #openstack-ironic | 08:56 | |
| *** pcaruana has joined #openstack-ironic | 08:56 | |
| *** lucasagomes has joined #openstack-ironic | 09:12 | |
| *** rpittau|afk is now known as rpittau | 09:16 | |
| rpittau | good morning ironic! o/ | 09:16 |
| *** k_mouza has joined #openstack-ironic | 09:18 | |
| *** dougsz has joined #openstack-ironic | 09:20 | |
| *** derekh has joined #openstack-ironic | 09:39 | |
| iurygregory | morning rpittau o/ | 09:46 |
| rpittau | hey iurygregory :) | 09:46 |
| *** alexmcleod has joined #openstack-ironic | 09:49 | |
| *** ociuhandu has quit IRC | 09:57 | |
| openstackgerrit | Merged openstack/bifrost master: Disable unbound in OpenDev CI https://review.opendev.org/702022 | 09:59 |
| *** ociuhandu has joined #openstack-ironic | 10:02 | |
| *** ociuhandu has quit IRC | 10:22 | |
| *** pcaruana has quit IRC | 10:23 | |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Correct bash syntax in test-bifrost script https://review.opendev.org/702390 | 10:31 |
| *** ociuhandu has joined #openstack-ironic | 10:35 | |
| *** yolanda has joined #openstack-ironic | 10:37 | |
| *** khansa has quit IRC | 10:59 | |
| *** pcaruana has joined #openstack-ironic | 11:02 | |
| *** khansa has joined #openstack-ironic | 11:04 | |
| *** rpittau is now known as rpittau|bbl | 11:11 | |
| *** Lucas_Gray has joined #openstack-ironic | 11:12 | |
| *** ociuhandu has quit IRC | 11:13 | |
| *** yedongcan has left #openstack-ironic | 11:15 | |
| *** ociuhandu has joined #openstack-ironic | 11:21 | |
| *** khansa has quit IRC | 11:25 | |
| *** khansa has joined #openstack-ironic | 12:01 | |
| *** dtantsur|afk is now known as dtantsur | 12:03 | |
| *** ociuhandu has quit IRC | 12:03 | |
| dtantsur | morning ironic | 12:04 |
| khansa | morning dtantsur o/ | 12:04 |
| iurygregory | morning dtantsur | 12:22 |
| *** Lucas_Gray has quit IRC | 12:23 | |
| *** Lucas_Gray has joined #openstack-ironic | 12:25 | |
| *** bfournie has joined #openstack-ironic | 12:29 | |
| *** ociuhandu has joined #openstack-ironic | 12:39 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Document using CentOS 8 DIB IPA images for Ussuri and newer https://review.opendev.org/702409 | 12:39 |
| *** ociuhandu has quit IRC | 12:43 | |
| openstackgerrit | Zygimantas Matonis proposed openstack/ironic-inspector master: Added scope to introspection rules. https://review.opendev.org/702215 | 12:57 |
| *** ociuhandu has joined #openstack-ironic | 13:03 | |
| *** rh-jelabarre has joined #openstack-ironic | 13:07 | |
| *** Lucas_Gray has quit IRC | 13:11 | |
| *** rloo has joined #openstack-ironic | 13:17 | |
| *** bfournie has quit IRC | 13:22 | |
| *** iurygregory has quit IRC | 13:23 | |
| *** iurygregory has joined #openstack-ironic | 13:23 | |
| *** rpittau|bbl is now known as rpittau | 13:26 | |
| *** jdandrea has joined #openstack-ironic | 13:52 | |
| *** jdandrea has quit IRC | 13:55 | |
| *** tkajinam has joined #openstack-ironic | 13:56 | |
| *** jdandrea has joined #openstack-ironic | 13:57 | |
| *** mbeierl has joined #openstack-ironic | 14:03 | |
| *** ociuhandu has quit IRC | 14:03 | |
| *** ociuhandu has joined #openstack-ironic | 14:05 | |
| *** iurygregory has quit IRC | 14:18 | |
| *** Lucas_Gray has joined #openstack-ironic | 14:27 | |
| TheJulia | good morning everyone | 14:29 |
| *** ociuhandu has quit IRC | 14:29 | |
| *** iurygregory has joined #openstack-ironic | 14:33 | |
| *** tzumainn has joined #openstack-ironic | 14:34 | |
| openstackgerrit | Merged openstack/ironic master: Document using CentOS 8 DIB IPA images for Ussuri and newer https://review.opendev.org/702409 | 14:35 |
| openstackgerrit | Julia Kreger proposed openstack/bifrost master: Switch to use a venv by default for testing https://review.opendev.org/701641 | 14:42 |
| dtantsur | morning TheJulia | 14:44 |
| rpittau | good morning TheJulia :) | 14:44 |
| rpioso | Good morning | 14:46 |
| TheJulia | So bifrost is close, just... packets seem to be ignored by dnsmasq on bionic | 14:46 |
| iurygregory | morning TheJulia | 14:46 |
| TheJulia | xenial appears to be detonating on installing pyyaml in the venv | 14:47 |
| rpittau | TheJulia: I had a look on bionic, I was not very lucky | 14:48 |
| TheJulia | it is basically in a case where it needs to be reproduced with tcpdump on hand | 14:48 |
| rpioso | derekh: I ran into the bug you fixed: https://review.opendev.org/#/c/698734/. Thank you! | 14:50 |
| patchbot | patch 698734 - ironic - Fix use of urlparse.urljoin (MERGED) - 2 patch sets | 14:50 |
| derekh | rpioso: your welcome :-) | 14:51 |
| rpioso | derekh: With that fix in place, the iDRAC seems to get a 403 Forbidden response when it posts the VirtualMedia.InsertMedia action. Did you encounter that? If so, how did you address it? | 14:51 |
| derekh | rpioso: I'm afraid that one doesn't sound familiar | 14:52 |
| *** vesper has quit IRC | 14:53 | |
| *** vesper11 has joined #openstack-ironic | 14:53 | |
| rpioso | derekh: Might I be missing some "common" web server config in local.conf or outside of it? | 14:53 |
| TheJulia | could it be an RBAC issue the bmc? Forbidden sounds like an access issue as opposed to input validation rejection | 14:54 |
| rpioso | derekh, TheJulia: I get the same when I visit the URL from my browser. | 14:55 |
| derekh | rpioso: is this a URL on the BMC or your ironic host? | 14:56 |
| TheJulia | URL for the image? | 14:56 |
| rpioso | derekh: It's created by ironic conductor and up by the devstack web server. | 14:57 |
| derekh | rpioso: I wasn't using devstack but sounds like something needs to be configured differently on your webserver | 14:58 |
| rpioso | TheJulia: Roughly .../redfish/boot-<UUID>?filename=whatever.iso | 14:58 |
| derekh | rpioso: are the file permissions ok ? | 14:58 |
| rpioso | derekh: Good question. Earlier in my testing, ironic was denied writing the iso to the directory. I chmod +w it. Not sure if that's a best practice, but, hey, it's devstack, so ... | 15:00 |
| TheJulia | hmm | 15:00 |
| TheJulia | rpioso: is selinux enabled? | 15:01 |
| * rpioso wonders if the devstack web server environment just works or needs config via local.conf or external means. | 15:01 | |
| dtantsur | TheJulia: there was some story around TFTP (?) connection trackers that derekh solved for devstack | 15:04 |
| dtantsur | can it affect bifrost now? | 15:04 |
| TheJulia | dtantsur: likely, but we're not even getting an IP address | 15:05 |
| TheJulia | inbound packet counter on virbr0 is ~20 packets, 0 transmitted | 15:05 |
| dtantsur | cd ironic && git log devstack --grep bionic | 15:05 |
| *** mbeierl has quit IRC | 15:06 | |
| *** ociuhandu has joined #openstack-ironic | 15:06 | |
| dtantsur | do we switch the network driver to e1000? | 15:06 |
| rpioso | TheJulia: Well, sestatus is not installed. Can I assume it's not enabled? | 15:06 |
| * iurygregory saw a patch to switch to e1000 | 15:07 | |
| rpioso | TheJulia: And should it be enabled for devstack? | 15:07 |
| iurygregory | dtantsur, https://github.com/openstack/bifrost/commit/e88031dd7838042b7b6e358027d312dce3bef6c2 =) | 15:07 |
| dtantsur | mmmm | 15:08 |
| *** tkajinam has quit IRC | 15:10 | |
| derekh | some details on the conntrack problem here (I havn't been following along so it mightn't be relevant) http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001451.html | 15:10 |
| *** ociuhandu has quit IRC | 15:11 | |
| rpittau | dtantsur: the workaround for the tftp traffic was done here -> https://opendev.org/openstack/ironic/commit/da966c9873535f52c75858f543af2a321b30543f | 15:20 |
| *** bfournie has joined #openstack-ironic | 15:27 | |
| etingof | rpioso, can you access that file under www-data (?) user? | 15:31 |
| TheJulia | rpioso: sestatus... I'd check proc/kernel log for signs of selinux running. Devstack, afaik, has no hooks or knowledge of it | 15:32 |
| etingof | rpioso, getenforce talls selinux status | 15:32 |
| etingof | rpioso, smth like this should work -- sudo -u www-data cat /var/www/html/redfish/boot-*.iso > /dev/null | 15:33 |
| iurygregory | TheJulia,do we want a CI job testing efibootmgr workflow? | 15:39 |
| rpioso | etingof: No such www-data user; however, drwx-w-r-t 2 stack stack 4096 Jan 14 02:08 /opt/stack/data/ironic/httpboot/redfish | 15:41 |
| *** ociuhandu has joined #openstack-ironic | 15:42 | |
| rpioso | etingof: -rw------- 1 stack stack 504457216 Jan 13 18:58 /opt/stack/data/ironic/httpboot/redfish/boot-76d582a5-656b-4892-8b2f-27518fa96560 | 15:42 |
| rpioso | etingof: And recall that I had to chmod +w the .../httpboot/redfish directory to enable ironic to write the boot files there. | 15:43 |
| *** Goneri has joined #openstack-ironic | 15:44 | |
| etingof | rpioso, will it work if you chmod 755 /opt/stack/data/ironic/httpboot/redfish ? | 15:49 |
| rpioso | etingof: I'll try that over the next few hours. | 15:51 |
| rpioso | derekh, etingof, TheJulia: Thank you, all! | 15:52 |
| *** Lucas_Gray has quit IRC | 15:57 | |
| *** TxGirlGeek has joined #openstack-ironic | 16:02 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-prometheus-exporter master: DevStack support and Redfish job https://review.opendev.org/685888 | 16:12 |
| arne_wiebalck | Is anyone actively using the extra_hardware collector for inspection? | 16:13 |
| arne_wiebalck | Or is aware of someone using it? | 16:13 |
| dtantsur | arne_wiebalck: we're using it both in tripleo and metal3 | 16:14 |
| dtantsur | I'm not sure which percentage of customers actually rely on the collected data | 16:15 |
| arne_wiebalck | dtantsur: from what I see, this is basically a dump of hardware-detect | 16:15 |
| dtantsur | it is, yes | 16:16 |
| arne_wiebalck | dtantsur: if we wanted to collect more data, should that go into 'data' as well or should there be a new collector/elelment? | 16:16 |
| dtantsur | arne_wiebalck: I think you either extend the underlying python-hardware library or write your own collector with your own destination. | 16:17 |
| dtantsur | what's your use case? | 16:17 |
| * rpittau sipping tea and following | 16:17 | |
| arne_wiebalck | dtantsur: I don't have a concrete example at hand, but it would be sth like extended the information about a memory module for instance | 16:17 |
| arne_wiebalck | dtantsur: we're looking at the missing delta atm | 16:18 |
| arne_wiebalck | dtantsur: to see what we're missing | 16:18 |
| dtantsur | if the extra data from python-hardware contains, say, 80% of what you need, it probably makes sense to extend it to cover the remaining 20 | 16:18 |
| dtantsur | if it's only 10%, a new collector may be more useful | 16:19 |
| arne_wiebalck | dtantsur: you may remember that together with Oath we created a list of things that would be sensible to collect | 16:19 |
| arne_wiebalck | dtantsur: right | 16:19 |
| dtantsur | for the hardware inventory service? | 16:19 |
| arne_wiebalck | yes | 16:19 |
| dtantsur | yeah, I seem to recall | 16:19 |
| arne_wiebalck | lol | 16:19 |
| dtantsur | I suggest checking what python-hardware supports. It collects thousands of facts on a real hardware. | 16:20 |
| dtantsur | (and rpittau keeps adding more! ;) | 16:20 |
| arne_wiebalck | the model has changed, but the idea of letting ironic getting the data is still alive | 16:20 |
| rpittau | well for now I keep removing stuff :P | 16:20 |
| dtantsur | you're removing python 2 stuff, but last time you touched the data you added NVMe (?) support | 16:20 |
| arne_wiebalck | rpittau: you maintain that lib? | 16:20 |
| rpittau | ah yeah, right :D | 16:20 |
| rpittau | arne_wiebalck: I'm a maintainer of that lib | 16:21 |
| arne_wiebalck | uuh, nice | 16:21 |
| dtantsur | arne_wiebalck: it's mostly rpittau, a bit of tonyb[m] and myself and another non-OpenStack person | 16:21 |
| * arne_wiebalck was just telling his co-workers that changing the lib is probably not an option | 16:21 | |
| * arne_wiebalck has now a different opinion | 16:22 | |
| tonyb[m] | Patches welcome! | 16:22 |
| dtantsur | extending it is very much an option | 16:22 |
| rpittau | arne_wiebalck: any change/update/help is very well appreciated :) | 16:22 |
| dtantsur | tonyb[m]: hey, isn't it the middle of the night for you? | 16:22 |
| tonyb[m] | Just don't break the API/user contract | 16:22 |
| arne_wiebalck | ok, let's see once we have the delta! | 16:22 |
| tonyb[m] | dtantsur: I'm in Brno | 16:22 |
| arne_wiebalck | thanks dtantsur rpittau tonyb[m] ! | 16:23 |
| dtantsur | tonyb[m]: oh, lovely! Have a Zebra for me :) | 16:23 |
| *** bobmel has joined #openstack-ironic | 16:23 | |
| tonyb[m] | dtantsur: zebra? | 16:23 |
| rpittau | mmm zebra... | 16:24 |
| tonyb[m] | Clearly I'm missing something | 16:24 |
| dtantsur | tonyb[m]: pork ribs | 16:24 |
| dtantsur | so it's not quite Zebra, but I cannot find the right symbol on my keyboard | 16:24 |
| tonyb[m] | Ahh from anywhere in particular? | 16:25 |
| iurygregory | pork ribs *-* | 16:25 |
| dtantsur | Pegas, I guess? | 16:25 |
| iurygregory | Pegas ++ | 16:25 |
| tonyb[m] | Okay | 16:25 |
| dtantsur | tonyb[m]: iurygregory can keep you a company? | 16:25 |
| dtantsur | :) | 16:25 |
| tonyb[m] | iurygregory: oh yeah? | 16:25 |
| iurygregory | if he needs =) | 16:25 |
| iurygregory | tonyb[m], I'm based in Brno | 16:26 |
| tonyb[m] | iurygregory: what are you doing for dinner tomorrow? | 16:26 |
| * dtantsur is connecting great people, nice! | 16:26 | |
| iurygregory | tonyb[m], no plans yet | 16:26 |
| tonyb[m] | iurygregory: pork ribs @ pegas? | 16:27 |
| iurygregory | tonyb[m], sure o/ | 16:27 |
| tonyb[m] | iurygregory: I'll be in the office tomorrow so we can make a plan then | 16:27 |
| tonyb[m] | iurygregory: invite whomever you want | 16:28 |
| iurygregory | tonyb[m], ack o/ | 16:28 |
| *** hamzy has quit IRC | 16:33 | |
| *** gyee has joined #openstack-ironic | 16:42 | |
| rpittau | good night! o/ | 17:00 |
| *** lucasagomes has quit IRC | 17:00 | |
| *** rpittau is now known as rpittau|afk | 17:01 | |
| iurygregory | strange CI gave +1 in https://review.opendev.org/#/c/685888/ but I see https://zuul.opendev.org/t/openstack/build/28e002db7a364b019a1440e6bf4921aa/log/job-output.txt#34472 | 17:06 |
| patchbot | patch 685888 - ironic-prometheus-exporter - DevStack support and Redfish job - 18 patch sets | 17:06 |
| iurygregory | D: | 17:06 |
| clarkb | iurygregory: its a string/name problem | 17:13 |
| iurygregory | clarkb, I'll check my bash script XD | 17:13 |
| clarkb | devstack@ironic-prometheus-exporter.service vs devstack@ironic_prometheus_exporter.service | 17:14 |
| clarkb | the first one is what is actually in systemd if I read that log correclty. The second is what it looks for at your link | 17:14 |
| iurygregory | tks! | 17:15 |
| *** ociuhandu_ has joined #openstack-ironic | 17:20 | |
| *** ociuhandu has quit IRC | 17:24 | |
| *** ociuhandu_ has quit IRC | 17:24 | |
| *** ociuhandu has joined #openstack-ironic | 17:26 | |
| *** ociuhandu has quit IRC | 17:28 | |
| *** hamzy has joined #openstack-ironic | 17:33 | |
| *** alexmcleod has quit IRC | 17:39 | |
| *** k_mouza has quit IRC | 17:40 | |
| rpioso | etingof: That didn't fix the issue. | 17:45 |
| rpioso | etingof: I changed the directory's mode: | 17:45 |
| rpioso | drwxr-xr-x 2 stack stack 4096 Jan 14 17:37 /opt/stack/data/ironic/httpboot/redfish | 17:46 |
| rpioso | etingof: The new boot-<UUID> created does not have read access for the world. | 17:46 |
| rpioso | -rw------- 1 stack stack 504457216 Jan 14 02:08 /opt/stack/data/ironic/httpboot/redfish/boot-b5f2add3-a169-41f3-850a-fbb8a6fc4b6f | 17:47 |
| *** dougsz has quit IRC | 17:56 | |
| *** derekh has quit IRC | 18:00 | |
| etingof | rpioso, on the other hand, may be stack ownership is sufficient? can you curl that file? | 18:02 |
| *** k_mouza has joined #openstack-ironic | 18:05 | |
| *** k_mouza has quit IRC | 18:09 | |
| rpioso | etingof: curl gets the same response, 403 Forbidden :-( | 18:13 |
| etingof | rpioso, if you chmod 644 that file, will curl work? | 18:13 |
| rpioso | etingof: That works | 18:22 |
| etingof | rpioso, a quick hack would be to add os.chmod() call before the image file is published... -- https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/redfish/boot.py#L470 | 18:31 |
| etingof | rpioso, alternatively, you can probably run http server under stack? | 18:32 |
| *** dtantsur is now known as dtantsur|afk | 18:32 | |
| dtantsur|afk | o/ | 18:33 |
| *** bobmel has quit IRC | 18:47 | |
| rpioso | etingof: Got by that one \o/ | 18:59 |
| etingof | rpioso, \o/ which one of those two? | 19:00 |
| *** bobmel has joined #openstack-ironic | 19:00 | |
| rpioso | etingof: VirtualMedia.InsertMedia receives a 204 response. | 19:01 |
| etingof | rpioso, right, how did you get by? | 19:01 |
| rpioso | etingof: I implemented the os.chmod() work around. | 19:01 |
| rpioso | etingof: How would I run the http server under stack? Is that a local.conf setting? | 19:02 |
| rpioso | etingof: Isn't it "owned" by devstack? | 19:02 |
| etingof | rpioso, I need to dig in, I do not know off the top of my head. My guess would be that it's up to systemd unit file... | 19:03 |
| rpioso | etingof: Understandable :-) Doesn't devstack install and/or configure the web server to suit its needs/ | 19:04 |
| rpioso | etingof: On to the next layer of the onion: iDRAC Redfish set boot device failed for node 8d5ad1ad-92dd-40c7-8054-1e9583d6d11d, because system 4c4c4544-0034-5010-8059-b4c04f484b32 has no manager which could. | 19:04 |
| rpioso | etingof: I previously pip -e installed sushy-oem-idrac | 19:05 |
| etingof | perhaps devstack could have a setting for http ownership, I do not know if it actually does. That might complicate things... | 19:05 |
| etingof | rpioso, I wonder how derekh got by this httpd access nuisance... | 19:06 |
| rpioso | etingof: I suspect he's testing on RDO. | 19:06 |
| etingof | but that's still devstack, no? | 19:06 |
| rpioso | etingof: Nope. More like OSP. | 19:07 |
| rpioso | etingof: It uses tripleo. | 19:07 |
| etingof | ah, not rdocloud! I see | 19:07 |
| etingof | so http there must have run under the same user as ironic then? | 19:08 |
| rpioso | etingof: I don't know | 19:08 |
| rpioso | etingof: sushy-oem-idrac issue ^^ ? | 19:08 |
| *** bobmel has quit IRC | 19:09 | |
| etingof | as for the new issue - try to inspect idrac document tree to make sure that the system of interest links a manager which has a virtual media device | 19:09 |
| etingof | that's the resolution order that ironic uses: system -> manager -> vmedia cd | 19:09 |
| etingof | you can see relevant JSONs in ironic debugging | 19:10 |
| etingof | or just curl them up | 19:10 |
| *** dsneddon has joined #openstack-ironic | 19:15 | |
| *** pcaruana has quit IRC | 19:16 | |
| rpioso | etingof: The system of interest does link a manager with virtual media devices, including /redfish/v1/Managers/iDRAC.Embedded.1/VirtualMedia/CD | 19:17 |
| etingof | is it /redfish/v1/Systems/4c4c4544-0034-5010-8059-b4c04f484b32 ? | 19:18 |
| rpioso | etingof: Not exactly. That's the UUID of the system. | 19:21 |
| etingof | is not the system addressable that way? | 19:22 |
| rpioso | etingof: Nope. I just tried it. The page can't be found. | 19:22 |
| * rpioso wonders if he correctly installed sushy-oem-idrac. | 19:23 | |
| etingof | sushy-oem-idrac is the most reliable piece of software ever! it has zero knobs and switches | 19:24 |
| rpioso | etingof: LOL | 19:24 |
| etingof | I'd check out ironic/sushy logs to see how ironic collects system document tree | 19:24 |
| etingof | that's where it gets hold of the manager | 19:25 |
| rpioso | etingof: False alarm. | 19:29 |
| rpioso | etingof: It tripped over the lacking Location header issue derekh encountered in December-ish. | 19:30 |
| rpioso | etingof: I'll change the sleep interval and hopefully profit. | 19:31 |
| etingof | heh | 19:31 |
| *** gmann is now known as gmann_afk | 19:36 | |
| TheJulia | iurygregory: whole disk image local boot uefi and it should be the default I think | 19:38 |
| rpioso | etingof: We appear to have ignition. The server is powering on \o/ | 19:55 |
| rpioso | etingof: Ooh ooh ... "Virtual CD Boot Requested by iDRAC" is displayed on the system console. | 19:57 |
| rpioso | etingof: There was an error briefly displayed on the console. Something like "Failed to allocate initrd. Hit any key to continue". I did _not_ respond. Instead, I waited. Now the console is dark, except for a blinking cursor. The node's provisioning state remains wait call-back. | 20:01 |
| *** hamzy_ has joined #openstack-ironic | 20:08 | |
| etingof | rpioso, this has something to do with boot image... | 20:10 |
| *** hamzy has quit IRC | 20:11 | |
| etingof | rpioso, there is a long discussion on similar problem, but w/o clear resolution -- https://bugzilla.redhat.com/show_bug.cgi?id=1608955 | 20:13 |
| openstack | bugzilla.redhat.com bug 1608955 in kernel "EFI stub: ERROR: Failed to alloc highmem for initrd" [Urgent,Closed: errata] - Assigned to lszubowi | 20:13 |
| etingof | rpioso, other sources refer to exceptionally large initrd | 20:14 |
| etingof | rpioso, could it be related to EFI boot loader version you are using? may be try to upgrade (or downgrade) relevant packages....? | 20:15 |
| rpioso | etingof: The deployment failed due to timeout. | 20:17 |
| rpioso | etingof: I'm trying boot_option:local. Did you, too? | 20:18 |
| etingof | rpioso, I understand you can't boot IPA ramdisk. that's the first step, if it fails, there is no hope | 20:19 |
| etingof | the immediate goal is to have ramdisk booted | 20:20 |
| rpioso | etingof: I previously saw the error message above while testing plain old PXE boot. Those deployments got beyond it. | 20:20 |
| rpioso | etingof: Understood. | 20:20 |
| etingof | the Dell machine I've been using booted the ramdisk that devstack prepared | 20:21 |
| etingof | so I am suspecting EFI tools - may be the versions you are using have some issues...? | 20:21 |
| etingof | other sources refer to some missing RAID drivers in the kernel, however I never had this problem with my Dell | 20:22 |
| rpioso | etingof: I believe mine is also booting from the ramdisk that devstack prepared. | 20:22 |
| rpioso | This devstack successfully deployed the same baremetal server via PXE boot and iSCSI deploy. That was a baseline test before trying vmedia. | 20:23 |
| rpioso | etingof: Aren't the EFI tools installed by devstack? | 20:25 |
| *** hamzy__ has joined #openstack-ironic | 20:43 | |
| *** hamzy_ has quit IRC | 20:46 | |
| *** tesseract has quit IRC | 20:50 | |
| *** hamzy_ has joined #openstack-ironic | 20:51 | |
| *** hamzy__ has quit IRC | 20:53 | |
| etingof | that makes me suspecting efi tools... | 20:55 |
| etingof | rpioso, EFI fw should be installed by devstack -- https://github.com/openstack/ironic/blob/master/devstack/files/debs/ironic#L26 | 20:57 |
| *** Anandhi has joined #openstack-ironic | 21:00 | |
| etingof | rpioso, also perhaps grub-efi | 21:01 |
| *** jtomasek has quit IRC | 21:01 | |
| *** hamzy__ has joined #openstack-ironic | 21:02 | |
| *** hamzy_ has quit IRC | 21:04 | |
| rpioso | etingof: So update or install edk2-ovmf and grub-efi on my devstack Fedora Bionic VM? | 21:06 |
| etingof | I am now thinking, grub-efi is important | 21:07 |
| etingof | rpioso, EFI system partition is created by grub toolchain and uploaded to glance for ironic to pull and burn into boot ISO -- https://github.com/openstack/ironic/blob/master/devstack/lib/ironic#L2600 | 21:07 |
| Anandhi | TheJulia: Thank you for your response to my question related to ironic discovery. I believe that the configuration file option "node_not_found_hook" should be set for autodiscovery according to theDiscovery section of this link: https://docs.openstack.org/ironic-inspector/queens/user/usage.html#discovery | 21:08 |
| Anandhi | https://docs.openstack.org/ironic-inspector/queens/user/usage.html#discovery | 21:08 |
| etingof | rpioso, I've been using 2.02-2ubuntu8.14 successfully with Dell hw | 21:08 |
| Anandhi | Sorry, pasted link multiple times. But I don't see this option, node-not-found-hook in ironic inspector config guide - https://docs.openstack.org/ironic-inspector/queens/configuration/ironic-inspector.html | 21:09 |
| *** hamzy_ has joined #openstack-ironic | 21:10 | |
| rpioso | 2.02-2ubuntu8.14 ? | 21:11 |
| etingof | rpioso, ovmf is intended for libvirt, I am not sure if grub-efi pulls it as a dependency | 21:11 |
| rpioso | etingof: ^^ | 21:11 |
| etingof | yes | 21:11 |
| rpioso | etingof: My devstack VM is running Ubuntu 18.04.3 LTS. | 21:11 |
| etingof | rpioso, I have the same ubuntu running devstack, grub-efi package version is reported as 2.02-2ubuntu8.14 (apt show grub-efi) | 21:13 |
| * rpioso realizes he mistakenly identified Ubuntu Bionic as Fedora Bionic earlier | 21:13 | |
| *** hamzy__ has quit IRC | 21:13 | |
| rpioso | etingof: Same version is displayed on mine -- 2.02-2ubuntu8.14 | 21:15 |
| rpioso | etingof: A error started appearing in the conductor log, perhaps after I updated the ironic repo to pick up derekh's bug fix: ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.InternalError) (1054, "Unknown column 'allocations.owner' in 'field list'") | 21:18 |
| rpioso | etingof: It's emitted when the conductor starts up. | 21:18 |
| rpioso | etingof: Should I update all of the other repos in my devstack? They date back to early December-ish. | 21:19 |
| rpioso | etingof: I did not restack after git pulling ironic. | 21:20 |
| etingof | that error appears as being caused by changed ORM->DB mapping | 21:20 |
| etingof | so either sqlalchemy migration or ironic respin should cure it... though your booting problem is irrelevant to this | 21:21 |
| rpioso | I don't know if I should restack after updating checkouts in one or more repos. | 21:21 |
| rpioso | etingof: Yeah, I thought it's orthogonal. | 21:22 |
| *** hamzy has joined #openstack-ironic | 21:22 | |
| rpioso | etingof: Did you extend the deploy timeout? | 21:22 |
| rpioso | etingof: Was the system console dark, except for a blinking cursor, during the boot of the ramdisk? | 21:23 |
| etingof | yes, it was dark for a couple of minutes | 21:23 |
| *** hamzy_ has quit IRC | 21:23 | |
| etingof | at that time I've seen idrac pulling boot image in httpd access log | 21:23 |
| rpioso | etingof: It was easily a couple of minutes. | 21:24 |
| etingof | did it boot in the end? | 21:24 |
| rpioso | etingof: I fished the iDRAC vmedia device resource during the boot and confirmed the device was connected to the devstack web server URL. | 21:25 |
| rpioso | etingof: I don't believe it did. deploy powers it off on failure, so I'm not absolutely certain. | 21:26 |
| rpioso | etingof: What did you see when it booted? Output on the console? | 21:26 |
| rpioso | etingof: Where's the httpd access log? | 21:26 |
| rpioso | etingof: I don't see anything obvious in /var/log/apache2/access.log. | 21:30 |
| etingof | rpioso, there should be linux kernel log on the console during boot | 21:32 |
| rpioso | etingof: I didn't see that, but I may have been looking at something else. | 21:32 |
| etingof | rpioso, re httpd log location - look at httpd configs in /etc/apache2 or /etc/http or something like that | 21:33 |
| rpioso | etingof: I think I found the logs of interest in ipxe_access.log, which seems strange. | 21:35 |
| etingof | well, may be that's alright | 21:36 |
| etingof | the logging is configured with ipxe http boot in mind? | 21:36 |
| * etingof -> sleep | 21:39 | |
| rpioso | etingof: First GET -- 2020-01-14 19:56:35.886, last GET 2020-01-14 20:04:45.243 of the boot-<UUID> file | 21:40 |
| rpioso | etingof: Sleep well | 21:40 |
| *** hamzy_ has joined #openstack-ironic | 21:50 | |
| etingof | that looks consistent with my Dell booting experience | 21:51 |
| rpioso | etingof: I'll give it another try while you're resting. | 21:52 |
| etingof | so the image was probably pulled by the node... | 21:52 |
| etingof | you can sum up all GETs size-wise to make sure the entire image gets pulled | 21:52 |
| *** Anandhi has quit IRC | 21:53 | |
| *** hamzy has quit IRC | 21:53 | |
| *** bfournie has quit IRC | 21:53 | |
| rpioso | etingof Was the last one smaller than the rest, like 4096, instead of 131072? | 21:53 |
| etingof | yes, unless image size is divisible by chunk size ;) | 21:54 |
| rpioso | etingof: There was like a six minute gap between the last chunk and next to last one. | 21:55 |
| rpioso | etingof: 6.5 minutes. | 21:55 |
| *** hamzy_ has quit IRC | 21:58 | |
| *** hamzy has joined #openstack-ironic | 21:59 | |
| openstackgerrit | Zane Bitter proposed openstack/ironic master: Fix API docs for target_power_state response https://review.opendev.org/702542 | 22:02 |
| *** hamzy_ has joined #openstack-ironic | 22:06 | |
| *** rcernin has joined #openstack-ironic | 22:08 | |
| *** hamzy has quit IRC | 22:09 | |
| openstackgerrit | khansa proposed openstack/sushy-cli master: Add .gitreview file https://review.opendev.org/702545 | 22:22 |
| openstackgerrit | khansa proposed openstack/sushy-cli master: add power command https://review.opendev.org/702546 | 22:22 |
| *** khansa has quit IRC | 22:23 | |
| *** bfournie has joined #openstack-ironic | 22:45 | |
| *** gmann_afk is now known as gmann | 22:53 | |
| *** igordc has joined #openstack-ironic | 22:58 | |
| *** igordc has quit IRC | 23:01 | |
| *** igordc has joined #openstack-ironic | 23:01 | |
| *** igordc has quit IRC | 23:02 | |
| *** igordc has joined #openstack-ironic | 23:03 | |
| *** igordc has quit IRC | 23:06 | |
| *** igordc has joined #openstack-ironic | 23:07 | |
| openstackgerrit | Akihiro Motoki proposed openstack/ironic-ui stable/train: Fix horizon dependency https://review.opendev.org/701685 | 23:08 |
| *** igordc has quit IRC | 23:09 | |
| *** igordc has joined #openstack-ironic | 23:10 | |
| *** igordc has quit IRC | 23:11 | |
| *** igordc has joined #openstack-ironic | 23:12 | |
| *** igordc has quit IRC | 23:14 | |
| *** igordc has joined #openstack-ironic | 23:15 | |
| *** hwoarang has quit IRC | 23:15 | |
| *** igordc has quit IRC | 23:19 | |
| *** hwoarang has joined #openstack-ironic | 23:22 | |
| *** ociuhandu has joined #openstack-ironic | 23:30 | |
| *** ociuhandu has quit IRC | 23:35 | |
| *** hamzy_ has quit IRC | 23:38 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!