| *** erbarr has quit IRC | 00:30 | |
| *** gyee has quit IRC | 00:38 | |
| *** ociuhandu has joined #openstack-ironic | 00:42 | |
| *** ijw has joined #openstack-ironic | 00:46 | |
| *** ociuhandu has quit IRC | 00:49 | |
| *** ijw has quit IRC | 00:51 | |
| kaifeng | TheJulia: wrt the inspector scope, I am ok to do it with a follow up, but patch owner doesn't respond for a while, so I am going to unvote. | 01:02 |
|---|---|---|
| kaifeng | TheJulia: ack, will do a rebase this week, I suspect we don't need additional work since the token is checked before command execution. | 01:08 |
| *** ijw has joined #openstack-ironic | 01:18 | |
| *** ijw has quit IRC | 01:23 | |
| *** ijw has joined #openstack-ironic | 01:49 | |
| *** mmethot has quit IRC | 01:53 | |
| *** mmethot has joined #openstack-ironic | 01:53 | |
| *** ijw has quit IRC | 01:54 | |
| *** yaawang has quit IRC | 01:56 | |
| *** yaawang has joined #openstack-ironic | 01:57 | |
| openstackgerrit | Verification of a change to openstack/ironic-python-agent failed: Rescan after restarting the md device https://review.opendev.org/706632 | 02:05 |
| *** ijw has joined #openstack-ironic | 02:20 | |
| *** ijw has quit IRC | 02:25 | |
| *** tzumainn has quit IRC | 02:50 | |
| *** ijw has joined #openstack-ironic | 02:52 | |
| *** mkrai has joined #openstack-ironic | 02:55 | |
| *** ijw has quit IRC | 02:56 | |
| *** ricolin_ has joined #openstack-ironic | 03:00 | |
| *** mmethot_ has joined #openstack-ironic | 03:05 | |
| *** mmethot has quit IRC | 03:07 | |
| *** yaawang has quit IRC | 03:08 | |
| *** yaawang has joined #openstack-ironic | 03:09 | |
| *** rh-jelabarre has quit IRC | 03:10 | |
| *** diurnalist has joined #openstack-ironic | 03:17 | |
| *** diurnalist has quit IRC | 03:22 | |
| *** ijw has joined #openstack-ironic | 03:23 | |
| *** ijw has quit IRC | 03:28 | |
| *** ijw has joined #openstack-ironic | 03:54 | |
| *** ijw has quit IRC | 03:59 | |
| *** yaawang has quit IRC | 04:24 | |
| *** yaawang has joined #openstack-ironic | 04:25 | |
| *** ijw has joined #openstack-ironic | 04:26 | |
| *** ijw has quit IRC | 04:31 | |
| *** tkajinam has quit IRC | 04:33 | |
| *** tkajinam has joined #openstack-ironic | 04:34 | |
| *** chenhaw has quit IRC | 04:49 | |
| *** ijw has joined #openstack-ironic | 04:58 | |
| *** ijw has quit IRC | 05:02 | |
| *** ricolin_ has quit IRC | 05:05 | |
| *** ricolin_ has joined #openstack-ironic | 05:05 | |
| *** ricolin_ has quit IRC | 05:05 | |
| openstackgerrit | Noor Muhammad Malik proposed openstack/ironic master: WIP: [RFE] Implement the bios-interface for idrac-wsman driver https://review.opendev.org/713450 | 05:09 |
| *** michchap has joined #openstack-ironic | 05:17 | |
| *** TxGirlGeek has quit IRC | 05:26 | |
| *** k_mouza has joined #openstack-ironic | 05:32 | |
| *** k_mouza has quit IRC | 05:36 | |
| *** early has quit IRC | 05:50 | |
| *** early has joined #openstack-ironic | 05:51 | |
| *** ijw has joined #openstack-ironic | 06:00 | |
| *** ijw has quit IRC | 06:04 | |
| arne_wiebalck | Good morning, ironic! | 06:25 |
| arne_wiebalck | kaifeng: TheJulia: re inspector scope, we will pick this up as soon as we can, gudrutis is not available atm | 06:26 |
| openstackgerrit | Mahendra Kamble proposed openstack/ironic master: DRAC : idrac-redfish inspect updates pxe port https://review.opendev.org/693328 | 06:30 |
| *** ijw has joined #openstack-ironic | 06:31 | |
| *** ijw has quit IRC | 06:35 | |
| *** ociuhandu has joined #openstack-ironic | 06:48 | |
| *** ociuhandu has quit IRC | 06:52 | |
| *** ijw has joined #openstack-ironic | 07:02 | |
| *** ijw has quit IRC | 07:07 | |
| *** mkrai has quit IRC | 07:33 | |
| *** ijw has joined #openstack-ironic | 07:34 | |
| dtantsur|afk | etingof: w00t! | 07:34 |
| dtantsur|afk | morning ironic | 07:34 |
| *** dtantsur|afk is now known as dtantsur | 07:34 | |
| *** ijw has quit IRC | 07:38 | |
| *** tesseract has joined #openstack-ironic | 07:50 | |
| etingof | dtantsur, o/ | 07:54 |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: Be moderate about disk space in the CI https://review.opendev.org/713586 | 07:55 |
| *** mkrai has joined #openstack-ironic | 07:58 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: Be moderate about disk space in the CI https://review.opendev.org/713586 | 08:00 |
| iurygregory | good morning ironic | 08:02 |
| *** mbeierl has quit IRC | 08:03 | |
| *** mbeierl has joined #openstack-ironic | 08:04 | |
| *** ijw has joined #openstack-ironic | 08:05 | |
| *** ijw has quit IRC | 08:10 | |
| *** yoctozepto9 has joined #openstack-ironic | 08:14 | |
| *** yoctozepto has quit IRC | 08:15 | |
| *** yoctozepto9 is now known as yoctozepto | 08:15 | |
| *** tkajinam has quit IRC | 08:18 | |
| *** rcernin has quit IRC | 08:21 | |
| *** amoralej|off is now known as amoralej | 08:24 | |
| *** Lucas_Gray has joined #openstack-ironic | 08:30 | |
| *** ijw has joined #openstack-ironic | 08:37 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Raise human-friendly messages on attempt to use pre-deploy steps drivers https://review.opendev.org/713594 | 08:39 |
| *** ijw has quit IRC | 08:42 | |
| *** ociuhandu has joined #openstack-ironic | 08:45 | |
| *** rpittau|afk is now known as rpittau | 08:45 | |
| rpittau | good morning ironic! o/ | 08:45 |
| *** lucasagomes has joined #openstack-ironic | 08:48 | |
| *** yoctozepto has quit IRC | 08:49 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: DNM test with cirros 0.5.1 https://review.opendev.org/712728 | 08:49 |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: Be moderate about disk space in the CI https://review.opendev.org/713586 | 08:53 |
| *** strigazi has joined #openstack-ironic | 08:57 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Use python3 binary for all operations https://review.opendev.org/713477 | 08:57 |
| *** Lucas_Gray has quit IRC | 08:58 | |
| *** yoctozepto has joined #openstack-ironic | 08:59 | |
| *** Lucas_Gray has joined #openstack-ironic | 09:00 | |
| *** ijw has joined #openstack-ironic | 09:09 | |
| *** iurygregory has quit IRC | 09:12 | |
| *** ociuhandu has quit IRC | 09:12 | |
| *** ociuhandu has joined #openstack-ironic | 09:12 | |
| *** ijw has quit IRC | 09:14 | |
| *** iurygregory has joined #openstack-ironic | 09:14 | |
| iurygregory | morning rpioso o/ | 09:14 |
| iurygregory | ops rpittau o/ | 09:14 |
| rpittau | hey iurygregory :) | 09:15 |
| iurygregory | rpittau, just wondering if you are ok with keeping "The default port to run the application is `9608`." after my explanation https://review.opendev.org/#/c/713274/1/doc/source/run.rst | 09:16 |
| patchbot | patch 713274 - ironic-prometheus-exporter - Add documentation - 1 patch set | 09:16 |
| iurygregory | before I push a new patch set =) | 09:17 |
| rpittau | iurygregory: if there's nothing in the code that set the port to 9608, then that phrase needs to be changed anyway | 09:18 |
| iurygregory | rpittau, maybe jsut say " The prometheus community defined the por for the ironic prometheus exporter as 9608 (see link), but you can choose any port for your deployments" | 09:20 |
| rpittau | iurygregory: that sounds better, my only doubt in the current version is that it seems like it will run on port 9608 by default, but instead you'll have to specify a port | 09:21 |
| iurygregory | rpittau, in devstack yes https://github.com/openstack/ironic-prometheus-exporter/blob/09f802160ec6e62be0aa16d4078369e985394cd3/devstack/plugin.sh#L5 | 09:22 |
| rpittau | iurygregory: it doesn't mention devstack in the docs | 09:23 |
| openstackgerrit | Merged openstack/ironic-python-agent master: Rescan devices before performing uefi checks https://review.opendev.org/706667 | 09:24 |
| iurygregory | yeah, my point in the docs was just to tell people that there is a port that should be used according to the prometheus community | 09:25 |
| rpittau | iurygregory: yep, then the change you proposed should be fine :) | 09:25 |
| iurygregory | people are free to choose the port they want since they will deploy using gunicorn/uwsgi etc | 09:25 |
| iurygregory | ack | 09:25 |
| *** derekh has joined #openstack-ironic | 09:33 | |
| *** ijw has joined #openstack-ironic | 09:40 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-prometheus-exporter master: Add documentation https://review.opendev.org/713274 | 09:42 |
| *** alexmcleod has joined #openstack-ironic | 09:43 | |
| *** ijw has quit IRC | 09:45 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent stable/train: Rescan devices before performing uefi checks https://review.opendev.org/713601 | 09:46 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent stable/stein: Rescan devices before performing uefi checks https://review.opendev.org/713602 | 09:47 |
| *** mmethot has joined #openstack-ironic | 09:52 | |
| *** mmethot_ has quit IRC | 09:54 | |
| * rpittau can't focus today | 09:54 | |
| dtantsur | ditto | 09:56 |
| dtantsur | etingof: we may need your help on https://review.opendev.org/#/c/712728/. the uefi-vmedia job refuses to pass, tempest cannot locate an ephemeral partition. | 09:56 |
| patchbot | patch 712728 - ironic - DNM test with cirros 0.5.1 - 5 patch sets | 09:56 |
| dtantsur | this is the only job that shows this problem, so dunno. somehow related to virtual media? | 09:57 |
| etingof | let me see | 09:57 |
| *** Lucas_Gray has quit IRC | 09:59 | |
| *** yoctozepto has quit IRC | 09:59 | |
| dtantsur | I have a lot of potential explanations, but none of them explains why only this job is affected.. | 10:00 |
| *** Lucas_Gray has joined #openstack-ironic | 10:00 | |
| *** mkrai_ has joined #openstack-ironic | 10:02 | |
| *** mkrai has quit IRC | 10:02 | |
| *** yoctozepto has joined #openstack-ironic | 10:03 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: Rescan after restarting the md device https://review.opendev.org/706632 | 10:05 |
| *** ociuhandu has quit IRC | 10:09 | |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 10:09 |
| etingof | dtantsur, is it that /dev/sda1 is not mounted? | 10:10 |
| dtantsur | etingof: probably? | 10:10 |
| *** ociuhandu has joined #openstack-ironic | 10:10 | |
| etingof | that's what tempest says | 10:10 |
| dtantsur | well, yes, it probably means that | 10:10 |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: [DNM] Test run inspector after ironic-api https://review.opendev.org/713608 | 10:10 |
| *** ijw has joined #openstack-ironic | 10:11 | |
| etingof | but it's there at least on the console: sda1 8:1 1073741824 ephemeral0 | 10:13 |
| *** yoctozepto2 has joined #openstack-ironic | 10:13 | |
| *** yoctozepto has quit IRC | 10:13 | |
| *** yoctozepto2 is now known as yoctozepto | 10:13 | |
| *** ociuhandu has quit IRC | 10:15 | |
| *** ijw has quit IRC | 10:16 | |
| etingof | is that the right host to ssh to - 172.24.5.250 ? on the console the eth0 seems to be on 10.1.0.27,26 | 10:19 |
| *** ociuhandu has joined #openstack-ironic | 10:21 | |
| *** ociuhandu has quit IRC | 10:27 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: Be moderate about disk space in the CI https://review.opendev.org/713586 | 10:28 |
| dtantsur | etingof: SSH commands before that command did work, so dunno | 10:38 |
| dtantsur | (incl the one that looks up the partition with label "ephemeral0") | 10:39 |
| etingof | so it can't be some other host, right? ;) | 10:39 |
| dtantsur | unlikely :) | 10:39 |
| dtantsur | and, as any idea that I can think of, it wouldn't explain why only this job is affected | 10:40 |
| etingof | tempest seems to check for mounted filesystem, not for partition... | 10:40 |
| dtantsur | I mean, the command before that | 10:41 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-prometheus-exporter master: Add documentation https://review.opendev.org/713274 | 10:42 |
| dtantsur | it does smth like `lsblk | grep ephemeral0` and learns the device name | 10:42 |
| dtantsur | then checks it in mounts | 10:42 |
| dtantsur | so, it correctly determines /dev/sda1, but then something goes sideways | 10:42 |
| *** ijw has joined #openstack-ironic | 10:42 | |
| etingof | yeah, the partition seems to be present, but somehow not mounted... | 10:44 |
| *** ijw has quit IRC | 10:47 | |
| *** khansa has joined #openstack-ironic | 10:50 | |
| *** jawad_axd has joined #openstack-ironic | 10:50 | |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 10:50 |
| mgoddard | hi rpittau, mind taking a look at https://review.opendev.org/#/c/713504 ? It fixes the kolla bifrost image | 10:55 |
| patchbot | patch 713504 - bifrost - Remove --ignore-installed from CentOS 8 pip 19 ins... - 1 patch set | 10:55 |
| rpittau | mgoddard: sure | 10:56 |
| rpittau | mmm interesting | 10:58 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost master: DNM: Testing pip > 19 on CentOS 8 https://review.opendev.org/713626 | 10:58 |
| dtantsur | I don't think --ignore-installed makes a lot of sense with precise versions anyway | 10:59 |
| *** k_mouza has joined #openstack-ironic | 11:04 | |
| etingof | weird: my xterm crashes when I run that tempest line -- set -eu -o pipefail; PATH=$PATH:/sbin:/usr/sbin; mount | grep -w '/dev/sda1' | cut -d' ' -f3 | 11:09 |
| etingof | perhaps it's the pipeline option | 11:09 |
| etingof | yes, that's a way to catch grep miss | 11:10 |
| *** ijw has joined #openstack-ironic | 11:14 | |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Fix dhcp job https://review.opendev.org/712304 | 11:15 |
| etingof | the only difference that vmedia brings on the table is that vmedia CD stays inserted... | 11:15 |
| *** tesseract-RH has joined #openstack-ironic | 11:16 | |
| *** tesseract has quit IRC | 11:18 | |
| *** ociuhandu has joined #openstack-ironic | 11:19 | |
| *** Wryhder has joined #openstack-ironic | 11:22 | |
| *** Lucas_Gray has quit IRC | 11:23 | |
| *** Wryhder is now known as Lucas_Gray | 11:23 | |
| openstackgerrit | Merged openstack/bifrost master: Remove --ignore-installed from CentOS 8 pip 19 install https://review.opendev.org/713504 | 11:29 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 11:29 |
| *** Lucas_Gray has quit IRC | 11:32 | |
| *** Lucas_Gray has joined #openstack-ironic | 11:34 | |
| *** Lucas_Gray has quit IRC | 11:35 | |
| *** ociuhandu has quit IRC | 11:35 | |
| *** Lucas_Gray has joined #openstack-ironic | 11:39 | |
| *** k_mouza has quit IRC | 11:40 | |
| *** k_mouza has joined #openstack-ironic | 11:44 | |
| *** mkrai_ has quit IRC | 11:46 | |
| *** Lucas_Gray has quit IRC | 11:47 | |
| etingof | dtantsur, with some other jobs, that partition is on /dev/vda1, is it because of different kernels? | 11:47 |
| *** Lucas_Gray has joined #openstack-ironic | 11:49 | |
| *** diurnalist has joined #openstack-ironic | 11:49 | |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 11:50 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-prometheus-exporter master: Add documentation https://review.opendev.org/713274 | 11:52 |
| *** rh-jelabarre has joined #openstack-ironic | 11:53 | |
| *** diurnalist has quit IRC | 11:54 | |
| *** rpittau is now known as rpittau|bbl | 12:01 | |
| *** Goneri has joined #openstack-ironic | 12:16 | |
| *** ijw has quit IRC | 12:17 | |
| dtantsur | etingof: oh, interesting! no, it's because of a different virtual drive type (scsi/sata vs virtio) | 12:21 |
| openstackgerrit | Verification of a change to openstack/ironic failed: Refactoring: split out wrap_ipv6 https://review.opendev.org/713457 | 12:25 |
| *** k_mouza has quit IRC | 12:32 | |
| openstackgerrit | Merged openstack/ironic master: Documentation clarifications for software RAID https://review.opendev.org/713404 | 12:40 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 12:44 |
| *** ijw has joined #openstack-ironic | 12:46 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/train: Documentation clarifications for software RAID https://review.opendev.org/713645 | 12:46 |
| *** k_mouza has joined #openstack-ironic | 12:47 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: Be moderate about disk space in the CI https://review.opendev.org/713586 | 12:47 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: [WIP] Raise human-friendly messages on attempt to use pre-deploy steps drivers https://review.opendev.org/713594 | 12:47 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Raise human-friendly messages on attempt to use pre-deploy steps drivers https://review.opendev.org/713594 | 12:48 |
| *** ijw has quit IRC | 12:50 | |
| *** amoralej is now known as amoralej|lunch | 12:53 | |
| *** rpittau|bbl is now known as rpittau | 13:01 | |
| *** sziviani_ has joined #openstack-ironic | 13:04 | |
| *** tesseract has joined #openstack-ironic | 13:05 | |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 13:07 |
| etingof | ah, right | 13:07 |
| *** tesseract-RH has quit IRC | 13:08 | |
| *** Lucas_Gray has quit IRC | 13:08 | |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 13:10 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: Run vbmcd as a Systemd service https://review.opendev.org/713648 | 13:10 |
| *** Lucas_Gray has joined #openstack-ironic | 13:12 | |
| *** Lucas_Gray has quit IRC | 13:16 | |
| *** ijw has joined #openstack-ironic | 13:17 | |
| *** Lucas_Gray has joined #openstack-ironic | 13:18 | |
| *** ijw has quit IRC | 13:21 | |
| iurygregory | dtantsur, about the policy for the stable branches, something like this http://paste.openstack.org/show/790849/ ? | 13:29 |
| *** rloo has joined #openstack-ironic | 13:29 | |
| dtantsur | iurygregory: nice! two nits: turn extended maintenance into a link to some openstack doc on subj, and "projects don't" | 13:30 |
| * dtantsur dives into the IPMI specification, sorry | 13:31 | |
| iurygregory | dtantsur, ack | 13:31 |
| etingof | dtantsur, with vmedia, scsi bus gets a cdrom device (strangely, the same lun as hdd?). but hdd seems to stay sda -- http://paste.openstack.org/show/790850/ | 13:32 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Workaround for missing adjtime file https://review.opendev.org/713651 | 13:33 |
| dtantsur | because cdrom's are named /dev/sr0 etc | 13:33 |
| *** ociuhandu has joined #openstack-ironic | 13:34 | |
| etingof | indeed. also interesting, that with vmedia this does not seem to happen: info: copying initramfs to /dev/sda3 | 13:35 |
| *** ociuhandu has quit IRC | 13:38 | |
| TheJulia | kaifeng: I was thinking they likely saw the +2 and didn't look at it again :( | 13:41 |
| *** k_mouza has quit IRC | 13:43 | |
| *** tzumainn has joined #openstack-ironic | 13:43 | |
| *** amoralej|lunch is now known as amoralej | 13:46 | |
| *** k_mouza has joined #openstack-ironic | 13:48 | |
| *** khansa has quit IRC | 13:48 | |
| *** ijw has joined #openstack-ironic | 13:48 | |
| openstackgerrit | Merged openstack/ironic-python-agent stable/train: Rescan devices before performing uefi checks https://review.opendev.org/713601 | 13:52 |
| openstackgerrit | Merged openstack/ironic-python-agent stable/stein: Rescan devices before performing uefi checks https://review.opendev.org/713602 | 13:53 |
| *** ijw has quit IRC | 13:53 | |
| *** jdandrea has joined #openstack-ironic | 13:55 | |
| dtantsur | morning TheJulia | 14:01 |
| *** diurnalist has joined #openstack-ironic | 14:02 | |
| *** bfournie has joined #openstack-ironic | 14:03 | |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Fix dhcp job https://review.opendev.org/712304 | 14:04 |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Fix dhcp job https://review.opendev.org/712304 | 14:04 |
| *** Lucas_Gray has quit IRC | 14:04 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Document our policies for stable branches https://review.opendev.org/713660 | 14:05 |
| dtantsur | my desire to burn IPMI with fire has substantially increased after carefully reading a part of its spec.. | 14:06 |
| iurygregory | can you quote the part that increased your desire dtantsur ? | 14:06 |
| dtantsur | iurygregory: you can pick any, really. I'm on 22.15. | 14:07 |
| * iurygregory gives a glass with whisky to dtantsur | 14:08 | |
| dtantsur | thanks | 14:08 |
| *** yoctozepto1 has joined #openstack-ironic | 14:12 | |
| *** yoctozepto has quit IRC | 14:13 | |
| *** yoctozepto1 is now known as yoctozepto | 14:13 | |
| dtantsur | okay, it seems that jjohnson2 will be so kind to look into the pyghmi issue for me. maybe I'll keep some of my sanity today! | 14:16 |
| *** kaifeng has quit IRC | 14:17 | |
| TheJulia | dtantsur: good morning | 14:19 |
| *** ijw has joined #openstack-ironic | 14:19 | |
| TheJulia | iurygregory: make that a bottle of whiskey... He may need to distill some for disinfectant | 14:19 |
| * iurygregory is not sure if a bottle of whiskey in one day is a good idea.. maybe for 1 month? XD | 14:21 | |
| *** khansa has joined #openstack-ironic | 14:22 | |
| * iurygregory grabs the bottle and gives to dtantsur | 14:22 | |
| dtantsur | yay! the 3rd bottle of whiskey in this house :) | 14:23 |
| iurygregory | I thought you had more bottles tbh o.o | 14:24 |
| *** ijw has quit IRC | 14:24 | |
| iurygregory | like 5 | 14:24 |
| dtantsur | I don't drink much nowadays.. | 14:24 |
| michchap | dtantsur: have you done a centos devstack build recently? The ironic_lib_prefix this gets https://review.opendev.org/#/c/684270/7/devstack/lib/ironic I think should be /usr/local for centos but it's getting /usr and failing for me | 14:25 |
| patchbot | patch 684270 - ironic - Drop rootwrap.d/ironic-lib.filters file (MERGED) - 7 patch sets | 14:25 |
| dtantsur | michchap: I haven't devstacked in a while | 14:25 |
| michchap | dtantsur: I don't know the packaging process well enough, I'm not sure how it decides where to put data so the best I can do is have a hardcoded fix for cent | 14:26 |
| dtantsur | honestly, I'm not sure either | 14:26 |
| * dtantsur checks the site module | 14:26 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Revamp grenade jobs https://review.opendev.org/713176 | 14:28 |
| openstackgerrit | Merged openstack/bifrost stable/train: Run vbmcd as a Systemd service https://review.opendev.org/713648 | 14:28 |
| michchap | dtantsur: actually I could use pip show to find it perhaps | 14:31 |
| dtantsur | michchap: maybe we're overthinking it and should just try two locations: in /usr/local and /usr | 14:31 |
| dtantsur | it's devstack, after all, it doesn't have to be super robust | 14:32 |
| michchap | dtantsur: can do, thanks! | 14:32 |
| dtantsur | patches are warmly welcome | 14:32 |
| TheJulia | ++ | 14:32 |
| TheJulia | plus side of the apocolypse, reason to make french toast using the left over bread from last nigth | 14:33 |
| dtantsur | heh | 14:34 |
| dtantsur | I'll probably go out to catch the sun while I'm allowed to | 14:34 |
| TheJulia | ++ | 14:34 |
| TheJulia | we shelter in place orders popping up all around us | 14:34 |
| TheJulia | the only virtue by where we live is none of the orders apply because we're between them all. | 14:35 |
| clarkb | the shelter in place rules for the bay area do allow you to go outside and do things like exercise fwiw | 14:36 |
| TheJulia | yeah, same here | 14:36 |
| etingof | I think here we are allowed to retreat back to the woods | 14:39 |
| TheJulia | go live in the woods with the bears and fishes? | 14:39 |
| TheJulia | fishies | 14:39 |
| etingof | or caves, previously occupied by the Neanderthals | 14:40 |
| TheJulia | to run 3 phase power to and begin to make into a data center? | 14:41 |
| iurygregory | here we can only go out if we have something covering the face (mask, scarf, whatever) | 14:41 |
| iurygregory | it's mandatory XD | 14:42 |
| dtantsur | we're still on a light version of quarantine (and it's probably bad because people don't give a damn) | 14:42 |
| rpittau | I didn't read very good things about how Germany is handling this :/ | 14:43 |
| dtantsur | but I like the ability to stretch my legs a bit (not on the balcony) | 14:43 |
| TheJulia | So, about those redfish/ | 14:43 |
| TheJulia | ? | 14:43 |
| dtantsur | rpittau: it's weird. we have a very low deaths number, maybe that's why the government thinks they're handling it. | 14:43 |
| dtantsur | etingof: JFYI Neantherthal is a super lovely place | 14:44 |
| dtantsur | * Neanderthal | 14:44 |
| dtantsur | It's 15 minutes by train from here, I'd totally visit it were it not for all this.. | 14:44 |
| TheJulia | etingof: on https://review.opendev.org/#/c/713315/1 what does "cb" stand for? | 14:45 |
| patchbot | patch 713315 - sushy - Add `Connector(..., response_cb=None)`` parameter - 1 patch set | 14:45 |
| TheJulia | oh! | 14:45 |
| TheJulia | nevermind | 14:45 |
| TheJulia | callback | 14:45 |
| TheJulia | etingof: response_callback would likely be better, I'll +2+a if you want to change it | 14:46 |
| dtantsur | Neanderthal: https://www.instagram.com/p/B5Qc6GjiN4w/ https://www.instagram.com/p/B5TXPB_CUwV/ | 14:46 |
| TheJulia | etingof: Also, have you heard back from dell folks on the latest sushy fixes? | 14:47 |
| *** dtantsur is now known as dtantsur|brb | 14:48 | |
| iurygregory | dtantsur|brb, the place looks amazing! | 14:48 |
| *** ijw has joined #openstack-ironic | 14:51 | |
| *** mkrai has joined #openstack-ironic | 14:52 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: Be moderate about disk space in the CI https://review.opendev.org/713586 | 14:53 |
| *** ijw has quit IRC | 14:55 | |
| etingof | TheJulia, sure, let be rename it | 14:56 |
| etingof | TheJulia, I do not think I've heard anything from Dell people in that regard so far | 14:56 |
| rpittau | mmm I see something weird happening sometimes with rabbitmq on ironic-inspector-api, seems issues with rabbitmq | 14:57 |
| iurygregory | TheJulia, just fyi the action item from the meeting https://review.opendev.org/#/c/713660/ | 14:58 |
| patchbot | patch 713660 - ironic - Document our policies for stable branches - 1 patch set | 14:58 |
| etingof | without trains, dtantsur|brb can just walk up to Neanderthal by bare foot like in good old times | 14:58 |
| *** priteau has joined #openstack-ironic | 14:59 | |
| rpittau | TheJulia: the thing you pointed out in https://review.opendev.org/672179 for the bifrost job, seems to be fixed starting inspector after ironic-api https://review.opendev.org/713608 | 15:00 |
| patchbot | patch 672179 - ironic-inspector - Use openstacksdk for ironic module - 57 patch sets | 15:00 |
| patchbot | patch 713608 - bifrost - [DNM] Test run inspector after ironic-api - 1 patch set | 15:00 |
| rpittau | I'm wondering if auth_type=none is causing that when using openstacksdk, as I'm having the same issue when running the functional tests | 15:02 |
| etingof | TheJulia, the whole purpose of that callback patch is to enable this - https://review.opendev.org/#/c/713418/ | 15:02 |
| patchbot | patch 713418 - sushy-cli - Add `--show-traffic` feature to aid Redfish diagno... - 1 patch set | 15:02 |
| TheJulia | rpittau: awesome | 15:06 |
| TheJulia | etingof: reading the comments, it feels like we're getting mixed signals | 15:07 |
| * etingof is not sure what TheJulia is referring to here | 15:08 | |
| TheJulia | the comment how using continious was more about mapping ipmi functionality and how it seems problematic with uefi impmenentations out there | 15:08 |
| TheJulia | sincec we're not explicitly looking at and leveraging the newer uefi boot item list | 15:08 |
| *** TxGirlGeek has joined #openstack-ironic | 15:11 | |
| *** dougsz has joined #openstack-ironic | 15:14 | |
| *** ijw has joined #openstack-ironic | 15:22 | |
| openstackgerrit | Noor Muhammad Malik proposed openstack/ironic master: WIP: [RFE] Implement the bios-interface for idrac-wsman driver https://review.opendev.org/713450 | 15:25 |
| *** ijw has quit IRC | 15:26 | |
| *** jawad_axd has quit IRC | 15:45 | |
| *** ijw has joined #openstack-ironic | 15:53 | |
| *** jcoufal has joined #openstack-ironic | 15:53 | |
| openstackgerrit | Noor Muhammad Malik proposed openstack/ironic master: WIP: [RFE] Implement the bios-interface for idrac-wsman driver https://review.opendev.org/713450 | 15:57 |
| openstackgerrit | Ilya Etingof proposed openstack/sushy master: Add `Connector(..., response_callback=None)`` parameter https://review.opendev.org/713315 | 15:59 |
| *** ijw has quit IRC | 16:00 | |
| *** mkrai has quit IRC | 16:02 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Do not use random to generate token https://review.opendev.org/713687 | 16:06 |
| *** dking_desktop has joined #openstack-ironic | 16:07 | |
| *** ijw has joined #openstack-ironic | 16:10 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-python-agent master: Use secrets to generate salt https://review.opendev.org/713690 | 16:15 |
| *** khansa has quit IRC | 16:20 | |
| openstackgerrit | Noor Muhammad Malik proposed openstack/ironic master: WIP: [RFE] Implement the bios-interface for idrac-wsman driver https://review.opendev.org/713450 | 16:23 |
| openstackgerrit | Ilya Etingof proposed openstack/sushy master: Lazily load message registries https://review.opendev.org/713512 | 16:25 |
| *** khansa has joined #openstack-ironic | 16:26 | |
| *** k_mouza has quit IRC | 16:27 | |
| *** diurnalist has quit IRC | 16:29 | |
| *** k_mouza has joined #openstack-ironic | 16:30 | |
| openstackgerrit | Ilya Etingof proposed openstack/sushy master: Add `Connector(..., response_callback=None)`` parameter https://review.opendev.org/713315 | 16:32 |
| openstackgerrit | Ilya Etingof proposed openstack/sushy master: Lazily load message registries https://review.opendev.org/713512 | 16:33 |
| openstackgerrit | Ilya Etingof proposed openstack/sushy-cli master: Add `--show-traffic` feature to aid Redfish diagnostics https://review.opendev.org/713418 | 16:33 |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Fix dhcp job https://review.opendev.org/712304 | 16:35 |
| *** diurnalist has joined #openstack-ironic | 16:35 | |
| etingof | dtantsur|brb, for sake of performance, should we initiate redfish session auth only if basic auth does not work? | 16:45 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 16:46 |
| etingof | otherwise we are repeating these (usually unsuccessful) session negotiation on every redfish api call | 16:47 |
| TheJulia | etingof: so the reason FOR session is some bmcs take basic auth as session creation | 16:47 |
| etingof | you need to got for session explicitly, right? | 16:48 |
| etingof | I propose to first attempt to do whatever you want with basic auth, and only divert to getting a session if that first request fails | 16:48 |
| etingof | what I currently observe with sushy-cli against idrac is that every request goes through always failing session negotiation | 16:49 |
| TheJulia | sesison and then fallback was the only way to not cause the bmc to lockup or refuse new requests | 16:50 |
| etingof | even failing session? | 16:51 |
| etingof | may be a mid-ground then would be to make it configurable in ironic e.g. basicauth or session...? | 16:53 |
| dking_desktop | TheJulia: You seem knowledgeable and have been very helpful. I have a rather complicated issue. Would you mind if I send you a private message to pick your brain a bit? | 16:56 |
| *** tesseract has quit IRC | 16:59 | |
| TheJulia | dking_desktop: I'd prefer to keep such discussion in the open, unless it concerns a vulnerability or other "fun" issues | 16:59 |
| TheJulia | Also, I'm crazy busy at times | 16:59 |
| iurygregory | dking_desktop, you can share the question here in the irc so everyone can help (if they have an idea) | 16:59 |
| *** lucasagomes has quit IRC | 17:01 | |
| TheJulia | etingof: if session, I think we should be falling back. A specific preference is likely a good middle ground but that is definitely something that is your milage may vary | 17:03 |
| dking_desktop | Okay, I'll give it a shot. I'm interested in trying to find a way to setup Ironic so that it can be used by parties who are not trusted with access to the switches or the BMC network directly. | 17:05 |
| *** dsneddon_ has joined #openstack-ironic | 17:05 | |
| openstackgerrit | Merged openstack/ironic master: Refactoring: split out wrap_ipv6 https://review.opendev.org/713457 | 17:07 |
| TheJulia | dking_desktop: sounds like why ironic was designed :) | 17:08 |
| dking_desktop | And they also need to be able to have access to tenant networks. | 17:09 |
| TheJulia | dking_desktop: what needs access to the tenant networks? | 17:09 |
| *** dtantsur|brb is now known as dtantsur | 17:09 | |
| dking_desktop | I would like the final baremetal servers to be able to be part of tenant networks. | 17:09 |
| TheJulia | dking_desktop: you need the neutron network interface in that case | 17:10 |
| TheJulia | https://docs.openstack.org/ironic/latest/admin/multitenancy.html | 17:10 |
| TheJulia | Although if your not using neutron, some more work may be required as a new network interface | 17:11 |
| TheJulia | we _DO_ have a data contraint in terms of the network ID value that can be saved for a vif, but we're happy to remove that constraint logic as we've already agreed to do so as a community once we have another interface that may do something like vlan number, but that is more a non-openstack use case | 17:11 |
| dking_desktop | As I understand it, it seems that in order to allow baremetal servers to connect to the tenant networks, Ironic has to use local_link_connection to connect directly to the switches. Also, Ironic itself needs to have access to the BMC network. I would not want the servers these customers can access directly to have that ability. | 17:11 |
| dking_desktop | And also, there would be multiple such customers sharing the same switches. | 17:12 |
| dtantsur | dking_desktop: you mean, you don't want ironic to have access to BMCs/switches or users using ironic? | 17:13 |
| dking_desktop | The users using ironic. | 17:13 |
| TheJulia | what acess do you think they have? | 17:14 |
| TheJulia | I think we have a perception disconnect | 17:14 |
| iurygregory | only operators would have access switches/bmcs ... | 17:15 |
| dking_desktop | Well, the way I have things setup at the moment is pretty stock, and Ironic is on my controller. That controller has to be able to at least connect to the BMC, but if I want to use tenant networks, then it also has to connect to the switch. I don't want the controllers to have that sort of access, since there will be many such users sharing the same switch. | 17:16 |
| dtantsur | I'm not sure how to work around it, to be honest. If you want a tool to manage networks, it needs access to switches.. | 17:18 |
| dking_desktop | Correct, there will have to be. I just want that tool to be shared somehow, and access restricted. I'm pretty sure there's not a simple solution. I've been trying to think of what way may be the less difficult to create. | 17:19 |
| TheJulia | dking_desktop: Why is neutron's ml2 plugin model not acceptable ? | 17:19 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 17:22 |
| dking_desktop | TheJulia: Ultimately, that will probably be what we use. However, whatever server is running that would have access to the switch to make changes, and I want to limit that access. Also, I want to limit access to the BMC network. | 17:22 |
| TheJulia | Realistically you have to establish controls for that your comfortable iwth | 17:23 |
| *** k_mouza has quit IRC | 17:23 | |
| TheJulia | What those controls are, is going to vary based upon your business requirements | 17:23 |
| dking_desktop | I was thinking about having a separate server running multiple instances of Ironic, one for each such stack, but each of those would have to communicate with the internal networks. | 17:23 |
| TheJulia | I'm not sure what problem that would really solve though | 17:24 |
| *** rmcall has joined #openstack-ironic | 17:26 | |
| *** gyee has joined #openstack-ironic | 17:27 | |
| dking_desktop | Let's say that I have 5 separate instances of OpenStack, each with their own Neutron, Nova, etc. But each of those also need to be able to use Ironic to provision baremetal servers which are connected to a shared switch, and none of them should be able to have any access to the other baremetal servers that do not belong to them, and none of those OpenStack instances should be able to directly access the BMC n | 17:28 |
| TheJulia | oh, I see | 17:31 |
| TheJulia | Yeah, that is a little hairy given it is five separate openstacks so 5 separate neutrons | 17:32 |
| dking_desktop | That's what I'm trying to accomplish. There's some other considerations, too, such as each of these stacks use separate VLANs. If I let any of them run Ironic inside of their own controller nodes, then they would gain access to manipulate the shared switch, and would also have access to the BMC network for those servers, so that would pose a security risk. | 17:32 |
| *** khansa has quit IRC | 17:33 | |
| TheJulia | yeah, the only idea that I have is five centrally managed ironic using keystone federation with configuration in the ironics pointing back to the tenant networks, and maybe a separate network interface driver in ironic to handle the switch config disconnected from neutron... maybe. OR maybe a separate neutron. The issue really becomes cross-tenant isolation since your hosting separate clusters then :( | 17:33 |
| dking_desktop | Yeah. That's what makes things unusual and more complicated. | 17:38 |
| dking_desktop | So, if there were such a network interface driver disconnected from neutron, and also if there were a separate keystone, would that be enough to isolate Ironic so that it doesn't need to talk directly to the other services on the local OpenStacks' internal networks? | 17:39 |
| *** priteau has quit IRC | 17:42 | |
| *** k_mouza has joined #openstack-ironic | 17:43 | |
| TheJulia | that those ironics would need to talk to some internal services based upon the config. You'd likely need a network_interface driver that could talk to neutron in the "tenant" openstacks, and then extract the vlan ID value and do $things | 17:44 |
| TheJulia | but that is just a high level thought | 17:44 |
| dking_desktop | Yeah. I'm sure it's an unusual thing I'm trying to do. I've been trying to brainstorm some ways to make any of this work. | 17:47 |
| openstackgerrit | Merged openstack/ironic stable/stein: Revamp grenade jobs https://review.opendev.org/713176 | 17:48 |
| etingof | rpioso, o/ | 17:48 |
| etingof | rpioso, iDRAC confuses me lately... | 17:48 |
| etingof | rpioso, I am trying to change BIOS BootMode via Redfish. It mostly stays the same despite reboots, however it changed once and I do not understand what caused that. | 17:49 |
| etingof | rpioso, All Redfish calls work out well, no errors. But the value does not change. | 17:50 |
| etingof | mostly | 17:51 |
| dking_desktop | TheJulia: Thanks for the suggestion. I might have to think on that for a bit. | 17:52 |
| *** derekh has quit IRC | 18:00 | |
| *** uzumaki has joined #openstack-ironic | 18:00 | |
| uzumaki | when implementing a periodic method in a driver interface, do I not have to call that method from anywhere? I'm confused | 18:02 |
| TheJulia | uzumaki: the periodic decorator takes care of that if memory serves | 18:02 |
| TheJulia | I think?!? | 18:02 |
| TheJulia | It has been almost 1.5 years since I looked closely at periodic jobs | 18:02 |
| uzumaki | so I create a method with periodic decorator, let that method do all the stuff it does (which is polling for job completion/failure and act accordingly), and I'm done? | 18:03 |
| TheJulia | I _think_ so | 18:03 |
| TheJulia | We start a pool of worker threads independently | 18:04 |
| TheJulia | and each periodic should be invoking a task to lock the worker thread | 18:04 |
| * TheJulia may have that slightly backwards | 18:04 | |
| uzumaki | haha i think so. The locking code goes into the periodic, which is fine. I'm just confused as to who's going to invoke that periodic guy | 18:05 |
| TheJulia | I think it is just the decorator that caues it to get picked up | 18:05 |
| TheJulia | it has to be on a driver interface though | 18:06 |
| uzumaki | another thing, handling failures in periodics, all that really needs to be done is to send a 'fail' event on the task? no need for manually handling state changes? | 18:07 |
| TheJulia | no, you have to manually trigger the state change if memory serves | 18:08 |
| uzumaki | the code I've seen in redfish it simply triggers a 'fail' event on the task, which I assume handles the state change to deploy/clean fail | 18:08 |
| uzumaki | something like a task.process_event('fail') and that's it | 18:10 |
| TheJulia | uzumaki: here is a good example: https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/drac/raid.py#L1180 | 18:10 |
| uzumaki | I was looking at _exactly_ the same example | 18:11 |
| *** dtantsur is now known as dtantsur|afk | 18:11 | |
| dtantsur|afk | "so I create a method with periodic decorator, let that method do all the stuff it does" yes | 18:11 |
| dtantsur|afk | g'night folks | 18:12 |
| uzumaki | g'night dtantsur|afk thanks for the comment :) | 18:12 |
| uzumaki | it does a _set_failed() which does the task.process_event('fail') to handle failures. Well, as long as nothing is outstanding in this, i can proceed with the coding then? | 18:12 |
| TheJulia | uzumaki: seems so :) | 18:12 |
| *** jawad_axd has joined #openstack-ironic | 18:13 | |
| uzumaki | and in the event everything goes fine in the clean/deploy step, all that step needs to do is specify the next state through deploy_utils.get_async_step_return_state(node), and the state change is taken care of | 18:13 |
| *** amoralej is now known as amoralej|off | 18:15 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: [WIP]: Burn configdrive into virtual media boot ISO https://review.opendev.org/687250 | 18:15 |
| TheJulia | uzumaki: I believe so, yes | 18:15 |
| uzumaki | Great. And I'm curious, who uses the async_step_flags that we set? I don't see those getting consumed by some custom periodic method | 18:17 |
| openstackgerrit | Mark Goddard proposed openstack/tenks master: Refactor virtualbmc-domain tasks into a module https://review.opendev.org/713043 | 18:19 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: WIP Raise human-friendly messages on attempt to use pre-deploy steps drivers https://review.opendev.org/713594 | 18:24 |
| *** k_mouza has quit IRC | 18:30 | |
| uzumaki | TheJulia, any ideas? | 18:37 |
| TheJulia | sorry, got distracted | 18:37 |
| TheJulia | I believe you would consume them yourself | 18:38 |
| TheJulia | if that makes sense | 18:38 |
| uzumaki | well, I thought so too initially, given that driver-internal-info is supposed to be for 'internal' work, but I fail to find any examples in the code where somebody has consumed them | 18:39 |
| uzumaki | that's the only missing piece of the puzzle, I'm close to finishing the BIOSInterface code otherwise | 18:43 |
| *** k_mouza has joined #openstack-ironic | 18:44 | |
| dking_desktop | Could somebody give me a rundown of exactly what happens when a baremetal server is requested with local_link_connection and using Networking Generic Switch? | 18:47 |
| *** k_mouza has quit IRC | 18:48 | |
| *** k_mouza has joined #openstack-ironic | 18:49 | |
| TheJulia | dking_desktop: sure in a little bit, I ahve a meeting soon though | 18:50 |
| dking_desktop | I tried adding that driver and setup configuration with some invalid switch creds to see where it would throw an error, but it only just timed out at the build. | 18:51 |
| dking_desktop | Thank you very much. Take your time. I'll stay logged in. The more I understand the process, the better I can look at making my own driver if necessary. | 18:51 |
| *** stevebaker has quit IRC | 18:52 | |
| *** stevebaker has joined #openstack-ironic | 18:53 | |
| *** k_mouza has quit IRC | 18:54 | |
| *** jawad_axd has quit IRC | 18:54 | |
| *** k_mouza has joined #openstack-ironic | 19:21 | |
| *** dougsz has quit IRC | 19:25 | |
| *** k_mouza has quit IRC | 19:26 | |
| *** uzumaki has quit IRC | 19:26 | |
| *** diurnalist has quit IRC | 19:30 | |
| *** ociuhandu has joined #openstack-ironic | 19:39 | |
| *** dougsz has joined #openstack-ironic | 19:40 | |
| rpittau | good night! o/ | 19:40 |
| *** rpittau is now known as rpittau|afk | 19:40 | |
| *** diurnalist has joined #openstack-ironic | 19:41 | |
| TheJulia | umm, uzumaki /quit :( | 19:41 |
| TheJulia | dking_desktop: o/ | 19:41 |
| dking_desktop | o/ | 19:41 |
| TheJulia | the tl'dr is the local_link_connection information is transmitted to neutron in the port binding profile information | 19:42 |
| TheJulia | from there the ML2 driver takes that, and does the needful | 19:42 |
| TheJulia | which typically, in most n-g-s drivers is just edit the individual port definition | 19:42 |
| dking_desktop | Okay. I think that right now, I'm interested in how to debug it, and where in the process does Ironic make the request to neutron | 19:44 |
| openstackgerrit | Mark Goddard proposed openstack/bifrost stable/train: WIP: CentOS 8 support (Train) https://review.opendev.org/713508 | 19:46 |
| *** jdandrea has quit IRC | 19:51 | |
| dking_desktop | I didn't see anything that looked relevant in any log, and I enabled debug in Neutron. | 19:51 |
| TheJulia | oh, so ironic does port binding as part of cleaning, rescue, and provisioning processes | 19:51 |
| TheJulia | so the first step is a port bind to start those processes, then the port is unbound, then the tenant vif is attached using the same port binding information | 19:52 |
| TheJulia | that information gets passed into the ml2 framework and I think the switch_id has to be known in the neutron configuration?!? | 19:52 |
| TheJulia | I put ?!? beacuse I think that is the case, but I jusst don't remember right now | 19:52 |
| dking_desktop | Before I start asking more questions, is there any good documentation on this? I saw enough info on the user end for if everything just works, but not a lot of documentation on what's going on under the hood without having to look through source code. | 19:55 |
| TheJulia | dking_desktop: the only thing would really be the ml2 standard and our multitenancy design spec | 19:56 |
| TheJulia | err, multitenant networking | 19:56 |
| *** ociuhandu has quit IRC | 19:57 | |
| *** gmann is now known as gmann_afk | 19:57 | |
| *** jcoufal has quit IRC | 19:57 | |
| *** ociuhandu has joined #openstack-ironic | 19:58 | |
| dking_desktop | Right now, I'm starting with provisioning a server, because that fails at the moment. I setup the node with the local_link_connection information, and I setup a test_switch. I used fake information because there's no driver for my switch yet, and I didn't want to test the cisco one directly without seeing what it would do first. Nothing really seems to happen at first, but when I "openstack server create", it | 19:59 |
| dking_desktop | haven't seen any packets yet on that test server, so I know I'm doing something wrong. | 19:59 |
| dking_desktop | It just stays in BUILD for a while, and then I get a message. I think it says that there's no available servers, and I think I saw something about a timeout. | 20:00 |
| *** ociuhandu has quit IRC | 20:02 | |
| *** ociuhandu has joined #openstack-ironic | 20:02 | |
| dking_desktop | I can't even find anything in any logs that indicate that it even picked up on the local_link_connection information. I'm assuming that it did because if I didn't add it, then I get an error that they are missing and required when using neutron networking. | 20:03 |
| *** k_mouza has joined #openstack-ironic | 20:14 | |
| TheJulia | dking_desktop: okay, so what is likely happening is neutron gets the port info binding info, and doesn't know what to do with it because it sounds like nothing in the neutron config matches the test_Switch | 20:19 |
| *** k_mouza has quit IRC | 20:19 | |
| dking_desktop | I added a section in the neutron configuration. I have a section for "[genericswitch:test_switch]" in the ml2_conf.ini. But I'm using kolla-ansible, and I'm not 100% certain that I put it in the right place. I added it to both /etc/kolla/neutron-server/ml2_conf.ini and also neutron-openvswitch-agent/ml2_conf.ini, just in case. | 20:22 |
| TheJulia | dking_desktop: is networking generic swtich loaded into neutron? | 20:24 |
| dking_desktop | The same file has "mechanism_drivers = openvswitch,baremetal,l2population,genericswitch" | 20:25 |
| TheJulia | hmmm | 20:27 |
| TheJulia | so, I would make sure debugging is turned on in the neutron-api service, and restart it, you may see more information then | 20:27 |
| dking_desktop | I also put that in both files, just in case. | 20:27 |
| dking_desktop | In neutron-server/neutron.conf, I set "[DEFAULT]\ndebug = True" and restarted it. I got lots of debug logs, but still nothing that seemed appropriate. | 20:30 |
| *** gmann_afk is now known as gmann | 20:33 | |
| TheJulia | dking_desktop: If memory serves, the neutron api server does log when it is trying to or initiating call outs to plugins to execute, so you may want to look at that. I'd also recommend looking at the port that ironic is creating for provisioning in neutron and backtracking from there, it should show up in the port list for the network which is defined as the provisioning_network for ironic | 20:34 |
| dking_desktop | I found this that looks like the failure in the ironic conductor logs: ERROR ironic.conductor.utils [req-823f7287-9f67-46c9-bb0d-b3744e65769a - - - - -] Timeout reached while waiting for callback for node 85ce002e-bdde-4de2-9df0-d84f3cec31d3 | 20:35 |
| TheJulia | so that is the conductor erroring, but you should find where it asked neutron to bind the port | 20:36 |
| TheJulia | that should also tell you the port id, and from there you should also be able to find in the neutron logs what is happening to that port request | 20:37 |
| *** ociuhandu has quit IRC | 20:37 | |
| TheJulia | once you find the ID of the port in the neutron logs, you'll want to switch to using the request id (req-UUID value) | 20:37 |
| dking_desktop | In the neutron logs, here's what it says at that time, and I see a few of these: DEBUG neutron_lib.callbacks.manager [req-e11d0bb6-90c6-4760-9fef-ce100a6955f3 - - - - -] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents--9223372036852909011', 'neutron.services.segments.db._update_segment_host_mapping_for_agent--9223363270609841502'] for agent, after_update _notify_loop /var | 20:37 |
| dking_desktop | /python2.7/site-packages/neutron_lib/callbacks/manager.py:193 | 20:37 |
| TheJulia | what is the network_interface set to on the node that your trying to provision? | 20:38 |
| *** ociuhandu has joined #openstack-ironic | 20:38 | |
| *** rmcall has quit IRC | 20:38 | |
| dking_desktop | I can check, but I believe neutron | 20:38 |
| *** ociuhandu has quit IRC | 20:39 | |
| *** ociuhandu has joined #openstack-ironic | 20:40 | |
| dking_desktop | Yes, that's what I used. I'm not finding the ID of the port in any logs, though. I'm going to check again. | 20:41 |
| TheJulia | which port are you talking about? | 20:41 |
| TheJulia | the ironic port? or the neutron port (vif)? | 20:41 |
| dking_desktop | The ironic port is what I was looking for. | 20:42 |
| TheJulia | yeah, you won't see that referenced anywhere | 20:44 |
| TheJulia | look for the vif id | 20:44 |
| TheJulia | ironic's port id is internal, the vif id is stored on the port | 20:44 |
| TheJulia | all inteactions with neutron use the vif or neutron port id | 20:44 |
| TheJulia | There may be a delay in responses soon, I'm feeling cold and under the weather | 20:45 |
| dking_desktop | This shows my ignorance, but where would I find that? I have several ports, but I don't see anything listed in the baremetal ports that seems to mention the vif id. | 20:45 |
| dking_desktop | I'm sorry to hear that. I may need to be done for the day shortly, so perhaps it may be best to pick back up tomorrow. But if I can find the vif id and find it in the logs, that may give me something to start checking. | 20:46 |
| TheJulia | dking_desktop: openstack node vif list shows tenant port vifs | 20:47 |
| TheJulia | if you do openstack baremetal port show <uuid> | 20:47 |
| TheJulia | it should be something like driver_internal_info field... a... | 20:47 |
| TheJulia | going to actually look up the label | 20:47 |
| dking_desktop | internal_info for me is {} | 20:48 |
| TheJulia | 'tenant_vif_port_id' | 20:48 |
| dking_desktop | I don't see that field from openstack baremetal port show <uuid> | 20:49 |
| dking_desktop | BTW, I'm using train. | 20:49 |
| TheJulia | well, it would be in internal_info | 20:49 |
| TheJulia | and actually for provisioning it should be 'provisioning_vif_port_id' | 20:49 |
| dking_desktop | Would you happen to know what should be setting internal_info? | 20:49 |
| TheJulia | so for provisionign | 20:50 |
| TheJulia | ironic is supposed to create a port on the configured provisioning network | 20:50 |
| TheJulia | and it stores that value in the logs. when the deploy fails, that information gets removed/detached | 20:50 |
| TheJulia | so that may be part of the problem | 20:50 |
| TheJulia | if you have multiple ports on a baremetal node, you'll also want to check them all | 20:51 |
| TheJulia | but this behavior all boils down to what the network_interface field is set to in openstack baremetal node show <uuid> | 20:51 |
| dking_desktop | Also, thank you for all of your help. I don't think that I would be able to track down any of this without many hours of looking through code. | 20:51 |
| TheJulia | if network_interface is set to flat, that would match the behavior your observing. Flat implies the physical and logical networking is basically all preconfigured | 20:53 |
| TheJulia | neutron does port creation and vif profile binding and all that fancy stuff | 20:54 |
| dking_desktop | There is another physical NIC, but I don't think I added any others in OpenStack. I might run it again and try to grab that ID so that I can check the logs for it later. Also, I have a provisioning network set in the ironic configuration but I didn't use it for the "create server..." command since I wanted to use a tenant network. | 20:55 |
| dking_desktop | With flat, things were working, but it only provisioned onto the provisioning network. | 20:55 |
| TheJulia | yup, that sounds about right | 20:56 |
| TheJulia | so you've got a basis, it just may be that since your debugging after the fact you haven't found the logs indicating the id you actually need to be hunting down so it is a little harder :( | 20:56 |
| *** ociuhandu has quit IRC | 20:56 | |
| *** ociuhandu has joined #openstack-ironic | 20:57 | |
| dking_desktop | Okay. I'll give it a try, maybe tomorrow, and check back then. Thank you again for all of your help. | 20:59 |
| dking_desktop | Try to get some rest and stay healthy. I know things are crazy some places these days. | 20:59 |
| *** ociuhandu has quit IRC | 21:02 | |
| *** openstackstatus has joined #openstack-ironic | 21:16 | |
| *** ChanServ sets mode: +v openstackstatus | 21:16 | |
| *** ociuhandu has joined #openstack-ironic | 21:18 | |
| *** ociuhandu has quit IRC | 21:27 | |
| *** ociuhandu has joined #openstack-ironic | 21:42 | |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: [WIP] Add `network_data` field to ports and portgroups https://review.opendev.org/687910 | 21:42 |
| *** ociuhandu has quit IRC | 21:46 | |
| *** dougsz has quit IRC | 21:50 | |
| *** ociuhandu has joined #openstack-ironic | 22:01 | |
| *** ociuhandu has quit IRC | 22:06 | |
| *** k_mouza has joined #openstack-ironic | 22:15 | |
| *** k_mouza has quit IRC | 22:19 | |
| *** ijw has quit IRC | 22:39 | |
| *** ijw has joined #openstack-ironic | 22:40 | |
| *** rcernin has joined #openstack-ironic | 22:44 | |
| *** tkajinam has joined #openstack-ironic | 22:50 | |
| *** threestrands has joined #openstack-ironic | 22:58 | |
| *** sziviani_ has quit IRC | 23:24 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!