| *** Lucas_Gray has quit IRC | 00:24 | |
| *** Lucas_Gray has joined #openstack-ironic | 00:27 | |
| *** gyee has quit IRC | 00:30 | |
| *** Lucas_Gray has quit IRC | 01:43 | |
| *** Lucas_Gray has joined #openstack-ironic | 01:49 | |
| *** diurnalist has joined #openstack-ironic | 02:20 | |
| *** diurnalist has quit IRC | 02:42 | |
| *** diurnalist has joined #openstack-ironic | 02:44 | |
| *** Yumeng has joined #openstack-ironic | 03:01 | |
| *** JamesBen_ has quit IRC | 03:03 | |
| *** rcernin has joined #openstack-ironic | 03:07 | |
| *** Lucas_Gray has quit IRC | 03:31 | |
| *** mkrai has joined #openstack-ironic | 03:32 | |
| *** JamesBenson has joined #openstack-ironic | 03:38 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Fix TypeError on agent lookup failure https://review.opendev.org/744077 | 03:43 |
|---|---|---|
| *** tkajinam has quit IRC | 03:51 | |
| *** tkajinam has joined #openstack-ironic | 03:52 | |
| *** diurnalist has quit IRC | 03:54 | |
| *** ociuhandu has joined #openstack-ironic | 04:18 | |
| *** ociuhandu has quit IRC | 04:23 | |
| *** JamesBenson has quit IRC | 05:01 | |
| *** diurnalist has joined #openstack-ironic | 05:01 | |
| *** rh-jelabarre has quit IRC | 05:05 | |
| *** diurnalist has quit IRC | 05:06 | |
| *** Yumeng has quit IRC | 06:00 | |
| *** yaawang has quit IRC | 06:02 | |
| *** yaawang has joined #openstack-ironic | 06:02 | |
| *** yaawang has quit IRC | 06:20 | |
| *** yaawang has joined #openstack-ironic | 06:21 | |
| *** Qianbiao has joined #openstack-ironic | 06:43 | |
| *** yaawang has quit IRC | 06:57 | |
| *** yaawang has joined #openstack-ironic | 06:58 | |
| *** rcernin has quit IRC | 06:58 | |
| *** rcernin_ has joined #openstack-ironic | 06:59 | |
| *** mkrai has quit IRC | 07:00 | |
| *** ntt_ has joined #openstack-ironic | 07:00 | |
| arne_wiebalck | Good morning, ironic! | 07:04 |
| *** rcernin_ has quit IRC | 07:05 | |
| rpittau | good morning ironic! o/ | 07:05 |
| janders | good morning arne_wiebalck rpittau o/ | 07:06 |
| *** rcernin has joined #openstack-ironic | 07:06 | |
| arne_wiebalck | hey janders rpittau o/ | 07:06 |
| janders | have we recently changed the default logging configuration for ironic in bifrost? | 07:06 |
| rpittau | hey janders arne_wiebalck :) | 07:06 |
| janders | i reinstalled bifrost and logging switched over from /var/log/ironic/ironic-conductor.log to /var/log/messages, confusing me greatly | 07:07 |
| janders | I thought my config is silently corrupted and the thing doesnt start because it stopped logging | 07:07 |
| janders | and actually it hasn't :) | 07:07 |
| rpittau | janders: https://review.opendev.org/742854 | 07:07 |
| patchbot | patch 742854 - bifrost - Make ironic logging more in line with other services. (MERGED) - 7 patch sets | 07:07 |
| janders | makes sense, thank you rpittau | 07:08 |
| rpittau | np | 07:08 |
| janders | I thought I remembered someone else getting caught out and asking here as well, hence I checked the old build and found/fixedit | 07:08 |
| janders | so didnt take me too long to work out :) | 07:09 |
| rpittau | clarkb: thanks! I had a test job running and it looks fine, I issued some rechecks, fingers crosse :) | 07:09 |
| *** Zempashi has joined #openstack-ironic | 07:26 | |
| *** dougsz has joined #openstack-ironic | 07:32 | |
| janders | https://bugzilla.redhat.com/show_bug.cgi?id=1854637 looks like the MariaDB issue that was hitting FC32 bifrost installs is confirmed and getting fixed in the upcoming release | 07:35 |
| openstack | bugzilla.redhat.com bug 1854637 in mariadb "bifrost fails when running ironic-dbsync create_schema" [High,New] - Assigned to mschorm | 07:35 |
| rpittau | great :) | 07:36 |
| *** mkrai has joined #openstack-ironic | 07:41 | |
| iurygregory | good morning everyone! | 07:42 |
| janders | good morning iurygregory o/ | 07:42 |
| rpittau | hey iurygregory :) | 07:42 |
| iurygregory | o/ | 07:42 |
| *** ntt_ has quit IRC | 07:58 | |
| *** ntt_ has joined #openstack-ironic | 07:58 | |
| *** dtantsur|afk is now known as dtantsur | 08:04 | |
| dtantsur | morning ironic | 08:04 |
| rpittau | hey dtantsur :) | 08:04 |
| janders | good morning dtantsur | 08:06 |
| rpittau | reviews make the meals of champions! -> https://review.opendev.org/742201 | 08:11 |
| patchbot | patch 742201 - bifrost - Consolidate env-setup - 4 patch sets | 08:11 |
| *** lucasagomes has joined #openstack-ironic | 08:15 | |
| dtantsur | seriously though, will anyone cry if I drop half of the code in sushy-tools? it's convoluted and broken in so many places, I cannot even count | 08:23 |
| *** k_mouza has joined #openstack-ironic | 08:29 | |
| rpittau | dtantsur: I won't, I'm also glad to help :) | 08:46 |
| openstackgerrit | Verification of a change to openstack/ironic failed: Make the final deploy step validation actually fail deploy https://review.opendev.org/743587 | 08:48 |
| *** derekh has joined #openstack-ironic | 08:51 | |
| *** sshnaidm|afk is now known as sshnaidm | 08:56 | |
| *** rcernin has quit IRC | 09:04 | |
| *** rcernin has joined #openstack-ironic | 09:04 | |
| *** rcernin has quit IRC | 09:05 | |
| *** rcernin has joined #openstack-ironic | 09:05 | |
| ajya | Hi, a Q about docs/code. The docs say that write_image deploy step is priority 80, but for ISCSI it is 90. Should it be 80? https://opendev.org/openstack/ironic/src/branch/master/ironic/drivers/modules/iscsi_deploy.py#L676 | 09:11 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Simplify the Manager driver https://review.opendev.org/744862 | 09:13 |
| dtantsur | rpittau: let's start with ^^^ | 09:13 |
| openstackgerrit | Doug Szumski proposed openstack/ironic-python-agent stable/ussuri: Fix bootloader install issue with MDRAID https://review.opendev.org/744863 | 09:13 |
| dtantsur | ajya: yeah, it's an oversight, should be 80 and 60 for write_image and prepare_instance_boot. Please propose a patch if you have a minute. | 09:14 |
| ajya | dtantsur: ok, will do | 09:14 |
| dtantsur | cool! no need for a release note since we (thanks god) haven't released it yet | 09:15 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Simplify the Manager driver https://review.opendev.org/744862 | 09:16 |
| *** ociuhandu has joined #openstack-ironic | 09:23 | |
| openstackgerrit | Aija Jaunteva proposed openstack/ironic master: Fix iscsi deploy steps priorities https://review.opendev.org/744866 | 09:25 |
| *** ntt__ has joined #openstack-ironic | 09:28 | |
| *** ntt_ has quit IRC | 09:28 | |
| *** Lucas_Gray has joined #openstack-ironic | 09:33 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Simplify the Manager driver https://review.opendev.org/744862 | 09:33 |
| *** yaawang has quit IRC | 09:35 | |
| *** yaawang has joined #openstack-ironic | 09:36 | |
| Qianbiao | hello ironic. | 09:40 |
| Qianbiao | hi, long time no see. | 09:40 |
| Qianbiao | I got an issue on our env when inspect a node. | 09:40 |
| Qianbiao | it says: no default or UI configuration directive found when boot. | 09:40 |
| Qianbiao | any hint how i should fix it? | 09:40 |
| openstackgerrit | Merged openstack/bifrost master: Consolidate env-setup https://review.opendev.org/742201 | 09:40 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: [WIP] Create a new fake manager driver mirroring system https://review.opendev.org/744868 | 09:45 |
| openstackgerrit | Doug Szumski proposed openstack/ironic-python-agent stable/train: Fix bootloader install issue with MDRAID https://review.opendev.org/744873 | 09:51 |
| Qianbiao | ^^^ I think maybe due to the pxeconfig, but i am not sure why metal3 generated config not working: http://paste.openstack.org/show/796594/ | 10:07 |
| Qianbiao | when i create node manually, inspect is ok, the pxeconfig is very different: http://paste.openstack.org/show/796595/ | 10:08 |
| *** tkajinam has quit IRC | 10:15 | |
| *** rcernin has quit IRC | 10:23 | |
| openstackgerrit | Merged openstack/ironic-python-agent master: Hint 404 lookup failures for Operators https://review.opendev.org/726976 | 10:28 |
| openstackgerrit | Merged openstack/ironic-python-agent master: Extends pci devices metrics https://review.opendev.org/744403 | 10:30 |
| rpittau | mmmm latest working uefi grub2 job that I could find is from July 28 :/ | 10:31 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Create a new fake manager driver mirroring system resources https://review.opendev.org/744868 | 10:33 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Simplify the Manager driver https://review.opendev.org/744862 | 10:36 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Create a new fake manager driver mirroring system resources https://review.opendev.org/744868 | 10:36 |
| rpittau | oh..... nice, we're hitting THE bug | 10:41 |
| rpittau | https://624c5ee4140c0e80a4e6-f0a55fc4957fe55450e72f1f6d277d79.ssl.cf2.rackcdn.com/743676/2/check/ironic-tempest-ipa-partition-uefi-pxe-grub2/4fdcd14/controller/logs/screen-dstat.txt | 10:41 |
| rpittau | that's not the root cause though | 10:42 |
| *** Zempashi has quit IRC | 10:43 | |
| *** ociuhandu has quit IRC | 10:54 | |
| *** ociuhandu has joined #openstack-ironic | 10:55 | |
| *** ociuhandu has quit IRC | 11:00 | |
| *** Zempashi has joined #openstack-ironic | 11:05 | |
| *** rcernin has joined #openstack-ironic | 11:25 | |
| *** rcernin has quit IRC | 11:30 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Log API errors https://review.opendev.org/744887 | 11:30 |
| janders | wrapping up for the night, see you tomorrow o/ | 11:31 |
| * iurygregory goes for lunch | 11:33 | |
| *** ociuhandu has joined #openstack-ironic | 11:36 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Simplify the Manager driver https://review.opendev.org/744862 | 11:36 |
| *** Zempashi has quit IRC | 11:37 | |
| *** JamesBenson has joined #openstack-ironic | 11:40 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Replace the static Manager driver with a new fake one https://review.opendev.org/744862 | 11:41 |
| *** ociuhandu has quit IRC | 11:42 | |
| *** ociuhandu has joined #openstack-ironic | 11:42 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Replace the static Manager driver with a new fake one https://review.opendev.org/744862 | 11:43 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: [DNM] test ci https://review.opendev.org/744888 | 11:45 |
| *** k_mouza has quit IRC | 11:54 | |
| *** uzumaki has joined #openstack-ironic | 11:55 | |
| *** aedc has joined #openstack-ironic | 11:56 | |
| *** rcernin has joined #openstack-ironic | 11:56 | |
| *** mkrai has quit IRC | 11:58 | |
| *** rh-jelabarre has joined #openstack-ironic | 11:58 | |
| *** rh-jelabarre has quit IRC | 11:58 | |
| *** rh-jelabarre has joined #openstack-ironic | 11:59 | |
| *** rcernin has quit IRC | 12:02 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Log the traceback of unexpected errors when verifying power credentials https://review.opendev.org/743941 | 12:05 |
| *** rcernin has joined #openstack-ironic | 12:17 | |
| *** Zempashi has joined #openstack-ironic | 12:18 | |
| *** uzumaki has quit IRC | 12:21 | |
| *** uzumaki has joined #openstack-ironic | 12:21 | |
| *** rcernin has quit IRC | 12:22 | |
| *** derekh has quit IRC | 12:24 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Replace the static Manager driver with a new fake one https://review.opendev.org/744862 | 12:37 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-lib master: Add logging to the HTTP basic auth code https://review.opendev.org/744899 | 12:48 |
| dtantsur | zaneb: ^^^ | 12:48 |
| *** JamesBenson has quit IRC | 12:49 | |
| dtantsur | TheJulia: when you have a minute: https://review.opendev.org/#/c/744700/ | 12:55 |
| patchbot | patch 744700 - ironic-inspector (bugfix/10.2) - Configure dsvm jobs for the bugfix branch - 6 patch sets | 12:55 |
| TheJulia | dtantsur ack, after my first meeting this morning | 12:56 |
| iurygregory | good morning TheJulia =) | 12:56 |
| *** rcernin has joined #openstack-ironic | 13:00 | |
| *** derekh has joined #openstack-ironic | 13:09 | |
| *** priteau has joined #openstack-ironic | 13:09 | |
| *** iurygregory has quit IRC | 13:12 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Replace the static Manager driver with a new fake one https://review.opendev.org/744862 | 13:14 |
| *** rcernin has quit IRC | 13:15 | |
| *** iurygregory has joined #openstack-ironic | 13:18 | |
| openstackgerrit | Merged openstack/ironic-python-agent stable/train: Ignore devices with size 0 when collecting inventory https://review.opendev.org/741172 | 13:28 |
| * dtantsur wants to cry from sushy-tools | 13:36 | |
| *** mraineri has joined #openstack-ironic | 13:39 | |
| *** mraineri has left #openstack-ironic | 13:39 | |
| *** mraineri has joined #openstack-ironic | 13:40 | |
| * TheJulia joins dtantsur but crying from people wanting to perform provisioning over LACP port channels | 13:43 | |
| iurygregory | so today is the official day to cry? | 13:43 |
| *** janders has quit IRC | 13:45 | |
| dtantsur | apparently? | 13:45 |
| *** k_mouza has joined #openstack-ironic | 13:46 | |
| TheJulia | seems as goo da day as any | 13:48 |
| TheJulia | good | 13:48 |
| * rpittau joins for solidarity | 13:48 | |
| * iurygregory joins | 13:49 | |
| *** JamesBenson has joined #openstack-ironic | 13:59 | |
| *** k_mouza has quit IRC | 14:02 | |
| *** k_mouza has joined #openstack-ironic | 14:04 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: [DNM] test ci https://review.opendev.org/744888 | 14:04 |
| *** ociuhandu has quit IRC | 14:04 | |
| *** ociuhandu has joined #openstack-ironic | 14:05 | |
| *** ociuhandu has quit IRC | 14:10 | |
| iurygregory | yay funny | 14:10 |
| iurygregory | people were trying the IPE with ipmi on VM's.... | 14:10 |
| iurygregory | and well, it doesn't work XD | 14:10 |
| TheJulia | ugh | 14:13 |
| TheJulia | what is it with $things today | 14:13 |
| iurygregory | well we knew it didn't work | 14:13 |
| iurygregory | thats also why we only test redfish in the IPE CI | 14:13 |
| iurygregory | I thought I had it documented somewhere =X | 14:14 |
| iurygregory | and probably need to have this information in some place on metal3... | 14:14 |
| *** uzumaki has quit IRC | 14:20 | |
| *** sshnaidm is now known as sshnaidm|afk | 14:24 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Get rid of Resources cache, store resources on the Application https://review.opendev.org/744911 | 14:25 |
| dtantsur | moar refactoring ^^ | 14:26 |
| *** JamesBen_ has joined #openstack-ironic | 14:27 | |
| *** JamesBenson has quit IRC | 14:30 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 14:31 |
| dtantsur | and this should finally be ready ^^ | 14:31 |
| rpittau | rmrrr the uefi grub2 job issue is really frustrating | 14:39 |
| rpittau | it's pxe at some point decided that the image is too big to load | 14:40 |
| rpittau | I checked changes in the last 7 days for dib images or ironic and there's literally nothing that could've lead to that | 14:41 |
| iurygregory | *magic* | 14:43 |
| TheJulia | it is likely wall clocking and if the timer runs out then things are bombing | 14:46 |
| TheJulia | we can't tell that from the screen though | 14:46 |
| *** JamesBenson has joined #openstack-ironic | 14:46 | |
| rpittau | yeah, that's the problem, but it's too sudden, and in both master and ussuri, they use different base os (focal/bionic), different images (master/ussuri), so I'm prone to think it's something in the infra | 14:48 |
| *** JamesBen_ has quit IRC | 14:48 | |
| * iurygregory loves rabbit broken in the multinode job -.-' | 14:48 | |
| iurygregory | https://zuul.opendev.org/t/openstack/build/dc99bafae9c14e35bbae709e2734aa43/log/controller/logs/screen-n-api.txt#2004 stein branch | 14:49 |
| openstackgerrit | Kaifeng Wang proposed openstack/ironic master: Fix console auto port allocation under IPv6 https://review.opendev.org/744518 | 14:49 |
| rpittau | iurygregory: that might be "normal", tou should check the rabbitmq logs | 14:50 |
| iurygregory | yeah looking now | 14:50 |
| rpittau | you see that afterwards it recovers ? | 14:50 |
| iurygregory | oh yeah | 14:51 |
| rpittau | anyway, I'm seriously thinking about moving the pxe grub2 job to tinyipa, at least temporary, the gate is completely broken like this | 14:51 |
| iurygregory | ++ | 14:51 |
| TheJulia | rpittau: well, "something in infra" is super vague. Are the failures all on a specific vendor cloud? | 14:53 |
| iurygregory | rax? =X | 14:54 |
| rpittau | TheJulia: they're on ovh as far as I can see, we don't use dib in rax anymore | 14:54 |
| iurygregory | yeah | 14:54 |
| iurygregory | oh ok the failures are in master also | 14:54 |
| rpittau | I know it's very vague, but I don't have anything else right now, at least nothing that can conduct to something we can act on | 14:54 |
| TheJulia | clarkb: anything special about ovh with nested virt? | 14:55 |
| rpittau | except for what we did with rax :) | 14:55 |
| dtantsur | rpittau: I think TheJulia has a patch to move this job to tinyipa, I +2'ed it today | 14:56 |
| rpittau | I just saw it....... | 14:56 |
| *** JamesBenson has quit IRC | 14:56 | |
| rpittau | I think I missed it because I was dealing with other type of sadness 2 days ago :( | 14:58 |
| TheJulia | CI sadness is toats a thing sadly | 14:58 |
| TheJulia | Just think of all the stable branch sadness! | 14:59 |
| * rpittau goes in a corner to cry | 14:59 | |
| TheJulia | No no, we cry together! | 14:59 |
| TheJulia | https://giphy.com/gifs/ZanWL0MNtv3y0 | 15:00 |
| iurygregory | stable branch sadness https://zuul.opendev.org/t/openstack/build/dc99bafae9c14e35bbae709e2734aa43/log/controller/logs/screen-ir-cond.txt#2268 | 15:01 |
| * iurygregory joins | 15:01 | |
| *** diurnalist has joined #openstack-ironic | 15:01 | |
| TheJulia | could we just disable neutron jobs on stable branches? | 15:02 |
| TheJulia | I'm only partially serious | 15:02 |
| clarkb | TheJulia: well we still recommend against nested virt because it is flaky in near impossible ways to debug | 15:04 |
| clarkb | TheJulia: we've enabled nested virt specific labels in nodepool hosted on clouds that intend to support it. The idea being jobs that intend to exercise it can then work with those clouds to debug problems when they arise (I don't think ovh is on that list) | 15:04 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-lib master: Add logging to the HTTP basic auth code https://review.opendev.org/744899 | 15:05 |
| TheJulia | clarkb: better question, is ovh fully emulated or are they running the VMs with on the processor with the virtualization flags? | 15:07 |
| clarkb | the VMs we get should be running on kvm virtualization not qemu emulation | 15:09 |
| TheJulia | other question for my fellow ironic troublemakers... our IPA with networkmanager, does it keep trying dhcp forever or does it give up? < Does anyone know? | 15:09 |
| dtantsur | I think there is a timeout | 15:10 |
| TheJulia | how long? | 15:10 |
| TheJulia | becasue this may explain a great many headaches | 15:10 |
| dtantsur | I don't remember for sure.. I do remember increasing some sort of a timeout | 15:15 |
| *** Qianbiao has quit IRC | 15:15 | |
| dtantsur | namely, https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/roles/ipa-build-dib-image/tasks/main.yaml#L30 | 15:15 |
| dtantsur | TheJulia: ^^ | 15:15 |
| TheJulia | bingo | 15:15 |
| TheJulia | dtantsur: thanks! | 15:15 |
| *** mkrai has joined #openstack-ironic | 15:15 | |
| *** JamesBenson has joined #openstack-ironic | 15:18 | |
| *** aedc has joined #openstack-ironic | 15:34 | |
| dtantsur | to top it all, opensuse is broken on bifrost.. | 15:35 |
| TheJulia | seriously, did we like... take a kids's lollipop? | 15:36 |
| TheJulia | And the universe is punishing us today? | 15:36 |
| *** aedc has quit IRC | 15:36 | |
| dtantsur | no clue. fortunately, it's not voting. | 15:36 |
| dtantsur | (unfortunately, we need to fix it) | 15:36 |
| dtantsur | trying locally, seeing "Update notifications were received from the following packages:" maybe that's the problem? | 15:39 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Clean up bindep.txt https://review.opendev.org/744930 | 15:49 |
| *** ntt__ has quit IRC | 15:52 | |
| openstackgerrit | Verification of a change to openstack/ironic-inspector failed: Configure dsvm jobs for the bugfix branch https://review.opendev.org/744700 | 15:52 |
| * dtantsur rolls eyes | 15:53 | |
| TheJulia | Anyone mange to get one of the flamethrowers from the boring company? | 15:58 |
| dtantsur | would be hard to get through the customs | 15:58 |
| *** lucasagomes has quit IRC | 16:00 | |
| gudrutis0 | TheJulia: are you also planning to set devstack on fire? | 16:05 |
| TheJulia | I was thinking all the thigns | 16:07 |
| *** JamesBenson has quit IRC | 16:10 | |
| *** diurnalist has quit IRC | 16:12 | |
| gudrutis0 | you guys are using these ubuntu images for devstack? | 16:13 |
| gudrutis0 | https://cloud-images.ubuntu.com/ | 16:13 |
| JayF | look, you don't destroy devstack with fire, you have to BASH it (hah!) | 16:13 |
| TheJulia | gudrutis0: the openstack infra team does make some minor changes afaik, but I've used those images in the past locally for VMs | 16:14 |
| *** JamesBenson has joined #openstack-ironic | 16:15 | |
| gudrutis0 | can I get the infra teams images from somewhere? | 16:16 |
| TheJulia | gudrutis0: unforutnately they annot be redistributed | 16:16 |
| clarkb | we actually do have them up | 16:16 |
| TheJulia | ubuntu has restrictions on that | 16:16 |
| TheJulia | oh really? | 16:16 |
| clarkb | yes | 16:16 |
| clarkb | I think ubuntu only cares if you've changed the distro packages themselves | 16:16 |
| clarkb | we don't | 16:16 |
| TheJulia | ahh | 16:17 |
| *** diurnalist has joined #openstack-ironic | 16:17 | |
| TheJulia | JayF: https://giphy.com/gifs/zck-kntr-bye-destroyed-crushed-PgzzOW3UgYuZftHuQZ <-- like this? | 16:17 |
| clarkb | https://nb01.opendev.org/images https://nb02.opendev.org/images and https://nb04.opendev.org/images | 16:17 |
| clarkb | we have several workers that can build the image so you want to check the three and grab the most recent | 16:17 |
| TheJulia | JayF: https://giphy.com/gifs/cheezburger-cars-crush-iXaluoRDb7WtW | 16:17 |
| clarkb | what we do is run debootstrap in a chroot against ubuntu pacakge repos, then copy those into an image with dib | 16:18 |
| clarkb | then we add glean instead of cloud-init, add unbound local dns forwarding resolver, and preintsall a couple of tools like tox in a virtualenv | 16:18 |
| JayF | TheJulia: if you get those pieces within 3 feet of each other, it'll still talk. Those things never die! | 16:19 |
| clarkb | but we aren't trying to be a different distro or anything | 16:19 |
| rpioso | dtantsur, TheJulia: Might you have time this week to review https://review.opendev.org/#/c/740721/ ? We're aiming to land an MVP this cycle. | 16:20 |
| patchbot | patch 740721 - ironic-specs - Operationally efficient & fast-track provisioning - 12 patch sets | 16:20 |
| dtantsur | will try, no promises though | 16:21 |
| TheJulia | I'm unsure I'll have any time this week, I'm slammed | 16:21 |
| * TheJulia goes back to the slide deck of doooooom | 16:22 | |
| gudrutis0 | clarkb: so these images will not work with `cloud-init`? | 16:22 |
| rpioso | dtantsur, TheJulia: Understood. If you find some time, we would much appreciate your feedback. We want to make sure we're heading down the right path. No red herrings ;-) | 16:23 |
| clarkb | gudrutis0: they do not have cloud init installed | 16:23 |
| clarkb | they have glean instead | 16:23 |
| openstackgerrit | Riccardo Pittau proposed openstack/sushy-tools master: Use property plus abstractmethod for abstractproperty https://review.opendev.org/744943 | 16:23 |
| *** mkrai has quit IRC | 16:24 | |
| openstackgerrit | Vishal Manchanda proposed openstack/ironic-ui master: DNM: testing gate on ubuntu focal https://review.opendev.org/744708 | 16:24 |
| gudrutis0 | Is the effect the same ? can I provide init script `--user-data` when creating server? | 16:24 |
| clarkb | gudrutis0: user data isn't supported. glean will write ssh keys to root's authorized keys and set up networking though | 16:24 |
| clarkb | it requires config drive for that and doesn't use metadata service | 16:25 |
| gudrutis0 | ah, dam, not what I want | 16:25 |
| gudrutis0 | but thanks | 16:25 |
| clarkb | gudrutis0: what are you trying to do? | 16:27 |
| gudrutis0 | Automate the creation of devstack machine by giving init.sh script. For some reason I can not ssh to ubunutu2004 images while everything works fine on ubuntu1804. | 16:27 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Use property plus abstractmethod for abstractproperty https://review.opendev.org/744945 | 16:28 |
| gudrutis0 | It takes to much time when you need to recreate it | 16:28 |
| TheJulia | rpioso: fwiw, you guys might not want to wait on posting the code | 16:28 |
| clarkb | gudrutis0: I would probably use a tool like disk image builder to do most of the prep before you even try to boot it. You can even write out user configs to ssh in that way | 16:29 |
| *** dougsz has quit IRC | 16:30 | |
| rpioso | TheJulia: We're driving toward posting code soon-ish :-) | 16:32 |
| dtantsur | folks, please review https://review.opendev.org/744862 it fixes a quite serious problem in sushy-tools (all nodes are using the same virtual media) | 16:33 |
| patchbot | patch 744862 - sushy-tools - Replace the static Manager driver with a new fake one - 9 patch sets | 16:33 |
| dtantsur | it only works in our CI because we test with one node :) | 16:34 |
| gudrutis0 | clarkb: but everything was working fine with ubuntu1804 and there was no need to repackaging it... Ok, I will see what I can do | 16:35 |
| clarkb | gudrutis0: ya, I don't know why/what has changed with 20.04. We haven't noticed because we don't use cloud-init | 16:35 |
| *** dtantsur is now known as dtantsur|afk | 16:39 | |
| dtantsur|afk | g'night | 16:39 |
| rpittau | o/ | 16:39 |
| gudrutis0 | clarkb: actually, you gave me an idea. I was relying on openstack on putting my .ssh keys on VM, but I can also use init-script for that. thx | 16:43 |
| clarkb | gudrutis0: note openstack doesn't put the keys on the VM. A tool like cloud-init does that | 16:43 |
| clarkb | openstack provides the information via metadata service or config drive though | 16:43 |
| gudrutis0 | `--key-name ` was working fine before though | 16:48 |
| *** gyee has joined #openstack-ironic | 16:48 | |
| clarkb | ya, could be a regression in 20.04's cloud init setup? | 16:55 |
| clarkb | if you get the console log of the booted instance it will usually log some hints at what went wrong | 16:56 |
| gudrutis0 | clarkb: actully I see in the logs that the key was correctly copied under ubuntu directory | 17:00 |
| *** derekh has quit IRC | 17:00 | |
| gudrutis0 | so the keys are not the issue | 17:00 |
| gudrutis0 | the issue is that I can not ping or ssh to the machine ( I get a timeout) | 17:00 |
| gudrutis0 | but, honestly I dont have time to debug it now, so I will stick to using 1804 | 17:02 |
| *** Lucas_Gray has quit IRC | 17:12 | |
| *** ociuhandu has joined #openstack-ironic | 17:18 | |
| *** sshnaidm|afk is now known as sshnaidm | 17:19 | |
| openstackgerrit | Vishal Manchanda proposed openstack/ironic-ui master: [goal] Migrate testing to ubuntu focal https://review.opendev.org/744708 | 17:20 |
| *** Zempashi has quit IRC | 17:22 | |
| *** ociuhandu has quit IRC | 17:22 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Fix iscsi deploy steps priorities https://review.opendev.org/744866 | 17:35 |
| *** priteau has quit IRC | 17:38 | |
| *** k_mouza has quit IRC | 17:43 | |
| TheJulia | folks, any reviews on https://review.opendev.org/#/c/744077/ would be appreciated | 17:53 |
| patchbot | patch 744077 - ironic-python-agent - Fix TypeError on agent lookup failure - 3 patch sets | 17:53 |
| *** SuiongNg|ITRI_TW has quit IRC | 17:58 | |
| *** k_mouza has joined #openstack-ironic | 18:13 | |
| *** k_mouza has quit IRC | 18:16 | |
| *** rh-jelabarre has quit IRC | 18:16 | |
| *** rh-jelabarre has joined #openstack-ironic | 18:16 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Log API errors https://review.opendev.org/744887 | 18:26 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent stable/rocky: Pin Ironic Tempest Plugin https://review.opendev.org/744773 | 18:27 |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Log API errors for easier debugging https://review.opendev.org/744887 | 18:28 |
| *** aedc has joined #openstack-ironic | 18:30 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent stable/queens: Pin Ironic Tempest Plugin https://review.opendev.org/744969 | 18:33 |
| *** antotala has joined #openstack-ironic | 18:36 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/sushy-tools master: Log API errors for easier debugging https://review.opendev.org/744887 | 18:36 |
| antotala | Hello All, | 18:38 |
| TheJulia | hello | 18:40 |
| openstackgerrit | Christopher Dearborn proposed openstack/sushy master: Fix retrieving software & firmware inventory https://review.opendev.org/744474 | 18:44 |
| antotala | I have cretaed a custom inspector image following the instructions https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html. | 18:52 |
| openstackgerrit | Christopher Dearborn proposed openstack/sushy master: Fix retrieving software & firmware inventory https://review.opendev.org/744474 | 18:53 |
| *** patchbot has quit IRC | 18:54 | |
| *** stevebaker has joined #openstack-ironic | 18:57 | |
| TheJulia | antotala: okay | 19:03 |
| TheJulia | antotala: What seems to be the issue? | 19:03 |
| antotala | TheJulia: I am looking for a way to test the image I have created, is there any way I can test the image without actually inspecting a real server ? | 19:06 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 19:10 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/stein: Pin ironic-tempest-plugin https://review.opendev.org/744547 | 19:15 |
| TheJulia | antotala: you could boot it in a vm? | 19:16 |
| *** sdanni has joined #openstack-ironic | 19:19 | |
| TheJulia | antotala: depending on version and settings and your environment (i.e. if you have multicast dns setup ) your test vm may show up in ironic, although you should just see IPA starting on the console in a somewhat confused state if you manually launch the kernel/ramdisk as a VM | 19:21 |
| stevebaker | morning Ironic! | 19:23 |
| TheJulia | guten Morgen | 19:23 |
| openstackgerrit | Merged openstack/ironic master: Change UEFI PXE job to use tinyipa https://review.opendev.org/744535 | 19:24 |
| openstackgerrit | Merged openstack/ironic-inspector bugfix/10.2: Configure dsvm jobs for the bugfix branch https://review.opendev.org/744700 | 19:24 |
| antotala | TheJulia: Created an iso and trying to boot it in a VM. | 19:25 |
| TheJulia | antotala: that should work | 19:25 |
| TheJulia | stevebaker: you haven't missed much. CI continues to make us sad | 19:32 |
| stevebaker | TheJulia: boo! is there anything I should look at? | 19:34 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/rocky: Pin ironic-tempest-plugin https://review.opendev.org/744567 | 19:39 |
| TheJulia | We've got patches that need to be backported to queens for a couple various bugs | 19:40 |
| TheJulia | so basically pinning is a thing to get CI funcitonal again on the older branches. As you can see I've also started proposing disabling jobs | 19:40 |
| TheJulia | I'm just not sure there is really value in digging deep into code approaching EM | 19:40 |
| TheJulia | beyond the bug fixes | 19:41 |
| openstackgerrit | Merged openstack/bifrost master: Clean up requirements https://review.opendev.org/743991 | 19:41 |
| TheJulia | stevebaker: rocky/queens IPA might be good | 19:43 |
| TheJulia | to look at | 19:43 |
| TheJulia | although, I think my fixes will do it once I fix a conditional | 19:43 |
| TheJulia | doh! | 19:43 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent stable/rocky: Pin Ironic Tempest Plugin https://review.opendev.org/744773 | 19:45 |
| stevebaker | TheJulia: ok I'll nurse those changes for a start | 19:45 |
| dking | I noticed that even though inspection is receiving multiple TLVs, local_link_connection is only being set for switch_id and port_id. Looking at the code, it seems like the other TLVs are only added if both the other TLVs are not processed? | 19:45 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent stable/queens: Pin Ironic Tempest Plugin https://review.opendev.org/744969 | 19:46 |
| dking | I'm trying to add at least TLV 5 to the port local_link_connection. I can see from the inspector logs that it is received. | 19:46 |
| TheJulia | dking: I suspect your best path is to open a bug and propose a patch | 19:49 |
| TheJulia | stevebaker: okay. I'm really wanting to get https://review.opendev.org/#/c/744077/ merged and backported soon. It is fairly nasty and I need to get that into queens :( | 19:50 |
| dking | TheJulia: Thank you for the recommendation. My first thought was that I was doing something wrong. I might first try going back through the git commits to try to find out what the reason was to only loop through LLDP_PROC_DATA_MAPPING, or why limit it to only two TLVs. | 19:51 |
| TheJulia | I unfortuantely don't know why | 19:51 |
| TheJulia | I think bfournie touched some of that at some point so he might remember why | 19:52 |
| *** antotala has quit IRC | 19:53 | |
| stevebaker | TheJulia: one day I aspire to being ironic-core, but that is a +1 from me for now | 19:53 |
| stevebaker | TheJulia: also, logging raising an exception, ouch | 19:54 |
| TheJulia | yeeaaaahhhh | 19:55 |
| * stevebaker reads the bug, wow | 19:56 | |
| TheJulia | yeah | 19:57 |
| *** Zempashi has joined #openstack-ironic | 19:59 | |
| bfournie | dking: the only TLVs that are processed are from the basic management, 802.1, and 802.3 TLV sets. All of the data for all of the TLVs is saved for the interface and can be retrieved. In addition the switch id and port id are saved for the Ironic port in the link_local_connection field | 20:00 |
| dking | bfournie: I see that it does collect the TLV I'm wanting (TLV 5) in lldp_basic.py, but in local_link_connection.py, it tries to parse the data collected from lldp_basic, but only does so for LLDP_CHASSIS_ID_NM and LLDP_PORT_ID_NM. I'd really like the other TLVs to be added there, most specifically because they unhexlify the data, rather than trying to figure out how to do that later with introspection rules. | 20:05 |
| dking | And since that's the place that already updates the data for the ports, I'd like to just expand that to collect all the TLVs. I can't see a reason to exclude them from the local_link_connection data, especially if they're already being processed. | 20:06 |
| openstackgerrit | Merged openstack/bifrost master: Install openstacksdk from pypi by default https://review.opendev.org/743989 | 20:10 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/ussuri: Change UEFI PXE job to use tinyipa https://review.opendev.org/744979 | 20:11 |
| bfournie | dking: the local_link_connection is specific data to identify the port only, will have to search for how its used but its not intended to included all of the data received for this interface | 20:17 |
| dking | bfournie: It looks like you're the person to talk to. That code was yours, 3 years ago. It looks like it was the first attempt to add lldp_basic processing to local_link_connection. | 20:17 |
| dking | Hmm.. Okay. Well, that's a shame. It would be super helpful for us. We are provisioning our nodes with Bifrost, and we'll need a way to determine what switches the nodes are connected to. The chassis MAC address isn't super helpful for us, as each port has it's own MAC. However, TLV 5 can contain SYS-NAME, which gives us the hostname of the switch. | 20:19 |
| bfournie | dking: local_link_connection always existed for the port and it could be set manually, this change was to retrieve the info from lldp and set it automatically | 20:19 |
| dking | Yeah, that was a handy change, and one which we could not live without. I'm just needing that extra piece of information, as it is a better indicator of the switch than the MAC address. | 20:20 |
| TheJulia | So I was bad and self-approved a stable backport that I also proposed but it was just a job change to unblock the gate | 20:20 |
| bfournie | dking: I see how if a mac isn't useful in that case if that is what the switch is sending | 20:20 |
| TheJulia | once it merges, we should be able to recheck all the ussuri stuff that is hung up | 20:21 |
| bfournie | dking: that may a useful change - to set sysname there if mac is received, otherwise use the chassis id | 20:21 |
| dking | Yeah, both of our switches, Supermicro and Aristas, send the chassis id as a MAC. But that can't be common, as the local_link_connection code specifically only accepts the mac_address subtype. | 20:23 |
| dking | s/common/uncommon/ | 20:24 |
| dking | The chassis ID is good to have, IMHO, as I believe it's a required TLV, whereas sys-name is not, and is absent by default on several switches. Of course, I think the code currently only adds any TLV if it exists, so I think that should be an easy change. | 20:26 |
| bfournie | dking: right, we did change to only accept if mac, but I've seen a switch level mac in this field on other switches, not per-port | 20:27 |
| dking | Ah. Sadly, ours don't work like that. | 20:27 |
| dking | I'll probably take a look and submit something tomorrow. Thanks for the input! I feel a bit better knowing why the decision was made. | 20:28 |
| bfournie | dking: k thanks, will look for it | 20:29 |
| *** gyee has quit IRC | 20:36 | |
| *** gyee has joined #openstack-ironic | 20:38 | |
| *** sdanni has quit IRC | 20:50 | |
| *** ociuhandu has joined #openstack-ironic | 20:51 | |
| *** ociuhandu has quit IRC | 20:56 | |
| *** gyee has quit IRC | 21:24 | |
| *** gyee has joined #openstack-ironic | 21:25 | |
| *** Zempashi has quit IRC | 21:32 | |
| *** rcernin has joined #openstack-ironic | 21:35 | |
| *** rcernin has quit IRC | 21:36 | |
| *** rcernin has joined #openstack-ironic | 21:36 | |
| *** markmcclain has quit IRC | 21:53 | |
| *** rcernin has quit IRC | 22:03 | |
| *** rcernin has joined #openstack-ironic | 22:17 | |
| *** janders has joined #openstack-ironic | 22:39 | |
| janders | g'day Ironic! o/ | 22:39 |
| *** rcernin has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-ironic | 22:40 | |
| *** rcernin has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-ironic | 22:44 | |
| *** tkajinam has joined #openstack-ironic | 23:00 | |
| *** ociuhandu has joined #openstack-ironic | 23:10 | |
| *** markmcclain has joined #openstack-ironic | 23:12 | |
| *** ociuhandu has quit IRC | 23:16 | |
| stevebaker | janders: hi | 23:23 |
| *** diurnalist has quit IRC | 23:24 | |
| *** irclogbot_3 has quit IRC | 23:34 | |
| *** irclogbot_2 has joined #openstack-ironic | 23:37 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Fix iscsi deploy steps priorities https://review.opendev.org/744866 | 23:43 |
| openstackgerrit | Merged openstack/ironic master: Log the traceback of unexpected errors when verifying power credentials https://review.opendev.org/743941 | 23:44 |
| openstackgerrit | Merged openstack/ironic master: Wipe agent token and URL on rescue and unrescue https://review.opendev.org/744655 | 23:44 |
| openstackgerrit | Merged openstack/ironic master: Convert v1 controller to plain, return JSON https://review.opendev.org/741065 | 23:50 |
| openstackgerrit | Merged openstack/ironic master: Break out collection functions for json usage https://review.opendev.org/743676 | 23:50 |
| openstackgerrit | Merged openstack/ironic stable/train: Use TLS for json_rpc when configured https://review.opendev.org/744132 | 23:50 |
| janders | hi stevebaker | 23:55 |
| janders | how was your break? | 23:55 |
| stevebaker | janders: really nice, toured some wineries, made some music | 23:58 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!