Tuesday, 2020-06-16

*** gmann_afk is now known as gmann00:06
*** Goneri has quit IRC00:28
*** cdearborn has quit IRC00:33
*** gyee has quit IRC01:02
*** Qianbiao has joined #openstack-ironic01:08
*** ricolin has joined #openstack-ironic01:27
*** threestrands_ has joined #openstack-ironic01:32
openstackgerritSteve Baker proposed openstack/ironic master: Enable HTTP Basic authentication for JSON-RPC  https://review.opendev.org/73575701:32
*** threestrands has quit IRC01:33
*** threestrands_ has quit IRC01:33
*** tzumainn has quit IRC01:33
*** threestrands_ has joined #openstack-ironic01:33
*** threestrands_ has quit IRC01:34
*** threestrands_ has joined #openstack-ironic01:35
*** threestrands_ has quit IRC01:36
*** threestrands_ has joined #openstack-ironic01:36
*** threestrands_ has quit IRC01:37
*** threestrands_ has joined #openstack-ironic01:38
*** threestrands_ has quit IRC01:39
*** threestrands_ has joined #openstack-ironic01:39
*** jdandrea has quit IRC01:39
*** threestrands_ has quit IRC01:40
*** threestrands_ has joined #openstack-ironic01:41
*** threestrands_ has quit IRC01:42
*** threestrands_ has joined #openstack-ironic01:42
*** threestrands_ has quit IRC01:43
*** threestrands_ has joined #openstack-ironic01:44
*** threestrands_ has quit IRC01:45
*** threestrands_ has joined #openstack-ironic01:45
*** threestrands_ has quit IRC01:46
*** threestrands_ has joined #openstack-ironic01:47
*** threestrands_ has quit IRC01:48
*** threestrands_ has joined #openstack-ironic01:48
*** threestrands_ has quit IRC01:49
*** threestrands_ has joined #openstack-ironic01:50
*** threestrands_ has quit IRC01:51
*** threestrands_ has joined #openstack-ironic01:51
*** threestrands_ has quit IRC01:52
*** threestrands_ has joined #openstack-ironic01:53
*** threestrands_ has quit IRC01:54
*** threestrands_ has joined #openstack-ironic01:54
*** threestrands_ has quit IRC01:55
*** threestrands_ has joined #openstack-ironic01:56
*** threestrands_ has quit IRC01:57
*** threestrands_ has joined #openstack-ironic01:57
*** threestrands_ has quit IRC01:58
*** threestrands_ has joined #openstack-ironic01:59
*** threestrands_ has quit IRC02:00
*** threestrands_ has joined #openstack-ironic02:00
*** threestrands_ has quit IRC02:01
*** threestrands_ has joined #openstack-ironic02:02
*** mkrai has joined #openstack-ironic02:14
*** rloo has quit IRC02:41
*** yaawang has quit IRC02:48
*** yaawang has joined #openstack-ironic02:52
openstackgerritRichard G. Pioso proposed openstack/ironic master: Fix Redfish handle no continuous override boot src  https://review.opendev.org/73164402:54
*** yaawang has quit IRC02:57
*** dking has quit IRC03:48
*** rh-jelabarre has quit IRC03:49
*** yaawang has joined #openstack-ironic03:50
*** diurnalist has quit IRC04:12
*** Qianbiao has quit IRC04:26
*** yaawang has quit IRC04:30
*** yaawang has joined #openstack-ironic04:31
*** mkrai has quit IRC04:58
*** mkrai_ has joined #openstack-ironic04:58
*** diurnalist has joined #openstack-ironic05:18
*** diurnalist has quit IRC05:23
*** yaawang has quit IRC05:31
*** yaawang has joined #openstack-ironic05:32
*** threestrands_ has quit IRC05:52
*** threestrands_ has joined #openstack-ironic05:53
*** threestrands_ has quit IRC05:54
*** threestrands_ has joined #openstack-ironic05:54
*** threestrands_ has quit IRC05:55
*** threestrands_ has joined #openstack-ironic05:56
*** threestrands_ has quit IRC05:57
*** threestrands_ has joined #openstack-ironic05:57
*** threestrands_ has quit IRC05:58
*** threestrands_ has joined #openstack-ironic05:59
*** threestrands_ has quit IRC06:00
*** threestrands_ has joined #openstack-ironic06:00
*** threestrands_ has quit IRC06:01
*** threestrands_ has joined #openstack-ironic06:02
*** threestrands_ has quit IRC06:03
*** threestrands_ has joined #openstack-ironic06:03
*** threestrands_ has quit IRC06:04
*** threestrands_ has joined #openstack-ironic06:05
*** threestrands_ has quit IRC06:06
*** threestrands_ has joined #openstack-ironic06:06
*** threestrands_ has quit IRC06:07
*** threestrands_ has joined #openstack-ironic06:08
*** threestrands_ has quit IRC06:09
*** threestrands_ has joined #openstack-ironic06:09
*** threestrands_ has quit IRC06:10
*** threestrands_ has joined #openstack-ironic06:11
*** Qianbiao has joined #openstack-ironic06:11
*** threestrands_ has quit IRC06:12
*** threestrands_ has joined #openstack-ironic06:12
*** threestrands_ has quit IRC06:13
*** threestrands_ has joined #openstack-ironic06:14
*** threestrands_ has quit IRC06:15
*** threestrands_ has joined #openstack-ironic06:15
*** threestrands_ has quit IRC06:16
*** threestrands_ has joined #openstack-ironic06:17
*** threestrands_ has quit IRC06:18
*** threestrands_ has joined #openstack-ironic06:18
*** threestrands_ has quit IRC06:19
*** threestrands_ has joined #openstack-ironic06:20
*** threestrands_ has quit IRC06:21
*** threestrands_ has joined #openstack-ironic06:21
*** threestrands_ has quit IRC06:22
*** threestrands_ has joined #openstack-ironic06:23
*** threestrands_ has quit IRC06:24
*** threestrands_ has joined #openstack-ironic06:24
*** threestrands_ has quit IRC06:25
*** threestrands_ has joined #openstack-ironic06:26
*** Qianbiao has quit IRC06:26
*** threestrands_ has quit IRC06:27
*** threestrands_ has joined #openstack-ironic06:27
*** threestrands_ has quit IRC06:28
*** threestrands_ has joined #openstack-ironic06:29
*** threestrands_ has quit IRC06:30
*** threestrands_ has joined #openstack-ironic06:30
*** threestrands_ has quit IRC06:31
*** threestrands_ has joined #openstack-ironic06:32
*** threestrands_ has quit IRC06:33
*** threestrands_ has joined #openstack-ironic06:33
*** threestrands_ has quit IRC06:34
*** threestrands_ has joined #openstack-ironic06:35
*** threestrands_ has quit IRC06:36
*** threestrands_ has joined #openstack-ironic06:36
*** threestrands_ has quit IRC06:37
*** threestrands_ has joined #openstack-ironic06:38
*** threestrands_ has quit IRC06:39
*** threestrands_ has joined #openstack-ironic06:39
*** threestrands_ has quit IRC06:40
*** threestrands_ has joined #openstack-ironic06:41
*** Qianbiao has joined #openstack-ironic06:41
*** threestrands_ has quit IRC06:42
*** threestrands_ has joined #openstack-ironic06:42
*** threestrands_ has quit IRC06:43
*** threestrands_ has joined #openstack-ironic06:44
*** threestrands_ has quit IRC06:45
*** threestrands_ has joined #openstack-ironic06:45
*** threestrands_ has quit IRC06:46
*** threestrands_ has joined #openstack-ironic06:47
*** threestrands_ has quit IRC06:48
*** threestrands_ has joined #openstack-ironic06:48
*** threestrands_ has quit IRC06:49
*** threestrands_ has joined #openstack-ironic06:50
*** threestrands_ has quit IRC06:51
*** threestrands_ has joined #openstack-ironic06:51
*** threestrands_ has quit IRC06:52
*** threestrands_ has joined #openstack-ironic06:53
*** threestrands_ has quit IRC06:54
*** threestrands_ has joined #openstack-ironic06:54
*** threestrands_ has quit IRC06:55
*** threestrands_ has joined #openstack-ironic06:56
*** threestrands_ has quit IRC06:57
*** rpittau|afk is now known as rpittau06:57
rpittaugood morning ironic! o/06:57
*** threestrands_ has joined #openstack-ironic06:57
*** threestrands_ has quit IRC06:58
*** threestrands_ has joined #openstack-ironic06:59
*** threestrands_ has quit IRC07:00
*** threestrands_ has joined #openstack-ironic07:00
iurygregorygood morning rpittau and Ironic o/07:00
*** threestrands_ has quit IRC07:01
*** threestrands_ has joined #openstack-ironic07:02
rpittauhey iurygregory :)07:02
*** threestrands_ has quit IRC07:03
*** threestrands_ has joined #openstack-ironic07:03
*** threestrands_ has quit IRC07:04
*** threestrands_ has joined #openstack-ironic07:05
*** threestrands_ has quit IRC07:06
*** threestrands_ has joined #openstack-ironic07:06
*** threestrands_ has quit IRC07:07
*** mkrai_ has quit IRC07:24
*** dougsz has joined #openstack-ironic07:28
*** rcernin has quit IRC07:31
*** dtantsur|afk is now known as dtantsur07:41
dtantsurmorning ironic07:42
iurygregorymorning dtantsur07:43
*** mkrai_ has joined #openstack-ironic07:48
rpittauhey dtantsur :)07:54
openstackgerritMerged openstack/ironic master: Document http_basic auth strategy  https://review.opendev.org/73548207:54
*** Habib-A has joined #openstack-ironic08:00
Habib-Ahello is anyone here ?!08:00
iurygregoryyup08:02
Habib-Athank god finally someone answered :D08:03
Habib-Acan you help me im having trouble to provision a physical server using ironic08:03
iurygregorywe can try to help08:04
Habib-Athe problem is that my server is ILO08:04
Habib-Ag808:04
Habib-Awith RAID08:04
Habib-Aand i need to build an image with proliant-tools08:04
iurygregorycorrect08:04
Habib-Aand the fedora image keeps failing to get IP and heartbeat08:05
Habib-Awith conducter08:05
Habib-Aevent though if i use the pre built image tinyipa08:05
Habib-Ait heartbeats but fails with erasing the disks saying something about sense key protected device about the /dev/sda08:06
*** lucasagomes has joined #openstack-ironic08:06
Habib-Aand the hdd not being ATA connected which makes snese08:06
Habib-Aso can you build me an image with proliant-tools and devuser for debug -- OS ubuntu/fedora for the support of the proliant-tools package08:07
iurygregoryhow are you trying to build your image?08:07
iurygregoryhttps://docs.openstack.org/ironic-python-agent/latest/admin/troubleshooting this guide helps if you need to add a key to your image so you can debug08:08
Habib-Ai will tryit08:09
*** mkrai_ has quit IRC08:10
Habib-Ai noticed  that the fedora image was  trying to use IPv6 and i only have v4 in my environment do you know if i could disable that ?08:10
Habib-Aim booting using virtual media08:10
*** Lucas_Gray has joined #openstack-ironic08:11
Habib-Afollowing is the command im using08:11
Habib-Aironic-python-agent-builder -o devall-proliant-agent-ramdisk -e proliant-tools fedora -e dhcp-all-interfaces -e baremetal -e grub2 -e iso -e devuser08:11
Habib-Aso i dont think it got an IP for me to try to login to08:15
iurygregoryfor ipv6 it seems you can disable if the second argument of the dhcp-all-interfaces is false https://github.com/openstack/diskimage-builder/blob/9b00377f31b094b1ca4dbc53b28058195b2ac987/diskimage_builder/elements/dhcp-all-interfaces/install.d/dhcp-all-interfaces.sh#L5008:18
iurygregorydid you build the virtual media with the ironic-python-agent-builder ?08:19
*** mkrai has joined #openstack-ironic08:22
Habib-Ayes i used  ironic-python-agent-builder cause DIB kept giving different error msgs while ipa builder successfully builds08:31
Habib-Awill try to disable dhcp-all-interfaces08:32
*** lucasagomes has quit IRC08:42
*** lucasagomes has joined #openstack-ironic08:43
*** priteau has joined #openstack-ironic08:45
*** lucasagomes has quit IRC09:06
*** lucasagomes has joined #openstack-ironic09:08
Qianbiaohello ironic.09:54
Qianbiaohi, tox-docs ci failed.09:54
Qianbiaohttps://zuul.opendev.org/t/openstack/build/1ebeb26a1f8d4f9ea814e3122e4ca13909:54
Qianbiaobut did not say why09:54
rpittauQianbiao: have you tried reproducing it locally ?09:58
rpittauQianbiao: btw the error is here https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_1eb/735276/1/check/openstack-tox-docs/1ebeb26/sphinx-build-pdf.log09:58
Qianbiaohmm rpittau i think this may cause by ci env?09:59
rpittaulooking at the error it seems so09:59
iurygregoryif I'm creating a node with boot_mode=uefi, do I also need to set this in the flavor or it should be ok?10:01
QianbiaoI think we should have a trigger word like "recheck-failed-jobs"10:04
*** k_mouza has joined #openstack-ironic10:06
dtantsurQianbiao: this has been repeatedly proposed and rejected by the infra team10:14
dtantsurI miss such a command too10:14
rpittauI miss that too :/10:14
iurygregoryits sad when docs/unit tests show failures10:15
QianbiaoIt must be very pretty, we all miss HE.10:15
iurygregorymaybe zuul could do something like (run unit/docs jobs) if all pass run functional jobs hehe10:16
dtantsuriurygregory: yep, this is possible10:16
iurygregoryreally?10:17
iurygregory=O10:17
dtantsurbut it will make testing WIP patches much harder since you will have to have unit tests passing before you get any results10:17
rpittauit's also kind of tricky from qa perspective though, because in the meantime requirements or other things can change10:17
iurygregorygotcha10:17
dtantsurrpittau: well, the gate pipeline will always run all jobs10:17
dtantsurit's more about the check pipeline10:17
rpittaubut in the end you will have to recheck anyway, no ?10:17
dtantsurI think some tripleo projects do this10:17
*** dmellado has quit IRC10:18
Qianbiaodtantsur the question i asked yestoday. about restart bmc when running steps like apply_raid_config10:20
Qianbiaois it possible?10:20
dtantsurQianbiao: sounds possible, what problems do you envision?10:20
Qianbiaoibmc has inband raid support, but it needs several restart of bmc.10:20
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in test_rpc_api and test_steps  https://review.opendev.org/73413210:24
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in test_base_manager  https://review.opendev.org/73481110:24
*** derekh has joined #openstack-ironic10:25
Qianbiaodtantsur: * Not restart bmc, but resatrt ComputerSystem.10:25
Qianbiaobut i think it's same, because both will quite tiny-os10:25
dtantsurQianbiao: I think idrac also restarts quite often, should be fine10:25
dtantsuryou can check the idrac code for inspiration10:26
Qianbiaook10:26
*** mkrai has quit IRC10:34
*** rpittau is now known as rpittau|bbl10:34
*** k_mouza has quit IRC10:36
Qianbiaodtantsur idrac is not "real" reboot.10:37
Qianbiaoit uses async job to finish step and check job status after reboot.10:37
Qianbiaowe are a bit different. we need a reboot to get storage info.10:38
Qianbiaoand then we can commit raid config, and then we need a reboot again to make it work ~~10:39
*** dmellado has joined #openstack-ironic10:41
openstackgerritMerged openstack/ironic-python-agent master: Fix apidoc build  https://review.opendev.org/73471210:41
*** sshnaidm is now known as sshnaidm|afk10:45
*** k_mouza has joined #openstack-ironic10:59
*** Qianbiao has quit IRC11:32
*** k_mouza has quit IRC11:34
*** k_mouza has joined #openstack-ironic11:35
*** sshnaidm|afk is now known as sshnaidm11:40
*** k_mouza has quit IRC11:45
*** priteau has quit IRC11:47
*** k_mouza has joined #openstack-ironic11:48
*** rh-jelabarre has joined #openstack-ironic12:14
*** rpittau|bbl is now known as rpittau12:28
hjensasHi Ironic, since yesterday my ironic provisioning is failing. Agent boot's and set's up iSCSI target, then it keeps heartbeating.12:29
hjensasI can see COMMAND=/usr/bin/ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m discovery -t st -p 192.168.25.24:3260 is executed on the conductor, but not devices are initialized.12:30
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector master: [WIP] Leader election for periodic sync with ironic  https://review.opendev.org/73552712:31
*** tkajinam has quit IRC12:31
hjensasOn the agent, I see 'ip_address': '[::]' in the 'portals' section. Is the agent listening with the wrong ip version?12:31
hjensasagent log snippets - http://paste.openstack.org/show/794808/12:31
dtantsur:: doesn't sound immediately incorrect12:33
*** derekh has quit IRC12:34
hjensasdtantsur: agree, it can be it's listening on all ips.12:34
*** rloo has joined #openstack-ironic12:39
*** derekh has joined #openstack-ironic12:39
*** Qianbiao has joined #openstack-ironic12:39
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Update IPA if update_repos is true (the default)  https://review.opendev.org/73589912:40
*** Goneri has joined #openstack-ironic13:03
*** cdearborn has joined #openstack-ironic13:14
TheJuliaI feel like I'm needing to drink more and more coffee to wake up13:22
TheJuliaI think I need a new plan in that regard13:22
*** Lucas_Gray has quit IRC13:27
*** ociuhandu has quit IRC13:31
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in test_neutron  https://review.opendev.org/73591513:36
*** uzumaki has joined #openstack-ironic13:40
openstackgerritMark Goddard proposed openstack/tenks stable/1.0: Add six to requirements.txt for wait_for_resources  https://review.opendev.org/73591713:42
*** mkrai has joined #openstack-ironic13:43
iurygregorygood morning TheJulia13:43
*** uzumaki has quit IRC13:45
*** mkrai has quit IRC13:47
*** tzumainn has joined #openstack-ironic13:48
*** ricolin has quit IRC13:53
*** jdandrea has joined #openstack-ironic13:54
*** tkajinam has joined #openstack-ironic13:57
*** mkrai has joined #openstack-ironic13:58
k_mouzaGood morning all! I'm having an issue with ironic when trying to create a new baremetal instance. I've a stein environment, installed with kolla-ansible, and already some baremetal nodes are running for some time. Now, when trying to create a new instance I'm getting the error listed here: https://pastebin.com/AsEEvmcP. Anyone faced this before?13:58
k_mouzaThe ironic_conductor container can reach the API IP fine, by the way (it has an interface directly connected to that network).13:59
*** Goneri has quit IRC14:00
TheJuliak_mouza: is the endpoint actually a a load balancer like haproxy?14:00
dtantsurTheJulia: morning14:02
k_mouzaTheJulia: yup, using haproxy, as I'm using 3 controller/network nodes. Ironic conductor is running only on one of them though.14:03
*** jtwill98 has joined #openstack-ironic14:06
*** Lucas_Gray has joined #openstack-ironic14:09
*** ociuhandu has joined #openstack-ironic14:11
*** v1k0d3n_ has joined #openstack-ironic14:12
*** dmellado_ has joined #openstack-ironic14:12
*** ianychoi_ has joined #openstack-ironic14:12
*** Goneri has joined #openstack-ironic14:14
*** mnaser has quit IRC14:14
*** ianychoi has quit IRC14:15
*** v1k0d3n has quit IRC14:16
*** dmellado has quit IRC14:16
TheJuliak_mouza: sounds like you have a load balancing config issue then because the client is opening a socket and and it is getting abruptly and unexpectedly hung up on14:16
*** portdirect has quit IRC14:16
*** gryf has quit IRC14:16
*** dmellado_ is now known as dmellado14:16
*** v1k0d3n_ is now known as v1k0d3n14:16
*** cdearborn has quit IRC14:16
*** rh-jelabarre has quit IRC14:16
*** mnaser has joined #openstack-ironic14:16
*** ociuhandu has quit IRC14:17
*** cdearborn has joined #openstack-ironic14:17
*** dsneddon has quit IRC14:18
*** dsneddon has joined #openstack-ironic14:18
openstackgerritMark Goddard proposed openstack/tenks stable/1.0: CI: multiple fixes  https://review.opendev.org/73591714:23
*** mkrai has quit IRC14:31
*** mkrai has joined #openstack-ironic14:31
openstackgerritMark Goddard proposed openstack/tenks stable/1.0: CI: multiple fixes  https://review.opendev.org/73591714:33
dtantsurTheJulia: do you remember why bifrost defaults to automated_clean=False?14:42
*** gryf has joined #openstack-ironic14:43
TheJuliadtantsur: that was before we had metadata only cleaning14:48
dtantsurah14:48
TheJuliaso it would be fine to switch it over14:48
dtantsurmaybe we should turn it on?14:48
dtantsuralso, have we got a solution for " Agent driver requires agent_url in driver_internal_info"?14:48
TheJuliaAlso, I nuked a number of hard disks in my home lab14:48
TheJuliapoor poor sata disks14:48
TheJuliadtantsur: got a bug or link for the latter item? I'm... kind of blanking on that at the moment14:49
dtantsurI guess fast track does not work because of ^^^ actually14:49
dtantsurTheJulia: nope, just trying fast track on bifrost14:49
TheJuliaahh14:49
dtantsurand indeed, there is no agent_url14:49
TheJuliahmm14:49
TheJuliasounds like it hasn't looked up/heartbeated yet14:49
dtantsurall other agent stuff is there, but not agent_url14:49
TheJuliaoh that is funky14:49
dtantsurTheJulia: oh, and something also wiped the agent token, so heartbeats stopped working14:51
TheJuliasteps your taking to reproduce?14:51
TheJuliaI thought i fixed that bug14:51
*** yaawang has quit IRC14:52
dtantsurTheJulia: 1. discover a node, 2. provide, 3. try to deploy14:52
*** ociuhandu has joined #openstack-ironic14:52
*** yaawang has joined #openstack-ironic14:52
TheJuliathat may be another distinct path issue14:52
TheJuliaon the provide verb I bet we're nuking it14:52
TheJuliaeww14:53
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Fix fast-track deployment after discovery/inspection  https://review.opendev.org/73593314:53
dtantsurfirst bug I found ^^14:53
TheJuliaI have 1 or two quickish meetings and then I should be caught up necessary bug replies/comments and go from there.14:55
TheJuliaand by go from there I could likely fix that issue since I bet I know what it is14:55
dtantsuroh nice14:55
*** Qianbiao has quit IRC14:58
*** Qianbiao has joined #openstack-ironic14:58
openstackgerritJames Denton proposed openstack/networking-generic-switch master: Support multiple links in link_local_information dict of portgroups  https://review.opendev.org/55705114:59
*** Qianbiao has quit IRC15:00
*** Qianbiao has joined #openstack-ironic15:00
*** diurnalist has joined #openstack-ironic15:04
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Make bifrost more metalsmith-friendly  https://review.opendev.org/73593615:05
*** Qianbiao has quit IRC15:10
*** Qianbiao has joined #openstack-ironic15:11
*** Qianbiao has quit IRC15:12
*** zbitter has joined #openstack-ironic15:19
*** zaneb has quit IRC15:20
dtantsurTheJulia: I've found that we wipe agent_token on tear down, looking more15:22
dtantsurmmm, okay, I've found all places15:23
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent-builder master: [WIP] Use script-dir option for pip ipa install  https://review.opendev.org/73594015:25
* dtantsur prepares a patch15:25
TheJuliaexcellent15:26
TheJuliaThat was basically what I was thinking, we do it on incoming verb as well, but I might have missed on the of the cases15:26
dtantsurI think I've caught 3 places total15:29
*** mkrai has quit IRC15:30
dtantsur4 now15:35
TheJuliawe'll need to carefully review it15:38
dtantsurI'm trying to make sure all code uses the helpers from conductor/utils15:39
TheJuliagood plan15:39
* TheJulia wishes she knew why she felt so tired today15:41
*** jtwill98 has quit IRC15:41
dtantsurTheJulia: do you recall why fast_track_able is False when node.last_error is not None?15:42
dtantsurI'm wondering if we should keep it this way or make it work after deploy/clean failure15:42
dtantsur... and how it all plays with credential-less deployments15:44
TheJuliaI think it was because even transient errors likely shouldn't proceed through15:45
* dtantsur brain boiling15:45
TheJuliabut we have few places where we log something15:45
dtantsuryeah, it's probably fair. it also means I should NOT rely on is_fast_track in the agent power interface15:45
TheJuliathat is likely a special case15:46
TheJuliaI'd need to think about, tbh15:46
dtantsuryeah, me too15:46
* dtantsur gets back to fixing what is supposed to work right now15:46
k_mouzaTheJulia: Thanks for the tip above about the load balancing config, although I can't see anything suspicious looking there... Another thing that I forgot to mention is that the error was initially a "Connection timeout" one, and the above appeared after I increased the timeout in the [neutron] part of the ironic conductor config. It's a weird error tbh...!15:47
TheJuliak_mouza: what did you set the timeout to?15:47
k_mouzaTheJulia: 360 !15:48
TheJuliak_mouza: too high15:48
TheJuliak_mouza: http keepalive allows the server to close the socket after like 240 seconds15:48
TheJuliaload balancers also may pre stage sockets/connections15:49
TheJuliak_mouza: I suspect the clients are not using http keepalive, fwiw15:49
openstackgerritRiccardo Pittau proposed openstack/ironic master: Convert ironic-tempest-ipa-wholedisk-bios-agent_ipmitool to dib  https://review.opendev.org/73475915:49
TheJuliaso combining the two features with a load balancer will get you some odd behaviors15:50
* TheJulia is resource of largely obscure knowledge apparently15:50
*** gyee has joined #openstack-ironic15:56
openstackgerritDmitry Tantsur proposed openstack/ironic master: Fix agent token and URL handling during fast-track deployment  https://review.opendev.org/73595516:07
dtantsurTheJulia: this ^^ should hopefully be it16:07
* dtantsur will test everything again tomorrow16:07
*** zbitter is now known as zaneb16:09
*** rh-jelabarre has joined #openstack-ironic16:09
rpittaugood night! o/16:12
*** rpittau is now known as rpittau|afk16:12
*** dtantsur is now known as dtantsur|afk16:14
dtantsur|afko/16:14
*** mkrai has joined #openstack-ironic16:16
iurygregoryyay I got a scenario working testing uefi with wholedisk and uefi with partition images \o/16:16
dtantsur|afkcongrats :)16:17
iurygregoryI had some headaches to get this working hehe16:18
*** ociuhandu has quit IRC16:20
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Add tests for uefi  https://review.opendev.org/73596016:20
Habib-Acan someone help me my node is stuck in clean wait i read the log but im not sure whats making it stuck it just keeps heartbeating after metadata destroy complete https://pastebin.com/aRcKEpmr16:23
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: [DNM] Testing uefi scenario  https://review.opendev.org/71445316:25
*** lucasagomes has quit IRC16:27
*** dking has joined #openstack-ironic16:29
*** mkrai has quit IRC16:32
*** Lucas_Gray has quit IRC16:32
*** dougsz has quit IRC16:32
*** ociuhandu has joined #openstack-ironic16:37
-openstackstatus- NOTICE: Zuul is being restarted for an urgent configuration change and may be offline for 15-30 minutes. Patches uploaded or approved during that time will need to be rechecked.16:38
openstackgerritMark Goddard proposed openstack/tenks stable/1.0: CI: multiple fixes  https://review.opendev.org/73591716:41
TheJuliadtantsur|afk: my last call of the day is almost over, I'll take a look :) Thanks!16:53
TheJuliaiurygregory: headaches?16:53
TheJuliaHabib-A: Hi16:54
TheJuliaHabib-A: so.. I have a little bad news for you: Your still doing erase devices and thus the wipe of the devices:  {'priority': 10, 'interface': 'deploy', 'step': 'erase_devices', 'abortable': True, 'reboot_requested': False}16:55
TheJuliaif you look look at the node *clean_step* fields, that would likely indicate it is doing that16:55
TheJuliai guess the question is how long has it been in that state?16:56
iurygregoryTheJulia, small ones =)  running devstack, running tempest, trying to figure out why tests were failing etc xD16:57
TheJuliaoh, the "normal" ones16:58
iurygregoryyeah16:58
TheJulia:(16:58
iurygregoryno need for medicine XD16:59
iurygregorytime to do some exercises, bye o/17:00
*** derekh has quit IRC17:00
TheJulia+1million17:00
k_mouzaTheJulia: I was playing with the timeout values for neutron in the conductor and it looks like that for anything larger than 60, the timeout error turns into the "Connection aborted" one. Totally buffled with this one...17:51
TheJuliadtantsur|afk: I think the change is good, makes sene and consolidates down the number of places so ++++17:57
TheJuliak_mouza: taking a step back, why adjust the timeouts?17:57
TheJuliak_mouza: I guess I'm trying to understand what set things down the path17:58
k_mouzaTheJulia: well, the default timeout value is giving me a timeout error in the logs when creating a new instance (https://pastebin.com/vrKK9bLR). So I tried to increase it to see if it goes through, but then I get the initial error I posted above (https://pastebin.com/AsEEvmcP)18:00
k_mouzabasically, trying to find what's broken when trying to add a new instance. (The APIs are reachable from within the docker containers btw)18:01
TheJuliaoh18:05
iurygregoryzuul is back =)18:05
TheJuliak_mouza: intentiaonlly are those from two different time windows?18:06
TheJuliak_mouza: master branch or ussuri?18:06
TheJuliaor older?18:07
-openstackstatus- NOTICE: Zuul is back online; changes uploaded or approved between 16:40 and 18:00 will need to be rechecked.18:07
TheJuliaoh, hmm18:09
TheJuliak_mouza: do you see anything in the neutron log corresponding to a post operation against the /ports resource18:09
TheJulia?18:09
*** mugsie has quit IRC18:11
*** mugsie has joined #openstack-ironic18:15
TheJuliastevebaker: for when your up, https://review.opendev.org/#/c/735757 a two super minor quick things you can fix and if there is nothing different I'll +2+A it18:19
patchbotpatch 735757 - ironic - Enable HTTP Basic authentication for JSON-RPC - 1 patch set18:19
TheJuliak_mouza: I guess my thought behind if we know the post is happening, we can at least begin to kind of sort it out18:20
TheJuliathe post should jsut be record creation in the db, which makes me suspect load balancer, even more, fwiw18:21
TheJuliaI'm going to take a short nap, maybe that will help me feel more human18:22
*** iurygregory has quit IRC18:33
*** dougsz has joined #openstack-ironic18:52
openstackgerritTzu-Mainn Chen proposed openstack/ironic master: Allow node lessee to see node's ports  https://review.opendev.org/73036619:04
*** naved001 has quit IRC19:06
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Allow operators to disable clean_up sync  https://review.opendev.org/73521219:34
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: DNM: Test change for disabling periodic sync  https://review.opendev.org/73525019:36
*** diurnalist has quit IRC19:43
openstackgerritJulia Kreger proposed openstack/ironic-specs master: Victoria Cycle Priorities  https://review.opendev.org/72010019:44
*** diurnalist has joined #openstack-ironic19:44
*** diurnalist has quit IRC19:50
*** diurnalist has joined #openstack-ironic19:51
*** diurnalist has quit IRC19:56
*** iurygregory has joined #openstack-ironic19:59
*** dougsz has quit IRC19:59
openstackgerritVerification of a change to openstack/ironic failed: agent: do not hard power off if soft power off actually succeeds  https://review.opendev.org/73167320:02
*** jdandrea has quit IRC20:03
*** diurnalist has joined #openstack-ironic20:03
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic stable/ussuri: devstack: Use uwsgi binary from path; temporary disable grenade  https://review.opendev.org/73601320:11
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic stable/train: devstack: Use uwsgi binary from path; temporary disable grenade  https://review.opendev.org/73601520:19
*** diurnalist has quit IRC20:20
*** diurnalist has joined #openstack-ironic20:21
iurygregoryussuri, train, stein... tks uwsgi =(20:22
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic stable/stein: devstack: Use uwsgi binary from path; temporary disable grenade  https://review.opendev.org/73601720:22
*** Lucas_Gray has joined #openstack-ironic20:23
TheJulia*sigh*20:34
TheJuliaon a plus side, we should be able to re-enable grenade20:34
iurygregoryyeah20:37
iurygregoryfor now train and stein are broken due ansible 2.9 and the fix is https://review.opendev.org/736006 , frickler told that uwsgi will also affect them so I pushed the backports20:39
patchbotpatch 736006 - devstack-gate - Ansible 2.9 fixes; fix match filter and sudo flags - 2 patch sets20:39
clarkbiurygregory: ya I'ev brought up pinning openstack zuul tenant back to ansible 2.8 in order to get uwsgi fixes in20:41
iurygregoryclarkb ++20:42
TheJulia*sigh*20:42
*** Lucas_Gray has quit IRC20:51
TheJuliawheeeeee http://paste.openstack.org/show/794834/20:58
*** Lucas_Gray has joined #openstack-ironic20:59
openstackgerritSteve Baker proposed openstack/ironic master: Enable HTTP Basic authentication for JSON-RPC  https://review.opendev.org/73575721:07
TheJuliastevebaker: thanks!21:12
stevebakerTheJulia: np!21:13
*** Lucas_Gray has quit IRC21:50
*** Lucas_Gray has joined #openstack-ironic21:52
*** bdodd has quit IRC21:54
*** bdodd has joined #openstack-ironic21:55
*** yaawang_ has joined #openstack-ironic21:59
*** yaawang has quit IRC21:59
*** alexmcleod_ has joined #openstack-ironic22:03
*** alexmcleod has quit IRC22:03
*** dtantsur|afk has quit IRC22:04
*** andy_ has quit IRC22:04
*** sziviani has quit IRC22:04
*** rh-jelabarre has quit IRC22:05
*** rh-jelabarre has joined #openstack-ironic22:05
*** irclogbot_1 has quit IRC22:07
*** benj_ has quit IRC22:07
*** stevebaker has quit IRC22:08
*** benj_ has joined #openstack-ironic22:09
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Add timeout operations to try and prevent hang on read()  https://review.opendev.org/73470922:12
*** andy_ has joined #openstack-ironic22:12
*** sziviani has joined #openstack-ironic22:12
*** dtantsur has joined #openstack-ironic22:12
*** irclogbot_2 has joined #openstack-ironic22:13
*** rh-jelabarre has quit IRC22:13
TheJuliadtantsur: I updated https://review.opendev.org/#/c/734709. We can't use eventlet to wrap things, monkey patching kind of blows things up and the risk is relatively low of the only thing that we CAN use eventlet for to capture the failure. But the stream itself should be good to detect and fail upon.22:13
patchbotpatch 734709 - ironic-python-agent - Add timeout operations to try and prevent hang on ... - 4 patch sets22:13
NobodyCamGood afternoon Ironic folks22:14
*** dtantsur has quit IRC22:18
*** dtantsur has joined #openstack-ironic22:18
*** k_mouza has quit IRC22:22
*** k_mouza has joined #openstack-ironic22:25
TheJuliagood afternoon NobodyCam22:25
*** andy_ has quit IRC22:30
*** sziviani has quit IRC22:30
*** iurygregory has quit IRC22:33
*** mugsie has quit IRC22:33
*** rloo has quit IRC22:33
*** mkowalski has quit IRC22:33
*** mkarpiarz has quit IRC22:33
*** maelk has quit IRC22:33
*** jlvillal has quit IRC22:33
*** fdegir has quit IRC22:33
*** baha has quit IRC22:33
*** seba has quit IRC22:33
*** PrinzElvis has quit IRC22:33
*** Habib-A has quit IRC22:33
*** benj_ has quit IRC22:33
*** michchap has quit IRC22:33
*** johnsom has quit IRC22:33
*** rpioso has quit IRC22:33
*** tonyb[m] has quit IRC22:33
*** johnhaan has quit IRC22:33
*** NobodyCam has quit IRC22:33
*** gyee has quit IRC22:33
*** gryf has quit IRC22:33
*** adrianc has quit IRC22:33
*** shadower has quit IRC22:33
*** TheJulia has quit IRC22:33
*** chihhsin has quit IRC22:33
*** dking has quit IRC22:33
*** zaneb has quit IRC22:33
*** mnaser has quit IRC22:33
*** tkajinam has quit IRC22:33
*** tzumainn has quit IRC22:33
*** eliaswimmer has quit IRC22:33
*** phrobb has quit IRC22:33
*** gmann has quit IRC22:33
*** jungleboyj has quit IRC22:33
*** vkmc has quit IRC22:33
*** arne_wiebalck has quit IRC22:33
*** rajinir has quit IRC22:33
*** andreaf has quit IRC22:33
*** samueldmq has quit IRC22:33
*** ab-a has quit IRC22:33
*** akahat has quit IRC22:33
*** spotz has quit IRC22:33
*** yaawang_ has quit IRC22:33
*** ociuhandu has quit IRC22:33
*** seongsoocho has quit IRC22:33
*** mnasiadka has quit IRC22:33
*** lifeless has quit IRC22:33
*** k_mouza has quit IRC22:33
*** Goneri has quit IRC22:33
*** dmellado has quit IRC22:33
*** zigo has quit IRC22:33
*** vesper11 has quit IRC22:33
*** hjensas has quit IRC22:33
*** erbarr has quit IRC22:33
*** melwitt has quit IRC22:33
*** logan- has quit IRC22:33
*** mcarden has quit IRC22:33
*** Lucas_Gray has quit IRC22:33
*** yolanda has quit IRC22:33
*** ddurst has quit IRC22:33
*** zzzeek has quit IRC22:33
*** stbenjam has quit IRC22:33
*** brtknr has quit IRC22:33
*** jhesketh has quit IRC22:33
*** ftarasenko has quit IRC22:33
*** gudrutis2 has quit IRC22:33
*** irclogbot_2 has quit IRC22:33
*** bfournie has quit IRC22:33
*** mbuil has quit IRC22:33
*** gregwork has quit IRC22:33
*** pas-ha has quit IRC22:33
*** buhman has quit IRC22:33
*** mgoddard has quit IRC22:33
*** dtruong has quit IRC22:33
*** cp- has quit IRC22:33
*** JayF has quit IRC22:33
*** zer0c00l has quit IRC22:33
*** m1p has quit IRC22:33
*** vdrok has quit IRC22:33
*** markmcclain has quit IRC22:33
*** persia has quit IRC22:33
*** ajya has quit IRC22:33
*** mgagne has quit IRC22:33
*** sshnaidm has quit IRC22:33
*** eandersson has quit IRC22:33
*** dansmith has quit IRC22:33
*** ianw has quit IRC22:33
*** radez has quit IRC22:33
*** early has quit IRC22:33
*** openstackgerrit has quit IRC22:33
*** sw3 has quit IRC22:33
*** whizzz has quit IRC22:33
*** johnthetubaguy has quit IRC22:33
*** dtantsur has quit IRC22:33
*** dsneddon has quit IRC22:33
*** cdearborn has quit IRC22:33
*** v1k0d3n has quit IRC22:33
*** bnemec has quit IRC22:33
*** donnyd has quit IRC22:33
*** hoonetorg has quit IRC22:33
*** rpittau|afk has quit IRC22:33
*** jroll has quit IRC22:33
*** fungi has quit IRC22:33
*** alexmcleod_ has quit IRC22:33
*** andrein has quit IRC22:33
*** strigazi has quit IRC22:33
*** SpamapS has quit IRC22:33
*** clarkb has quit IRC22:33
*** bdodd has quit IRC22:33
*** jtomasek has quit IRC22:33
*** dhellmann has quit IRC22:33
*** trident has quit IRC22:33
*** mraineri has quit IRC22:33
*** ildikov has quit IRC22:33
*** jmccrory has quit IRC22:33
*** patchbot has quit IRC22:33
*** rnoriega_ has quit IRC22:33
*** larsks has quit IRC22:33
*** hjensas has joined #openstack-ironic22:37
*** vesper11 has joined #openstack-ironic22:37
*** zigo has joined #openstack-ironic22:37
*** dmellado has joined #openstack-ironic22:37
*** Goneri has joined #openstack-ironic22:37
*** k_mouza has joined #openstack-ironic22:37
*** johnhaan has joined #openstack-ironic22:37
*** NobodyCam has joined #openstack-ironic22:37
*** tonyb[m] has joined #openstack-ironic22:37
*** rpioso has joined #openstack-ironic22:37
*** johnsom has joined #openstack-ironic22:37
*** michchap has joined #openstack-ironic22:37
*** benj_ has joined #openstack-ironic22:37
*** fungi has joined #openstack-ironic22:37
*** jroll has joined #openstack-ironic22:37
*** rpittau|afk has joined #openstack-ironic22:37
*** hoonetorg has joined #openstack-ironic22:37
*** bnemec has joined #openstack-ironic22:37
*** donnyd has joined #openstack-ironic22:37
*** v1k0d3n has joined #openstack-ironic22:37
*** cdearborn has joined #openstack-ironic22:37
*** dsneddon has joined #openstack-ironic22:37
*** dtantsur has joined #openstack-ironic22:37
*** gudrutis2 has joined #openstack-ironic22:37
*** ftarasenko has joined #openstack-ironic22:37
*** jhesketh has joined #openstack-ironic22:37
*** brtknr has joined #openstack-ironic22:37
*** stbenjam has joined #openstack-ironic22:37
*** zzzeek has joined #openstack-ironic22:37
*** ddurst has joined #openstack-ironic22:37
*** yolanda has joined #openstack-ironic22:37
*** Lucas_Gray has joined #openstack-ironic22:37
*** logan- has joined #openstack-ironic22:37
*** mcarden has joined #openstack-ironic22:37
*** melwitt has joined #openstack-ironic22:37
*** erbarr has joined #openstack-ironic22:37
*** chihhsin has joined #openstack-ironic22:37
*** TheJulia has joined #openstack-ironic22:37
*** shadower has joined #openstack-ironic22:37
*** adrianc has joined #openstack-ironic22:37
*** gryf has joined #openstack-ironic22:37
*** gyee has joined #openstack-ironic22:37
*** larsks has joined #openstack-ironic22:37
*** rnoriega_ has joined #openstack-ironic22:37
*** patchbot has joined #openstack-ironic22:37
*** jmccrory has joined #openstack-ironic22:37
*** ildikov has joined #openstack-ironic22:37
*** mraineri has joined #openstack-ironic22:37
*** trident has joined #openstack-ironic22:37
*** dhellmann has joined #openstack-ironic22:37
*** jtomasek has joined #openstack-ironic22:37
*** bdodd has joined #openstack-ironic22:37
*** PrinzElvis has joined #openstack-ironic22:37
*** mkarpiarz has joined #openstack-ironic22:37
*** maelk has joined #openstack-ironic22:37
*** jlvillal has joined #openstack-ironic22:37
*** fdegir has joined #openstack-ironic22:37
*** baha has joined #openstack-ironic22:37
*** seba has joined #openstack-ironic22:37
*** iurygregory has joined #openstack-ironic22:37
*** mugsie has joined #openstack-ironic22:37
*** rloo has joined #openstack-ironic22:37
*** sshnaidm has joined #openstack-ironic22:38
*** eandersson has joined #openstack-ironic22:38
*** radez has joined #openstack-ironic22:38
*** dansmith has joined #openstack-ironic22:38
*** ianw has joined #openstack-ironic22:38
*** early has joined #openstack-ironic22:38
*** openstackgerrit has joined #openstack-ironic22:38
*** sw3 has joined #openstack-ironic22:38
*** whizzz has joined #openstack-ironic22:38
*** johnthetubaguy has joined #openstack-ironic22:38
*** mkowalski has joined #openstack-ironic22:38
*** yaawang_ has joined #openstack-ironic22:38
*** ociuhandu has joined #openstack-ironic22:38
*** lifeless has joined #openstack-ironic22:38
*** m1p has joined #openstack-ironic22:38
*** vdrok has joined #openstack-ironic22:38
*** markmcclain has joined #openstack-ironic22:38
*** mgagne has joined #openstack-ironic22:38
*** persia has joined #openstack-ironic22:38
*** ajya has joined #openstack-ironic22:38
*** irclogbot_2 has joined #openstack-ironic22:38
*** bfournie has joined #openstack-ironic22:38
*** mbuil has joined #openstack-ironic22:38
*** gregwork has joined #openstack-ironic22:38
*** pas-ha has joined #openstack-ironic22:38
*** buhman has joined #openstack-ironic22:38
*** mgoddard has joined #openstack-ironic22:38
*** dtruong has joined #openstack-ironic22:38
*** cp- has joined #openstack-ironic22:38
*** zer0c00l has joined #openstack-ironic22:38
*** JayF has joined #openstack-ironic22:38
*** Habib-A has joined #openstack-ironic22:38
*** dking has joined #openstack-ironic22:39
*** zaneb has joined #openstack-ironic22:39
*** mnaser has joined #openstack-ironic22:39
*** tkajinam has joined #openstack-ironic22:39
*** tzumainn has joined #openstack-ironic22:39
*** eliaswimmer has joined #openstack-ironic22:39
*** phrobb has joined #openstack-ironic22:39
*** gmann has joined #openstack-ironic22:39
*** jungleboyj has joined #openstack-ironic22:39
*** vkmc has joined #openstack-ironic22:39
*** arne_wiebalck has joined #openstack-ironic22:39
*** rajinir has joined #openstack-ironic22:39
*** andreaf has joined #openstack-ironic22:39
*** samueldmq has joined #openstack-ironic22:39
*** ab-a has joined #openstack-ironic22:39
*** spotz has joined #openstack-ironic22:39
*** akahat has joined #openstack-ironic22:39
*** erbarr has quit IRC22:39
* TheJulia blinks22:39
*** mkowalski has quit IRC22:40
*** gregwork has quit IRC22:40
*** samueldmq has quit IRC22:41
*** andy_ has joined #openstack-ironic22:41
*** sziviani has joined #openstack-ironic22:41
*** mnaser has quit IRC22:41
*** gmann has quit IRC22:41
*** jungleboyj has quit IRC22:41
*** samueldmq has joined #openstack-ironic22:42
*** andrein has joined #openstack-ironic22:42
*** strigazi has joined #openstack-ironic22:42
*** SpamapS has joined #openstack-ironic22:42
*** clarkb has joined #openstack-ironic22:42
*** jungleboyj has joined #openstack-ironic22:43
TheJuliarloo: https://review.opendev.org/665835 <-- updated agai22:43
patchbotpatch 665835 - ironic - Block port deletions where vif is present - 13 patch sets22:43
TheJuliaagain22:43
*** gregwork has joined #openstack-ironic22:43
*** erbarr has joined #openstack-ironic22:43
*** gmann has joined #openstack-ironic22:43
*** mnaser has joined #openstack-ironic22:48
*** rcernin has joined #openstack-ironic22:51
NobodyCamAfternoon TheJulia :)22:52
*** rcernin has quit IRC22:53
*** rcernin has joined #openstack-ironic22:54
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Remove cimc_address from address field list  https://review.opendev.org/73606122:56
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Remove cimc_address from address field list  https://review.opendev.org/73606122:57
TheJuliahow goes it NobodyCam22:57
*** mnasiadka has joined #openstack-ironic23:01
*** mkowalski has joined #openstack-ironic23:03
*** k_mouza has quit IRC23:15
*** k_mouza has joined #openstack-ironic23:22
*** k_mouza has quit IRC23:22
*** Goneri has quit IRC23:24
*** diurnalist has quit IRC23:29
*** diurnalist has joined #openstack-ironic23:31
*** diurnalist has quit IRC23:46
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: WIP: Add full download retries  https://review.opendev.org/73606523:46
TheJuliadtantsur: I guess ^^^ is the next logical change beyond identifying a mid-stream failure, make it so we retry and thus the agent tries for a while before a failure is raised.23:47
TheJuliadtantsur: and yes, it is largely a duplication of the retry logic you had. Just seems to "fit the bill"23:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!