Wednesday, 2020-06-17

*** gyee has quit IRC00:01
*** diurnalist has joined #openstack-ironic00:17
*** cdearborn has quit IRC00:20
*** stevebaker has joined #openstack-ironic00:31
*** Goneri has joined #openstack-ironic00:54
*** stevebaker has quit IRC01:38
*** tzumainn has quit IRC01:40
*** Goneri has quit IRC01:48
*** ricolin has joined #openstack-ironic02:01
*** early has quit IRC02:11
*** rloo has quit IRC02:11
*** Lucas_Gray has quit IRC02:12
*** early has joined #openstack-ironic02:30
*** rcernin has quit IRC02:49
*** stevebaker has joined #openstack-ironic02:51
*** rcernin has joined #openstack-ironic02:54
*** mkrai has joined #openstack-ironic02:59
*** rcernin has quit IRC03:08
*** Qianbiao has joined #openstack-ironic03:16
*** rcernin has joined #openstack-ironic03:45
*** rcernin has quit IRC03:45
*** rcernin has joined #openstack-ironic03:50
*** diurnalist has quit IRC05:10
*** Qianbiao has quit IRC05:19
*** threestrands has joined #openstack-ironic05:27
*** threestrands has quit IRC05:33
arne_wiebalckGood morning, ironic!05:39
*** Qianbiao has joined #openstack-ironic06:08
*** Qianbiao has quit IRC06:29
*** Habib-A92 has joined #openstack-ironic06:31
openstackgerritKaifeng Wang proposed openstack/ironic-inspector master: Accept IPv6 link local address during interface validation  https://review.opendev.org/73611006:34
Habib-A92TheJuliathe server had been running for 3 hours in that state before i restarted it in debug mode06:39
Habib-A92do you advise me running it again and waiting more ?!06:40
Habib-A92i had ata erase disabled as the server has 2 7200rpm hdd in raid 1 over hpe controller server model proliant g806:41
Habib-A92i couldn't build an image with proliant-tools or it worked and booted it wouldn't get an IP keeps trying v6 which i don't have implemented06:42
Habib-A92so instead i went for disable ata cause it fails erasing and hope that shred package would erase the disk successfully06:43
*** Qianbiao has joined #openstack-ironic06:47
Habib-ATheJulia :D06:48
Habib-Aim using tinyipa-stable-train06:49
*** rpittau|afk is now known as rpittau06:56
rpittaugood morning ironic! o/06:57
Habib-AGood morning :D06:59
dtantsurmorning ironic07:07
*** diurnalist has joined #openstack-ironic07:07
*** mkrai has quit IRC07:08
dtantsurHabib-A: shredding large disks can take VERY long, hours is normal07:08
rpittaugood morning dtantsur :)07:10
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Disable automatic updates in dnf-based systems  https://review.opendev.org/73533507:11
iurygregorygood morning arne_wiebalck rpittau dtantsur and Ironic o/07:21
rpittauhey iurygregory :)07:21
arne_wiebalckhey iurygregory o/07:21
Habib-Adtantsur its only 1TB disks i will try again and wait07:27
iurygregoryonly 1TB xd07:29
iurygregorythis can take some time07:29
dtantsurby default it means writing 2TiB of random data and 1TiB of zeroes IIRC07:29
iurygregoryyeah07:30
iurygregoryso you can go do other things hehe07:30
rpittauHabib-A: for 1TB 3 hours is kind of normal, can be even more07:30
openstackgerritDmitry Tantsur proposed openstack/ironic master: Fix internal server error on deleting nodes with allocations  https://review.opendev.org/73611807:30
* dtantsur facepalm ^^07:30
iurygregoryI know at the university in Brazil 2TB took 8hrs07:31
iurygregoryyay now we will have stable/ussuri green again since the devstack change is merged \o/07:32
*** dougsz has joined #openstack-ironic07:32
rpittauyeah, I rechecked one patch in ironic-lib, hopefully it will pass07:33
iurygregoryrpittau, probably only after our fix is merged in stable/ussuri also07:33
rpittauiurygregory: in master ?07:34
iurygregoryussuri07:34
iurygregoryI thought you meant ussuri since it was the branch I mentioned =)07:35
* iurygregory is waking up07:35
rpittauoh no, it was a general thing as devstack fix is merged, the recheck is in master07:35
* iurygregory needs a second mug of coffee07:36
iurygregoryrpittau, gotcha =)07:36
*** mkrai has joined #openstack-ironic07:36
*** diurnalist has quit IRC07:40
rpittaummm I'm seeing an 'Invalid or missing agent token received.' in a dib job07:41
openstackgerritDmitry Tantsur proposed openstack/ironic master: Fix internal server error on deleting nodes with allocations  https://review.opendev.org/73611807:44
dtantsurrpittau: maybe https://review.opendev.org/735955 ?07:47
patchbotpatch 735955 - ironic - Fix agent token and URL handling during fast-track... - 1 patch set07:47
*** rcernin has quit IRC07:47
dtantsurI need to test it locally though07:47
rpittaudtantsur: yeah, might be that, the logs are in the ipmitool converted job here in case https://review.opendev.org/73475907:48
patchbotpatch 734759 - ironic - Convert ironic-tempest-ipa-wholedisk-bios-agent_ip... - 7 patch sets07:48
dtantsurrpittau: try rebasing on top of my patch?07:48
rpittauyep07:48
openstackgerritRiccardo Pittau proposed openstack/ironic master: Convert ironic-tempest-ipa-wholedisk-bios-agent_ipmitool to dib  https://review.opendev.org/73475907:52
*** mcarden has quit IRC08:02
openstackgerritRiccardo Pittau proposed openstack/metalsmith master: Use is instead of ambiguous pipe  https://review.opendev.org/73612408:04
*** lucasagomes has joined #openstack-ironic08:05
rpittau^^ this should fix the error in the metalsmith jobs we're seeing08:06
rpittau'Missing sudo password' error, ok....08:07
openstackgerritOpenStack Proposal Bot proposed openstack/ironic-inspector master: Imported Translations from Zanata  https://review.opendev.org/73612608:07
*** Lucas_Gray has joined #openstack-ironic08:24
*** k_mouza has joined #openstack-ironic08:31
*** Lucas_Gray has quit IRC08:35
*** k_mouza has quit IRC08:35
rpittaugerrit seems particularly slow for me today....08:36
openstackgerritRiccardo Pittau proposed openstack/ironic master: Fix mock callable for Python 3.6 and precedent  https://review.opendev.org/73613108:36
*** Lucas_Gray has joined #openstack-ironic08:36
openstackgerritMerged openstack/ironic-inspector master: Remove cimc_address from address field list  https://review.opendev.org/73606108:38
*** priteau has joined #openstack-ironic08:50
*** Habib-A has quit IRC08:51
*** ociuhandu has quit IRC08:51
iurygregoryrpittau, same here08:51
iurygregoryyesterday was slow also =(08:51
dtantsurand storyboard too08:53
iurygregoryouch =(08:56
*** tkajinam has quit IRC08:57
openstackgerritDmitry Tantsur proposed openstack/ironic master: Networking boot fallback for local boot  https://review.opendev.org/73449008:57
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Agent power interface  https://review.opendev.org/73559108:57
arne_wiebalckiurygregory, rpittau Could one of you have a look at the (trivial doc) change in https://review.opendev.org/#/c/734806, please?09:00
patchbotpatch 734806 - ironic - [doc] Fix indentation in admin/troubleshooting - 2 patch sets09:00
*** Habib-A92 has quit IRC09:00
iurygregoryarne_wiebalck, doing now09:00
arne_wiebalckiurygregory: thanks!09:00
iurygregorynova valid host OMG09:00
arne_wiebalck:-D09:01
arne_wiebalck(non-intentional)09:01
* rpittau gives coffee to gerrit09:02
*** mkrai has quit IRC09:08
*** mkrai_ has joined #openstack-ironic09:08
*** mkrai_ has quit IRC09:09
*** mkrai has joined #openstack-ironic09:09
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Agent power interface  https://review.opendev.org/73559109:10
Qianbiaohello ironic.09:38
*** yaawang_ has quit IRC09:40
*** diurnalist has joined #openstack-ironic09:40
openstackgerritQianBiao Ng proposed openstack/ironic master: Fix: review from dtantsur of 728123  https://review.opendev.org/73527609:45
arne_wiebalckdtantsur: the 'standalone' option in inspector.conf means there is only a single inspector in the deployment? (I am not clear on what "standalone: Whether to run ironic-inspector as a standalone service." means.)09:52
arne_wiebalckdtantsur: this is in the context of the leader election patch09:52
dtantsurarne_wiebalck: it's not an ideal name, it rather means API+conductor in the same provess09:52
dtantsur* process09:52
arne_wiebalckdtantsur: ah, ok09:52
dtantsurusually, this is standalone since for an HA deployment we kind of expect the split to be used. but it's likely not mandatory, especially in your case.09:53
arne_wiebalckdtantsur: this is why I was confused when kaifeng and you asked me about this the other day09:53
arne_wiebalckdtantsur: yeah, we have several standalone inspectors09:53
arne_wiebalckdtantsur: :-D09:53
arne_wiebalckdtantsur: splitting these is still experimental?09:54
dtantsurI guess it is, we haven't fully settled down on AMQP-vs-anything-else09:55
arne_wiebalckin standalone, there is no need for messaging, I guess?09:55
dtantsurcorrect09:56
arne_wiebalckright, so standalone inspector would also allow for easier removal of amqp from a deployment ... but would then block the way to a non-standalone inspector09:58
* arne_wiebalck is thinking about trying out json rpc09:58
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: Add api-ref for indicators API  https://review.opendev.org/73214909:59
Qianbiaorpittau thanks for the review. :)10:03
rpittauQianbiao: yw :)10:03
Qianbiaodtantsur you got a minute for https://review.opendev.org/#/c/735276 ?10:03
patchbotpatch 735276 - ironic - Fix: review from dtantsur of 728123 - 2 patch sets10:03
* arne_wiebalck just realised the non-standalone mode for the inspector came only with Train ... and are nicely documented :)10:06
Qianbiaois there any ironic client that could visit from mobile?10:07
Qianbiaosimple web-based client or mobile app.10:07
iurygregorysomething happened to devstack and the logs doesn't show tempest.conf D: or /etc =(10:07
openstackgerritRiccardo Pittau proposed openstack/ironic master: Fix mock callable for Python 3.6 and precedent  https://review.opendev.org/73613110:09
rpittauiurygregory: is it actually running tempest ?10:09
dtantsurQianbiao: we have ironic-ui which is a horizon panel. not sure how mobile-friendly it is.10:09
Qianbiaodtantsur thanks will have a look at it.10:10
*** diurnalist has quit IRC10:10
iurygregoryrpittau, well https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_024/727480/5/check/ironic-standalone/024304f/testr_results.html =010:11
iurygregory=)10:11
iurygregoryhttps://zuul.opendev.org/t/openstack/build/024304fb457843aca01d2759eb5a6a81/log/job-output.txt#26060 =(10:11
iurygregorysome jobs have10:11
iurygregorybut others dont .-.10:11
iurygregoryI think something changed10:11
rpittauiurygregory: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_024/727480/5/check/ironic-standalone/024304f/controller/logs/tempest_conf.txt10:12
rpittau?10:12
iurygregoryhttps://review.opendev.org/#/c/735955/10:13
patchbotpatch 735955 - ironic - Fix agent token and URL handling during fast-track... - 1 patch set10:13
iurygregorythis one if you look at standalone doesn't has10:13
rpittauthere's no ironic-bm-logs either, something else wrong there10:14
iurygregoryyup10:14
iurygregoryno ironic-bm-logs no etc10:14
iurygregoryno tempest.conf10:14
rpittauno, I mean, something wrong in ironic10:15
rpittauand indeed Jun 17 08:02:36.096098 ubuntu-bionic-vexxhost-ca-ymq-1-0017188447 ironic-conductor[25404]: ERROR ironic.conductor.manager [None req-40bb5c66-6647-48e5-8a45-c255a723c255 None None] Invalid agent_token receieved for node d51d098d-db38-4d77-9b67-806d173034a710:15
rpittauiurygregory: https://83cd06ba4ab6fcb10ece-e95fa72ee90c44566fe24f9777fa290e.ssl.cf1.rackcdn.com/735955/1/check/ironic-standalone/da4a72a/controller/logs/screen-ir-cond.txt10:15
rpittausame issue I saw before, dtantsur patch should fix that hopefully10:15
*** derekh has joined #openstack-ironic10:17
*** yaawang_ has joined #openstack-ironic10:17
*** rpittau is now known as rpittau|bbl10:18
iurygregorydo you think thats why we are not getting the full logs?10:18
rpittau|bblyeah, I think the node doesn't finish even the first run and it just doesn't log anything, it sounds weird but it might be, I saw it happening in other jobs too10:20
* rpittau|bbl lunch ttyl10:20
iurygregoryhumm gotcha10:20
iurygregoryI asked qa if they know something10:20
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in test_rpc_api and test_steps  https://review.opendev.org/73413210:20
*** k_mouza has joined #openstack-ironic10:20
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in test_rpc_api and test_steps  https://review.opendev.org/73413210:21
openstackgerritMerged openstack/ironic-lib master: Use unittest.mock instead of mock  https://review.opendev.org/73435210:24
openstackgerritMerged openstack/ironic master: [doc] Fix indentation in admin/troubleshooting  https://review.opendev.org/73480610:24
iurygregorywell not only our project is affected10:25
iurygregoryso something changed that is not in our control hehe10:26
iurygregorydtantsur, I noticed that metalsmith-integration-glance-localboot-centos7 is failling quite a lot in stable/ussuri with RETRY_LIMIT10:30
iurygregorynot quite a lot, but it's falling sorry =(10:31
dtantsuriurygregory: maybe the same problem that rpittau|bbl fixed recently?10:32
* iurygregory looks for the patch10:32
iurygregoryhttps://review.opendev.org/#/c/736124/ ?10:33
patchbotpatch 736124 - metalsmith - Use is instead of ambiguous pipe - 1 patch set10:33
dtantsuryeah10:35
openstackgerritVerification of a change to openstack/tenks failed: CI: multiple fixes  https://review.opendev.org/73591710:36
iurygregorysad "that seems to be a general issue, possibly related to the ansible upgrade" =(10:39
iurygregoryno config files or ipa logs in ironic-bm-logs for us10:40
openstackgerritMerged openstack/metalsmith master: Use is instead of ambiguous pipe  https://review.opendev.org/73612410:47
openstackgerritIury Gregory Melo Ferreira proposed openstack/metalsmith stable/ussuri: Use is instead of ambiguous pipe  https://review.opendev.org/73615310:49
*** Lucas_Gray has quit IRC10:49
iurygregorydtantsur, I pushed the backport to stable/ussuri10:49
*** mkrai has quit IRC10:51
*** mkrai_ has joined #openstack-ironic10:51
*** derekh has quit IRC11:00
*** derekh has joined #openstack-ironic11:00
*** k_mouza has quit IRC11:04
*** k_mouza has joined #openstack-ironic11:08
openstackgerritMerged openstack/ironic stable/train: agent: do not hard power off if soft power off actually succeeds  https://review.opendev.org/73167311:08
openstackgerritMerged openstack/ironic-inspector master: Imported Translations from Zanata  https://review.opendev.org/73612611:08
openstackgerritVerification of a change to openstack/tenks failed: CI: multiple fixes  https://review.opendev.org/73591711:08
*** dougsz has quit IRC11:09
*** mkrai_ has quit IRC11:10
*** ociuhandu has joined #openstack-ironic11:12
dtantsuriurygregory: thanks, let's see how it goes11:14
*** dougsz has joined #openstack-ironic11:24
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Fix fast-track with the direct deploy interface  https://review.opendev.org/73616411:27
dtantsurTheJulia: moar fast-track fixes ^^^11:27
openstackgerritDmitry Tantsur proposed openstack/ironic master: Networking boot fallback for local boot  https://review.opendev.org/73449011:30
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Agent power interface  https://review.opendev.org/73559111:30
*** sshnaidm_ has joined #openstack-ironic11:32
*** sshnaidm has quit IRC11:33
*** dougsz has quit IRC11:35
*** sshnaidm_ is now known as sshnaidm11:35
*** mkrai_ has joined #openstack-ironic11:42
*** Lucas_Gray has joined #openstack-ironic12:02
openstackgerritIury Gregory Melo Ferreira proposed openstack/metalsmith stable/train: Use is instead of ambiguous pipe  https://review.opendev.org/73617012:04
openstackgerritIury Gregory Melo Ferreira proposed openstack/metalsmith stable/stein: Use is instead of ambiguous pipe  https://review.opendev.org/73617212:05
*** rpittau|bbl is now known as rpittau12:06
*** diurnalist has joined #openstack-ironic12:07
*** Lucas_Gray has quit IRC12:16
*** Lucas_Gray has joined #openstack-ironic12:19
rpittauI was too optimistic for the ironic-bm-logs missing :(12:22
*** Wryhder has joined #openstack-ironic12:23
iurygregoryrpittau, yup =)12:24
*** Lucas_Gray has quit IRC12:24
iurygregorynow we need to use black magic to debug failures \o/12:24
*** Wryhder has quit IRC12:28
rpittaudtantsur: even rebasing on  top of your patch I'm still seeing 'Invalid or missing agent token received'12:30
rpittauand of course not having the ipa ramdisk logs doesn't really help....12:30
dtantsurugh12:31
*** mkrai_ has quit IRC12:31
iurygregory=(12:31
*** Lucas_Gray has joined #openstack-ironic12:32
*** dougsz has joined #openstack-ironic12:36
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Make get_partition_uuids work with whole disk images  https://review.opendev.org/73619112:40
*** diurnalist has quit IRC12:40
dtantsur"Major events are to remain banned until at least the end of October in Germany", so much for the Summit :(12:44
rpittauwow12:46
rpittauwell it's going to be same here as well probably, still don't know if I will be able to go back to Sardegna this summer...12:46
TheJuliadtantsur: where did you hear/red that?12:46
dtantsurTheJulia: https://www.thelocal.de/20200617/major-events-to-be-banned-in-germany-until-end-of-october12:47
dtantsurrpittau: personal travels should be fine soon12:48
rpittaudtantsur: as we say 'bucca tua santa' ~bless your words :)12:49
iurygregorydtantsur, they can do the summit in CZ12:49
dtantsurlol12:49
iurygregoryhere you can already have events with 2500 people12:49
dtantsurwell, I'd not mind a summit in Prague or Vienna, really12:49
iurygregory=(12:49
rpittaudepends if CZ accepts foreigners12:49
iurygregoryrpittau, it does already12:49
dtantsurwell, I'm not sure how long it will take until EU starts accepting people from the USA, Russia or China12:49
rpittauah cool12:50
dtantsuriurygregory: inside EU - yes. for outsiders it's much more complicated.12:50
iurygregorydepending on the country you need to show the negative test12:50
TheJuliawell, I'm fairly sure nobody will be allowing folks from the US, so the question then becomes how many people does that cut out of an event?12:50
dtantsurTheJulia: minus USA, minus China, minus former USSR, a half would be optimistic12:51
rpittauor from China, considering recent events.....12:51
iurygregoryyeah12:51
iurygregorymy feeling is that CZ is moving to fast allowing such things (but maybe is just me)12:51
dtantsurit probably is12:51
rpittaubtw good morning TheJulia :D12:52
dtantsurand yes, good (heh) morning TheJulia12:52
TheJuliagood morning12:52
iurygregorygood morning TheJulia =)12:52
iurygregorywelcome to the CI without useful logs =)12:52
TheJuliaseriously?12:53
dtantsurlogs are overrated *drops the mic and walks out*12:53
rpittauuntil yesterday the logs were there, then a black hole happened12:53
iurygregoryyup no ironic-bm-logs folder , no /etc folder, no tempest.log12:53
iurygregorytks zuul \o/12:54
dtantsurexample?12:54
rpittaueverything looks fine here https://549a2824969dd0dbcdfc-714725c6ff301dc1cc967cdae554d0b2.ssl.cf5.rackcdn.com/731159/2/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/5da04c5/job-output.txt12:54
rpittauwhich is actually from the 1512:54
iurygregoryI think the reason is the ansible update they did12:54
dtantsurmmm, right, no logs in https://zuul.opendev.org/t/openstack/build/45d730d881484a5fbcc8c1f49a07ee4d/logs12:54
*** rloo has joined #openstack-ironic12:54
rpittaulooks like it's selectively not copying some dirs12:55
iurygregoryyesterday at 4 UTC I think12:55
dtantsurcan someone please raise it to #openstack-infra?12:55
iurygregorydtantsur, frickler now about the issue12:55
iurygregoryI talked with him on openstack-qa12:55
iurygregorythey were talking in #opendev12:56
dtantsurmmm, ok12:56
dtantsurthank you\12:56
* dtantsur desperately tries to make fast-track work12:56
iurygregoryin the begin I thought it was something on the devstack-tempest job but no =(12:57
TheJuliadtantsur: It did work at some point, what is not working for you now?12:57
dtantsurTheJulia: https://review.opendev.org/73616412:58
patchbotpatch 736164 - ironic - [WIP] Fix fast-track with the direct deploy interface - 1 patch set12:58
dtantsurnot sure when and how the issue got introduced, but it's pretty badly broken now12:58
dtantsur+ that agent URL/token problems from yesterday (I'm past them now)12:58
rpittauTheJulia: not sure if it's related, but we're seeing some errors related to the token, like 'Invalid or missing agent token received'12:58
TheJuliaugh, I put the partitioning call an  hour too early on my calendar13:03
TheJulia#couldhavesleptin13:03
dtantsurouch13:03
rpittau:/13:03
TheJuliarpittau: older agents with newer ironics?13:03
rpittaummmmm13:04
rpittauit's in a recent job13:04
openstackgerritDmitry Tantsur proposed openstack/ironic master: Networking boot fallback for local boot  https://review.opendev.org/73449013:04
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Agent power interface  https://review.opendev.org/73559113:04
TheJuliarpittau: got a link13:07
TheJulia?13:07
rpittauthe image seems to be from yesterday13:07
rpittauyeah13:07
rpittauTheJulia: conductor logs https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b61/734759/8/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool/b61a274/controller/logs/screen-ir-cond.txt13:07
rpittauwe don't have ipa ramdisk logs :/13:08
TheJulialooks like we asked the agent for its logs before that13:10
TheJuliathen it heartbeated again13:10
TheJuliaYeah, we can ignore that it looks like. We were not in fast track so we collected the logs before successfully exiting cleaning13:14
TheJuliaWe went to tear down, but it triggered a heartbeat between tear_down_cleaning and token removal13:14
TheJuliahmm13:14
TheJuliayup, ignorable13:15
TheJuliajust looks nasty13:15
rpittauok13:17
rpittauwithout other logs hard to say what went wrong13:18
rpittauthe only difference is in the ramdisk13:18
openstackgerritJulia Kreger proposed openstack/ironic master: Minor agent token log handling  https://review.opendev.org/73621413:20
TheJuliarpittau: ^^^13:20
TheJuliathat should make the backtrace disappear13:20
rpittauthanks :)13:20
TheJuliaand then the world will be a happier place with less red herrings13:20
dtantsurhmm, my bifrost environment decided to stop PXE booting, lovely13:20
openstackgerritRiccardo Pittau proposed openstack/ironic master: Convert ironic-tempest-ipa-wholedisk-bios-agent_ipmitool to dib  https://review.opendev.org/73475913:21
TheJuliadtantsur: well, I can tell you bifrost was not written with irish cream13:22
TheJuliaso it may be demanding an irish whiskey13:23
dtantsurouch, I only have scotch here :(13:23
dtantsurc'mon, boot the freaking kernel finally :(13:24
dtantsurthe strange thing, I don't know what made it stop booting. it worked like an hour ago. and PXE itself works, it just stops right afterwards..13:24
TheJuliadtantsur: I have vague memorys of having to bounce/restart dnsmasq at some point, maybe it is hung or out of reservations?13:26
TheJulias/reservations/possible leases/13:26
gudrutis2Hi people13:27
gudrutis2Is there a snippet how to create alembic migration script?13:27
dtantsurTheJulia: dnsmasq looks happy :(13:29
TheJuliadtantsur: networking in general?13:29
dtantsurhard to say. I don't know what it's doing even, it has downloaded IPA alright13:29
TheJuliagudrutis2: One an do it manually if one knows the style and the head point but generally there is alembic helper utility to create the skeleton13:29
TheJuliadtantsur: so the ramdisk is booting?13:30
TheJuliagudrutis2: sadly, and I suspect this is the case for all of us long time ironic devs... we change the db model so rarely that we surely have to look it up completely every single time13:30
dtantsurTheJulia: maybe? it stopped right after iPXE and stays there for tens of minutes already13:30
dtantsurgudrutis2: https://docs.openstack.org/ironic-inspector/latest/contributor/#making-changes-to-the-database13:31
TheJuliadtantsur: is console=ttyS0 in the pxe command line?13:31
TheJulia\o/13:31
dtantsurTheJulia: I haven't changed anything recently, so should be (will double check)13:31
TheJuliairccloud != vim13:31
dtantsuryeah, it's there13:32
* dtantsur tries again13:33
*** diurnalist has joined #openstack-ironic13:33
TheJuliacorrupt kernel/ramdisk?13:38
TheJuliaI guess that is a risk if you re-ran a complete install and maybe grabbed an inflight copy on a mirror13:38
TheJuliaplus side of breaking tests you wrote in the previous retry patch. your tests are easy to fix!13:39
*** rpittau is now known as rpittau|brb13:39
dtantsurI didn't reinstall, only updated the ironic source...13:39
dtantsuroh, which tests are you talking about? :)13:39
*** priteau has quit IRC13:40
*** ricolin has quit IRC13:40
TheJuliadtantsur: well, further enhancing mid stream download failure retries13:40
TheJuliaso if a major break kills the download, we will completely retry13:40
dtantsurah, right13:40
dtantsuryeah, I remember now. sorry, my brain is overloaded with this discovery+fast-track+agent-power thing13:41
TheJuliano worries13:41
TheJuliano worries, I think this should take care of the downstream retry headaches even though I'm pushing back on lacp it should all help the agent navigate hard breaks that last the period of port hold-downs13:42
*** tzumainn has joined #openstack-ironic13:43
openstackgerritMerged openstack/metalsmith stable/ussuri: Use is instead of ambiguous pipe  https://review.opendev.org/73615313:45
gudrutis2TheJulia: yep, I found the utility, and it generated the script and ... It would have been more useful to clone and modify existing script because it just generated an empty body :)13:45
TheJuliagudrutis2: the key is the positional data at the top of the migration script though :)13:46
dtantsurgudrutis2: there is some gotcha there13:46
dtantsurlike, you need to do 'upgrade', then apply your change to models, then generate the migration13:46
dtantsurif you run upgrade *after* modifying models, you'll get an empty migration13:46
dtantsuror something like that13:46
TheJuliaI think there is yet another caveat, but it has been so long that I simply don't remember anymore13:47
TheJulia(shows how often I touch our data models)13:48
gudrutis2well I am following this tutorial https://alembic.sqlalchemy.org/en/latest/tutorial.html13:48
gudrutis2I will mess around and see if I will corupt my testack DB or not :)13:49
TheJuliaeasy enough to recreate if you do13:49
tzumainnhi! I have a question about a one-line change in the database layer - https://review.opendev.org/#/c/730366/2 - which allows ports to be seen not only by node owners, but also node lessees13:50
patchbotpatch 730366 - ironic - Allow node lessee to see node's ports - 2 patch sets13:50
TheJuliaspeaking of databases!13:50
gudrutis2:D13:50
tzumainnthe change updates a filter called add_port_filter_by_node_owner - so my question would be - should I change the name of the filter function?13:50
tzumainnor should I create a new one?13:51
tzumainnalso the change means that a bunch of functions going up to the API layer is no longer filtering by owner, but filtering by project, which would then match owner or lessee13:51
tzumainncan I change the name of the parameter being used in those functions from owner to project, or do I have to add a new parameter alongside owner?13:52
*** rpittau|brb is now known as rpittau13:53
*** priteau has joined #openstack-ironic13:56
*** mkrai has joined #openstack-ironic13:59
TheJulialikely new parameter and method, I think13:59
tzumainnTheJulia, that makes sense the more I think about it - thanks!14:00
dtantsurJayF: hey, joining the partitioning discussion?14:01
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Add full download retries  https://review.opendev.org/73606514:02
openstackgerritMerged openstack/ironic-inspector master: Accept IPv6 link local address during interface validation  https://review.opendev.org/73611014:08
openstackgerritShivanand Tendulker proposed openstack/ironic master: Adds boot mode support to iLO management interface  https://review.opendev.org/73623214:15
*** rh-jelabarre has joined #openstack-ironic14:30
*** jovial[m] has joined #openstack-ironic14:34
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector stable/ussuri: Accept IPv6 link local address during interface validation  https://review.opendev.org/73624414:45
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector stable/train: Accept IPv6 link local address during interface validation  https://review.opendev.org/73624514:46
openstackgerritMark Goddard proposed openstack/tenks stable/1.0: CI: multiple fixes  https://review.opendev.org/73591714:53
openstackgerritMark Goddard proposed openstack/tenks master: WIP: test CI  https://review.opendev.org/73624914:54
*** Goneri has joined #openstack-ironic14:57
JayFdtantsur: Letting zer0c00l lead the way on this one; but I should be active in reviews once specs are up. I know just enough about our downstream use cases and current upstream support to be dangerous :)15:11
dtantsur:D15:11
dtantsuryou can check the etherpad to see where it's going15:11
JayFI have been :D15:11
openstackgerritMerged openstack/metalsmith stable/stein: Use is instead of ambiguous pipe  https://review.opendev.org/73617215:12
JayFYou should know by now I'm getting to professional-levels of lurking :D15:12
*** Lucas_Gray has quit IRC15:12
dtantsurindeed :)15:12
dtantsurrloo: it seems that we can support preseed with more or less the same code, which solves the problem of supporting non-RH systems (at least some of them)15:13
JayFThat's basically my only concern ^... although I'll note in reality, kickstart is 100x better than any other product for automated OS installs15:14
JayFso while it's good to be generic (and I want to advocate for that), admittedly the reality is that may not be actually super useful15:14
dtantsurright15:14
rpittauI took the liberty to add an Advanced Partitioning entry under Priorities in the Ironic whiteboard15:17
rpittauL39215:17
TheJuliarpittau: thanks15:17
*** sshnaidm is now known as sshnaidm|bbl15:17
TheJuliafwiw, if we ever have a service that does super advanced configs, I believe glitter may be required15:18
JayFis that a utility, or are you referring to the infectious crafting material?15:19
TheJuliainfectious crafting material15:19
JayF[nope]15:19
dtantsurheh15:19
rpittaulike...? https://media.giphy.com/media/xTiTnEHBh7qapyuvwQ/giphy.gif15:20
dtantsur:D15:21
rloodtantsur: ++ i think supporting kickstart and preseed (and others in the future probably?) with similar interface/mechanism would be good.15:21
rloomoar drivers/interfaces!15:21
dtantsurwell, I haven't done the driver composition reform for nothing ;)15:22
dtantsurI still wonder if we should permit the users to change the deploy interface somehow..15:22
TheJuliahttps://usercontent.irccloud-cdn.com/file/UJaGWALX/glitter.gif15:26
rpittauwow15:26
JayFdtantsur: I was thinking, maybe more like let glance metadata (or some external method for standalone) help dictate the deploy interface / deploy interface settings15:28
NobodyCamI like the glitterBoots15:28
NobodyCamnd OOFC Good Morning all15:28
JayFdtantsur: booting into an image that's tagged:is_ramdisk? Then use ramdisk driver, same for "kickstart" or "preseed", etc15:28
* JayF doesn't think glance calls it tags? Maybe metadata?15:29
TheJuliadtantsur: I keep wondering that....15:29
TheJuliametadata I believe15:29
TheJuliaFor non-openstack users I guess they would stsill have to change the drivers, but I suspect that is somewhere in the neighborhood of perfectly okay15:29
TheJuliaNobodyCam: I wanted it to color it like pixie boots, but the editor didn't listen and... *shrugs*15:31
TheJuliafor a 30 second amusement, it is not bad :)15:31
JayFwe may now have a :pixieglitter: in VzM slack15:32
dtantsurTheJulia: I'm thinking of including deploy_interface to a post-MVP iteration of the deployment API (re standalone users)15:36
JayFonly thing that gets awkward about that, is unless you change data model (currentDeployInterface vs defaultDeployInterface) you can make deployments to a node inconsistent based on what was previously used to deploy them15:38
JayFe.g. I have a saved script that calls deployment API without deploy interface, run it, it works. Someone else does a deploy, changes the deploy interface. Now my previous tested-working script broke, despite me using the API identically.15:39
dtantsurJayF: I was thinking about instance_info/deploy_interface that overrides node.deploy_interface (I've already done a similar thing with root_device)15:39
dtantsur(loading drivers will get funny at this point, of course)15:39
JayFthat makes a lot of sense +115:40
openstackgerritTzu-Mainn Chen proposed openstack/ironic master: Allow node lessee to see node's ports  https://review.opendev.org/73036615:48
NobodyCamTheJulia: I like it!15:54
openstackgerritMerged openstack/ironic-inspector master: CI: make grenade voting again  https://review.opendev.org/73557915:57
*** gyee has joined #openstack-ironic15:58
*** lucasagomes has quit IRC16:01
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Add timeout operations to try and prevent hang on read()  https://review.opendev.org/73470916:02
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Add full download retries  https://review.opendev.org/73606516:04
*** dtantsur is now known as dtantsur|afk16:12
dtantsur|afko/16:12
rpittaubye everyone! o/16:17
*** rpittau is now known as rpittau|afk16:17
TheJuliagoodnigth16:17
*** mkrai has quit IRC16:18
TheJuliadtantsur|afk: for tomorrow, those download retry patches are goign to be tried out by our field folks in that environment they are fighting with.16:21
arne_wiebalckbye everyone o/16:23
*** Qianbiao has quit IRC16:32
*** dougsz has quit IRC16:34
*** ociuhandu_ has joined #openstack-ironic16:38
*** ociuhandu has quit IRC16:42
*** ociuhandu_ has quit IRC16:43
*** ociuhandu has joined #openstack-ironic16:47
iurygregoryfinally zuul gave +1 in the backport to fix stable/ussuri \o/16:48
TheJuliawoot16:50
*** ociuhandu has quit IRC16:51
*** uzumaki has joined #openstack-ironic16:54
*** derekh has quit IRC17:01
*** k_mouza has quit IRC17:16
*** k_mouza has joined #openstack-ironic17:16
*** k_mouza has quit IRC17:21
openstackgerritJulia Kreger proposed openstack/ironic master: iPXE ISO Ramdisk booting  https://review.opendev.org/72946417:33
openstackgerritVerification of a change to openstack/ironic-inspector failed: Accept IPv6 link local address during interface validation  https://review.opendev.org/73624418:00
*** naved001 has joined #openstack-ironic18:25
TheJulianeat hostname :)18:41
*** diurnalist has quit IRC19:00
*** sshnaidm|bbl is now known as sshnaidm19:09
*** eliaswimmer has quit IRC19:09
*** eliaswimmer has joined #openstack-ironic19:13
openstackgerritMerged openstack/bifrost master: Migrate pip_install.yml to a new role bifrost-pip-install  https://review.opendev.org/73546119:28
*** diurnalist has joined #openstack-ironic19:30
openstackgerritMerged openstack/ironic stable/ussuri: devstack: Use uwsgi binary from path; temporary disable grenade  https://review.opendev.org/73601319:38
iurygregory\o/19:39
*** ociuhandu has joined #openstack-ironic19:52
*** pas-ha_ has joined #openstack-ironic20:11
*** Lucas_Gray has joined #openstack-ironic20:11
*** mbuil_ has joined #openstack-ironic20:13
*** gyee has quit IRC20:17
*** cp-- has joined #openstack-ironic20:18
*** irclogbot_2 has quit IRC20:18
*** bfournie has quit IRC20:18
*** mbuil has quit IRC20:18
*** pas-ha has quit IRC20:18
*** buhman has quit IRC20:18
*** mgoddard has quit IRC20:18
*** dtruong has quit IRC20:18
*** cp- has quit IRC20:18
*** JayF has quit IRC20:18
*** zer0c00l has quit IRC20:18
*** pas-ha_ is now known as pas-ha20:18
*** dtruong has joined #openstack-ironic20:19
*** irclogbot_2 has joined #openstack-ironic20:21
*** mgoddard has joined #openstack-ironic20:21
*** buhman has joined #openstack-ironic20:23
openstackgerritMerged openstack/ironic-inspector stable/train: Accept IPv6 link local address during interface validation  https://review.opendev.org/73624520:26
*** gyee has joined #openstack-ironic20:31
*** ildikov has quit IRC20:35
*** pas-ha has quit IRC20:35
*** rpioso has quit IRC20:38
*** tonyb[m] has quit IRC20:43
openstackgerritJulia Kreger proposed openstack/ironic-tempest-plugin master: Ramdisk iso boot tempest scenario test  https://review.opendev.org/73637820:47
*** ociuhandu has quit IRC20:48
*** PrinzElvis has quit IRC20:49
*** priteau has quit IRC20:49
*** PrinzElvis has joined #openstack-ironic20:52
*** PrinzElvis has quit IRC20:57
*** JasonF has joined #openstack-ironic21:06
*** JasonF is now known as JayF21:06
*** diurnalist has quit IRC21:06
*** diurnalist has joined #openstack-ironic21:17
*** ociuhandu has joined #openstack-ironic21:19
openstackgerritJulia Kreger proposed openstack/ironic master: ISO ramdisk virtual media test enablement  https://review.opendev.org/73638321:22
*** ociuhandu has quit IRC21:24
*** rh-jelabarre has quit IRC21:24
* TheJulia hopes that was enough but will take a long time to get through initial CI21:24
*** ildikov has joined #openstack-ironic21:33
*** pas-ha has joined #openstack-ironic21:34
*** PrinzElvis has joined #openstack-ironic21:35
openstackgerritSteve Baker proposed openstack/ironic master: Enable HTTP Basic authentication for JSON-RPC  https://review.opendev.org/73575721:36
*** rpioso has joined #openstack-ironic21:37
*** tonyb[m] has joined #openstack-ironic21:38
*** k_mouza has joined #openstack-ironic21:45
stevebakerTheJulia: hey I notice the ironic-standalone devstack job still uses keystone. Should I look at switching that to http_basic, at least for the json-rpc calls?21:45
TheJuliastevebaker: I'm afraid that the tempest suite is heavily baked into the ci jobs for that21:47
stevebakerTheJulia: yeah but maybe json-rpc is sufficiently internal to not make a difference to tempest21:48
*** k_mouza has quit IRC21:49
TheJuliaoh, for jsonrpc. I guess that is doable21:50
TheJuliathat would make a lot of sense actually21:50
TheJuliaI wasn't just thinking json-rpc21:50
stevebakerTheJulia: yeah API would be nice, but tempest issues aside, glance would still need keystone. I could take a look at what it would take for tempest to use file:// based images, then we could maybe switch off glance and swift?21:53
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Extend retry interval to 30 seconds  https://review.opendev.org/73639021:54
TheJuliaI was just actually looking at that and I put up a patch to just pass a URL. Lets see if it works!21:54
TheJulia#popcorn21:54
stevebakerwoop woop21:54
TheJuliahttps://zuul.opendev.org/t/openstack/stream/ba4df76c875043e4b11753cf862193e3?logfile=console.log21:55
openstackgerritVladyslav Drok proposed openstack/ironic master: Stop allocating double space for raw images  https://review.opendev.org/73639122:02
*** mrda has quit IRC22:27
*** mrda has joined #openstack-ironic22:30
stevebakerTheJulia: only one test failed22:41
*** rcernin has joined #openstack-ironic22:42
TheJulia\o/22:50
TheJuliahmm... the tests skipped :\22:51
*** seongsoocho has joined #openstack-ironic22:51
*** gyee has quit IRC22:52
openstackgerritSteve Baker proposed openstack/ironic master: devstack json-rpc, use http basic auth  https://review.opendev.org/73639822:53
*** tkajinam has joined #openstack-ironic22:54
*** Lucas_Gray has quit IRC23:07
*** hoonetorg has quit IRC23:10
openstackgerritVerification of a change to openstack/ironic failed: devstack: Use uwsgi binary from path; temporary disable grenade  https://review.opendev.org/73601523:20
*** Lucas_Gray has joined #openstack-ironic23:22
*** hoonetorg has joined #openstack-ironic23:23
*** diurnalist has quit IRC23:30

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!